124 results on '"Güçlütürk, Y."'
Search Results
2. Brain2GAN: Reconstructing perceived faces from the primate brain via StyleGAN3
- Author
-
Dado, T.M., Papale, P., Lozano, A., Le, L., Wang, F., Gerven, M.A.J. van, Roelfsema, P.R., Güçlütürk, Y., Güçlü, U., Dado, T.M., Papale, P., Lozano, A., Le, L., Wang, F., Gerven, M.A.J. van, Roelfsema, P.R., Güçlütürk, Y., and Güçlü, U.
- Abstract
ICLR 2023: The Eleventh International Conference on Learning Representations (Kigali, Rwanda, May 1-5, 2023), Item does not contain fulltext, Neural coding characterizes the relationship between stimuli and their corresponding neural responses. The usage of synthesized yet photorealistic reality by generative adversarial networks (GANs) allows for superior control over these data: the underlying feature representations that account for the semantics in synthesized data are known a priori and their relationship is perfect rather than approximated post-hoc by feature extraction models. We exploit this property in neural decoding of multi-unit activity responses that we recorded from the primate brain upon presentation with synthesized face images in a passive fixation experiment. The face reconstructions we acquired from brain activity were astonishingly similar to the originally perceived face stimuli. This provides strong evidence that the neural face manifold and the disentangled w-latent space conditioned on StyleGAN3 (rather than the z-latent space of arbitrary GANs or other feature representations we encountered so far) share how they represent the high-level semantics of the high-dimensional space of faces.
- Published
- 2023
3. End-to-end reconstruction of natural images from multi-unit recordings with Brain2Pix
- Author
-
Le, L., Papale, P., Lozano, A., Dado, T.M., Wang, F., Gerven, M.A.J. van, Roelfsema, P.R., Güçlütürk, Y., Güçlü, U., Le, L., Papale, P., Lozano, A., Dado, T.M., Wang, F., Gerven, M.A.J. van, Roelfsema, P.R., Güçlütürk, Y., and Güçlü, U.
- Abstract
Item does not contain fulltext, Reconstructing naturalistic images from brain signals has been a challenging task for scientists, with successful results largely limited to large human fMRI datasets. In this study, we apply the brain2pix reconstruction model to multi-unit activity (MUA) data from the macaque brain, providing a novel extension of the model. This approach allows for investigation of information representation in different brain regions and time windows with greater spatial and temporal precision. Our results offer insights into the neural basis of visual perception, showing that V1 neurons represent texture and color, V4 neurons exhibit symmetric representations, and IT neurons reveal concept-like features. We also demonstrate that the model can be used to decode features at different layers of a neural network, with V1 more strongly correlated with initial layers and V4 and IT with deeper layers. Overall, our approach provides a valuable tool for studying brain representations in high temporal and spatial detail., CCN 2023: Conference on Cognitive Computational Neuroscience (Oxford, UK, August 24 - 27, 2023)
- Published
- 2023
4. Feature-disentangled reconstruction of perception from multi-unit recording
- Author
-
Dado, T.M., Papale, P., Lozano, A., Le, L., Gerven, M.A.J. van, Roelfsema, P.R., Güçlütürk, Y., Güçlü, U., Dado, T.M., Papale, P., Lozano, A., Le, L., Gerven, M.A.J. van, Roelfsema, P.R., Güçlütürk, Y., and Güçlü, U.
- Abstract
Item does not contain fulltext, Here, we aimed to explain neural representations of perception, for which we analyzed the relationship between multi-unit activity (MUA) recorded from the primate brain and various feature representations of visual stimuli. Our encoding analysis revealed that the $w$-latent representations of feature-disentangled generative adversarial networks (GANs) were the most effective candidate for predicting neural responses to images. Importantly, the usage of synthesized yet photorealistic images allowed for superior control over these data as their underlying latent representations were known a priori rather than approximated post-hoc. As such, we leveraged this property in neural reconstruction of the perceived images. Taken together with the fact that the (unsupervised) generative models themselves were never optimized on neural data, these results highlight the importance of feature disentanglement and unsupervised training as driving factors in shaping neural representations., CCN 2023: Conference on Cognitive Computational Neuroscience (Oxford, UK, August 24 - 27, 2023)
- Published
- 2023
5. First impressions: A survey on vision-based apparent personality trait analysis
- Author
-
Jacques Junior, J.C.S., Güçlütürk, Y., Pérez, M., Güçlü, U., Andujar, C., Baró, X., Escalante, H.J., Guyon, I., Gerven, M.A.J. van, Lier, R.J. van, Escalera, S., Jacques Junior, J.C.S., Güçlütürk, Y., Pérez, M., Güçlü, U., Andujar, C., Baró, X., Escalante, H.J., Guyon, I., Gerven, M.A.J. van, Lier, R.J. van, and Escalera, S.
- Abstract
Contains fulltext : 219469.pdf (Publisher’s version ) (Closed access), Personality analysis has been widely studied in psychology, neuropsychology, and signal processing fields, among others. From the past few years, it also became an attractive research area in visual computing. From the computational point of view, by far speech and text have been the most considered cues of information for analyzing personality. However, recently there has been an increasing interest from the computer vision community in analyzing personality from visual data. Recent computer vision approaches are able to accurately analyze human faces, body postures and behaviors, and use these information to infer apparent personality traits. Because of the overwhelming research interest in this topic, and of the potential impact that this sort of methods could have in society, we present in this paper an up-to-date review of existing vision-based approaches for apparent personality trait recognition. We describe seminal and cutting edge works on the subject, discussing and comparing their distinctive features and limitations. Future venues of research in the field are identified and discussed. Furthermore, aspects on the subjectivity in data labeling/evaluation, as well as current datasets and challenges organized to push the research on the field are reviewed.
- Published
- 2022
6. Hyperrealistic neural decoding for reconstructing faces from fMRI activations via the GAN latent space
- Author
-
Dado, T.M., Güçlütürk, Y., Ambrogioni, L., Ras, G.E.H., Bosch, S.E., Gerven, M.A.J. van, Güçlü, U., Dado, T.M., Güçlütürk, Y., Ambrogioni, L., Ras, G.E.H., Bosch, S.E., Gerven, M.A.J. van, and Güçlü, U.
- Abstract
Contains fulltext : 246460.pdf (Publisher’s version ) (Open Access), Neural decoding can be conceptualized as the problem of mapping brain responses back to sensory stimuli via a feature space. We introduce (i) a novel experimental paradigm that uses well-controlled yet highly naturalistic stimuli with a priori known feature representations and (ii) an implementation thereof for HYPerrealistic reconstruction of PERception (HYPER) of faces from brain recordings. To this end, we embrace the use of generative adversarial networks (GANs) at the earliest step of our neural decoding pipeline by acquiring fMRI data as participants perceive face images synthesized by the generator network of a GAN. We show that the latent vectors used for generation effectively capture the same defining stimulus properties as the fMRI measurements. As such, these latents (conditioned on the GAN) are used as the in-between feature representations underlying the perceived images that can be predicted in neural decoding for (re-)generation of the originally perceived stimuli, leading to the most accurate reconstructions of perception to date.
- Published
- 2022
7. Real-world indoor mobility with simulated prosthetic vision: The benefits and feasibility of contour-based scene simplification at different phosphene resolutions
- Author
-
Ruyter van Steveninck, J. de, Gestel, T. van, Koenders, P., Ham, G. van der, Vereecken, F., Güçlü, U., Gerven, M.A.J. van, Güçlütürk, Y., Wezel, R.J.A. van, Ruyter van Steveninck, J. de, Gestel, T. van, Koenders, P., Ham, G. van der, Vereecken, F., Güçlü, U., Gerven, M.A.J. van, Güçlütürk, Y., and Wezel, R.J.A. van
- Abstract
Contains fulltext : 246314.pdf (Publisher’s version ) (Open Access), Neuroprosthetic implants are a promising technology for restoring some form of vision in people with visual impairments via electrical neurostimulation in the visual pathway. Although an artificially generated prosthetic percept is relatively limited compared with normal vision, it may provide some elementary perception of the surroundings, re-enabling daily living functionality. For mobility in particular, various studies have investigated the benefits of visual neuroprosthetics in a simulated prosthetic vision paradigm with varying outcomes. The previous literature suggests that scene simplification via image processing, and particularly contour extraction, may potentially improve the mobility performance in a virtual environment. In the current simulation study with sighted participants, we explore both the theoretically attainable benefits of strict scene simplification in an indoor environment by controlling the environmental complexity, as well as the practically achieved improvement with a deep learning-based surface boundary detection implementation compared with traditional edge detection. A simulated electrode resolution of 26 x 26 was found to provide sufficient information for mobility in a simple environment. Our results suggest that, for a lower number of implanted electrodes, the removal of background textures and within-surface gradients may be beneficial in theory. However, the deep learning-based implementation for surface boundary detection did not improve mobility performance in the current study. Furthermore, our findings indicate that, for a greater number of electrodes, the removal of within-surface gradients and background textures may deteriorate, rather than improve, mobility. Therefore, finding a balanced amount of scene simplification requires a careful tradeoff between informativity and interpretability that may depend on the number of implanted electrodes.
- Published
- 2022
8. A biologically plausible phosphene simulator for the optimization of visual cortical prostheses
- Author
-
Grinten, M. van der, Ruyter van Steveninck, J. de, Lozano, A., Roelfsema, P.R., Gerven, M.A.J. van, Wezel, R.J.A. van, Güçlü, U., Güçlütürk, Y., Grinten, M. van der, Ruyter van Steveninck, J. de, Lozano, A., Roelfsema, P.R., Gerven, M.A.J. van, Wezel, R.J.A. van, Güçlü, U., and Güçlütürk, Y.
- Abstract
Item does not contain fulltext
- Published
- 2022
9. Modeling, recognizing, and explaining apparent personality from videos
- Author
-
Escalante, H.J., Kaya, H., Salah, A.A., Escalera, S., Güçlütürk, Y., Güçlü, U., Baró, X., Guyon, I., Jacques Junior, J.C.S., Madadi, M., Ayache, S., Viegas, E., Gurpinar, F., Sukma Wicaksana, A., Liem, C.C.S., Gerven, M.A.J. van, Lier, R.J. van, Escalante, H.J., Kaya, H., Salah, A.A., Escalera, S., Güçlütürk, Y., Güçlü, U., Baró, X., Guyon, I., Jacques Junior, J.C.S., Madadi, M., Ayache, S., Viegas, E., Gurpinar, F., Sukma Wicaksana, A., Liem, C.C.S., Gerven, M.A.J. van, and Lier, R.J. van
- Abstract
Contains fulltext : 221841.pdf (postprint version ) (Open Access), Explainability and interpretability are two critical aspects of decision support systems. Despite their importance, it is only recently that researchers are starting to explore these aspects. This paper provides an introduction to explainability and interpretability in the context of apparent personality recognition. To the best of our knowledge, this is the first effort in this direction. We describe a challenge we organized on explainability in first impressions analysis from video. We analyze in detail the newly introduced data set, evaluation protocol, proposed solutions and summarize the results of the challenge. We investigate the issue of bias in detail. Finally, derived from our study, we outline research opportunities that we foresee will be relevant in this area in the near future.
- Published
- 2022
10. Brain2Pix: Fully convolutional naturalistic video frame reconstruction from brain activity
- Author
-
Le, L., Ambrogioni, L., Seeliger, K., Güçlütürk, Y., Gerven, M.A.J. van, Güçlü, U., Le, L., Ambrogioni, L., Seeliger, K., Güçlütürk, Y., Gerven, M.A.J. van, and Güçlü, U.
- Abstract
Contains fulltext : 270653.pdf (Publisher’s version ) (Open Access), Reconstructing complex and dynamic visual perception from brain activity remains a major challenge in machine learning applications to neuroscience. Here, we present a new method for reconstructing naturalistic images and videos from very large single-participant functional magnetic resonance imaging data that leverages the recent success of image-to-image transformation networks. This is achieved by exploiting spatial information obtained from retinotopic mappings across the visual system. More specifically, we first determine what position each voxel in a particular region of interest would represent in the visual field based on its corresponding receptive field location. Then, the 2D image representation of the brain activity on the visual field is passed to a fully convolutional image-to-image network trained to recover the original stimuli using VGG feature loss with an adversarial regularizer. In our experiments, we show that our method offers a significant improvement over existing video reconstruction techniques.
- Published
- 2022
11. Congruency effects in crossmodal art perception: Differential cortical activations
- Author
-
Yilmaz, F., Leeuwen, T.M. van, Wintermans, A.A.P.M., Güçlü, U., Güçlütürk, Y., Lier, R.J. van, Yilmaz, F., Leeuwen, T.M. van, Wintermans, A.A.P.M., Güçlü, U., Güçlütürk, Y., and Lier, R.J. van
- Abstract
Item does not contain fulltext, 1 p.
- Published
- 2022
12. Hyperrealistic neural decoding of faces
- Author
-
Dado, T.M., Güçlütürk, Y., Ambrogioni, L., Ras, G.E.H., Bosch, S.E., Gerven, M.A.J. van, Güçlü, U., Dado, T.M., Güçlütürk, Y., Ambrogioni, L., Ras, G.E.H., Bosch, S.E., Gerven, M.A.J. van, and Güçlü, U.
- Abstract
NER'21: 10th International IEEE EMBS Conference on Neural Engineering (4-6 May 2021), Item does not contain fulltext, We show how generative modeling approximates the neural (face) manifold by obtaining state-of-the-art reconstructions of perceived face images from fMRI activations.
- Published
- 2021
13. Improved processing strategy for head pose detection in phosphene vision
- Author
-
Lierop, S.N.C. van, Borne, E.W.P. van den, Güçlütürk, Y., Lierop, S.N.C. van, Borne, E.W.P. van den, and Güçlütürk, Y.
- Abstract
NER'21: 10th International IEEE EMBS Conference on Neural Engineering (4-6 May 2021), Item does not contain fulltext, Cortical visual prostheses are emerging as a technology for restoring limited visual perception abilities for blind people in the form of phosphene vision. Due to limitations related to hardware and brain physiology, intelligent processing strategies are needed to transform images into meaningful phosphene representations. Here we present a strategy to improve head pose perception in phosphene vision, utilizing methods from computer graphics and computer vision.
- Published
- 2021
14. Recognition of humans using simulated prosthetic vision
- Author
-
Hartjes, J., Güçlütürk, Y., Hartjes, J., and Güçlütürk, Y.
- Abstract
NER'21: 10th International IEEE EMBS Conference on Neural Engineering (4-6 May 2021), Item does not contain fulltext, Focusing on one of the most significant stimulus types that we encounter in our daily lives, we introduce and compare several methods of representing freely moving humans in prosthetic phosphene vision via realistic simulations and preliminary behavioral experiments.
- Published
- 2021
15. End-to-end neural system identification with neural information flow
- Author
-
Seeliger, K., Ambrogioni, L., Güçlütürk, Y., Güçlü, U., Gerven, M.A.J. van, Seeliger, K., Ambrogioni, L., Güçlütürk, Y., Güçlü, U., and Gerven, M.A.J. van
- Abstract
Contains fulltext : 230081.pdf (preprint version ) (Open Access), Neural information flow (NIF) provides a novel approach for system identification in neuroscience. It models the neural computations in multiple brain regions and can be trained end-to-end via stochastic gradient descent from noninvasive data. NIF models represent neural information processing via a network of coupled tensors, each encoding the representation of the sensory input contained in a brain region. The elements of these tensors can be interpreted as cortical columns whose activity encodes the presence of a specific feature in a spatiotemporal location. Each tensor is coupled to the measured data specific to a brain region via low-rank observation models that can be decomposed into the spatial, temporal and feature receptive fields of a localized neuronal population. Both these observation models and the convolutional weights defining the information processing within regions are learned end-to-end by predicting the neural signal during sensory stimulation. We trained a NIF model on the activity of early visual areas using a large-scale fMRI dataset recorded in a single participant. We show that we can recover plausible visual representations and population receptive fields that are consistent with empirical findings.
- Published
- 2021
16. End-to-end neural system identification with neural information flow
- Author
-
Seeliger, K., primary, Ambrogioni, L., additional, Güçlütürk, Y., additional, van den Bulk, L. M., additional, Güçlü, U., additional, and van Gerven, M. A. J., additional
- Published
- 2021
- Full Text
- View/download PDF
17. Psychology Meets Machine Learning: Interdisciplinary Perspectives on Algorithmic Job Candidate Screening
- Author
-
Liem, C.C.S., Langer, Markus, Demetriou, A.M., Hiemstra, Annemarie M.F., Achmadnoer Sukma Wicaksana, Sukma, Born, Marise Ph., König, Cornelis J., Jair Escalante, H., Escalera, S., Guyon, I., Baró, X., Güçlütürk, Y, and Work and Organizational Psychology
- Subjects
business.industry ,General problem ,05 social sciences ,Methodology ,Interdisciplinarity ,02 engineering and technology ,Job candidate screening ,Machine learning ,computer.software_genre ,Explainability ,Transparency (behavior) ,Multimodal analysis ,0502 economics and business ,Scalability ,0202 electrical engineering, electronic engineering, information engineering ,Psychology ,020201 artificial intelligence & image processing ,Artificial intelligence ,business ,computer ,050203 business & management - Abstract
In a rapidly digitizing world, machine learning algorithms are increasingly employed in scenarios that directly impact humans. This also is seen in job candidate screening. Data-driven candidate assessment is gaining interest, due to high scalability and more systematic assessment mechanisms. However, it will only be truly accepted and trusted if explainability and transparency can be guaranteed. The current chapter emerged from ongoing discussions between psychologists and computer scientists with machine learning interests, and discusses the job candidate screening problem from an interdisciplinary viewpoint. After introducing the general problem, we present a tutorial on common important methodological focus points in psychological and machine learning research. Following this, we both contrast and combine psychological and machine learning approaches, and present a use case example of a data-driven job candidate assessment system, intended to be explainable towards non-technical hiring specialists. In connection to this, we also give an overview of more traditional job candidate assessment approaches, and discuss considerations for optimizing the acceptability of technology-supported hiring solutions by relevant stakeholders. Finally, we present several recommendations on how interdisciplinary collaboration on the topic may be fostered.
- Published
- 2020
- Full Text
- View/download PDF
18. Modeling, recognizing, and explaining apparent personality from videos
- Author
-
Escalante, H.J., Kaya, H., Salah, A.A., Escalera, S., Güçlütürk, Y., Güçlü, U., Baró, X., Guyon, I., Jacques Junior, J.C.S., Madadi, M., Ayache, S., Viegas, E., Gurpinar, F., Sukma Wicaksana, A., Liem, C.C.S., Gerven, M.A.J. van, Lier, R.J. van, Escalante, H.J., Kaya, H., Salah, A.A., Escalera, S., Güçlütürk, Y., Güçlü, U., Baró, X., Guyon, I., Jacques Junior, J.C.S., Madadi, M., Ayache, S., Viegas, E., Gurpinar, F., Sukma Wicaksana, A., Liem, C.C.S., Gerven, M.A.J. van, and Lier, R.J. van
- Abstract
14 februari 2020, Contains fulltext : 221841.pdf (postprint version ) (Open Access), Explainability and interpretability are two critical aspects of decision support systems. Despite their importance, it is only recently that researchers are starting to explore these aspects. This paper provides an introduction to explainability and interpretability in the context of apparent personality recognition. To the best of our knowledge, this is the first effort in this direction. We describe a challenge we organized on explainability in first impressions analysis from video. We analyze in detail the newly introduced data set, evaluation protocol, proposed solutions and summarize the results of the challenge. We investigate the issue of bias in detail. Finally, derived from our study, we outline research opportunities that we foresee will be relevant in this area in the near future.
- Published
- 2020
19. Guest editorial: Image and video inpainting and denoising
- Author
-
Escalera, S., Escalante, H.J., Baró, X., Guyon, I., Madadi, M., Wan, J., Ayache, S., Güçlütürk, Y., Güçlü, U., Escalera, S., Escalante, H.J., Baró, X., Guyon, I., Madadi, M., Wan, J., Ayache, S., Güçlütürk, Y., and Güçlü, U.
- Abstract
Contains fulltext : 217792.pdf (publisher's version ) (Closed access)
- Published
- 2020
20. Explanation methods in deep learning: Users, values, concerns and challenges
- Author
-
Ras, G.E.H., Gerven, M.A.J. van, Haselager, W.F.G., Escalante, H.J., Escalera, S., Guyon, I., Baró, X., Güçlütürk, Y., Güçlü, U., Gerven, M. van, Escalante, H.J., Escalera, S., Guyon, I., Baró, X., Güçlütürk, Y., Güçlü, U., and Gerven, M. van
- Subjects
Computer science ,business.industry ,Deep learning ,Springer Series on Challenges in Machine Learning ,Perception, Action and Control [DI-BCB_DCC_Theme 2] ,020206 networking & telecommunications ,Context (language use) ,02 engineering and technology ,Cognitive artificial intelligence ,Data science ,Bridge (nautical) ,Brain Networks and Neuronal Communication [DI-BCB_DCC_Theme 4] ,020204 information systems ,General Data Protection Regulation ,Taxonomy (general) ,0202 electrical engineering, electronic engineering, information engineering ,Deep neural networks ,Artificial intelligence ,business ,Interpretability - Abstract
Item does not contain fulltext Issues regarding explainable AI involve four components: users, laws and regulations, explanations and algorithms. Together these components provide a context in which explanation methods can be evaluated regarding their adequacy. The goal of this chapter is to bridge the gap between expert users and lay users. Different kinds of users are identified and their concerns revealed, relevant statements from the General Data Protection Regulation are analyzed in the context of Deep Neural Networks (DNNs), a taxonomy for the classification of existing explanation methods is introduced, and finally, the various classes of explanation methods are analyzed to verify if user concerns are justified. Overall, it is clear that (visual) explanations can be given about various aspects of the influence of the input on the output. However, it is noted that explanation methods or interfaces for lay users are missing and we speculate which criteria these methods/interfaces should satisfy. Finally it is noted that two important concerns are difficult to address with explanation methods: the concern about bias in datasets that leads to biased DNNs, as well as the suspicion about unfair outcomes.
- Published
- 2018
21. Emotion Recognition with Simulated Phosphene Vision
- Author
-
Bollen, C.J.M., Wezel, R.J.A. van, Gerven, M.A.J. van, Güçlütürk, Y., Liu, X., Min, R., McDaniel, T., Liu, X., Min, R., and McDaniel, T.
- Subjects
Facial expression ,Landmark ,business.industry ,Computer science ,media_common.quotation_subject ,Biophysics ,Image processing ,Cognitive artificial intelligence ,Edge detection ,Visual field ,Phosphene ,Perception ,Computer vision ,Artificial intelligence ,Emotion recognition ,business ,media_common - Abstract
Contains fulltext : 215179.pdf (Publisher’s version ) (Open Access) Electrical stimulation of retina, optic nerve or cortex is found to elicit visual sensations, known as phosphenes. This allows visual prosthetics to partially restore vision by representing the visual field as a phosphene pattern. Since the resolution and performance of visual prostheses are limited, only a fraction of the information in a visual scene can be represented by phosphenes. Here, we propose a simple yet powerful image processing strategy for recognizing facial expressions with prosthetic vision, supporting communication and social interaction in the blind. A psychophysical study was conducted to investigate whether a landmark-based representation of facial expressions could improve emotion detection with prosthetic vision. Our approach was compared to edge detection, which is commonly used in current retinal prosthetic devices. Additionally, the relationship between the number of phosphenes and accuracy of emotion recognition was studied. The landmark model improved accuracy of emotion recognition, regardless of the number of phosphenes. Secondly, the accuracy improved with an increasing number of phosphenes up to a saturation point. The performance saturated with fewer phosphenes with the landmark model than with edge detection. These results suggest that landmark-based image pre-processing allows for a more efficient use of the limited information that can be stored in a phosphene pattern, providing a route towards more meaningful and higher-quality perceptual experience in subjects with prosthetic vision. MM '19: The 27th ACM International Conference on Multimedia (Nice, France, 21-25 October 2019)
- Published
- 2019
22. Wasserstein Variational Inference
- Author
-
Ambrogioni, L., Güçlü, U., Güçlütürk, Y., Hinne, M., van Gerven, M., Maris, E., Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R., and Psychologische Methodenleer (Psychologie, FMG)
- Subjects
Statistics::Other Statistics - Abstract
This paper introduces Wasserstein variational inference, a new form of approximate Bayesian inference based on optimal transport theory. Wasserstein variational inference uses a new family of divergences that includes both f-divergences and the Wasserstein distance as special cases. The gradients of the Wasserstein variational loss are obtained by backpropagating through the Sinkhorn iterations. This technique results in a very stable likelihood-free training method that can be used with implicit distributions and probabilistic programs. Using the Wasserstein variational inference framework, we introduce several new forms of autoencoders and test their robustness and performance against existing variational autoencoding techniques.
- Published
- 2019
23. Current advances in neural decoding
- Author
-
Gerven, M.A.J. van, Seeliger, K., Güçlü, U., Güçlütürk, Y., Samek, W., Montavon, G., Vedaldi, A., Hansen, L.K., Muller, K.R., Samek, W., Montavon, G., Vedaldi, A., Hansen, L.K., and Muller, K.R.
- Subjects
Visual perception ,medicine.diagnostic_test ,business.industry ,Computer science ,Brain activity and meditation ,Nonlinear methods ,Cognitive artificial intelligence ,Machine learning ,computer.software_genre ,Linear methods ,medicine ,Deep neural networks ,Artificial intelligence ,business ,Functional magnetic resonance imaging ,computer ,Neural decoding - Abstract
Contains fulltext : 207719.pdf (Publisher’s version ) (Closed access) Neural decoding refers to the extraction of semantically meaningful information from brain activity patterns. We discuss how advances in machine learning drive new advances in neural decoding. While linear methods allow for the reconstruction of basic stimuli from brain activity, more sophisticated nonlinear methods are required when reconstructing complex naturalistic stimuli. We show how deep neural networks and adversarial training yield state-of-the-art results. Ongoing advances in machine learning may one day allow the reconstruction of thoughts from brain activity patterns, providing a unique insight into the contents of the human mind.
- Published
- 2019
24. Simulating neuroprosthetic vision for emotion recognition
- Author
-
Bollen, C.J.M., Güçlü, U., Wezel, R.J.A. van, Gerven, M.A.J. van, Güçlütürk, Y., Bollen, C.J.M., Güçlü, U., Wezel, R.J.A. van, Gerven, M.A.J. van, and Güçlütürk, Y.
- Abstract
2019 8th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW) (Cambridge, UK, 3-6 September 2019), Contains fulltext : 215183.pdf (publisher's version ) (Closed access), We developed a phosphene vision simulator to assist in the development of image processing strategies for implementation in visual prosthetics. This simulation runs on a mobile phone, which can be placed in an AR headset to provide the experience of having prosthetic phosphene vision to individuals with normal vision. This setup allows the participants to experience the future of cortical visual neuroprostheses, while allowing us to evaluate and compare different signal processing algorithms to provide guidelines for the optimal perceptual experience. In this demo we will show how intelligent algorithms can improve the quality of perception with prosthetic vision with an image processing pipeline that allows for accurate emotion expression recognition.
- Published
- 2019
25. Forward amortized inference for likelihood-free variational marginalization
- Author
-
Chaudhuri, K., Sugiyama, M., Ambrogioni, L., Güçlü, U., Berezutskaya, Y., Borne, E.W.P. van den, Güçlütürk, Y., Hinne, M., Maris, E.G.G., Gerven, M.A.J. van, Chaudhuri, K., Sugiyama, M., Ambrogioni, L., Güçlü, U., Berezutskaya, Y., Borne, E.W.P. van den, Güçlütürk, Y., Hinne, M., Maris, E.G.G., and Gerven, M.A.J. van
- Abstract
International Conference on Artificial Intelligence and Statistics (AISTATS) 2019 (Naha, Okinawa, Japan, April 16 - 18, 2019), Contains fulltext : 203406.pdf (preprint version ) (Open Access), In this paper, we introduce a new form of amortized variational inference by using the forward KL divergence in a joint-contrastive variational loss. The resulting forward amortized variational inference is a likelihood-free method as its gradient can be sampled without bias and without requiring any evaluation of either the model joint distribution or its derivatives. We prove that our new variational loss is optimized by the exact posterior marginals in the fully factorized mean-field approximation, a property that is not shared with the more conventional reverse KL inference. Furthermore, we show that forward amortized inference can be easily marginalized over large families of latent variables in order to obtain a marginalized variational posterior. We consider two examples of variational marginalization. In our first example we train a Bayesian forecaster for predicting a simplified chaotic model of atmospheric convection. In the second example we train an amortized variational approximation of a Bayesian optimal classifier by marginalizing over the model space. The result is a powerful meta-classification network that can solve arbitrary classification problems without further training.
- Published
- 2019
26. Decomposing complexity preferences for music
- Author
-
Güçlütürk, Y., Lier, R.J. van, Güçlütürk, Y., and Lier, R.J. van
- Abstract
Contains fulltext : 202341.pdf (publisher's version ) (Open Access), Recently, we demonstrated complexity as a major factor for explaining individual differences in visual preferences for abstract digital art. We have shown that participants could best be separated into two groups based on their liking ratings for abstract digital art comprising geometric patterns: one group with a preference for complex visual patterns and another group with a preference for simple visual patterns. In the present study, building up on these results, we extended our investigations for complexity preferences from highly controlled visual stimuli to ecologically valid stimuli in the auditory modality. Similar to visual preferences, we showed that music preferences are highly influenced by stimulus complexity. We demonstrated this by clustering a large number of participants based on their liking ratings for song excerpts from various musical genres. Our results show that, based on their liking ratings, participants can best be separated into two groups: one group with a preference for more complex songs and another group with a preference for simpler songs. Finally, we considered various demographic and personal characteristics to explore differences between the groups, and reported that at least for the current data set age and gender to be significant factors separating the two groups.
- Published
- 2019
27. Current advances in neural decoding
- Author
-
Samek, W., Montavon, G., Vedaldi, A., Hansen, L.K., Muller, K.R., Gerven, M.A.J. van, Seeliger, K., Güçlü, U., Güçlütürk, Y., Samek, W., Montavon, G., Vedaldi, A., Hansen, L.K., Muller, K.R., Gerven, M.A.J. van, Seeliger, K., Güçlü, U., and Güçlütürk, Y.
- Abstract
Contains fulltext : 207719.pdf (publisher's version ) (Closed access), Neural decoding refers to the extraction of semantically meaningful information from brain activity patterns. We discuss how advances in machine learning drive new advances in neural decoding. While linear methods allow for the reconstruction of basic stimuli from brain activity, more sophisticated nonlinear methods are required when reconstructing complex naturalistic stimuli. We show how deep neural networks and adversarial training yield state-of-the-art results. Ongoing advances in machine learning may one day allow the reconstruction of thoughts from brain activity patterns, providing a unique insight into the contents of the human mind.
- Published
- 2019
28. Emotion recognition with simulated phosphene vision
- Author
-
Liu, X., Min, R., McDaniel, T., Bollen, C.J.M., Wezel, R.J.A. van, Gerven, M.A.J. van, Güçlütürk, Y., Liu, X., Min, R., McDaniel, T., Bollen, C.J.M., Wezel, R.J.A. van, Gerven, M.A.J. van, and Güçlütürk, Y.
- Abstract
MM '19: The 27th ACM International Conference on Multimedia (Nice, France, 21-25 October 2019), Contains fulltext : 215179.pdf (publisher's version ) (Closed access), Electrical stimulation of retina, optic nerve or cortex is found to elicit visual sensations, known as phosphenes. This allows visual prosthetics to partially restore vision by representing the visual field as a phosphene pattern. Since the resolution and performance of visual prostheses are limited, only a fraction of the information in a visual scene can be represented by phosphenes. Here, we propose a simple yet powerful image processing strategy for recognizing facial expressions with prosthetic vision, supporting communication and social interaction in the blind. A psychophysical study was conducted to investigate whether a landmark-based representation of facial expressions could improve emotion detection with prosthetic vision. Our approach was compared to edge detection, which is commonly used in current retinal prosthetic devices. Additionally, the relationship between the number of phosphenes and accuracy of emotion recognition was studied. The landmark model improved accuracy of emotion recognition, regardless of the number of phosphenes. Secondly, the accuracy improved with an increasing number of phosphenes up to a saturation point. The performance saturated with fewer phosphenes with the landmark model than with edge detection. These results suggest that landmark-based image pre-processing allows for a more efficient use of the limited information that can be stored in a phosphene pattern, providing a route towards more meaningful and higher-quality perceptual experience in subjects with prosthetic vision.
- Published
- 2019
29. Effects of complexity in perception: From construction to reconstruction
- Author
-
Güçlütürk, Y., Bekkering, H., Lier, R.J. van, and Radboud University Nijmegen
- Subjects
Action, intention, and motor control ,Perception, Action and Control [DI-BCB_DCC_Theme 2] - Abstract
Contains fulltext : 195176.pdf (Publisher’s version ) (Open Access) Radboud University, 29 juni 2018 Promotor : Bekkering, H. Co-promotor : Lier, R.J. van 211 p.
- Published
- 2018
30. Forward Amortized Inference for Likelihood-Free Variational Marginalization
- Author
-
Ambrogioni, L., Güçlü, U., Berezutskaya, Y., Borne, E.W.P. van den, Güçlütürk, Y., Hinne, M., Maris, E.G.G., Gerven, M.A.J. van, Chaudhuri, K., Sugiyama, M., Chaudhuri, K., and Sugiyama, M.
- Subjects
FOS: Computer and information sciences ,Computer Science - Learning ,Statistics - Machine Learning ,Action, intention, and motor control ,Machine Learning (stat.ML) ,Cognitive artificial intelligence ,Machine Learning (cs.LG) - Abstract
In this paper, we introduce a new form of amortized variational inference by using the forward KL divergence in a joint-contrastive variational loss. The resulting forward amortized variational inference is a likelihood-free method as its gradient can be sampled without bias and without requiring any evaluation of either the model joint distribution or its derivatives. We prove that our new variational loss is optimized by the exact posterior marginals in the fully factorized mean-field approximation, a property that is not shared with the more conventional reverse KL inference. Furthermore, we show that forward amortized inference can be easily marginalized over large families of latent variables in order to obtain a marginalized variational posterior. We consider two examples of variational marginalization. In our first example we train a Bayesian forecaster for predicting a simplified chaotic model of atmospheric convection. In the second example we train an amortized variational approximation of a Bayesian optimal classifier by marginalizing over the model space. The result is a powerful meta-classification network that can solve arbitrary classification problems without further training., Comment: 9 pages, 3 figures
- Published
- 2018
- Full Text
- View/download PDF
31. Explanation methods in deep learning: Users, values, concerns and challenges
- Author
-
Escalante, H.J., Escalera, S., Guyon, I., Baró, X., Güçlütürk, Y., Güçlü, U., Gerven, M. van, Ras, G.E.H., Gerven, M.A.J. van, Haselager, W.F.G., Escalante, H.J., Escalera, S., Guyon, I., Baró, X., Güçlütürk, Y., Güçlü, U., Gerven, M. van, Ras, G.E.H., Gerven, M.A.J. van, and Haselager, W.F.G.
- Abstract
Item does not contain fulltext, Issues regarding explainable AI involve four components: users, laws and regulations, explanations and algorithms. Together these components provide a context in which explanation methods can be evaluated regarding their adequacy. The goal of this chapter is to bridge the gap between expert users and lay users. Different kinds of users are identified and their concerns revealed, relevant statements from the General Data Protection Regulation are analyzed in the context of Deep Neural Networks (DNNs), a taxonomy for the classification of existing explanation methods is introduced, and finally, the various classes of explanation methods are analyzed to verify if user concerns are justified. Overall, it is clear that (visual) explanations can be given about various aspects of the influence of the input on the output. However, it is noted that explanation methods or interfaces for lay users are missing and we speculate which criteria these methods/interfaces should satisfy. Finally it is noted that two important concerns are difficult to address with explanation methods: the concern about bias in datasets that leads to biased DNNs, as well as the suspicion about unfair outcomes.
- Published
- 2018
32. Representations of naturalistic stimulus complexity in early and associative visual and auditory cortices
- Author
-
Güçlütürk, Y., Güçlü, U., Gerven, M.A.J. van, Lier, R.J. van, Güçlütürk, Y., Güçlü, U., Gerven, M.A.J. van, and Lier, R.J. van
- Abstract
Contains fulltext : 183759.pdf (publisher's version ) (Open Access), The complexity of sensory stimuli has an important role in perception and cognition. However, its neural representation is not well understood. Here, we characterize the representations of naturalistic visual and auditory stimulus complexity in early and associative visual and auditory cortices. This is realized by means of encoding and decoding analyses of two fMRI datasets in the visual and auditory modalities. Our results implicate most early and some associative sensory areas in representing the complexity of naturalistic sensory stimuli. For example, parahippocampal place area, which was previously shown to represent scene features, is shown to also represent scene complexity. Similarly, posterior regions of superior temporal gyrus and superior temporal sulcus, which were previously shown to represent syntactic (language) complexity, are shown to also represent music (auditory) complexity. Furthermore, our results suggest the existence of gradients in sensitivity to naturalistic sensory stimulus complexity in these areas.
- Published
- 2018
33. Effects of complexity in perception: From construction to reconstruction
- Author
-
Bekkering, H., Lier, R.J. van, Güçlütürk, Y., Bekkering, H., Lier, R.J. van, and Güçlütürk, Y.
- Abstract
Radboud University, 29 juni 2018, Promotor : Bekkering, H. Co-promotor : Lier, R.J. van, Contains fulltext : 195176.pdf (publisher's version ) (Open Access)
- Published
- 2018
34. Wasserstein variational inference
- Author
-
Bengio, S., Wallach, H.M., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Ambrogioni, L., Güçlü, U., Güçlütürk, Y., Hinne, M., Gerven, M.A.J. van, Maris, E.G.G., Bengio, S., Wallach, H.M., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Ambrogioni, L., Güçlü, U., Güçlütürk, Y., Hinne, M., Gerven, M.A.J. van, and Maris, E.G.G.
- Abstract
NIPS'18: 32nd International Conference on Neural Information Processing Systems (Montreal, Canada, 3-8 December2018), Item does not contain fulltext, This paper introduces Wasserstein variational inference, a new form of approximate Bayesian inference based on optimal transport theory. Wasserstein variational inference uses a new family of divergences that includes both f-divergences and the Wasserstein distance as special cases. The gradients of the Wasserstein variational loss are obtained by backpropagating through the Sinkhorn iterations. This technique results in a very stable likelihood-free training method that can be used with implicit distributions and probabilistic programs. Using the Wasserstein variational inference framework, we introduce several new forms of autoencoders and test their robustness and performance against existing variational autoencoding techniques.
- Published
- 2018
35. Explainable and interpretable models in computer vision and machine learning
- Author
-
Escalante, H.J., Escalera, S., Guyon, I., Baró, X., Güçlütürk, Y., Güçlü, U., Gerven, M.A.J. van, Escalante, H.J., Escalera, S., Guyon, I., Baró, X., Güçlütürk, Y., Güçlü, U., and Gerven, M.A.J. van
- Abstract
Item does not contain fulltext
- Published
- 2018
36. Multimodal first impression analysis with deep residual networks
- Author
-
Güçlütürk, Y., Güçlü, U., Baró, X., Escalante, H.J., Guyon, I., Escalera, S., Gerven, M.A.J. van, Lier, R.J. van, Güçlütürk, Y., Güçlü, U., Baró, X., Escalante, H.J., Guyon, I., Escalera, S., Gerven, M.A.J. van, and Lier, R.J. van
- Abstract
Contains fulltext : 182126.pdf (Publisher’s version ) (Closed access), People form first impressions about the personalities of unfamiliar individuals even after very brief interactions with them. In this study we present and evaluate several models that mimic this automatic social behavior. Specifically, we present several models trained on a large dataset of short YouTube video blog posts for predicting apparent Big Five personality traits of people and whether they seem suitable to be recommended to a job interview. Along with presenting our audiovisual approach and results that won the third place in the ChaLearn First Impressions Challenge, we investigate modeling in different modalities including audio only, visual only, language only, audiovisual, and combination of audiovisual and language. Our results demonstrate that the best performance could be obtained using a fusion of all data modalities.
- Published
- 2018
37. Reconstructing perceived faces from brain activations with deep adversarial neural decoding
- Author
-
Güçlütürk, Y., Güçlü, U., Seeliger, K., Bosch, S.E., Lier, R.J. van, Gerven, M.A.J. van, Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R., Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R.
- Subjects
Brain Networks and Neuronal Communication [DI-BCB_DCC_Theme 4] ,Quantitative Biology::Neurons and Cognition ,Action, intention, and motor control ,Perception, Action and Control [DI-BCB_DCC_Theme 2] ,Cognitive artificial intelligence - Abstract
Contains fulltext : 179505.pdf (Publisher’s version ) (Open Access) Here, we present a novel approach to solve the problem of reconstructing perceived stimuli from brain responses by combining probabilistic inference with deep learning. Our approach first inverts the linear transformation from latent features to brain responses with maximum a posteriori estimation and then inverts the nonlinear transformation from perceived stimuli to latent features with adversarial training of convolutional neural networks. We test our approach with a functional magnetic resonance imaging experiment and show that it can generate state-of-the-art reconstructions of perceived faces from brain activations. NIPS 2017: 31st Annual Conference on Neural Information Processing Systems (Long Beach, California, December 4-9, 2017)
- Published
- 2017
38. Bayesian model ensembling using meta-trained recurrent neural networks
- Author
-
Ambrogioni, L., Berezutskaya, Y., Güçlü, U., Borne, E.W.P. van den, Güçlütürk, Y., Gerven, M.A.J. van, and Maris, E.G.G.
- Subjects
Language in Interaction ,Brain Networks and Neuronal Communication [DI-BCB_DCC_Theme 4] ,ComputingMethodologies_PATTERNRECOGNITION ,Action, intention, and motor control ,Perception, Action and Control [DI-BCB_DCC_Theme 2] ,Cognitive artificial intelligence - Abstract
Contains fulltext : 180405.pdf (Author’s version preprint ) (Open Access) In this paper we demonstrate that a recurrent neural network meta-trained on an ensemble of arbitrary classification tasks can be used as an approximation of the Bayes optimal classifier. This result is obtained by relying on the framework of e-free approximate Bayesian inference, where the Bayesian posterior is approximated by training a neural network using synthetic samples. We denote the resulting model as neural ensembler. We show that a single neural ensembler trained on a large set of synthetic data achieves competitive classification performance on multiple real-world classification problems without additional training. 31st Conference on Neural Information Processing Systems (NIPS 2017) (Long Beach, CA, USA, December 4-9, 2017)
- Published
- 2017
39. Algorithmic composition of polyphonic music with the WaveCRF
- Author
-
Güçlü, U., Güçlütürk, Y., Ambrogioni, L., Maris, E.G.G., Lier, R.J. van, and Gerven, M.A.J. van
- Subjects
Brain Networks and Neuronal Communication [DI-BCB_DCC_Theme 4] ,Action, intention, and motor control ,Perception, Action and Control [DI-BCB_DCC_Theme 2] ,Cognitive artificial intelligence - Abstract
Contains fulltext : 179506.pdf (Publisher’s version ) (Open Access) Here, we propose a new approach for modeling conditional probability distributions of polyphonic music by combining WaveNET and CRF-RNN variants, and show that this approach beats LSTM and WaveNET baselines that do not take into account the statistical dependencies between simultaneous notes. NIPS 2017: 31st Annual Conference on Neural Information Processing Systems (Long Beach, California, December 4-9, 2017)
- Published
- 2017
40. Generative adversarial networks for reconstructing natural images from brain activity
- Author
-
Seeliger, K., primary, Güçlü, U., additional, Ambrogioni, L., additional, Güçlütürk, Y., additional, and van Gerven, M. A. J., additional
- Published
- 2017
- Full Text
- View/download PDF
41. Visualizing apparent personality analysis with deep residual networks
- Author
-
Güçlütürk, Y., Güçlü, U., Pérez, M., Escalante, H.J., Baró, X., Guyon, I., Andujar, C., Jacques, J., Madadi, M., Escalera, S., Gerven, M.A.J. van, Lier, R.J. van, Güçlütürk, Y., Güçlü, U., Pérez, M., Escalante, H.J., Baró, X., Guyon, I., Andujar, C., Jacques, J., Madadi, M., Escalera, S., Gerven, M.A.J. van, and Lier, R.J. van
- Abstract
2017 IEEE International Conference on Computer Vision (ICCV) (Venice, Italy, 22 Oct - 29 Oct 2017), Contains fulltext : 179512.pdf (publisher's version ) (Open Access), Automatic prediction of personality traits is a subjective task that has recently received much attention. Specifically, automatic apparent personality trait prediction from multimodal data has emerged as a hot topic within the filed of computer vision and, more particularly, the so called "looking at people" sub-field. Considering "apparent" personality traits as opposed to real ones considerably reduces the subjectivity of the task. The real world applications are encountered in a wide range of domains, including entertainment, health, human computer interaction, recruitment and security. Predictive models of personality traits are useful for individuals in many scenarios (e.g., preparing for job interviews, preparing for public speaking). However, these predictions in and of themselves might be deemed to be untrustworthy without human understandable supportive evidence. Through a series of experiments on a recently released benchmark dataset for automatic apparent personality trait prediction, this paper characterizes the audio and visual information that is used by a state-of-the-art model while making its predictions, so as to provide such supportive evidence by explaining predictions made. Additionally, the paper describes a new web application, which gives feedback on apparent personality traits of its users by combining model predictions with their explanations.
- Published
- 2017
42. Reconstructing perceived faces from brain activations with deep adversarial neural decoding
- Author
-
Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R., Güçlütürk, Y., Güçlü, U., Seeliger, K., Bosch, S.E., Lier, R.J. van, Gerven, M.A.J. van, Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R., Güçlütürk, Y., Güçlü, U., Seeliger, K., Bosch, S.E., Lier, R.J. van, and Gerven, M.A.J. van
- Abstract
NIPS 2017: 31st Annual Conference on Neural Information Processing Systems (Long Beach, California, December 4-9, 2017), Contains fulltext : 179505.pdf (publisher's version ) (Open Access), Here, we present a novel approach to solve the problem of reconstructing perceived stimuli from brain responses by combining probabilistic inference with deep learning. Our approach first inverts the linear transformation from latent features to brain responses with maximum a posteriori estimation and then inverts the nonlinear transformation from perceived stimuli to latent features with adversarial training of convolutional neural networks. We test our approach with a functional magnetic resonance imaging experiment and show that it can generate state-of-the-art reconstructions of perceived faces from brain activations.
- Published
- 2017
43. Design of an explainable machine learning challenge for video interviews
- Author
-
Escalante, H.J., Guyon, I., Escalera, S., Jacques, J., Madadi, M., Baró, X., Ayache, S., Viegas, E., Güçlütürk, Y., Güçlü, U., Gerven, M.A.J. van, Lier, R.J. van, Escalante, H.J., Guyon, I., Escalera, S., Jacques, J., Madadi, M., Baró, X., Ayache, S., Viegas, E., Güçlütürk, Y., Güçlü, U., Gerven, M.A.J. van, and Lier, R.J. van
- Abstract
2017 International Joint Conference on Neural Networks (IJCNN) (Anchorage, Alaska, 14-19 May 2017), Contains fulltext : 179525.pdf (publisher's version ) (Closed access), This paper reviews and discusses research advances on "explainable machine learning" in computer vision. We focus on a particular area of the "Looking at People" (LAP) thematic domain: first impressions and personality analysis. Our aim is to make the computational intelligence and computer vision communities aware of the importance of developing explanatory mechanisms for computer-assisted decision making applications, such as automating recruitment. Judgments based on personality traits are being made routinely by human resource departments to evaluate the candidates' capacity of social insertion and their potential of career growth. However, inferring personality traits and, in general, the process by which we humans form a first impression of people, is highly subjective and may be biased. Previous studies have demonstrated that learning machines can learn to mimic human decisions. In this paper, we go one step further and formulate the problem of explaining the decisions of the models as a means of identifying what visual aspects are important, understanding how they relate to decisions suggested, and possibly gaining insight into undesirable negative biases. We design a new challenge on explainability of learning machines for first impressions analysis. We describe the setting, scenario, evaluation metrics and preliminary outcomes of the competition. To the best of our knowledge this is the first effort in terms of challenges for explainability in computer vision. In addition our challenge design comprises several other quantitative and qualitative elements of novelty, including a "coopetition" setting, which combines competition and collaboration.
- Published
- 2017
44. Convolutional sketch inversion
- Author
-
Güçlütürk, Y., Güçlü, U., Lier, R.J. van, Gerven, M.A.J. van, Hua, G., Jégou, H., Hua, G., and Jégou, H.
- Subjects
Brain Networks and Neuronal Communication [DI-BCB_DCC_Theme 4] ,Action, intention, and motor control ,Cognitive artificial intelligence - Abstract
Contains fulltext : 159908.pdf (Publisher’s version ) (Closed access) In this paper, we use deep neural networks for inverting face sketches to synthesize photorealistic face images. We first construct a semi-simulated dataset containing a very large number of computer-generated face sketches with different styles and corresponding face images by expanding existing unconstrained face data sets. We then train models achieving state-of-the-art results on both computer-generated sketches and hand-drawn sketches by leveraging recent advances in deep learning such as batch normalization, deep residual learning, perceptual losses and stochastic optimization in combination with our new dataset. We finally demonstrate potential applications of our models in fine arts and forensic arts. In contrast to existing patch-based approaches, our deep-neural-network-based approach can be used for synthesizing photorealistic face images by inverting face sketches in the wild. Computer Vision – ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8-10 and 15-16, 2016, Proceedings, Part I. ECCV 2016: European Conference on Computer Vision (Amsterdam, The Netherlands, October 8-10 and 15-16, 2016)
- Published
- 2016
45. Deep impression: Audiovisual deep residual networks for multimodal apparent personality trait recognition
- Author
-
Güçlütürk, Y., Güçlü, U., Gerven, M.A.J. van, Lier, R.J. van, Hua, G., Jégou, H., Hua, G., and Jégou, H.
- Subjects
Brain Networks and Neuronal Communication [DI-BCB_DCC_Theme 4] ,Action, intention, and motor control ,ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION ,Perception, Action and Control [DI-BCB_DCC_Theme 2] ,Cognitive artificial intelligence - Abstract
Item does not contain fulltext Here, we develop an audiovisual deep residual network for multimodal apparent personality trait recognition. The network is trained end-to-end for predicting the Big Five personality traits of people from their videos. That is, the network does not require any feature engineering or visual analysis such as face detection, face landmark alignment or facial expression recognition. Recently, the network won the third place in the ChaLearn First Impressions Challenge with a test accuracy of 0.9109. Computer Vision - ECCV 2016 Workshops, Amsterdam, The Netherlands, October 8-10 and 15-16, 2016
- Published
- 2016
46. Liking versus complexity: Decomposing the inverted U-curve
- Author
-
Güçlütürk, Y., Jacobs, R.H.A.H., Lier, R.J. van, Güçlütürk, Y., Jacobs, R.H.A.H., and Lier, R.J. van
- Abstract
Contains fulltext : 157468.pdf (publisher's version ) (Open Access), The relationship between liking and stimulus complexity is commonly reported to follow an inverted U-curve. However, large individual differences among complexity preferences of participants have frequently been observed since the earliest studies on the topic. The common use of across-participant analysis methods that ignore these large individual differences in aesthetic preferences gives an impression of high agreement between individuals. In this study, we collected ratings of liking and perceived complexity from 30 participants for a set of digitally generated grayscale images. In addition, we calculated an objective measure of complexity for each image. Our results reveal that the inverted U-curve relationship between liking and stimulus complexity comes about as the combination of different individual liking functions. Specifically, after automatically clustering the participants based on their liking ratings, we determined that one group of participants in our sample had increasingly lower liking ratings for increasingly more complex stimuli, while a second group of participants had increasingly higher liking ratings for increasingly more complex stimuli. Based on our findings, we call for a focus on the individual differences in aesthetic preferences, adoption of alternative analysis methods that would account for these differences and a re-evaluation of established rules of human aesthetic preferences.
- Published
- 2016
47. Deep impression: Audiovisual deep residual networks for multimodal apparent personality trait recognition
- Author
-
Hua, G., Jégou, H., Güçlütürk, Y., Güçlü, U., Gerven, M.A.J. van, Lier, R.J. van, Hua, G., Jégou, H., Güçlütürk, Y., Güçlü, U., Gerven, M.A.J. van, and Lier, R.J. van
- Abstract
Computer Vision - ECCV 2016 Workshops, Amsterdam, The Netherlands, October 8-10 and 15-16, 2016, Item does not contain fulltext, Here, we develop an audiovisual deep residual network for multimodal apparent personality trait recognition. The network is trained end-to-end for predicting the Big Five personality traits of people from their videos. That is, the network does not require any feature engineering or visual analysis such as face detection, face landmark alignment or facial expression recognition. Recently, the network won the third place in the ChaLearn First Impressions Challenge with a test accuracy of 0.9109.
- Published
- 2016
48. Convolutional sketch inversion
- Author
-
Hua, G., Jégou, H., Güçlütürk, Y., Güçlü, U., Lier, R.J. van, Gerven, M.A.J. van, Hua, G., Jégou, H., Güçlütürk, Y., Güçlü, U., Lier, R.J. van, and Gerven, M.A.J. van
- Abstract
ECCV 2016: European Conference on Computer Vision (Amsterdam, The Netherlands, October 8-10 and 15-16, 2016), Contains fulltext : 159908.pdf (publisher's version ) (Closed access), In this paper, we use deep neural networks for inverting face sketches to synthesize photorealistic face images. We first construct a semi-simulated dataset containing a very large number of computer-generated face sketches with different styles and corresponding face images by expanding existing unconstrained face data sets. We then train models achieving state-of-the-art results on both computer-generated sketches and hand-drawn sketches by leveraging recent advances in deep learning such as batch normalization, deep residual learning, perceptual losses and stochastic optimization in combination with our new dataset. We finally demonstrate potential applications of our models in fine arts and forensic arts. In contrast to existing patch-based approaches, our deep-neural-network-based approach can be used for synthesizing photorealistic face images by inverting face sketches in the wild. Computer Vision – ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8-10 and 15-16, 2016, Proceedings, Part I.
- Published
- 2016
49. What you need is what you like: Knowing target and distractor categories is sufficient for distractor devaluation
- Author
-
Jacobs, R.H.A.H., Güçlütürk, Y., Lier, R.J. van, Jacobs, R.H.A.H., Güçlütürk, Y., and Lier, R.J. van
- Abstract
Contains fulltext : 157480.pdf (publisher's version ) (Open Access), 38th European Conference on Visual Perception (ECVP) 2015 Liverpool
- Published
- 2015
50. The visual cortex in the blind but not the auditory cortex in the deaf becomes multiple-demand regions.
- Author
-
Duymuş H, Verma M, Güçlütürk Y, Öztürk M, Varol AB, Kurt Ş, Gezici T, Akgür BF, Giray İ, Öksüz EE, and Farooqui AA
- Abstract
The fate of deprived sensory cortices - visual regions in the blind and auditory regions in the deaf - exemplifies the extent to which experience can change brain regions. These regions are frequently seen to activate during tasks involving other sensory modalities, leading many accounts to infer that these regions have started processing sensory information of other modalities. However, such observations can also imply that these regions are now activating to any task event regardless of the sensory modality. Activating to task events, irrespective of the sensory modality involved, is a feature of the multiple-demands (MD) network. These are a common set of regions within the frontal and parietal cortices that activate in response to any kind of control demand. Thus, demands as diverse as attention, perceptual difficulty, rule-switching, updating working memory, inhibiting responses, decision-making, and difficult arithmetic - all activate these same set of regions that are thought to instantiate domain-general cognitive control and underpin fluid intelligence. We investigated if deprived sensory cortices, or foci within them, become part of the MD network. We tested if the same foci within the visual regions of the blind and auditory regions of the deaf activated to different control demands. We found that control demands related to updating auditory working memory, difficult tactile decisions, time-duration judgments, and sensorimotor-speed - all activated the entire bilateral occipital regions in the blind but not in the sighted. These occipital regions in the blind were the only regions outside the canonical fronto-parietal MD regions to show such activation to multiple control demands. Further, compared to the sighted, these occipital regions in the blind had higher functional connectivity with fronto-parietal MD regions. Early deaf, in contrast, did not activate their auditory regions to different control demands, showing that auditory regions do not become MD regions in the deaf. We suggest that visual regions in the blind do not take a new sensory role but become part of the MD network, and this is not a response of all deprived sensory cortices but a feature unique to the visual regions., (© The Author(s) 2024. Published by Oxford University Press on behalf of the Guarantors of Brain.)
- Published
- 2024
- Full Text
- View/download PDF
Catalog
Discovery Service for Jio Institute Digital Library
For full access to our library's resources, please sign in.