We outline general theoretical and practical implications of what we promote as enactive cinema f... more We outline general theoretical and practical implications of what we promote as enactive cinema for the neuroscientific study of online socio-emotional interaction. In a real-time functional magnetic resonance imaging (rt-fMRI) setting, participants are immersed in cinematic experiences that simulate social situations. While viewing, their physiological reactions-including brain responses-are tracked, representing implicit and unconscious experiences of the ongoing social situations. These reactions, in turn, are analyzed in real-time and fed back to modify the cinematic sequences they are viewing while being scanned. Due to the engaging cinematic content, the proposed setting focuses on living-by in terms of shared psycho-physiological epiphenomena of experience rather than active coping in terms of goal-oriented motor actions. It constitutes a means to parametrically modify stimuli that depict social situations and their broader environmental contexts. As an alternative to studying the variation of brain responses as a function of a priori fixed stimuli, this method can be applied to survey the range of stimuli that evoke similar responses across participants at particular brain regions of interest.
In the newly founded Soviet Union, Aleksandr A. Bogdanov and Segei M. Eisenstein, each in his own... more In the newly founded Soviet Union, Aleksandr A. Bogdanov and Segei M. Eisenstein, each in his own way, struggled to make sense of the world by means of the most recent findings in the sciences. Both were driven by a desire to describe the universal laws of organization that would embrace the dynamics of the human mind and society, mutually, in arts and sciences. Bogdanov, a leading theoretician of political, economical, cultural, and educational revolution, is today also recognized internationally as one of the pioneering systems scientists of the early twentieth century. Eisenstein began to establish an international reputation thanks to the originality of his films and his eclectic theoretical writings that have remained a rich source of continuing discoveries for film scholars. I propose that both of these thinkers, in their own right, and by way of their common synergy, can contribute to a systemic understanding of today’s complex world and its cultural reflections.
Phenomenological Considerations on Time Consciousness under Neurocinematic Search Light
Film narratives are intrinsically time-dependent designs. This article proposes a model of narrat... more Film narratives are intrinsically time-dependent designs. This article proposes a model of narrative nowness, based on Husserl's concepts of retention and protention on one hand, and Francisco Varela's neurophenomenological exploration of time consciousness on the other, relating this further to narrative experience and its neural epiphenomena. Only recently has brain research been equipped with the possibility of dealing with temporal frames relevant for time consciousness in the scope of whole narratives. The study of cinema using neuroscientific methods and insights is referred to as neurocinematics. We promote neurocinematics as a complementary method of traditional film research, rather than an approach of brain sciences in general. Neurocinematic methods may provide film studies with new tools for re-evaluating established filmmaking conventions and developing new ways to study, for instance, the film viewer's experience and related aspects of time consciousness.
The essay relates the cinema author’s creative processes to the systems intelligence approach. Th... more The essay relates the cinema author’s creative processes to the systems intelligence approach. The underpinning assumption is that cinema stands forth as an intersubjective frame of sensemaking. This idea is reflected against the early systemic views of the Russian filmmaker and theoretician Sergei M. Eisenstein. In its unfolding, the cinema author’s creative processes are described from a particular point of view, that is, that of the enactive mind point by means of introducing the neuroscientific concept of embodied simulation as the bodily basis of these processes. This is applied in the hypothetical model of cinema author’s mental workspace, the embodied simulatorium as it is termed. In this paper it will be discussed how embodied processes constitute what in this volume is referred to as systems intelligence.
While the internet has democratized and accelerated content creation and sharing, it has also mad... more While the internet has democratized and accelerated content creation and sharing, it has also made people more vulnerable to manipulation and misinformation. Also, the received information can be distorted by psychological biases. This is problematic especially in health-related communications which can greatly affect the quality of life of individuals. We assembled and analyzed 364 texts related to nutrition and health from Finnish online sources, such as news, columns and blogs, and asked non-experts to subjectively evaluate the texts. Texts were rated for their trustworthiness, sentiment, logic, information, clarity, and neutrality properties. We then estimated individual biases and consensus ratings that were used in training regression models. Firstly, we found that trustworthiness was significantly correlated to the information, neutrality and logic of the texts. Secondly, individual ratings for information and logic were significantly biased by the age and diet of the raters. Our best regression models explained up to 70% of the total variance of consensus ratings based on the low-level properties of texts, such as semantic embeddings, presence of keyterms and part-of-speech tags, references, quotes and paragraphs. With a novel combination of crowdsourcing, behavioral analysis, natural language processing and predictive modeling, our study contributes to the automated identification of reliable and high-quality online information. While critical evaluation of truthfulness cannot be surrendered to the machine only, our findings provide new insights into automated evaluation of subjective text properties and analysis of morphologically-rich languages in regards to trustworthiness.
Narratives surround us in our everyday life in different forms. In the sensory brain areas, the p... more Narratives surround us in our everyday life in different forms. In the sensory brain areas, the processing of narratives is dependent on the media of presentation, be that in audiovisual or written form. However, little is known of the brain areas that process complex narrative content mediated by various forms. To isolate these regions, we looked for the functional networks reacting in a similar manner to the same narrative content despite different media of presentation. We collected 3-T fMRI whole brain data from 31 healthy human adults during two separate runs when they were either viewing a movie or reading its screenplay text. The independent component analysis (ICA) was used to separate 40 components. By correlating the components' time-courses between the two different media conditions, we could isolate 5 functional networks that particularly related to the same narrative content. These TOP-5 components with the highest correlation covered fronto-temporal, parietal, and occipital areas with no major involvement of primary visual or auditory cortices. Interestingly, the top-ranked network with highest modality-invariance also correlated negatively with the dialogue predictor, thus pinpointing that narrative comprehension entails processes that are not language-reliant. In summary, our novel experiment design provided new insight into narrative comprehension networks across modalities.
Movie viewing allows human perception and cognition to be studied in complex, real-life-like situ... more Movie viewing allows human perception and cognition to be studied in complex, real-life-like situations in a brain-imaging laboratory. Previous studies with functional magnetic resonance imaging (fMRI) and with magneto- and electroencephalography (MEG and EEG) have demonstrated consistent temporal dynamics of brain activity across movie viewers. However, little is known about the similarities and differences of fMRI and MEG or EEG dynamics during such naturalistic situations. We thus compared MEG and fMRI responses to the same 15-min black-and-white movie in the same eight subjects who watched the movie twice during both MEG and fMRI recordings. We analyzed intra- and intersubject voxel-wise correlations within each imaging modality as well as the correlation of the MEG envelopes and fMRI signals. The fMRI signals showed voxel-wise within- and between-subjects correlations up to r = 0.66 and r = 0.37, respectively, whereas these correlations were clearly weaker for the envelopes of ...
How does the human brain recall and connect relevant memories with unfolding events? To study thi... more How does the human brain recall and connect relevant memories with unfolding events? To study this, we presented 25 healthy subjects, during functional magnetic resonance imaging, the movie 'Memento' (director C. Nolan). In this movie, scenes are presented in chronologically reverse order with certain scenes briefly overlapping previously presented scenes. Such overlapping "key-frames" serve as effective memory cues for the viewers, prompting recall of relevant memories of the previously seen scene and connecting them with the concurrent scene. We hypothesized that these repeating key-frames serve as immediate recall cues and would facilitate reconstruction of the story piece-by-piece. The chronological version of Memento, shown in a separate experiment for another group of subjects, served as a control condition. Using multivariate event-related pattern analysis method and representational similarity analysis, focal fingerprint patterns of hemodynamic activity wer...
The dancer, choreographer, and filmmaker Maya Deren can be seen as one of the pioneers of screend... more The dancer, choreographer, and filmmaker Maya Deren can be seen as one of the pioneers of screendance. Her experimental films have challenged conventional plot-driven mainstream cinema by emphasizing an ambiguous experience, open for multiple interpretations. For Deren film viewing is a socially determined ritual embodying intersubjectively shared experiences of participants. This makes her films particularly interesting for today’s neurocinematic studies. Deren’s ideas also anticipate the recent enactive mind approach, according to which the body-brain system is in an inseparable manner situated and coupled with the world through interaction. It assumes that both private, such as perception and cognition, and intersubjective aspects of human enactment, such as culture, sciences, or the arts, are based on the embodiment of life experience. Reflecting this discourse, Deren’s film At Land is analyzed as an expression of a human body-brain system situated and enactive within the world,...
Observation of another person's actions and feelings activates brain areas that support similar f... more Observation of another person's actions and feelings activates brain areas that support similar functions in the observer, thereby facilitating inferences about the other's mental and bodily states. In real life, events eliciting this kind of vicarious brain activations are intermingled with other complex, ever-changing stimuli in the environment. One practical approach to study the neural underpinnings of real-life vicarious perception is to image brain activity during movie viewing. Here the goal was to find out how observed haptic events in a silent movie would affect the spectator's sensorimotor cortex. The functional state of the sensorimotor cortex was monitored by analyzing, in 16 healthy subjects, magnetoencephalographic (MEG) responses to tactile finger stimuli that were presented once per second throughout the session. Using canonical correlation analysis and spatial filtering, consistent single-trial responses across subjects were uncovered, and their waveform changes throughout the movie were quantified. The long-latency (85-175 ms) parts of the responses were modulated in concordance with the participants' average moment-by-moment ratings of own engagement in the haptic content of the movie (correlation r 5 0.49; ratings collected after the MEG session). The results, obtained by using novel signal-analysis approaches, demonstrate that the functional state of the human sensorimotor cortex fluctuates in a fine-grained manner even during passive observation of temporally varying haptic events.
Cognitive neurosciences have made significant progress in learning about brain activity in situat... more Cognitive neurosciences have made significant progress in learning about brain activity in situated cognition, thanks to adopting stimuli that simulate immersion in naturalistic conditions instead of isolated artificial stimuli. In particular, the use of films in neuroscientific experiments, a paradigm often referred to as neurocinematics, has contributed to this success. The use of cinematic stimuli, however, has also revealed a fundamental shortcoming of neuroimaging studies: The lack of conceptual and methodological means to handle the viewers' experience of narrative events in their temporally extended contexts in the scale of full cinematic narrative, not to mention life itself. In order to give a conceptual structure to the issue of temporal contexts, we depart from the neurophenomenological approach to time consciousness by neurobiologist Francisco Varela, which in turn builds on Husserl's phenomenology of time. More specifically, we will discuss the experience of nar...
One of the challenges of naturalistic neurosciences using movie-viewing experiments is how to int... more One of the challenges of naturalistic neurosciences using movie-viewing experiments is how to interpret observed brain activations in relation to the multiplicity of time-locked stimulus features. As previous studies have shown less inter-subject synchronization across viewers of random video footage than story-driven films, new methods need to be developed for analysis of less story-driven contents. To optimize the linkage between our fMRI data collected during viewing of a deliberately non-narrative silent film 'At Land' by Maya Deren (1944) and its annotated content, we combined the method of elastic-net regularization with the modeldriven linear regression and the well-established data-driven independent component analysis (ICA) and inter-subject correlation (ISC) methods. In the linear regression analysis, both IC and region-of-interest (ROI) time-series were fitted with time-series of a total of 36 binary-valued and one real-valued tactile annotation of film features. The elastic-net regularization and cross-validation were applied in the ordinary least-squares linear regression in order to avoid over-fitting due to the multicollinearity of regressors, the results were compared against both the partial least-squares (PLS) regression and the un-regularized full-model regression. Nonparametric permutation testing scheme was applied to evaluate the statistical significance of regression. We found statistically significant correlation between the annotation model and 9 ICs out of 40 ICs. Regression analysis was also repeated for a large set of cubic ROIs covering the grey matter. Both IC-and ROI-based regression analyses revealed activations in parietal and occipital regions, with additional smaller clusters in the frontal lobe. Furthermore, we found elastic-net based regression more sensitive than PLS and un-regularized regression since it detected a larger number of significant ICs and ROIs. Along with the ISC ranking methods, our regression analysis proved a feasible method for ordering the ICs based on their functional relevance to the annotated cinematic features. The novelty of our method isin comparison to the hypothesis-driven manual preselection and observation of some individual regressors biased by choicein applying data-driven approach to all content features simultaneously. We found especially the combination of regularized regression and ICA useful when analyzing fMRI data obtained using non-narrative movie stimulus with a large set of complex and correlated features.
Earlier studies have shown considerable intersubject synchronization of brain activity when subje... more Earlier studies have shown considerable intersubject synchronization of brain activity when subjects watch the same movie or listen to the same story. Here we investigated the across-subjects similarity of brain responses to speech and non-speech sounds in a continuous audio drama designed for blind people. Thirteen healthy adults listened for ,19 min to the audio drama while their brain activity was measured with 3 T functional magnetic resonance imaging (fMRI). An intersubjectcorrelation (ISC) map, computed across the whole experiment to assess the stimulus-driven extrinsic brain network, indicated statistically significant ISC in temporal, frontal and parietal cortices, cingulate cortex, and amygdala. Group-level independent component (IC) analysis was used to parcel out the brain signals into functionally coupled networks, and the dependence of the ICs on external stimuli was tested by comparing them with the ISC map. This procedure revealed four extrinsic ICs of which two-covering non-overlapping areas of the auditory cortex-were modulated by both speech and nonspeech sounds. The two other extrinsic ICs, one left-hemisphere-lateralized and the other right-hemisphere-lateralized, were speech-related and comprised the superior and middle temporal gyri, temporal poles, and the left angular and inferior orbital gyri. In areas of low ISC four ICs that were defined intrinsic fluctuated similarly as the time-courses of either the speech-sound-related or all-sounds-related extrinsic ICs. These ICs included the superior temporal gyrus, the anterior insula, and the frontal, parietal and midline occipital cortices. Taken together, substantial intersubject synchronization of cortical activity was observed in subjects listening to an audio drama, with results suggesting that speech is processed in two separate networks, one dedicated to the processing of speech sounds and the other to both speech and non-speech sounds.
This is an electronic reprint of the original article. This reprint may differ from the original ... more This is an electronic reprint of the original article. This reprint may differ from the original in pagination and typographic detail.
Uploads
Papers by Pia Tikka