James Davey1, Hannah E. Thompson1,Glyn Hallam1, Theodoros Karapanagiotidis1, Charlotte Murphy1, Irene De Caso1, Katya Krieger-Redwood1, Boris C. Bernhardt2, Jonathan Smallwood1, Elizabeth Jefferies1*
1Deparment of Psychology and York Neuroimaging Centre, University of York, UK
2McConnell Brain Imaging Centre, Montreal Neurological Institute and Hospital, McGill University, Montreal, QC, H3A 2B4, Canada
Postal address: Professor Beth Jefferies,
Department of Psychology,
University of York,
+44 1904 324368
Making sense of the world around us depends upon selectively retrieving information relevant to our current goal or context. However, it is unclear whether selective semantic retrieval relies exclusively on general control mechanisms recruited in demanding non-semantic tasks, or instead on systems specialised for the control of meaning. One hypothesis is that the left posterior middle temporal gyrus (pMTG) is important in the controlled retrieval of semantic (not non-semantic) information; however this view remains controversial since a parallel literature links this site to event and relational semantics. In a functional neuroimaging study, we demonstrated that an area of pMTG implicated in semantic control by a recent meta-analysis was activated in a conjunction of (i) semantic association over size judgements and (ii) action over colour feature matching. Under these circumstances the same region showed functional coupling with the inferior frontal gyrus – another crucial site for semantic control. Structural and functional connectivity analysis demonstrated that this site is at the nexus of networks recruited in automatic semantic processing (the default mode network) and executively demanding tasks (the multiple-demand network). Moreover, in both task and task-free contexts, pMTG exhibited functional properties that were more similar to ventral parts of inferior frontal cortex, implicated in controlled semantic retrieval, than more dorsal inferior frontal sulcus, implicated in domain-general control. Finally, the pMTG region was functionally correlated at rest with other regions implicated in control-demanding semantic tasks, including inferior frontal gyrus and intraparietal sulcus. We suggest that pMTG may play a crucial role within a large-scale network that allows the integration of automatic retrieval in the default mode network with executively-demanding goal-oriented cognition, and that this could support our ability to understand actions and non-dominant semantic associations, allowing semantic retrieval to be ‘shaped’ to suit a task or context.
Across our lifetime we acquire a large body of conceptual knowledge, only a subset of which is relevant for any given task or context; thus automatic spreading activation within semantic representations is often insufficient for efficient semantic cognition (Thompson-Schill et al., 1997; Badre et al., 2005; Jefferies, 2013). Automatic spreading activation can facilitate the retrieval of features and associations that are dominant for a particular concept (e.g., carrot-peel). When semantic retrieval needs to be focussed on aspects of knowledge that are not the strongest response for the inputs, additional control mechanisms can be engaged to guide semantic retrieval. For example, control is needed to recover weak associations (carrot-horse) and to match words on the basis of specific sensory-motor features, such as actions or colour (e.g., carrot with traffic cone), since the functional characteristics of these concepts are more central to their meaning (Thompson-Schill et al., 1997; Badre et al., 2005; Whitney et al., 2011; Noonan et al., 2013; Davey et al., 2015a).
Different brain regions have been implicated in the representation and controlled retrieval of semantic information. The ventral anterior temporal lobes (ATL) have been argued to form a key repository of conceptual information, following studies of patients with semantic dementia (SD). These patients have relatively focal bilateral atrophy focussed on ATL, associated with a gradual deterioration of knowledge and multimodal semantic deficits, first affecting fine-grained distinctions between concepts, and then eroding more basic distinctions (Mummery et al., 2000; Hodges and Patterson, 2007; Patterson et al., 2007). Deficits in SD patients suggest a loss of central semantic information (Bozeat et al., 2000; Jefferies and Lambon Ralph, 2006) and studies employing inhibitory transcranial magnetic stimulation (TMS) in healthy participants have provided converging evidence for a necessary role of this region in comprehension (Pobric et al., 2007; Pobric et al., 2010). Functional magnetic resonance imaging (fMRI) studies reveal activation of ATL during diverse semantic judgements (Binder et al., 2009; Visser et al., 2010; Rice et al., 2015). Finally, analyses of inter-regional signal correlations during task free (i.e. resting-state) functional scans have shown that ATL is part of a large scale assembly that includes medial prefrontal and posterior cingulate cortices, commonly referred to as the default mode network (DMN, Raichle et al., 2001; Buckner et al., 2008; Yeo et al., 2011; Jackson et al., 2016).
Converging neuroscientific methods have also identified brain regions beyond ATL which are important for multimodal semantics, specifically left inferior frontal gyrus (LIFG) and posterior middle temporal gyrus (pMTG). These regions are thought to contribute to the control of semantic retrieval. Patients with semantic aphasia (SA), who have lesions affecting these regions following stroke, fail the same range of verbal and non-verbal semantic tasks as SD patients; however, unlike SD cases, they often retrieve information that is irrelevant or inappropriate for the task, show strong effects of cues and miscues, and perform poorly in the face of strong distracters or ambiguous meanings (Thompson-Schill et al., 2002; Jefferies and Lambon Ralph, 2006; Jefferies et al., 2008; Corbett et al., 2009; Jefferies et al., 2010). Converging evidence from fMRI (Poldrack et al., 1999; Badre et al., 2005; Snijders et al., 2010; Noonan et al., 2013; Davey et al., 2015b) and TMS (Hoffman et al., 2010; Whitney et al., 2011; Davey et al., 2015a) supports the view that both of these regions contribute to semantic control. Indeed, in a recent neuroimaging meta-analysis, LIFG and pMTG were the sites activated most strongly and consistently across many different contrasts designed to tap semantic control (Noonan et al., 2013). In addition, when high-control semantic tasks were contrasted with demanding phonological judgements, pMTG and the anterior part of LIFG showed a specifically semantic response, suggesting that these two regions lie outside of the multiple-demand network (MDN), which is recruited during executively-demanding tasks across domains (Duncan, 2010).
These findings therefore provide some evidence that semantic cognition may be underpinned by at least three component processes, supported by distinct brain networks. (1) Domain-general executive control implemented by the MDN (Duncan, 2010) and the fronto-parietal control system (Power and Petersen, 2013) may support the capacity to engage and sustain a particular type of semantic retrieval in line with the task instructions, as well as the application of top-down constraints to support goal-driven aspects of cognition beyond semantics (Duncan and Owen, 2000; Duncan, 2010; Fedorenko et al., 2013; Noonan et al., 2013). For example, in a feature-matching task (in which globally unrelated words must be linked together on the basis that they both have a particular feature specified in the task instructions), there is a need to apply a pre-specified goal during semantic retrieval, and the implementation of this goal may involve the executive system. (2) Activation is thought to spread automatically between highly-related concepts within the representational system (underpinning semantic priming effects for strong associates). This allows dominant features and associations to be retrieved in the absence of executive control, and is supported by ATL and potentially other regions in the DMN (Wirth et al., 2011; Lau et al., 2013; Power and Petersen, 2013; Jackson et al., 2016). (3) A third network might support situations in which there is no explicit goal to indicate which aspect of knowledge should be brought to the fore, but the pattern of retrieval that is required for success is not the dominant one given the stimuli – i.e., semantic retrieval must be controlled to identify and sustain a linking context. The retrieval of relatively weak global associations is a good example of such a task: here, the instructions do not establish which types of associations or features should be the focus for retrieval – instead, it is necessary to establish a linking context from the concepts themselves and retrieve features relevant to this context.
Figure 1 illustrates the spatial distribution for these three putative networks (MDN, DMN, and semantic control) from prior published investigations. This figure shows that regions implicated in semantic control by the meta-analysis of Noonan et al. (2013, in green) are only partially overlapping with the MDN (from Fedorenko et al., 2013, in red). Non-overlapping areas in LIFG and pMTG appear to be important for demanding semantic tasks (relative to easier semantic judgements) but not executive control across domains. Moreover, these semantic control regions are spatially intermediate between the MDN (implicated in executive control) and the DMN (implicated in automatic retrieval, from Yeo et al., 2011, in blue); this location could allow semantic control regions to integrate two distributed networks that are anti-correlated at rest and yet both crucial for semantic cognition, e.g., when semantic knowledge, not a task goal, defines the attentional focus.
Figure 1. Spatial maps of the Default Mode Network (DMN, blue, from Yeo et al., 2011), Multiple-Demand Network (MDN, red, from Fedorenko et al., 2013) and Semantic Control Network (green, from Noonan et al., 2013), presented on a rendered MNI-152 brain and on axial, coronal, and sagittal slices. The key for overlapping areas between different networks is presented on the right hand side of the figure. Images are shown with fully saturated colours to maximise the visibility of the overlapping regions. Regions implicated in semantic control and also found in the MDN include dlPFC (dorsolateral prefrontal cortex), dIFG (dorsal inferior frontal gyrus), pre-SMA (pre-supplementary motor area), IPS (intraparietal sulcus) and LOC (lateral occipital cortex). Regions implicated in semantic control and also found in the DMN include vIFG (ventral inferior frontal gyrus); vMPFC (ventral medial prefrontal cortex) and pMTG (posterior middle temporal gyrus).
The proposal that the control of semantic retrieval is partially distinct from executive control is broadly consistent with functional dissociations that have been identified within left inferior frontal cortex. Within the language domain, studies have reported a functional gradient in left inferior frontal gyrus (IFG), with ventral anterior aspects of IFG implicated in semantic control specifically, and dorsal posterior IFG contributing more broadly to language control, including phonological tasks (Poldrack et al., 1999; Wagner et al., 2001a; Wagner et al., 2001b; Devlin et al., 2003; Gough et al., 2005; Snyder et al., 2007). Dorsal IFG, bordering inferior frontal sulcus (IFS), is recruited when participants select specific aspects of knowledge in line with an externally-specified goal (i.e., instructions to match words by colour or shape in the absence of a global semantic relationship; Badre et al., 2005). This selection process may be important for many language tasks, such as lexical and phonological retrieval. In contrast, ventral/anterior IFG shows an increased response when weak and strong semantic associations are contrasted (e.g., salt-grain > salt-pepper) – i.e., when participants shape retrieval to converge on a distant link between two concepts in the absence of an explicit goal. This ability to recover a non-dominant conceptual link does not generalise easily to other aspects of language processing. Recent work using single-subject analyses identified regions within the multiple-demand network, in dorsal and posterior IFG/IFS, that respond to difficult verbal working memory judgements involving non-words (Fedorenko et al., 2013): these regions are adjacent to, but spatially distinct from, areas of IFG that show a greater response to easier meaning-based trials involving words in sentences (Fedorenko et al., 2012; Blank et al., 2014). Moreover, analyses of resting-state connectivity have implicated anterior aspects of prefrontal cortex in a cingulo-opercular control system, which includes regions that display sustained activity during task-set maintenance, whilst dorsal prefrontal regions couple with a fronto-parietal system engaged by ongoing selection and implementation (Power and Petersen, 2013): this pattern may relate to the functional distinction between anterior and dorsal LIFG. Thus, a more semantic response in anterior/ventral parts of IFG may be broadly in line with the proposal that anterior areas in IFG establish and maintain priorities for what is to be retrieved, while the short-term process of selection itself is implemented in posterior regions of IFG (Badre and D'Esposito, 2007). Badre and colleagues referred to this functional specialisation within IFG as “controlled retrieval” and “selection” respectively (Badre et al., 2005).
The functional contribution of IFG has been considered in detail while the significance of the second region identified by Noonan and colleagues, pMTG, remains controversial. Although this site is implicated in semantic control, a parallel literature links pMTG, together with angular gyrus (AG), to the comprehension of actions and events (Johnson-Frey et al., 2005; Liljeström et al., 2008), and to relational semantics (Humphreys and Lambon Ralph, 2014; Price et al., 2015), and these adjacent areas of temporoparietal cortex can show a similar response to contrasts tapping event knowledge (Wagner et al., 2005; Sachs et al., 2008; Kim, 2011). One theoretical account suggests that AG and/or pMTG provide a “thematic hub”, capturing aspects of knowledge relating to the associations between concepts – such as knowledge about which concepts are found and used together (Schwartz et al., 2011). However, while pMTG often shows increased activation in harder semantic tasks (Noonan et al., 2013), mid-AG typically shows deactivation in semantic and other tasks relative to rest, especially for harder judgements (Binder et al., 2008; Humphreys and Lambon Ralph, 2014; Humphreys et al., 2015). Moreover, these sites showed a double dissociation in a recent TMS study (Davey et al., 2015a): inhibitory stimulation of pMTG disrupted weak more than strong associations, while TMS to AG showed the opposite pattern. These data are not easily reconciled with a simple “thematic hub” account and suggest instead that pMTG and AG support different components of semantic cognition, although ones that can at times function in a cooperative manner.
The current study focussed on understanding the functional contribution of pMTG to semantic control and event/relational semantics: we explored the hypothesis that this region acts to integrate information from the MDN and also the DMN, which are anti-correlated at rest. First, we identified whether there are regions of pMTG that show a common response to the retrieval of action features (contrasted with colour feature judgements which have not been historically linked to pMTG; Badre et al., 2005) and global associations (relative to feature judgements). Both of these event/relational contrasts depend on retrieving information in line with a specific stimulus-driven context, as opposed to the application of a specific goal specified in the instructions. We compared the location of this response to regions implicated in semantic control and domain-general control in previous meta-analyses and also used psychophysiological interaction (PPI) analysis to understand the functional coupling of this region under these conditions. Second, to explore whether pMTG could integrate executive and automatic mechanisms contributing to semantic retrieval, we used resting state fMRI and diffusion MRI tractography to examine if the spatial networks that corresponded to peaks in our easy and hard semantic decisions (i.e., the contrasts of relatively global associations over the harder feature selection and vice versa), converged on the region of cortex identified as important in event/relational semantics. Third, we considered whether the response in pMTG could be linked to the proposed functional gradient in IFG (Badre et al., 2005) by examining whether ventral/anterior as opposed to dorsal/posterior IFG had greater resting state connectivity with this region.
This research was approved by the ethics committee of the York Neuroimaging Centre, University of York, UK. All participants were right-handed, native English speaking, with normal or corrected-to-normal vision. For the experimental task, we recruited 22 neurologically healthy participants from the University of York (Cohort 1). Two participants were removed due to movement artefacts during fMRI data acquisition (Mean age = 24.8, SD = 3.8, range 21 – 35 years, 9 males). Resting-state scans were collected at York for two cohorts (Mean Age = 21.3, SD = 2.7, Range 18 – 31 years, 38 males); 39 participants were recruited into Cohort 2, with 48 recruited into Cohort 3. These two samples were collected as part of different projects; however the resting-state scan was collected before task-based scans in both cases and the data sets are combined in the analysis below. For Cohort 3, we also obtained diffusion MRI. Finally, we also used two publicly-available data sets to provide independent confirmation of the resting-state connectivity patterns observed in this study: (i) 141 participants from the Nathan Kline Institute (NKI)/Rockland Enhanced Sample (Nooner et al., 2012) were used to relate the connectivity patterns of ventral/anterior and dorsal/posterior IFG to the response we observed in pMTG. Full details of this sample can be found in Gorgolewski et al. (2014). (ii) Data from Yeo, Krienen and colleagues (2011), implemented in Neurosynth (Yarkoni, 2011), was used in a final step to investigate the functional connectivity of the pMTG site we obtained across different analyses. We compared the functional behaviour of the pMTG across these different cohorts to minimise the duration of specific testing sessions and to allow us to capitalize on the power of large scale publicly available data sets.
Semantic knowledge for items from two semantic categories (animals and tools) was probed using three tasks (see Figure 2). (i) The first task (global associations) involved matching probe words to a semantically-related target (e.g., selecting honeycomb for the probe bee, as opposed to an unrelated distracter). This task did not require participants to apply a specific goal or instruction to constrain semantic retrieval; instead it was necessary to identify a semantic link from the items presented on each trial. (ii) In the second task, participants were asked to identify a target which had similar dimensions to the probe concept (size matching) (e.g., selecting flannel for the probe sandpaper, as these items are a similar size/shape, even though they are not globally related). (iii) The final tasks required participants to match items based on highly specific features (specific feature matching). For tool items, participants were asked to select the target word that had a similar action to the probe word (e.g., selecting screwdriver for the probe key, as these tools involve similar turning actions). For animals, participants matched items on the basis of colour similarity (e.g., selecting basketball for the probe tiger, as both are orange and black). Both tasks ii and iii required participants to match items based on a feature given to them in the task instructions, as opposed to identifying a link from the stimuli themselves. However, the specific feature matching tasks, based on colour and action, were harder (see behavioural data below). Given our research question, we focussed on the following contrasts: first, the conjunction of action > colour features (localising voxels that respond to action understanding) and global associations > size features (localising voxels that respond to relational judgements), expected to converge in pMTG. By looking at the conjunction of these two sets of contrasts, we can rule out task difficulty as a confound in our localisation of the pMTG (since the global association task was easier than size matching). Secondly, we contrasted the hardest feature selection tasks (action and colour matching) with easier global associations, to identify brain regions responding differentially to control demands (and the reverse contrast for more automatic spreading activation).
The experiment was organised into a total of 36 blocks divided equally among the 6 experimental conditions (i.e., the 3 tasks probed using 2 categories). There were 5 trials per block resulting in 30 trials per experimental condition. Before each block commenced, an instruction slide was presented stating the task to be performed (global, size, action, or colour) for 1000ms. A reminder of the instructions was also present on each trial in parentheses under the probe word. A two-alternative forced choice paradigm (Figure 2) was used; participants were instructed to match the centrally presented probe word to one of two potential targets. Probe words were presented for 1000ms, followed by the response options which remained on screen until a response was recorded via button press, with maximum trial duration set to 4500ms. The inter-trial interval was 4000-6000ms, with 10 seconds of rest between each experimental block. One null event was present in each experimental block to increase the amount of rest which was used as a baseline in the analysis of the fMRI data; the screen was blank for 4500ms plus jitter (4000-6000ms) with the location of the null event randomised in each experimental block. Before the fMRI experiment, participants were given a practice session consisting of two blocks for each condition. The task was presented in the scanner using NBS Presentation version 16 (Neurobehavioral Systems inc., 2013). Participants viewed the words via a front silvered mirror and responded using a Lumina Response Pad (Cedrus Corporation), placed in their left hand.
Figure 2. Example trial structure for all conditions in the task-based fMRI study. The study employed a 2x3 design, with three types of judgements (about global semantic associations, size feature matching and specific feature matching) for animal and tool concepts.
A copy of the stimuli used in this experiment is provided using the Open Science Framework (OSF, https://osf.io/); https://osf.io/5pq8z/.All words used in the experiment were concrete nouns denoting manipulable objects or animals. There were 30 animal probes and 30 man-made probes, repeated across the three tasks (global association, size matching and specific-feature matching), each with a unique target word for each task. The distracters in all conditions were target items from other trials that did not have overlapping features or a global association with the probe. No restrictions were placed on the number of times a word could be used (mean number of repetitions = 2, SD = 1.4, range = 6); however the number of repetitions was equivalent between conditions. For the words in each trial (probe, target, and distracter) we collected measures of familiarity, imageability, manipulability, lexical frequency, word length, and number of words, averaging across all the words in a single trial, and compared trials across relevant conditions. Ratings of familiarity and imageability were taken from the MRC psycholinguistic database (Wilson, 1987). Ratings of manipulability, familiarity, and imageability were also collected on a 7 point scale (1 – low, 7 – high) from a separate cohort of 11 healthy adult participants who did not take part in the scanning sessions (familiarity and imageability ratings were collected for targets with missing values in existing databases). Lexical frequency was taken from the SUBLEX-UK database (van Heuven et al., 2014). Table 1 contains the psycholinguistic variables for the experimental conditions. Trials were matched across conditions for word length (action vs. colour: t(58) = 1.1, p = .276; global vs. size:, t(58) = 1.89, p = .076 ; global vs. feature: t(58) = 1.46, p = .179). They were also matched for number of letters (action vs. colour: t(58) = .14, p = .886; global vs. size: t(58) = .23, p = .818; global vs. feature: t(58) = .75, p = .449), and lexical frequency (action vs. colour: t(58)= 1.48, p = .114; global vs. size: t(58)= 1.13, p = .261; global vs. feature: t(58)= 1.43, p = .156). Manipulability ratings were higher for action than colour trials as expected (t(58) = 9.47, p <.001). Imageability was also higher for colour than action trials (t(58) = 7.61, p <.001). No significant differences were observed for manipulability and imageability for global vs. size (manipulability: t(58) = .027, p = .979; imageability: t(58)= 1.55, p = .123), or for global vs. feature (manipulability: t(58)= .50, p = .616; imageability: t(58)= .322, p = .748). Finally, a different set of 13 participants rated the extent to which they found it necessary to generate a spatiotemporal context to complete the specific feature matching conditions (e.g., colour and action judgements), on a seven point scale (1 – not very useful, 7 – retrieving the context was very helpful). Retrieval of a spatiotemporal context was significantly more important for tool action trials (M = 4.13, SD = .50) than animal colour trials (M = 2.49, SD = .43; t(28) = 13.87 p <.001).
Table 1. Psycholinguistic variables for the stimuli used in the task-based fMRI study
Number of words
Specific feature: action
Specific feature: colour
Structural and functional data were acquired using a 3T GE HDx Excite MRI scanner utilising an eight-channel phased array head coil (GE) tuned to 127.4 MHz, at the York Neuroimaging Centre, University of York. Structural MRI acquisition in all participants was based on a T1-weighted 3D fast spoiled gradient echo sequence (TR = 7.8 ms, TE = minimum full, flip angle 20°, matrix size = 256 x 256, 176 slices, voxel size = 1.13 x 1.13 x 1 mm3). Task-based and resting-state activity was recorded from the whole brain using single-shot 2D gradient-echo echo planar imaging (EPI) with a flip angle = 90°, matrix size = 64 x 64, and field of view (FOV) = 192 x 192 mm2. Other scan parameters slightly varied for task-based fMRI in Cohort 1 (TR = 2000 ms, TE = 30 ms, 32 slices, voxel size = 3 x 3 x 4.5 mm 3, 12 min), resting-state fMRI for Cohort 2 (TR = 2000 ms, TE = minimum full, 32 slices with 0.5 mm gap, voxel size = 3 x 3 x 3 mm3, 7 min) and resting-state fMRI for Cohort 3 (TR = 3000 ms, TE = minimum full, 60 slices, voxel size = 3 x 3 x 3 mm3, 9 min). An intermediary FLAIR scan with the same orientation as the functional scans was collected to improve the co-registration between subject-specific structural and functional scans. In Cohort 3, we also collected diffusion weighted MRI data using a 2D single-shot pulsed gradient spin-echo EPI sequence (TR = 15000ms, TE = 86ms, matrix = 96 x 96, 59 slices, voxel size = 2 x 2 x 2 mm3; b = 1000 s/mm2, 45 diffusion directions, 7 B0 volumes, 13 minutes). Parameters of the independent (NKI)/Rockland Enhanced Sample are described in detail by Gorgolewski et al. (2014) and Smallwood et al. (2016).
Data pre-processing and analysis
a) Task-based fMRI. Analyses were conducted at the first and higher level using FSL-FEAT version 4.1.9 (Smith et al., 2004; Woolrich et al., 2009; Jenkinson et al., 2012). Pre-processing included slice timing correction, linear motion correction (Jenkinson et al., 2002), high-pass temporal filtering (sigma = 100s), brain extraction (Smith, 2002), linear co-registration to the corresponding T1-weighted image followed by linear co-registration to MNI152 standard space (Jenkinson and Smith, 2001), spatial smoothing using a Gaussian kernel with full-width-half-maximum (FWHM) of 5mm and grand-mean intensity normalisation of the entire 4D dataset by a single multiplicative factor.
Pre-processed time series data were modelled using a general linear model correcting for local autocorrelation (Woolrich et al., 2001) using a block design. The linear model included the six experimental conditions modelling block start time and block duration. fMRI scanning was split into two separate scanner runs collected sequentially; both runs were analysed independently at the lower level then combined using a fixed-effects higher-level analysis. Six contrasts were defined; individual conditions > rest (animal/tool global, animal/tool size, tool action, animal colour). We focussed our subsequent analysis on the comparison of easy global associations vs. harder specific feature selection (building on the approach of Badre et al., 2005) and on a conjunction of action > colour and global associations > size to identify regions engaged by event/relational semantics (Nichols et al., 2005). All analyses were cluster corrected using a z-statistic threshold of 2.3 to define contiguous clusters. Multiple comparisons were controlled using Gaussian Random Field Theory at a threshold of p <.05.
b) Psychophysiological Interaction (PPI). A conjunction analysis of action > colour and global > size revealed an area of pMTG that responded to event semantics. We used this pMTG region as a mask and extracted the time-course (for each participant and each run) within this area to examine psychophysiological interactions (PPI; O'Reilly et al., 2012) between the pMTG and other brain regions involved in event semantics. The extracted time-course of pMTG and the interaction were included in a GLM model as explanatory variables (at the lower level, for each participant and each task individually, for each run). As with the functional analysis, the two runs were combined, and the results were submitted to a group level analysis, with the same cluster-forming threshold and significance level (Z = 2.3, p < .05). The contrasts included in this analysis were action > colour and global > size as before – and, as with the functional data, a formal conjunction of these contrasts was conducted. c) Resting-state fMRI. Pre-processing steps were as for task fMRI, apart from the addition of Gaussian low pass temporal filtering, with sigma = 2.8s, and spatial smoothing using a Gaussian kernel with full-width-half-maximum (FWHM) of 6mm. We extracted the time series from 3mm spheres placed at regions of interest (ROIs, see below) and used these as explanatory variables in connectivity analyses at the single subject level. In each analysis, we entered 11 nuisance regressors; the top five principal components extracted from white matter (WM) and cerebrospinal fluid (CSF) masks based on the CompCor method (Behzadi et al., 2007) and six head motion parameters. WM and CSF masks were generated from each individual's high resolution structural image (Zhang et al., 2001). No global signal regression was performed, following the method implemented in Murphy et al. (2009). At the group-level, analyses were carried out using FMRIB's Local Analysis of Mixed Effects (FLAME1), the same cluster correction method used for the functional fMRI was used at the group level.
d) Diffusion MRI. Subject-wise diffusion MRI processing was carried out in native diffusion space using FSL (version 4.1.9). Pre-processing of the DTI data involved eddy-current distortion correction and motion correction using FDT v2.0 (part of FSL), as well as brain extraction using BET. A probabilistic diffusion model was then fitted on the corrected data using BEDPOSTX: the Bayesian estimation of diffusion parameters obtained using sampling techniques toolbox (Behrens et al., 2003). BEDPOSTX uses Monte Carlo Markov chain sampling to generate parameters for probabilistic tractography. Up to 2 fibres were modelled per voxel using a burn-in of 1000 iterations before starting the sampling of diffusion parameters. Next, probabilistic tractography was performed to reconstruct fibres passing through our seed masks using PROBTRACKX. This technique repeatedly samples from the diffusion parameters calculated in BEDPOSTX to build a distribution of the likely tracts from each seed region. The seed masks were transformed from MNI standard space to diffusion space using nonlinear registration. 5000 sample tracts were generated per seed voxel. We used the standard parameters of a curvature threshold of 0.2 (corresponding to a minimum angle of approximately ±80 degrees), a step length of 0.5mm and a maximum number of steps of 2000. No waypoint or termination masks were included. The resulting individual maps were transformed back to MNI standard space, thresholded at 0.02% of total samples sent from the mask and concatenated into a single 4D file. Nonparametric voxelwise statistical testing was performed using FSL Randomize with 25000 permutations in order to get a group tractography map (Nichols and Holmes, 2002). The resulting maps were thresholded at p < 0.01, Family-Wise Error (FWE) corrected, using the Threshold-Free Cluster Enhancement (TFCE) technique (Smith and Nichols, 2009).
Selection of seeds and ROIs
For the psychophysiological interaction (PPI) analysis we used the cluster generated by the conjunction of action > colour and global > size mask. For the resting state analysis we used two functional peaks from our task-based fMRI analyses (data from Cohort 1) as seeds in the analyses of resting state connectivity and diffusion MRI (data from Cohorts 2 and 3): one functional peak was linked to relatively automatic semantic retrieval (easy global associations > hard feature selection, in inferior ATL, MNI co-ordinates -48 2 -38) and one was linked to executive control (hard feature selection > easy global associations, in IFS, MNI co-ordinates -42 28 16). In a further resting-state connectivity analysis (using the NKI data), we placed seeds at peaks responding to different aspects of semantic control taken from Badre et al. (2005), allowing us to link the response in pMTG to the previously-reported functional distinction between “selection” in dorsal/posterior IFG (MNI -48 18 18) and “controlled retrieval” in ventral/anterior IFG (MNI -51 27 3).
We also performed an ROI analysis of the task-based fMRI data using 8mm spheres, focused on these ventral and dorsal LIFG peaks from Badre et al. (2005) and the pMTG peak for semantic control taken from the Noonan et al. (2013) meta-analysis (-58 -49 -8). The FEATquery tool in FSL was used to extract average percentage signal change across all the voxels in each ROI for all six conditions across participants.
Following Simmons et al., (2011) we report the design choices that our study depends on. The sample size of 22 for the functional data (Cohort 1) was based on the assumption that approximately 20 participants with useable data would be necessary to provide a stable measurement of the semantic processes in question. We used samples of approximately 50 participants for the diffusion MRI and 90 participants for the resting state analysis (Cohorts 2 and 3) reflecting the data that was available; moreover, prior studies conducted in our laboratory that have successfully revealed positive results with samples that range from 40 - 90 participants (e.g. Smallwood et al., 2013; Smallwood et al., 2016). For the NKI data, we used the same participants as in a previous investigation (Gorgolewski et al., 2014), since the data was already available. We did not perform a formal power calculation for any of these decisions.
The participants in Cohorts 2 and 3 who provided resting state and diffusion MRI subsequently performed a behavioural battery of tasks in the laboratory. These measures were not directly related to the current experimental question and were not explored in the current study. The relationship between these measures and individual variation in cognitive performance is an ongoing focus in our laboratory (for examples in the public record see Baker et al., 2015; Konishi et al., 2015; Smallwood et al., 2016).
Behavioural performance (reaction time, accuracy and response efficiency) is shown in Table 2. A 2 (category; animals vs. tools) by 3 (task; global association, size feature, and specific feature) repeated-measures analysis of variance (ANOVA) was conducted on response efficiency, revealing no significant differences between item categories (F (1,29) = 1.33, p = .258), and a significant main effect of task (F (2,58) = 19.28, p < .001), demonstrating poorest performance in the specific feature condition (M = 2313), followed by the size feature (M = 2088), and global association (M = 1655) matching conditions. No significant interaction was observed (F (2,58) = .42, p = .662). This pattern of results justifies the comparison of specific feature matching vs. global associations as a way of identifying regions responding to difficult judgements (cf. Badre et al., 2005).
Table 2 - Behavioural results (RT, accuracy, and response efficiency)
Mean (% correct)
Footnote: SE = standard error
The unthresholded statistical maps can be found on Neurovault; http://neurovault.org/collections/WLSYBFYI/. This collection contains the uncorrected z-statistic maps for the 6 experimental conditions (animal/tool global, animal/tool size, animal colour, and tool action) contrasted against rest.
We identified the region of pMTG important for event/relational semantics through a conjunction of two contrasts that commonly involved generating a spatiotemporal or thematic context to identify a link between items: (i) Global semantic associations (i.e., whether chicken goes with egg) were compared with feature decisions about object size (i.e., whether a tortoise is the same size as a helmet), since global associations to both animals and tools require a linking context to be recovered, while size matching does not. (ii) Decisions about action features (i.e., whether the motion used by a key is similar to a screwdriver), were compared with decisions about colour features (i.e., whether a Tiger is the same colour as a Basketball), since tool action judgements involved generating a spatiotemporal framework to support retrieval, while animal colour judgements did not. While these contrasts are different in many ways, pMTG was expected to respond to both since it has been implicated in understanding actions and thematic associations. Importantly, a response to this conjunction cannot be explained in terms of global task difficulty – since the global association task was easier than the size judgement task, according to behavioural performance.
The result of these analyses is presented in Figure 3. When compared to size judgements, global associations activated posterior aspects of the temporal lobes extending from the lateral occipital cortex, along the middle temporal gyrus into the anterior temporal lobes. Activation was also observed in the left frontal lobe focused on middle frontal gyrus, superior frontal gyrus and frontal pole (contrast shown in blue in top panel of Figure 3). Relative to colour judgements, action judgements activated a large left temporoparietal cluster including inferior lateral occipital cortex, posterior middle and inferior temporal gyrus, supramarginal gyrus, superior parietal lobule and angular gyrus. A second inferior frontal cluster revealed activation in precentral gyrus, LIFG (opercularis and triangularis) and frontal orbital cortex (contrast shown in red in top panel of Figure 3). The formal conjunction analysis between these contrasts revealed the hypothesised pattern of shared activation in pMTG, indicating that this region was common to both action and relational semantic judgements (shown in green in Figure 3, both top and bottom panels).