What? Research teams are solicited to participate in a project that is examining the variability of neuroimaging results when the same data are analyzed by different research teams. Each analysis team will receive access to a raw imaging dataset including fMRI data of 108 participants that performed two versions of a mixed gamble task. Each analysis team will freely analyze the data using their standard analysis techniques. Preprocessed data (using fmriprep) will also be provided for any team that wishes to use them.
Each group will be asked to provide a binary (yes/no) decision regarding nine specific a priori hypotheses (regarding activation in specific areas for specific contrasts). In addition, groups will be asked to submit both thresholded and unthresholded statistical maps for each of the contrasts of interest, and a description of their analysis pipeline (according to the COBIDAS guidelines). All analysis teams will be anonymized prior to reporting or subsequent sharing of the submitted results.
When? Estimated start date is November 2018, when access to the data would be given. Analysis teams will be given three months to perform the analysis and report the results.
Why? In addition to being part of a fascinating landmark project, all investigators who submit data (up to three individuals per group) will be listed as co-authors on the initial paper.
Contact. In case you have any questions, please contact Rotem Botvinik-Nezer via [email protected].
Overview. We collected fMRI data on two versions of a mixed gambles task (Tom et al., 2007). On each trial, a mixed gamble was presented (one gain amount, one loss amount) and the participants decided whether to accept the gamble or not. Each participant was assigned to one of two conditions: In the equal indifference condition, the matrix of gambles included potential gains twice the range of potential losses (Tom et al., 2007); in the equal range condition, the matrix included an equal range of potential gains and losses (De Martino et al., 2010).
Data. 119 healthy participants completed the experiment (n=60 from the equal indifference group and n=59 from the equal range group). Nine participants were excluded prior to fMRI analysis based on pre-registered exclusion criteria: Five did not show a significant effect of both gains and loses on their choices (Bayesian logistic regression, p < 0.05; reflecting a lack of understanding of the task) and four missed over 10% of trials (in one or more runs). Data of two additional participants is currently under QA. Thus, at least 108 participants will be included in the final dataset sent to the analysis teams (n=54 from the equal indifference group and n=54 from the equal range group).
MRI protocols. MRI was performed on a 3T Siemens Prisma scanner at Tel Aviv University. MRI scanning included the following acquisitions:
Data transfer and processing. Data were converted to NIFTI format using dcm2nii and transformed into the Brain Imaging Data Structure (BIDS). Data will be provided to participating researchers using the Globus data transfer service.
Data to be shared for analysis across groups. For the analysis project, the following data will be shared according to the BIDS format: (i) Task fMRI and structural data, and (ii) Behavioral data for mixed gambles task (including trial timing for fMRI analysis). Each analysis team will be provided with the raw data. In addition, data preprocessed with FMRIPREP will be made available to any researchers who wish to use them.
Upon arrival, as soon as they signed the consent forms, participants were endowed with 20 ILS (~$5.5 USD) cash payment for their participation. The experimenter explained that the money is theirs to keep, and is part of the full amount they would receive at the end of the experiment. Next, the participants received general instructions regarding behavior inside the scanner and performed a shortened version of the full task (i.e. a demo).
Then, participants entered the MRI scanner. Participants completed four runs of the mixed gambles task, each consisting of 64 trials. On each trial, a mixed gamble was presented, entailing a 50/50 chance of gaining one amount of money or losing another amount. Possible gains ranged from 10-40 ILS (in increments of 2 ILS) or 5-20 ILS (equal indifference and equal range conditions, respectively) and possible losses ranged from 5-20 ILS (in increments of 1 ILS). All 256 possible combinations of gains and losses were presented across the four runs.
Stimulus presentation was similar to Tom et al. (2007). Timing of all stimuli and response events were computed using Matlab 2014b and the Psychtoolbox on an Apple MacBookPro running Mac OS X Yosemite version 10.10.5 (Apple Computers, Cupertino, CA). The timing and order of stimulus presentation was optimized for estimation efficiency using a tailored code from the creators of neuropowertools. Due to the fact that this experiment involved only one trial type, which is beyond the scope of the existing tool, a custom solution was required. The efficiency calculations assumed a 32s HRF, a 1s TR and a truncated exponential distribution of ITIs (min=6s, max=10s mean=7s, lambda was extrapolated from these parameters), the minimal ITI encompass a potential trial duration of 4s and 2s intermission.
To even the gambles between different runs, the full matrix of gambles was divided into 16 4X4 sub matrices, which were independently scrambled and allocated to the different runs. This procedure facilitated the overall similarity between runs. Eight different onsets were created using this procedure for each experimental condition.
As in Tom et al. (2007), participants were asked to evaluate whether or not they would like to play each of the gambles presented to them (strongly accept, weakly accept, weakly reject or strongly reject). They were told that one trial from each of the runs would be selected at random, and if they had accepted that gamble during the task, the outcome would be decided with a coin toss; if they had rejected the gamble, then the gamble would not be played.
Following imaging, participants were presented with questionnaires regarding gambling attitudes and made a number of choices involving hypothetical gambles.
Parametric effect of gain:
Parametric effect of loss:
Equal range vs. equal indifference:
The NARPS data are initially shared under a limited data use agreement; the principal restriction is that users of the data will not be allowed to release, publicize, or discuss their results until the end of a specified embargo period. The primary data will become publicly available once the prediction markets close.
All result images submitted for this project will be shared openly via neurovault.org upon completion of the project, with no restriction on reuse. The analysis reports will also be shared. The research team generating the images and report will be labeled via a unique code (team ID) but will not be identified; however, we cannot guarantee that it will not be possible to identify the lab generating the results on the basis of other information such as features of the image or the analysis pipeline used.
For the primary papers to be produced from this project (including initial analysis of prediction market outcomes and neuroimaging analyses), members of the primary project team will draft the manuscripts. All members of each analysis team will be offered co-authorship on the papers; each analysis team is limited to no more than three participants (recommended size of two, with one PI and one trainee). Authorship will be limited to analysis teams who submit their results and report by the deadline. Co-authors from the analysis teams will be given two weeks to review any drafts of papers prior to submission.