Student Project: Are expectation effects on reward-based learning performance based on the model-based or model-free system?

This is a project suitable for MSc or Forskerlinje students. It is also possible to participate in the research in the form of a BSc or semester project.


When we make decisions, we rely on our estimate of the value associated with possible outcomes. Learning these values is called reward-based learning and is influenced by two processes: A model-free process characterized by slow and cumulative learning from errors and a model-based process based on building a mental model of the environment to predict the optimal decision. Previous research showed that expectations influence performance in reward learning. We want to investigate the influence of non-invasive brain stimulation (NIBS) and expectations on model-free and model-based reward learning. This is relevant for understanding how NIBS and expectations influence cognition.

Espen Bjørkedal, PhD, University lecturer, Cognitive Neuroscience Research Group.
Matthias Mittner, PhD, Associate Professor, Cognitive Neuroscience Research Group.

Theoretical background
Experimental research on reward learning utilizes learning tasks where participants have to make decisions that lead to different outcomes. Their task is to learn the value associated with the outcomes of their decisions in order to perform optimally. By using computational modeling, it has been shown that performance is guided by two fundamentally different processes: by building a mental model of the environment and try to predict the best option (model-based) or relying on results from choices that we have previously made in the assumption that these positive or negative outcomes will generalize to the new choice (model-free). Previous research shows that expectations increase performance on the learning task by improving the model-free system. On the other hand, expectations can both improve or impair the model-based process. Previous research has not been able to separate the influence of expectations on these two processes in a single experiment. We propose a novel experiment which investigates the respective effects on both of these systems to get a better understanding of these learning-related effects. An important aspect in this regard is the involvement of the dorsolateral prefrontal cortex (DLPFC), the area associated with model-based learning, which can be stimulated using non-invasive brain stimulation techniques (NIBS).

Research question/ research hypothesis

  1. Is it possible to differentiate between model-based and model-free performance using a modified version of a reinforcement-learning task?
  2. Can we induce expectation-effect on either of the two modes of processing?
  3. What is the relative strength of these effects on the model-free and model-based systems, respectively, and is this related to the application of NIBS?

Design, Procedure, and Method
In order to estimate the relative usage of the model-free or model-based strategies, we developed a special gambling game that involved choosing between different lotteries. This experimental task can be used, by applying computational cognitive models of reinforcement learning, to disentangle the use of the two different learning systems. In addition, previous research recently showed that performance in this task is positively correlated with usage of the model-based system, thus allowing separate indices for involvement. The relevant feature of this task is a two-step decision requirement, meaning that a user has to make to interrelated choices. Model-based performance is indicated when the participant uses relevant information from previous encounters of the second-stage choices to inform her first-stage choice. We will use a reinforcement procedure together with an active NIBS intervention to investigate this. This will result in an experimental design composed of three different groups undergoing slightly different versions of the task. Transcranial direct current stimulation, a well-researched NIBS technique, will be used to stimulate the dorsolateral prefrontal cortex using a high-definition, 4×1 pattern of electrode positioning. We will use both conventional frequentist statistics and Bayesian approaches to analyze the data.

Student’s tasks and learning outcomes
The student will be responsible for recruiting participants, conducting data collection, and contributing to data analysis. She/he will learn about the psychological and neurobiological background of reward-based decision making and transcranial direct current stimulation, a non-invasive brain stimulation technique. The student will have the opportunity to actively participate in design decisions, task/lab preparation, and all aspects of the research. In addition, the student will have the opportunity to study computational cognitive models of reinforcement making (although this is not a requirement). The student will be also named as a co-author on all publications related to this project.

Research environment and research group
The study will be performed within the Research Group for Cognitive Neuroscience at IPS. This research group has a good track record of conducting high-quality research (both in the field of tDCS and value-based decision making) and publishing in international peer-reviewed journals. The project will be supervised by Espen Bjørkedal and co-supervised by Matthias Mittner. All technical aspects of the study (i.e., lab equipment) are already available at IPS. The research proposed here will be conducted in close collaboration with our long-term collaborator, Dr. Zsolt Turi from the University Hospital of Göttingen. Our group has already conducted two studies using similar stimuli and tasks and we are therefore confident about its suitability as a productive student project.

Comments are closed.