Decision making refers to the cognitive and neural mechanisms underlying processes that generate choices. A well characterized decision-making paradigm is that of evidence accumulation or evidence integration referring to the process by which the subject gradually processes evidence for or against different choices until making a well defined choice. Most behavioral studies to date have focused on evidence accumulation in stationary environments. However, real world environments change over time. Crucially, in a dynamic environment older observations may no longer reflect the current state of the world, and an observer needs to modify their inference processes to discount older evidence.
Along with Alex Piet (Piet et. al. 2018), I have developed a behavioral paradigm called "Dynamics Clicks Task" . The task is explained in the following figure:
Dynamic Clicks Task structure and example trial. a Schematic of task events and timing. A center light illuminates indicating the rat may initiate a trial by poking its nose into a center port. Auditory clicks are generated from state-dependent Poisson processes (the two states are schematized by light green and light blue backgrounds) and played concurrently from left and right speakers. The hidden state toggles between two states according to a telegraph process with hazard rate h. When the auditory clicks end, and the center light turns off, the rats must infer which of the two states the trial ended in and report their decision by poking into one of two reward ports. Trials have random durations so the rat must be prepared to answer at all time points. b An example trial. The hidden state transitions randomly, and the auditory clicks (black triangles) are generated accordingly. The optimal inference process (black line; see text for its derivation) accumulates clicks, and discounts accumulated evidence proportionally to the volatility of the environment and click statistics. For the optimal process, a choice is generated at the end of the trial according to whether the optimal inference variable is above or below 0
Using a high-throughput automated rat training, we trained rats to accumulate and discount pulse-based evidence in a dynamic environment.we formalized the optimal behavior on our task, which critically involves discounting evidence on a timescale proportional to the environmental volatility and the reliability of each click. Importantly, our pulse-based analysis allowed us to separate evidence reliability into experimenter imposed stimulus statistics and sensory noise. We find that once sensory noise is taken into account, the rats have timescales consistent with the optimal inference process. We used behavioral modeling to investigate rat to rat variability, and to predict a moment-by-moment estimate of the rats’ accumulated evidence. Finally, we demonstrated rats can adjust their integration timescales in response to changing environmental statistics. Our findings open new questions into complex rodent behavior and the underlying neural mechanisms of decision making in dynamic environments.
Trained behaviors in the context of decision making gives us a stereotypical trial structure that one can use in order to unravel the input - output relation between stimulus and behavior on hand , stimulus and neural activity on the other hand. Nevertheless, these behaviors does not reflect the natural decisions that the animal is taking in their natural habitat. Ultimately, one would mimic natural like decision making in a laboratory setting. The prime candidate for this type of naturalistic decision making is : foraging.
Foraging is one of the most ubiquitous behaviors that animals exhibit, as search for food is essential for survival. Foraging can be considered a natural cognitive behavior comprising: navigation, spatial memory, statistical inference, route planning, motor control, arousal system, reward system and threat monitoring system. Thus, providing the opportunity to understand how these processes and their interplay have been shaped by natural selection to optimize returns in the face of environmental and physiological constraints. The patch-leaving problem is a canonical foraging task where an animal must decide when to leave a resource to search for another.
Along with Jacob Davidson (Max Planck Institute for Animal Behavior), we formulated a mechanistic model of patch-leaving decisions by linking ecological models of the patch-leaving task with models of evidence accumulation that are used in systems neuroscience. We call this model the foraging drift-diffusion model (FDDM). In our model, patch-leaving decisions are described by a drift-diffusion process, which represents the noisy process through which an animal accumulates evidence (by finding food), and how this experience is used to decide when to leave the patch. Evidence accumulation and decisions within a patch are coupled to a moving average process that keeps track of the average rate of energy available from the environment.