Learn how to plan and structure your stimulus presentation order in Pro Lab
In a few cases, the structure of your experiment is very simple: just add the stimuli (the images and videos) you want to track the gaze on and run the experiment. However, in most experiments it's not that simple. One reason for this is that we have order effects. This means that how we react to things is influenced by the order in which we see them.
For example, at the beginning of an experiment you don’t know what to expect, you perhaps don’t know what to look for to solve your task, or you have not formed an idea of what the researcher is after. So how you view an image is different at the beginning of the experiment, compared to the end of the experiment. We could call this a 'training effect', i.e., you become better at solving the task with more training and experience of the task. Alternatively, you could also consider an effect of the opposite kind. At the beginning of the experiment, you are full of excitement and wonder at being part of your first psychological experiment. You are attentive and quick to respond. However, after 30 minutes of responding to close to a hundred stimuli, you find yourself low on energy and slow to respond. You are now experiencing a fatigue effect.
The consequence of these two order effects is that how people respond to stimuli which are presented at the beginning of a test is different to how they respond to stimuli at the end, and this difference is not driven by the individual stimuli as such so you run the risk of falsely concluding that the first half of the stimuli was much easier than the latter half of the stimuli.
We want to prevent one or several stimulus items from carrying a disproportionate amount of these order effects, because otherwise it's unclear which viewing behavior is driven by the stimulus and which is driven by inexperience or fatigue. Ideally, we want to distribute the learning and fatigue effects evenly across all stimulus items. One way to do this is to randomize the presentation order of the stimuli. If the experiment consists of a group of images, then we would simply shuffle the order, so for one participant an item could end up right at the beginning of the experiment, but for another, it could end up in the middle or at the end of the experiment. On average, after making many recordings with this experiment, the stimuli will end up roughly equally distributed in all positions.
Figure 1. Shows a set of stimuli, here ordered 1-6, that is then shuffled in a random order for every new recording.
Depending on the experiment and the type of stimuli, different presentation orders are relevant. The simplest possible form would be to simply show all the stimuli in whatever order they have been put in the group, from the first to the last. Such a sequential order would be very appropriate if you have a group of images containing instructions for the participants. Obviously, they have a reading order and should not come randomly.
For other stimuli, like experimentally manipulated images, a random order makes the most sense, for the aforementioned reasons of order effects. In many experiments however, the trial does not consist of just a group of images. You may have a fixation cross in the center of the screen to start off the participants in a search task from the same point. In this experiment, we do not want to just randomize the presentation order of everything, so we end up with several images or fixation crosses in a row. Instead, we want to have the fixation cross fixed at the beginning of every trial, and then just sample one image from a group of images, one at a time.
Figure 2. Shows a study design where a fixation cross is displayed at the beginning of the test and is then followed by a randomly sampled image from a pool of different stimuli images.
This sampling, however, can occur in different ways. The simplest way to set it up, is to sequentially sample one item at a time. If you have a particular order you want to present the images, but every image should be preceded by some fixed element, then this choice makes sense. Then the fixed elements will occur, and then an image from the group will be sampled, until all of them have been sampled.
The most common choice however, is 'random without replacement'. This means you randomly sample one item from this group, and once it is consumed, it is not put back in the group. So, it will be impossible to pick the same item twice in a row (unless the list is repopulated with items again). A clear benefit of this approach is that it guarantees that the data set is balanced, in the sense that you show every item the same number of times.
The third way of sampling items is 'random with replacement'. The difference, as revealed by the name, is that after sampling a particular element it goes back into the group again. This means you can pick the same element again during the next draw.
If we have a group of stimuli, say three of them, then we want to iterate our grouping showing a fixation cross and a sample a new image. One way to do this is to wrap this group in another group with a repetition behavior and repeat it three times so all the stimuli are exhausted.
Figure 3. When presenting the design shown in this figure to the experiment participant, they will see; Trial 1 - a fixation cross followed by a random visual search array; Trial 2 - a fixation cross again, and one of the remaining arrays and; Trial 3 - a fixation cross followed by the last array stimulus.
In many experiments, showing something once does not give you enough data to uncover the effect of experimental manipulation. For an experiment where repeat viewing of the same material is not problematic, it would make sense to create several cycles of the same material. To add another cycle, and have two presentations of every stimulus, then one approach is to simply double the number of repetitions from 3 to 6. Note that for sampling randomly without replacement, the stimuli are not actually removed forever when they are sampled, they are simply temporarily excluded from selection. If the repetition continues with a fourth repetition, then all stimuli are back as available for selection as again. The second approach to produce two cycles is to wrap the group with three repetitions in another group with two repetitions. This would produce 2x3 = 6 stimulus presentations.
If the experiment simply consists of showing a random order of all the stimuli, then it may be an inelegant way to add a group with sample behavior and another group with repetition to be able to display all the material. The simpler way is simply to create a group with a shuffle behavior. This makes the overall build less complicated because there is one less group to keep track of.
However, it is often productive to consider several of these mechanisms together, to be able to build up a richer presentation flow in the experiment. For example, a trial can be seen as a group itself, containing a fixed part, and then a random part that samples from another group. This trial can then exist as different copies with small changes to the variants. Each of these trial groups can then be located inside a group which repeats, until all the material it samples is consumed. These groups are then contained in a single group which shuffles the presentation order of the repeated trial groups. This way it is possible to scale up an experiment from a pilot test to a full experiment very quickly. Such an example is illustrated below.
Figure 4. Shows a more complex presentation order in a visual search experiment. This experiment starts randomly with a block of either X/O arrays or T/L arrays, and then performs three trials of these, with a fixation cross before each. Then it continues with the next block, with a fixation cross before each search array.
Watch the video above to see how to set up your stimulus presentation in Pro Lab using Group elements.
Once you have designed your study structure, you are ready to implement it in Pro Lab. To start this process, go to the Design module of the software and start to organize your stimulus presentation elements on the timeline. The timeline is a linear representation of your presentation sequence. Besides being able to add stimuli elements directly to it, e.g. images or videos files, you can also add a "Group". A “Group” is a container, with its own timeline, to which you can add other presentation elements. You can assign different presentation rules or actions to each "Group". These allow you to "break" the linear or sequential presentation of the timeline and introduce presentation loops, sampling rules, and shuffling (randomization) of presentation elements. You can use "Groups" to apply the rules and actions discussed in the previous sections to the stimulus presentation.
Let's see how you can implement the example in Figure 2 in Pro Lab. This time create an "Instruction" and "Thank you" element and add it to the experiment:
You are now ready to run your first test recording. Every time you run a recording the subject will see the following presentation structure: Instruction element > "Fixation cross" element > Randomly selected "Search array" element > "Thank you" element.
"Group" elements is a versatile feature that allows you to combine different presentation rules to implement the different levels of experiment design complexity in Tobii Pro Lab.
