Getting started / Making an experiment

Making and running an experiment – Visual method

In the visual method of making an experiment we still work with objects, properties, and MATLAB values. However, we lay them out in tables. You can use any spreadsheet app for this, but Google Sheets is most common since it gives you dropdown menus and quick docs. Options allow making even complex experiments in short tables. Then in MATLAB use the command loadExperiment to load a spreadsheet file, followed by commands like runExperiment.

We’ll look at the basic steps first. These are enough to make simpler (but still realistic) experiments. Then we’ll look at options you can add in for more complex experiments. See also example experiments and a tutorial video.

Making an experiment – General

(1) Set up an experiment file – makeExperiment

To start you can use makeExperiment at the MATLAB command line to open a new experiment spreadsheet.

▸ If you choose Google Sheets, this includes tables, formatting, dropdown menus, quick docs, documentation links, etc.

▸ If you choose a spreadsheet app on your computer (Microsoft Excel, Apple Numbers, etc.), it's the same except it doesn't include dropdown menus or quick docs.

▸ makeExperiment is optional. You can always use a blank spreadsheet in any app.

(2) Define trials

The main part of making an experiment is making its trial definitions. For each trial this just means setting property values for all the element objects to run in it. You can also set trial object properties if you want to change any away from default (e.g. preTrialInterval). To make trial definitions, use a table in your spreadsheet file with columns = object properties and rows = trial definitions—see example below. You can also use more than one sheet/table if convenient.

The usual approach is to ignore trial repetition and order to run in when defining trials. We’ll add that in separately later (Trial list). Instead just write each distinct table row once, and in any convenient order. Often you can make all the trial definitions you need in just a few rows.

• ‍Heading row 1 is where you specify what objects will be in the trial definitions in the table. Each heading is an object type name (e.g. picture).

Optionally you can add an object variable name (e.g. stimulus), separated by a space or line break. Object variable names can be any MATLAB variable names. They are identifiers that distinguish multiple objects of the same type. They appear in places like experiment results output. You can add an index after an object variable name if you want to organize multiple objects into the same variable, e.g. stimuli(1), stimuli(2). If you don’t specify an object variable name, default is same as object type name.

• ‍Heading row 2 is where you specify names of properties you want to set for the objects in row 1. You can also use expressions to set parts of properties, e.g. end.duration to set field duration of struct property end, position(1) and position(2) to set components of vector property position in separate columns, etc.

Typically group property columns for an object under the same object heading. You can also set properties for an object in groups of columns that aren't next to each other—just repeat the object heading at each group (see examples below).

(Google Sheets)
In Google Sheets you can use dropdown menus for object and property headings. However, you can also type over any dropdown to make a custom heading (e.g. add an object variable name/index in a row 1 heading).

• ‍Rows 3+ are your trial definitions. In each row set values for the targets you made headings for. These are just MATLAB values in MATLAB syntax, e.g. <cd>2<cd>, <cd>true<cd>, <cd>false<cd>, <cds>"hello"<cds>, <cds>'goodbye'<cds>, <cd>[1 2 3]<cd>, <cd>1:3<cd>, <cd>{<cd><cds>'a' 'b' 'c'<cds><cd>}<cd>, etc. More generally they can be any MATLAB expression, e.g. <cd>randomNum(0, 1)<cd> to set a property to a random number between 0–1. PsychBench evaluates all values/expressions in MATLAB when you load the experiment later.

Auto repeat: If you want the same value/expression for all trial definitions in a column, you can just write it once for the first trial definition. When you load the experiment in MATLAB, PsychBench will automatically repeat the cell down for however many trial definitions there are. Note by default it repeats the expression in the cell, then those expressions evaluate independently. Usually this is the same as repeating a value, but not always—for example, if you auto repeat randomNum(0, 1) then each repetition will evaluate to a different random number, which is typically what you want. (Optionally you can tell it to evaluate once and repeat the actual value instead.)

Little notes about how PsychBench reads spreadsheets:

• Curly quotes “” and ‘’ are read as straight quotes "" '', so you don’t need to worry about them when specifying string values <cds>"x"<cds>, <cds>'x'<cds>.
• TRUE/FALSE are read as true/false.
• Sometimes spreadsheet apps hide or remove a quote at the start of a cell (' and/or "). If yours does and you're trying to set a string value, just use two opening quotes (<cds>''x'<cds> or <cds>""x"<cds>) or add the missing quote back in, whatever works.
• PsychBench ignores formatting, sheet names, comments added using your spreadsheet app’s comment or note features, spreadsheet data types, and any empty rows and columns at table edges. So you can use or not worry about these, as you like.

(3) Set trial repetition and order – Trial list

After you have your distinct trial definitions, you can add repetition and order to run in. The usual way is to set a trial list, which is just a list of trial definitions for the experiment to run through. To do this, use a separate sheet/table where you fill in just one cell (you can name the sheet if you like). By default PsychBench automatically numbers trial definitions 1, 2, 3, … in the order they appear across your trial definition tables, and the trial list is just a MATLAB vector with numbers drawn from them. You can repeat numbers in the vector. You can order numbers however you want, including randomizing order in parts or in whole. As usual you can use MATLAB expressions to set the vector—little functions rep and randomOrder in <><PsychBench folder><>/tools are useful here. There are options you can apply here which can be important in more complex experiments: see Different trial structures, Custom trial definition numbering/names later.

Like for auto repeat above: By default when PsychBench repeats running a trial definition, it copies the expressions in it, then those copies evaluate independently. e.g. if a trial definition has randomNum(0, 1) for a property then each time it runs will give a different random number. (Optionally you can tell it to evaluate once and copy the actual value instead.)

The trial list is optional. If you don’t set it, the experiment just runs each trial definition once in number/name order. Or for unusual cases you can set repetition and order directly in trial definitions.

(4) Define objects not specific to trial – experiment, devices, staircases

If you need to set any properties of objects that are not specific to trial—e.g. an experiment object, screen or other device objects, staircase objects, etc.—do this in a separate sheet/table. This is like a trial definition table except it only has one row of values.

Making an experiment – Options

There are various options you can add to the basic steps above to make more complex experiments. In any given experiment you won’t use most of these. But you can use any of them as needed:

Table shortcuts

In addition to standard features in your spreadsheet app (e.g. "="), there are some PsychBench-specific shortcuts you can use in your tables. When you load the experiment in MATLAB, PsychBench will automatically apply these to fill out cells in tables. You can combine any number of these shortcuts: They work in the order listed below. Further if you combine *#, *, ? for a column, you can specify the order they work in by the order you list them in the shortcut heading.

Sometimes a shortcut or two can save hundreds of rows of typing. But other times the logic of combining shortcuts can become complicated and just writing things out the long way can be easier or clearer. Generally use shortcuts only as far as they make things easier for the experiment you're working on! However, at minimum you will probably want to use the auto repeat shortcut often.

• Fill cells in a column from a vector/array – ^

You can tell PsychBench to fill cells in a column with values from a vector or array, one value per cell. To do this, write ^<><values><> in any cell, where <><values><> is an expression for the array to draw from. e.g. ^1:100 to fill 100 cells with the numbers 1–100. You can also use a variable name to draw from an external variable, e.g. ^myArray. PsychBench will fill starting at the cell containing ^ and into any empty cells below it, adding rows after that if needed.

If filling from a row or column array, values draw from across its length. If filling from a multi-dimensional array, values draw from across its rows (dimension 1). If filling from a row/column cell array, values are extracted from cells.

• Repeat in a column – *#

You can tell PsychBench to repeat all the values/expressions in a column. To do this, add a third heading row (see examples below) and put *# in it for the column, where is # number of times to repeat, e.g. *5. PsychBench will repeat into any empty cells below the values, adding rows after that if needed.

• Generate combinations across columns – *

If you want to make a trial definition for each combination of values/expressions across two or more columns, you don’t need to make them explicitly. Instead you can just write the distinct values/expressions in each column, then tell PsychBench to generate combinations. To do this, add a third heading row (see examples below) and put * in it for each of the columns. PsychBench will replace the columns with one combination per row, adding rows to the table if needed. In case it matters, order of combinations is what would be generated by for loops nested (outer to inner) in the order the * columns appear. See example below.

Treat columns as one column – *[ ]
If you want cells across some * columns to keep lined up in the combinations, use *[name] instead of just * for those columns, where name is any text label you want to identify the group, e.g. *[b]. Columns in a [ ] group act like a single column for generating combinations (they must have the same number of values/expressions). They don’t need to be next to each other in the table or even under the same object.

Generate combinations within groups
You can generate combinations independently for different sets of columns. To do this, use *name, where name is any text label, e.g. *A. Note you can combine this with [ ] groups, e.g. *A[b] = generate combinations with columns marked with A, treating columns marked with [b] as one column.

• Randomize order in a column – ?

You can tell PsychBench to randomize the order of all values/expressions in a column. To do this, add a third heading row (see examples below) and put ? in it for the column. (Note ? is useful for randomizing order relative to other columns. To randomize the order whole trial definitions run in, typically just randomize their order in the trial list.)

Keep cells across columns lined up – ?[ ]
Like with * above, if you want cells in some ? columns to keep lined up in the randomization, use ?[name] for those columns, e.g. ?[b].

• Auto repeat

If you want the same value/expression for all trial definitions in a column, you can just write it once for the first trial definition. PsychBench will automatically repeat the cell down for however many trial definitions there are (after applying all shortcuts above in all columns, which can add trial definitions). More generally you can use auto repeat anywhere in a column: any empty cells will copy down the last non-empty cell above them.

• Set multiple objects in the same columns

If you have multiple objects of the same type for which you want to set a property/target to the same value/expression in each row (possibly different across rows), you can use a single object heading (row 1). In the object heading list multiple object variables separated by spaces/commas/line breaks, e.g. picture stimulus distractor, or using multiple object indexes picture stimuli(1:2). This repeats all the property columns under that object heading for each of the objects. Note you can mix this with separate object headings for properties where you want to set different values across the objects.

By default shortcuts that repeat cells (*#, *, auto repeat) repeat the expressions in the cells, then those expressions evaluate independently. Usually this is the same as repeating values, but not always. e.g. if randomNum(0, 1) gets repeated then each repetition will evaluate to a different random number, which is typically what you want. (Optionally you can tell it to evaluate once and repeat the actual value instead.)

Repeat value, not expression – $

By default table shortcuts that repeat cells (*#, *, auto repeat) repeat the expressions in the cells, then those expressions evaluate independently. This also applies for cells in trial definitions that repeat running in an experiment. Usually this is the same as repeating values, but not always. For example, if randomNum(0, 1) gets repeated then each repetition will evaluate to a different random number, which is typically what you want.

Optionally you can tell PsychBench to evaluate once and repeat the actual value instead. To do this, add $ at the end of the expression, e.g. randomNum(0, 1)$. In this example you would then get the same random number repeated.

Dependent values

When you write expressions in your tables, you can reference other object property values in the table, make and reference free variables, and use conditional code.

• Reference other properties

You can make property values dependent on other property values. To do this, reference object variable + property in the expression you set, e.g. stimulus.color, stimuli(2).color, etc. The referenced value must be in an earlier column in the same table. In a trial definition table the reference is to the value in the same row (trial definition). 

• Reference variables

You can also make property values dependent on "free variables" which you set with any names and values. To define a variable, add a column anywhere before the first object in a table, leave heading row 1 (object) blank, and specify a variable name in heading row 2. As always these will be evaluated in MATLAB, so names must be valid MATLAB variable names. Set values in the column just like you would for a property. You can then reference the variable by name in expressions in later columns in the table. In a trial definition table the reference is to the value in the same row (trial definition).

Note when you define variables you can set them in parts just like properties—e.g. use separate column headings var(1), var(2) if you want to set the numbers in a 1×2 vector variable separately. You can also use table shortcuts and other options for them like for property columns.

• Reference external variables

You can also reference external variables which you will set values for later in MATLAB when you load the experiment. Provide these variables as additional inputs to loadExperiment. In expressions in your tables reference them by the names they will have in the workspace where you call loadExperiment and that you will input them with. The order you input them in to loadExperiment doesn't matter, only their names. Note you can mix this with the ^ shortcut to fill cells in a column from an external array, e.g. ^myArray. You can use external variables anywhere, even in the trial list.

• Conditional code

In addition to values and expressions you can write little blocks of MATLAB code in table cells. Code must make a variable called ans containing the value to set (or if you just state a value without assigning it to a variable, it will go in MATLAB's default variable ans). Note you can reference other variables/properties as above. Typically do this to make a value conditional using an if block. Use commas and semicolons to separate lines. e.g. <cdk>if<cdk><cd> num == 2, true; <cd><cdk>else<cdk><cd>, false; <cd><cdk>end<cdk>

Two notes:

(1) If you use table shortcuts that repeat/reorder cells (*#, *, ?, auto repeat), by default any property/variable references in those cells evaluate after the shortcuts apply. In other words shortcuts fill the table out, then expressions get evaluated in whatever row they are afterward. See examples below.

(2) Property/variable references are all within expressions which get evaluated in MATLAB, so they reference actual values. For example, if you use randomNum(0, 1) to set a property and then reference that property in another property later in the row, the two properties will always use the same random number.

Leave properties at default/unset

You don't need to include columns for properties that you want to leave at default (unset) in all rows in a table, which is usually most properties. Aside from that, in trial definition tables there are further options for unusual needs:

• Leave a property/target at default in some but not all trial definitions – <cd>[]<cd> / -

→ Set the property = <cd>[]<cd> (MATLAB empty).
→ More generally you can put - in a cell to mean don’t set the property/target at all.
Note both <cd>[]<cd> and - work with auto repeat and other table shortcuts above (e.g. they can be in combinations generated using *).

• Leave all properties of an object at default in all trial definitions

→ Make one column for the object and set the object heading (row 1) but leave the property heading (row 2) blank. This makes the object in each trial definition but doesn’t set any properties, so all are left at default. (Additionally you can put - in cells in the column where you don’t want to make the object at all in those trial definitions.)

• Define an empty trial

→ If you set every cell in a row = -, the trial is still defined (takes a trial definition number) but will skip running wherever it is used in the experiment because no objects are set in it.

Different trial structures

The trials (rows) × properties (columns) layout of a trial definition table is good for the common case where every trial has the same structure, i.e. similar objects with just a few property values differing. If you need some trials with different structures—e.g. intros, breaks, outros, syncs, training trials, etc.—you can use multiple trial definition sheets/tables with different heading rows. PsychBench continues numbering trial definitions 1, 2, 3, … across tables for use in the trial list. See example below.

Custom trial definition numbering/names

You can give some trial definitions custom numbering or names for use in the trial list. This is often important in more complex experiments in order to have trial definition identifiers that are easy to work with (for example, giving trial definitions that will run together adjacent numbers so that you can use <cd>a:b<cd> vector syntax in the trial list).

To use custom trial definition numbering/names, add a column with no headings anywhere before the first object in a table. You can also give it the heading n_def (optional). You can use table shortcuts and other options for it just like for property columns. There are three options for values in this column:

• Integer ≥ 0 (group number)‍

PsychBench numbers trial definitions that have the same group number n as n+1, n+2, n+3, … in the order they appear across tables. For example, you can use 100 in a set of trials to tell PsychBench to number them 101, 102, 103, … (Default = 0 → 1, 2, 3, …). Note you can use auto repeat on a group number, like any value.

• Vector of integers ≥ 0‍

Like group number above except here group number is the sum of the numbers in the vector. This can be useful if you want to organize trial definitions into groups in multiple dimensions (e.g. block + condition within block). Often it is convenient to encode the group dimensions in different decimal digits.

e.g.
<cd>[4000 200]<cd> → 4200 → trial definitions numbered 4201, 4202, 4203, ...
This might represent trial definitions in block 4 condition 2.

Note you can set the numbers in a vector in separate columns by using the optional n_def heading and adding indexes—e.g. use column headings n_def(1), n_def(2) if you want to set the numbers in a 1×2 vector n_def separately.

• String <cds>""<cds> or <cds>''<cds> (name)‍

PsychBench names the trial definition with the string. Useful for one-off special trials like intros, breaks, sync with scanner, etc. Then in the trial list you can mix numbers and strings using a cell array of vectors and strings, e.g.
<cd>{<cd><cds>"intro"<cds><cd> 1:8 <cd><cds>"break"<cds><cd> 1:8}<cd>

Set trial repetition and order in trial definitions

For unusual cases you can set trial repetition and order to run in directly in trial definitions instead of using a trial list. To do this, in each trial definition table add a column anywhere before the first object with n in a heading row (row 1 or 2). Then for each trial definition set a trial number to run as during the experiment (1, 2, 3, … in any order). If you want to repeat a trial definition, you can give it a vector of numbers. You can use table shortcuts and other options for an n column just like for property columns.

Ignore columns/sheets

You can tell PsychBench to ignore any column in a table by putting % in heading row 1 for it, before the heading if any. You can also tell it to ignore a whole sheet by putting % before its name. (Like commenting out in MATLAB.)

Randomization

There are several basic ways to randomize in PsychBench. We'll demonstrate with some examples. In most cases little functions in <><PsychBench folder><>/tools are helpful, e.g. randomNum, randomChoose, randomRoll, randomBalancePerms, randomOrder, etc.

Randomize order

One common need is to sample values from a discrete set a certain number of times each. Often the most natural way to do this is to make trial definitions for all the values or combinations of values, then apply randomization to the order of presentation:

Randomize values

Another common need is independent random sampling, i.e. just re-rolling the dice each time. For this you need to randomly generate the values themselves:

Counterbalance across subjects

To counterbalance randomization across subjects, you can generate the random orders and/or values for all experiment runs (subjects) in MATLAB beforehand, save them in a .mat file, and then load it into MATLAB and draw from it at each experiment run using references to external variables and maybe the ^ shortcut.

Loading and running an experiment

When you have your experiment made, save it to an Excel file (in Google Sheets: File → Download → Microsoft Excel), Then you can use the following commands in MATLAB to load and run it. As usual type help <><function name><> in MATLAB for usage of each.

Load an experiment – loadExperiment

Start with the command loadExperiment. This loads an Excel experiment file and evaluates all the MATLAB values/expressions in it to build the experiment in memory. Typically use loadExperiment once for each experiment run/subject so that any randomization in the spreadsheet is refreshed.

View an experiment – viewExperiment

Before running an experiment for the first time, it can be helpful to check it in table form in MATLAB. Call viewExperiment at the MATLAB command line to see the trials that will run. You can also call viewExperiment -d to sort them by trial definition.

Test an experiment using auto response – runExperiment -a

Another useful testing feature is running an experiment in auto response mode to simulate a subject. To do this call runExperiment -a. You can change auto response options in properties autoResponse, autoResponseLatency which all response handler elements have.

Run an experiment – runExperiment

Use the command runExperiment to run the experiment in memory. runExperiment outputs results in variables, and optionally saves them to .mat and .csv files.

Quit/Resume an experiment – Ctrl + Esc, saveExperiment, loadExperiment

You can press Ctrl + Esc to quit an experiment part way through.

If you quit an experiment part way through and then call runExperiment again, it asks if you want to resume the experiment at the trial you quit. Results output and time measurements will be as if the experiment was never quit.

You can only resume an experiment if it’s still in memory, i.e. up until the next loadExperiment, newExperiment (coding method), clearExperiment, clear all, or quitting MATLAB. However, most commonly you’ll want to resume in a later session. To do this, use saveExperiment to save the state of the experiment to a .mat file (not a .xlsx file), then loadExperiment to load it back into memory later.

Other tools – showElements, recordElements, etc.

A few other tools can be used at the MATLAB command line for various tasks. For example, showElements shows/runs element objects without needing to make an experiment, recordElements captures element objects to image or movie files, etc. Also don't forget pb, pb_prefs, pb_update, etc. These are all in the root PsychBench folder—type help PsychBench for an overview.