Synchronizing Arduino with Brain Recordings

Experiment
I'm running a motor learning experiment with human subjects. Subject watches the computer screen, and when a visual cue appears, subject rapidly types a previously-memorized sequence on a keyboard. The subject's responses cause the computer program (matlab) to proceed to the next visual cue. This repeats for 30 minutes. While the subject is performing the experiment, we record their brain activity. Sensor data from sensors connected to the arduino are synchronized with this brain activity, which is recorded by a separate device.

Arduino Board: Uno Rev3, but buying a different one is arbitrary if necessary
Computer: Windows laptop with 16 to 32 gb ram and latest processors

Sensors I'm using, connected to arduino:
(1) a custom made 4-key cherry mx keyboard plugged directly into arduino - detect keypress actuation time
(2) four force sensitive resistors, adhered to top of keys on custom keyboard - detect whether subject is touching key even if they are not pressing
(3) a simple light sensor - detect when visual cue appears on screen

Actuator, connected to arduino:
(1) vibration motor?

Other sensors, connected to a separate neural recording device:
(1) electrodes that record brain activity
(2) accelerometer integrated into the neural recording device

Requirements/Notes
Neural recording setup is already all taken care of - just explained above for the sake of people understanding the bigger picture goals of the arduino project. Let us focus on the arduino data. I need all the arduino sensor data recorded as rapidly as possible, and I would like to know the time in microseconds that each value is recorded. All data needs to be written to my laptop in real time, including the microseconds time stamps so that I can save it all as a datafile at the end of the experiment. It's fine if the data from the keyboard and the light sensor only write through the serial connection when a threshold is reached (so just when a key is pressed or the light being sensed goes dark). But, the FSR data need to be continuously written from arduino over USB to laptop because we need fully raw (not thresholded) data from the FSRs.

The ideal precision is single or double digit microseconds, and it needs to be able to do this for at least 30 minutes. The ideal datafile would be a text document where each line says Sensor Number x, Value y, Time z

We are aligning the arduino data to with the recorded brain data, using the brain data as the master clock data. We'll have the vibration motor of the arduino vibrate the neural recording device (which has a built-in accelerometer, so vibration shows up in neural data) at the start and end of the experiment so that we can scale the arduino sensor time series data to the neural recording time series data. Thus, it is also important that we know the exact times that the motor is vibrating, since this is how we will align our two device clocks.

Also, I know matlab is slow, which is why we are only using it to present visual cues, not relying on it for timing information about those visual cues or the keypresses.

Here are my questions. Any insight is greatly appreciated here!
(1) What is the best way to get and record the most accurate timing information for each sensor reading? Would it be fine to call micros() and write the returned value to serial communication along with every single sensor reading, or would that induce a delay that cumulatively worsens time estimates or significantly slows sensor sampling rates? Would it be better to only call micros() at the start of each loop through the sensors? Or some other approach all together?

(2) Is there some considerable time or resources requirement that would motivate me to only record when light sensor and keypress surpass certain values as opposed to recording their values at every loop?

(3) Do tiny rotating motors actuate rapidly, on the scale of microseconds? If we are using a vibration from the arduino to the neural recording device to align the two clocks, I am wondering if I should also use an accelerometer or vibration sensor on the arduino to sense when the vibration actually begins. It wouldn't be identical to the one on the neural recording device, but it might be better than nothing.

(4) Is there an off-the-shelf actuator that either generates electrical stimulation or vibration that would actuate more rapidly than one of those tiny coin-sized vibrating motors? (something that can be picked up by an electrical recording device that has a built-in accelerometer)

(5) I'm aware that people using keyboards with arduino usually use an arduino with ATmega32u4 because that can have keyboard input directly interface with the computer like a real keyboard. However, if I were to use an arduino with ATmega32u4 so that the keypresses could be read like a keyboard in matlab, would I still be able to simultaneously stream all of the sensor data (including keypress information) into a file? If not, would it be better to stick with arduino uno and have matlab constantly reading the serial log information looking for keystrokes? Yes there would be more lag in matlab in this latter case, but lag in matlab is relatively fine whereas lag in keypress or other sensor data is not okay.

(6) If I make a custom, 4-key keyboard, would it be better (in terms of lag and time stamps for all sensors) to have the keys directly plugged into the arduino, which would perform debouncing etc., or to have a little chip dedicated to the keyboard that would do the debouncing and key scanning and then would be plugged in as a single input to the arduino?

I know this is a lot, so I tried to keep it organized. If possible, please address the question you're responding to so that it is easier for me to understand your comments. Thank you so much in advance to anyone who can provide help!

There IS a lot there and its possible some issues might be better addressed in a different section of the forum.

subject rapidly types a previously-memorized sequence on a keyboard.

What I would like to clarify is this.

You wish to synchronize a mechanical input (ie typing) with a brain electrical activity recording.

A typical typing speed is around 60wpm or 1 word/sec say 6 char / sec.

Brain electrical activity recordings are normally band limited below 50Hz
(I used to be an electroencephalographer)

The average human reaction time to a visual stimulus is around 200msec.

What do you expect to gain by recording synch times in microseconds?

Thanks for your response! I'm trying to synchronize the mechanical keyboard input, as well as the light sensor data, FSR data and motor output with the brain data.

In the reaction time experiments, difference in reaction time on the order of 10 milliseconds is considered to be quite a large difference, and smaller differences still matter to us.

Our neural recordings are up to 1000Hz

We need to be able to account for the full range of human visual processing speeds.

How long it takes someone to react or how long it takes someone to process a stimulus is largely irrelevant, since we are interested in all time points, including before they have time to process the cue or respond, i.e., we are interested in all the processes occurring within those first milliseconds.

What I see as the really relevant thing here is that our neural recordings only go up to 1000Hz, but even so, since they are being aligned with the sensor data, the lowest lag possible in the sensor data wrt ground truth would be ideal, as I can always downsample the sensor data during post processing. We could sample every 1 millisecond on the arduino, but I am worried about cumulative delays. Unless there is strong reason to believe that doing time grabs in micros() is inferior to millis() wrt clock drift issues or lag or something else I'm not thinking of (please do tell me if so- I am new to arduino), I don't see why we would use millis over micros

Use an interrupt to note the time in millis() or micros() of when the visual stimulus appears and again when they react. Then it doesn't matter how long it takes for the data to be sent because you have the time stamp and plenty of synch points.

I hope you are making good progress. I've been thinking about your vibration motors, on the timescale you are describing they will be very slow to respond.

If you want a tactile cue, then the fastest would be an electrical impulse. Or perhaps you could use a visual stimulus - a flash from an LED, or an auditory stimulus - eg a tone pitch change.

I'm assuming that will be your initial stimulus, from which all timing will be measured.

You COULD potentially sample every few microseconds with the arduino, it would depend on how complex your code was. Then save the data say to an SD card. I dont know how fast that is.

Serial communication would likely slow down your sampling rate, so you'ld want to transfer all the data when sampling was finished.

The uno is pretty primitive for this type of application, you may be better choosing a faster and more powerful device such as an ESP32 based system, that would also support a wifi data transfer.

In fact I'm beginning to wonder if eg a rasperry pi 4 might be a better option? (shh thats heresy on this forum)

This topic was automatically closed 120 days after the last reply. New replies are no longer allowed.