While looking for architectures that decode electroencephalography without discarding either its timing or its spatial structure, I came across the EEG Conformer, proposed by Song et al. What follows is a short high level summary of what it does, and of where it still falls short.
Preliminaries:
- EEG, i.e., electroencephalography, records the electrical activity of the brain through electrodes placed on the scalp. The signal carries spatial information, meaning where the activity occurs, and temporal information, meaning how it changes over time.
- The amplitudes involved are very small, and the recording is contaminated by eye blinks, muscle tension and electrical noise, so a considerable part of the work happens before the model sees anything at all.
- Motor imagery, i.e., imagining a movement without performing it, is the task used in the standard benchmark, where a classifier has to decide which hand the participant was imagining moving.
Convolutional networks capture local structure in EEG well and struggle with dependencies that unfold over longer periods. Transformers are the opposite, and a growing number of researchers have begun applying them to EEG, which I covered in a post on attention models in EEG signal processing for the ai-driller blog we ran at HABS. The EEG Conformer places the two in sequence rather than choosing between them.
The architecture
The pipeline has three stages. A temporal convolution acts as a bank of forty detectors, each tuned to a pattern in the timing of the activity. A spatial convolution then asks what happens across the other channels when a given pattern appears in one of them. Finally the self-attention layers let every point in the signal weigh its relationship with every other point, which is what allows dependencies over longer stretches to be recovered, and a small classifier decides what the segment represents.
The division of labour is the interesting part. The convolutions compress the raw signal into something with fewer, more meaningful dimensions, and only then does attention operate, which keeps the quadratic cost of attention from being paid on the raw sampling rate.
Evaluation
The authors evaluate the model on the BCI Competition IV Dataset 2a, the standard benchmark for motor imagery classification.
| Method | Accuracy | What This Means |
|---|---|---|
| Traditional FBCSP | 69.3% | Decent, but room for improvement |
| EEGNet | 73.8% | Good CNN performance |
| ShallowConvNet | 75.7% | Previous state-of-the-art |
| EEG Conformer | 80.0% | New benchmark achieved |
Interpretability
The authors also propose Class Activation Topography, i.e., class activation mapping weights displayed on the EEG topography, so that the regions and intervals a decision rests on can be read off directly rather than inferred.
In practice
Preprocessing matters more here than the architecture does. Bandpass filtering, typically between 4 and 38 Hz, removes electrical noise and irrelevant frequencies; normalisation standardises amplitudes across channels; artifact removal clears eye blinks and muscle tension. The model is relatively forgiving, but the difference between a mediocre and a good result is usually made before training starts.
Limitations
- Performance still varies between individuals, even where it is consistent within one.
- The model is more expensive to run than a plain convolutional network, though not unreasonably so.
- It is sensitive to its hyperparameters and needs careful tuning.
- A model trained on one person does not transfer to another without further work.
The EEG Conformer is therefore a promising approach rather than a solved problem, achieving something neither convolutional networks nor transformers manage on their own. Several obstacles remain before it can be used in practice: acquiring enough labelled data, establishing robust preprocessing protocols, and adapting the model to settings where a participant sees a stimulus only a handful of times.
At HABS the difficulty we keep returning to is a different one. Experimental protocols need to be ecologically valid, i.e., close to the conditions people are actually in, while remaining controlled enough for the results to be comparable with those reported on ideal laboratory datasets. Those two requirements pull in opposite directions, and most of the published accuracy figures sit at the controlled end of that range.
Resources
- Original paper: IEEE Transactions on Neural Systems and Rehabilitation Engineering
- Code: GitHub repository
- Framework integration: available in Braindecode
- Datasets: BCI Competition IV, SEED, DEAP
If any of this is useful, take a look at the paper itself!
