Towards a passive brain-computer interface (BCI) using hybrid EEG-fNIRS measurements: From experiments under lab conditions to the classification of sparse events in real-life applications


Neural networks have recently gained interest in the context of predictive maintenance as they can deal with massive amounts of sensor data without the need to provide a priori knowledge. However, supervised learning methods require labelled data streams for training. We aim for a passive BCI capable of tagging time points at which individuals detect machine failures in assembly lines, to provide a simple method for labelling sensor data.

Currently, most BCIs are based on electroencephalography (EEG) signals. However, EEG has a poor spatial resolution and is highly sensitive regarding interferences such as subject motion and insufficient electrode attachment et cetera. To enhance the robustness and performance of BCIs, a second measurement modality – functional near-infrared spectroscopy (fNIRS) – was successfully implemented [1-3]. With this optical technique, the neural activity is indirectly accessed via measuring the local relative concentration changes in oxygenated and deoxygenated hemoglobin.

So far, EEG and fNIRS data is processed for most applications with classical signal processing and machine learning techniques using hand-crafted features. Since both measurement modalities are affected by various sources of noise, we aim for bypassing long research cycles that are required to extract reliable and robust features by using convolutional neural networks as classifiers [4-6].

Aim & Research Methods

Your task is to…

  • design and perform classification / error detection experiments to create test and training datasets
  • evaluate different classification strategies, e.g. neural networks and classical techniques for our specific application
  • implement a real-time classification prototype
  • investigate the influence of interferences such as motion, mental workload, physical stress et cetera on the performance

Preferred knowledge

  • programming skills in Matlab and Python
  • dedication and motivation to work on an interdisciplinary research topic self-reliantly

Possible starting date & further information

Potential starting date is as soon as possible. For further details and application contact Lennart Weiß in person or via email.


  1. Liu Y et al. Towards a Hybrid P300-Based BCI Using Simultaneous fNIR and EEG. Foundations of Augmented Cognition: 7th International Conference, AC 2013, HCI International 2013, Las Vegas, NV, USA, July 21-26, 2013. Proceedings (pp.335-344)
  2. Khan MJ and Hong KS, Hybrid EEG-fNIRS-based eight-command decoding for BCI: Application to quadrocopter control. Frontiers in Neurorobotics 2017, 11, 6
  3. Fazli S et al. Enhanced performance by a hybdrif NIRS-EEG brain computer interface, NeuroImage 2012, 59, 519-529
  4. Trakoolwilaiwan T. et al. Convolutional neural network for high-accuracy functional near-infrared spectroscopy in a brain-computer-interface: three-class classification of rest, right-, and left-hand motor execution, Neurophoton. 2017, 5, 1
  5. Lawhern VJ et al. EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces. J. Neural Eng. 2018; 1 5: 05601 3
  6. Schirrmeister RT et al. Deep learning with convolutional neural networks for EEG decoding and visualization. Hum. Brain Mapp. 2017; 38: 5391 -5420