Brain-computer interface based on high frequency steady-state visual evoked potentials: A feasibility study Ulrich Hoffmann

Eric J. Fimbel

Thierry Keller

virtual car over a racing track [3], a system for controlling an electrical prosthesis [4], and a system for environment control [5]. A review of the state-of-the-art in SSVEP-based BCIs can be found in [6]. Compared to other types of BCIs, SSVEP-based BCIs have the advantage that relatively high information transfer rates can be achieved, that only little time is necessary to calibrate the system for new users, and that a small number of electrodes is sufficient for achieving good performance [6]. An unsolved problem is however the visual stimulation that is used in stateof-the-art systems: typically checkerboards or light-emitting diodes (LEDs) blinking with frequencies between 5 Hz and 30 Hz are used. Flicker in this frequency range evokes SSVEPs with a large amplitude but is annoying and tiring for the user. Another, even more worrying problem is that flicker in this frequency range can evoke epileptic seizures [7]. A possible solution to these problems is to use high frequency flicker. Depending on the exact frequency that is used, high frequency flicker is barely visible or even completely imperceptible and thus does not cause the problems of low frequency flicker. However, it is well known that the amplitude of SSVEPs strongly decreases at high frequencies [8], and so building a BCI using high frequency flicker is a challenge. To date, attempts to use high frequency flicker in a BCI are reported in only two publications. In [9] an experiment using LEDs flickering with frequencies in the range 20 – 45 Hz is described. An algorithm based on independent component analysis (ICA) is used for selecting an optimal electrode pair, and it is shown that the recorded signals can be correctly classified for many subjects, even for stimulus frequencies above 40 Hz. In [10] LEDs flickering with frequencies in the range 40 – 49 Hz are used. The main focus of [10] is on the comparison of two methods for estimating the amplitude of SSVEPs, namely traditional Fourier analysis and a method called phase-rectified signal averaging (PRSA). Results obtained from two subjects and from a fixed bipolar electrode pair show that PRSA outperforms Fourier analysis and that reliable SSVEP detection is possible only in a narrow frequency band around 45 Hz. The present paper presents an experiment in which subjects watched two checkerboards flickering at different frequencies on a cathode ray tube (CRT) monitor. The checkerboards flickered at frequencies between 15.45 Hz and 85 Hz and thus allowed us to explore the frequency range above 50 Hz. During the experiment subjects were asked if they perceived

un co

rre

Abstract—Brain-computer interfaces (BCIs) based on steadystate visual evoked potentials (SSVEPs) are systems in which virtual or physical objects are tagged with flicker of different frequencies. When a user focuses on one of the objects its flicker frequency becomes visible in the electroencephalogram (EEG) and so the object on which the user focuses can be determined from brain activity alone. A significant problem inherent to such systems is that typically flicker with frequencies in the range 5 – 30 Hz is used. Flicker in this frequency range is known to elicit easily detectable SSVEPs but is very tiring and annoying for users and can possibly trigger epileptic seizures. In this paper we study the feasibility of using higher frequencies for which the perceived flicker is less intensive. We compare the classification accuracy that can be achieved for stimuli flickering with low frequencies (15 – 20 Hz), medium frequencies (30 – 45 Hz), and high frequencies (50 – 85 Hz). The classification of the data is done with a Bayesian algorithm that learns classification rules and selects optimal electrode pairs. The results show that the medium frequency range can be used to build a high-performance BCI for which the flicker is hardly visible. We also found that for some subjects even high frequency flicker evokes reliably detectable SSVEPs.

cte d

Fatronik - Tecnalia , Biorobotics Department Paseo Mikeletegi 7, 20009 Donostia - San Sebastian, Spain

I. I NTRODUCTION

dra

ft -

Steady-state visual evoked potentials (SSVEPs) are oscillations in the EEG that are generated in the visual cortex when a subject views a periodically flickering stimulus. An interesting characteristic of these oscillations is that their amplitude can be modulated by visual attention. Subjects can increase the amplitude of the SSVEPs by concentrating on the stimulus or decrease the amplitude by ignoring it. A typical example for a SSVEP-based BCI is the system described in [1]. In this system two virtual buttons are displayed on the left and right side of a monitor. The luminance of the virtual buttons is modulated with different frequencies (17.56 Hz and 23.42 Hz) to produce SSVEPs and users can select one of the two buttons simply by looking at it. If during a certain time interval the amplitude of the EEG oscillations at one of the stimulus frequencies is significantly higher than the amplitude at the nearby frequencies, the system concludes that the user is probably looking at the corresponding button. Several variations of the system described in [1] can be found in the literature. For example in [2], a system is described that allows to control a character in an immersive gaming environment. The system allows to steer the gaming character to the left or to the right by focusing on one out of two checkerboards flickering at 17 Hz and 20 Hz. Other examples of SSVEP-based BCIs are a system for steering a

II. M ATERIALS AND M ETHODS A. Stimuli

Fig. 1. Screenshot of the display used in the experiment. The areas in the lower left and right corner of the display served to measure temporal precision of the stimuli and flickered at the same frequency as the checkerboards. These areas were covered by foam pads which held two control photodiodes in place.

rre

Two black and white checkerboards displayed on the left and right side of a 19 inch CRT monitor were used as stimuli (see Fig. 1). The checkerboards were viewed from a distance of 55 cm and consisted of 16 × 16 fields, each field having a size of 8×8 millimeters. The viewing angle subtended by each checkerboard was 6.64 degrees horizontally and 6.64 degrees vertically. The viewing angle subtended by single fields in the checkerboards was 0.42 degrees horizontally and 0.42 degrees vertically. The pixels in the white fields of the checkerboards were set to an RGB value of (255,255,255), which resulted in a luminance of about 125 cd/m2 , and the pixels in the black fields were set to an RGB value of (10,10,10), which resulted in a luminance of about 13 cd/m2 . Small fixation crosses appeared in the center of each checkerboard. The checkerboards were surrounded by a uniform background set to an RGB value of (10,10,10). The ambient illuminance in the field of view of the users was approximately 100 lux. To generate flicker, the checkerboards were only shown once every n frames, i.e. the possible flicker frequencies were divisors of the refresh rate of the monitor (170 Hz). Stimuli with frequencies of 15.45 Hz (n = 11), 17 Hz (n = 10), 34 Hz (n = 5), 42.5 Hz (n = 4), 56.6 Hz (n = 3), and 85 Hz (n = 2) were used. The stimuli were split into three groups, a low frequency group (15.45 Hz and 17 Hz), a medium frequency group (34 Hz and 42.5 Hz), and a high frequency group (56.6 Hz and 85 Hz). In each group the stimulus with the lower frequency was displayed on the left side of the screen while the stimulus with the higher frequency was displayed on the right. The temporal precision of stimuli was verified by means of two photodiodes placed at the bottom of the monitor.

cte d

the stimuli as flickering or static, which allowed us to get an estimate of the flicker fusion threshold1 . and to relate this estimate to the classification results. The data recorded during the experiment was classified by a Bayesian algorithm, which was also used to select optimal electrode pairs. The layout of the rest of the paper is as follows. In Section II the experimental setup and the methods used for recording the data are described. Then, in Section III the algorithms used for data anlysis are described. Section IV contains a description of the results and conclusions are drawn in Section V.

C. Procedure

dra

ft -

un co

After installation of the electrodes participants sat down in front of the monitor with the head placed on a chin rest. The chin rest allowed controlling the viewing conditions and minimized head and neck movements and the attendant electromyogram (EMG) artifacts. The experiment consisted of 3 × 20 trials in which the subject either watched stimuli from the low frequency group, from the medium frequency group, or from the high frequency group. In half of the trials the subject was instructed to concentrate on the checkerboard on the right side of the screen and in the other half the subject was instructed to concentrate on the checkerboard on the left side of the screen. The order of trials was randomized and each trial was structured as follows: 1) The subject was instructed by an on-screen message and by the experimenter to concentrate on the left or on the right checkerboard. 2) When the subject was ready the trial started and two flickering checkerboards were displayed during 20 seconds. 3) After each trial the subject was asked if the target checkerboard was perceived as flickering or static.

B. EEG Recording

The EEG was recorded with a g.Tec g.USBamp amplifier using a sampling rate of 512 Hz. The ground electrode was placed at position Cz and the reference electrode at position Fz. Recording electrodes were placed at positions Fp2, P7, P3, Pz, P4, P8, PO5, POz, PO6, O1, O2, and Oz. Electrode impedance was kept below 5 kOhm in all recordings. 1 The flicker fusion threshold is a concept from psychophysics and is defined as the frequency at which a flickering image begins to appear as steady.

D. Subjects Data was recorded from four male subjects with mean age 30.75 ± 3.9 years. All subjects had normal or corrected-tonormal vision. III. DATA A NALYSIS To test the feasibility of using high frequency flicker in an SSVEP-based BCI, we analyzed to what extent the recorded EEG allowed to discriminate concentration on the left checkerboard from concentration on the right checkerboard. This analysis was performed separately for each of the frequency

A. Preprocessing

iopt = arg max p(Y |X, Mi ).

rre

1) Bipolar Lead Expansion: To prepare for the selection of an optimal electrode pair, the 12 channel data recorded in the experiment was expanded to 78 channel data containing signals from the 12 original electrodes and 66 bipolar leads. The signals for each of the bipolar leads were computed by subtracting the raw signals from the two electrodes constituting the lead. 2) Filtering: The data were filtered with Chebyshev type I bandpass filters with a bandwidth of 2 Hz and a stopband attenuation of 60 dB. Each EEG segment was filtered with filters centered at frequencies fl , fr , fl − 3, fl + 3, fr − 3, and fr + 3, where fl and fr denote the flicker frequencies of the left and right checkerboards. The output of the six filters was used to estimate the amplitude of the SSVEPs (cf. Section III-A4). 3) Single Trial Extraction: Single trials of length 0.5 s, 1 s, 2 s, 4 s, or 8 s were extracted from the filtered data. To avoid effects caused by the onset of the stimulation pattern and to take into account the delay introduced by the filtering, only the last 16 s of each 20 s long segment were used for single trial extraction. 4) SSVEP Amplitude Estimation: To estimate the amplitude ai (f ) of the SSVEPs in channel i at frequency f , the following equation was used2 :

(BLDA) algorithm described in [12] was used. Given a training data set containing D-dimensional feature vectors X = {x1 , x2 , . . . , xN }, xi ∈ RD and class labels Y = {y1 , y2 , . . . , yN } , yi ∈ {−1, 1}, this algorithm computes classification rules which allow to classify new feature vectors not belonging to the training set. In addition, the BLDA algorithm can be used to perform Bayesian model selection, i.e. BLDA can be used to compute the probability p(Y |X, M ) of the set of class labels Y , given the set of feature vectors X and a model M . Here, model selection is used to choose an electrode pair which yields good classification performance. To this end, the probability p(Y |X, Mi ) for different models Mi is computed, where each model corresponds to classification based on only the amplitude estimates ai (fl ) and ai (fr ) from channel i. An optimal channel iopt can then be selected with the help of the following equation:

cte d

groups, i.e for the EEG recorded during stimulation with low, medium, and high frequency flicker. To analyze the data, the EEG recordings were preprocessed and then classifiers were trained and tested in a crossvalidation procedure in order to estimate classification accuracy. Details about the methods used for preprocessing, classification, and crossvalidation are given in the following subsections.

C. Crossvalidation

The learning of classifiers and the selection of an optimal electrode pair was embedded in a ten-fold crossvalidation. To this end, first the data from each frequency group was split into ten mutually exclusive subsets. Then, in each fold of the crossvalidation, a classifier was learnt and an electrode pair was selected using only nine of the ten subsets of data. The classifier was then tested on the subset of data that was not used for training.

un co

ai (f ) =

2σi (f ) . σi (f − 3) + σi (f + 3)

(1)

ft -

Here, σi (f ) denotes the standard deviation of the signal from channel i filtered at frequency f . The frequency was set to the stimulus frequencies fl and fr and hence two amplitude estimates were obtained for each channel and each trial.

dra

B. Classification and Lead Selection The electrode configuration used in a BCI determines the suitability of the system for daily use. Clearly, systems that use only few electrodes take less time for setup and are more user friendly than systems with many electrodes. In addition, analyzing signals recorded from only a few electrodes is faster and easier than analyzing signals recorded from many electrodes. Given these considerations we have developed a method that allows to automatically select an optimal electrode pair for the classification of SSVEPs. To select an optimal electrode pair and to perform classification, the Bayesian Linear Discriminant Analysis 2 Note that a similar method has been proposed in [11] and that the estimates of SSVEP amplitude obtained with this method are immune to broadband power variations as for example caused by EMG contamination.

(2)

i

IV. R ESULTS

A. Classification Accuracy The average classification accuracy, computed with the crossvalidation procedure for each subject, for different trial lengths, and for different stimulus frequencies, is shown in Table I. One can see that the accuracy for medium frequencies is slightly lower than that for the low frequency stimuli. However, an increase of trial length leads to accuracies close to 100% for both frequency groups and for all subjects. The classification accuracy for the high frequency stimuli is significantly lower than that for the other two frequency groups. In addition, the differences between subjects are bigger for the high frequency stimuli. This is especially evident when comparing the performance of subjects S2 and S4, and S1 and S3. While for subjects S1 and S3, classification accuracy increases over 70% with increasing trial length, it remains close to 50% for subjects S2 and S4. B. Perception of Flicker During the experiments the subjects were asked after each run if the checkerboard they concentrated on appeared to be flickering or if it appeared as a steady image. For the checkerboards flickering with frequencies of 15.45 Hz, 17 Hz, and 34 Hz all subjects perceived the flickering in 100% of the runs. The checkerboard flickering with a frequency of 42.5 Hz was perceived to be flickering in only 57.5% of the runs

Low frequency S1 S2 S3 S4 Avg.

Medium frequency

High frequency

0.5

1

2

4

8

0.5

1

2

4

8

0.5

1

2

4

8

85 92 97 71 86

88 96 98 78 90

94 96 100 85 94

100 100 100 94 98

100 100 100 90 98

75 75 85 71 77

81 81 90 76 82

83 86 95 82 86

93 89 99 90 93

93 93 100 98 96

55 55 67 50 57

56 52 72 55 59

63 54 77 51 61

75 53 83 55 66

73 60 90 52 69

(averaged over subjects). The checkerboards flickering with frequencies of 56 Hz and 85 Hz were always perceived as steady, i.e. the subjects perceived the flicker in 0% of the runs. V. C ONCLUSION

R EFERENCES

[1] M. Middendorf, G. McMillan, G. Calhoun, and K. S. Jones, “Brain computer interfaces based on the steady-state visual-evoked response,” IEEE Transactions on Rehabilitation Engineering, vol. 8, pp. 211–214, 2000. [2] E. C. Lalor, S. P. Kelly, C. Finucane, R. Burke, R. Smith, R. B. Reill, and G. McDarby, “Steady-state VEP-based brain-computer interface control in an immersive 3D gaming environment,” EURASIP Journal on Applied Signal Processing, vol. 19, pp. 3156–3164, 2005. [3] P. Martinez, H. Bakardjian, and A. Cichocki, “Fully online multicommand brain-computer interface with visual neurofeedback using SSVEP paradigm,” Computational Intelligence and Neuroscience, vol. 2007, 2007. [4] G. M. Putz and G. Pfurtscheller, “Control of an electrical prosthesis with an SSVEP-based BCI,” IEEE Transactions on Biomedical Engineering, vol. 55, pp. 361–364, 2008. [5] X. Gao, D. Xu, M. Cheng, and S. Gao, “A BCI-based environmental controller for the motion-disabled,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 11, pp. 137–140, 2003. [6] Y. Wang, X. Gao, B. Hong, C. Jia, and S. Gao, “Brain-computer interfaces based on visual evoked potentials: Feasibility of practical system designs,” IEEE Engineering in Biology and Medicine Magazine, vol. 27, pp. 64–71, 2008. [7] R. S. Fisher, G. Harding, G. Erba, G. L. Barkley, and A. Wilkins, “Photic- and pattern-induced seizures: A review for the epilepsy foundation of america working group,” Epilepsia, vol. 46, pp. 1426–1441, 2005. [8] M. A. Pastor, J. Artieda, J. Arbizu, M. Valencia, and J. C. Masdeu, “Human cerebral activation during steady-state visual-evoked responses,” The Journal of Neuroscience, vol. 23, p. 1162111627, 2003. [9] Y. Wang, R. Wang, and X. Gao, “Brain-computer interface based on the high-frequency steady-state visual evoked potential,” in Proceedings of International Conference on Neural Interface and Control, 2005. [10] G. G. Molina, “Detection of high-frequency steady state visual evoked potentials using phase rectified reconstruction,” in Proceedings EUSIPCO, 2008. [11] T. Meigen and M. Bach, “On the statistical significance of electrophysiological steady-state responses.” Documenta Ophthalmologica, vol. 98, pp. 207–232, 1999. [12] U. Hoffmann, J.-M. Vesin, T. Ebrahimi, and K. Diserens, “An efficient P300-based brain-computer interface for disabled subjects,” Journal of Neuroscience Methods, vol. 167, pp. 115–125, 2008. [13] C. S. Herrmann, “Human EEG responses to 1–100 Hz flicker: resonance phenomena in visual cortex and their potential correlation to cognitive phenomena,” Experimental Brain Research, vol. 137, pp. 346–353, 2001. [14] P. E. Williams, F. Mechler, J. Gordon, R. Shapley, and M. J. Hawken, “Entrainment to video displays in primary visual cortex of macaque and humans,” The Journal of Neuroscience, vol. 24, pp. 8278–8288, 2004.

dra

ft -

un co

rre

The results described in the last section show that when designing an SSVEP-based BCI there exists a trade-off between the perceived intensity of flicker and the classification accuracy. In our experiments only the stimuli with frequencies above 50 Hz were completely imperceptible for all subjects but unfortunately for these stimuli an acceptable classification accuracy was obtained for only two out of the four subjects. A much better classification accuracy was achieved for the stimuli flickering at frequencies of 34 Hz and 42.5 Hz. This is in accordance with the results described in [9] and indicates that the frequency range around 40 Hz is a good choice for building an SSVEP-based BCI with only barely visible flicker and good classification accuracy. In general, the literature about experiments with high frequency SSVEPs [8], [13], [14] and also the data from our experiments seem to indicate that stimuli which are far beyond the threshold of perception elicit only very little or no brain activity. Stimuli, however, which are only slightly beyond the threshold of perception might still elicit some brain activity and can potentially be used to build a BCI. Therefore a good strategy for future research in the area of SSVEP-based BCIs might be to adapt stimuli to users, such that the stimuli are just slightly beyond their individual threshold of perception. Together with classification methods that can make full use of multichannel signals, such as for example the common spatial patterns method used in motor imagery BCIs, it might then be possible to build efficient BCIs in which the visual stimulation is much less annoying and tiring then in current systems. ACKNOWLEDGMENT

The authors would like to express their gratitude to the subjects for participating in the experiments.

AND DIFFERENT STIMULUS FREQUENCIES .

cte d

TABLE I M EAN CLASSIFICATION ACCURACY FOR ALL SUBJECTS (S1 - S4), DIFFERENT TRIAL LENGTHS (0.5 S - 8 S ),

Brain-computer interface based on high frequency steady-state visual ...

visual evoked potentials: A feasibility study ... state visual evoked potentials (SSVEPs) are systems in which virtual or physical ... The classification of the data is.

362KB Sizes 0 Downloads 108 Views

Recommend Documents

high frequency privileges chart - QSL.net
SSB PHONE. NOV./TECH CW. CW. DIGITAL. PHONE. HIGH FREQUENCY. PRIVILEGES CHART. By Gordon West For Kenwood Communications. # 062200.

Impact of Delays on a Consensus-based Primary Frequency Control ...
for AC Systems Connected by a Multi-Terminal HVDC Grid. Jing Dai, Yannick ...... with the network topology,. i.e., each edge in the figure also represents a bi-.

high frequency privileges chart - QSL.net
80. METERS. Best Evenings. & Nights. 160. METERS. Night Owl. Band. 15. METERS. Best Days. 20. METERS ... CW & Data. CW. DIGITAL. CW BEACONS.

Steerable local frequency based multispectral ...
fusion is addressed using the phase information of the source image pixels at different orientations. We make .... More improved multiresolution transform tech-.

Bootstrapping high-frequency jump tests
Dec 19, 2016 - of a standardized version of intraday returns (such as in Lee and Mykland (2008, 2012)). In addition, tests ... show that although truncation is not needed for the bootstrap jump test to control the asymptotic size ... version. Our res

A High-Frequency Decimal Multiplier
represented in binary, these applications often store data in decimal format and process data using decimal arithmetic software [1]. Although decimal arithmetic.

Bootstrapping high-frequency jump tests
Jun 26, 2015 - In this paper, we consider bootstrap jump tests based on functions of .... Section 6 gives simulations while Section 7 provides an empirical application. ..... improves finite sample performance of the bootstrap jump tests.