Controlling Video Game Using Brainwaves (EEG)

Hey everyone, this project is made by Umang Bansal who is a 4rd year BTech Computer Science student from Vellore Institute of Technology, Bhopal.

The project focuses on the development of a Brain-Computer Interface (BCI) system leveraging electroencephalography (EEG) signals. But before moving forward let's get familiar with BCI and EEG.

What is electroencephalography (EEG)?

Electroencephalography (EEG) is a method to record an electrogram of the spontaneous electrical activity of the brain. It is typically non-invasive, with the EEG electrodes placed along the scalp (commonly called "scalp EEG") using the International 10–20 system.

What is Brain Computer Interface (BCI)?

A brain computer interface (BCI), sometimes called a brain machine interface (BMI), is a direct communication link between the brain's electrical activity and an external device, most commonly a computer or robotic limb.

About the project

In this project, Umang aims to explore the potential of BCI technology by designing a system that interprets EEG signals to control external devices in real time. You will get to see how he will be controlling a car racing game by classifying EEG signals into two states: attentive and relaxed.

Why he made this project?

He decided to dive into this project to learn more about Brain Computer Interface (BCI) technology. The main goal is very simple: to see what BCI systems can do and how they work. He was really curious about how he could use signals from the brain to control external devices like computers or gadgets. It was a chance to learn new things and be part of the exciting world of neuroscience and technology. He hoped to figure out how to make BCI even better and find new ways to use it in future.

HARDWARE:

1 x BioAmp EXG Pill (with JST PH 2.0 connector and a header pin)

1 x BioAmp Cable v3

3 x Gel Electrodes

3 x Jumper Cables

1 x Arduino Uno / Maker Uno with USB Cable (You can also use any other microcontroller board with an ADC)

1 x Nuprep Skin Preparation Gel

1 x Wet wipe

1 x Brain BioAmp Band (optional)

1 x Electrode Gel (only if using Brain BioAmp Band)

SOFTWARE:

• Arduino IDE
• Visual Studio Code
• Google Collab

Which kit to buy?

You can get any of the kit mentioned below from our online stores (Shipping Worldwide)

1. DIY Neuroscience Kit - Basic
2. DIY Neuroscience Kit - Pro

These DIY Neuroscience Kits includes everything you need to get started.

The BioAmp EXG Pill comes pre-soldered with DIY Neuroscience Kit Basic and Pro but in case you are getting BioAmp EXG Pill separately then you will have to assemble it for this project by soldering the header pins and JST PH 2.0 connector as shown in the diagram.

Apply Nuprep Skin Preparation Gel on the skin surface where electrodes would be placed to remove dead skin cells and clean the skin from dirt. After rubbing the skin surface thoroughly, clean it with a wet wipe or an alcohol swab.

About Nuprep Gel:

Nuprep skin preparation gel is a mildly abrasive, highly conductive gel that should be applied before placing the electrodes on the skin to improve measurements. When applied gently, it strips away the top layer of skin and moistens the underlying skin layer which reduces the skin impedance with minimal skin irritation and discomfort.

Connect the BioAmp Cable v3 to BioAmp EXG Pill as shown above.

Let's understand the electrode placements before moving forward in this project. For recording EEG from prefrontal cortex part of brain, you have to place the electrodes on your forehead, specifically on Fp1 and Fp2 (refer International 10-20 system for recording EEG)

What is International 10-20 system for recording EEG?

It is an internationally recognized method to describe and apply the location of electrodes in the context of an EEG exam or voluntary lab research. This method was developed to maintain standardized testing methods ensuring that a subject's study outcomes (clinical or research) could be compiled, reproduced, and effectively analyzed and compared using the scientific method.

Options to measure EEG

You have 2 options to measure the EEG signals, either using the gel electrodes or using a dry electrode-based EEG band. You can try both of them one by one.

Option 1 - Measuring EEG using Gel electrodes:

1. Connect the BioAmp Cable to gel electrodes,
2. Peel the plastic backing from the electrodes.
3. Place the IN+ and IN- cables on the forehead & REF (reference) on the bony part, on the back side of your earlobe as shown in the video above.

Option 2 - Measuring EEG using Brain BioAmp Band:

1. Connect the BioAmp Cable to the Brain BioAmp Band in such a way that IN+ and IN- are placed on the forehead.
2. In this case, the REF (reference) should be connected using a gel electrode. So connect the reference of BioAmp Cable to a gel electrode, peel off the plastic backing, and place it on the bony part, on the back side of your earlobe.
3. Now put a small drop of electrode gel on the dry electrodes (IN+ and IN-) between the skin and metallic part of BioAmp Cable to get the best results.

Connect BioAmp EXG Pill to Arduino Uno/Maker Uno using the jumper cables as directed below:

• VCC to 5V
• GND to GND
• OUT to A0

Note: BE VERY CAREFUL and follow the above diagram while making the connections between your BioAmp EXG Pill & Arduino Uno/Maker Uno, especially the GND and VCC else it may damage the sensor.

Download the zip file of the GitHub Repository mentioned below:

https://github.com/Umang-Bansal/BCI

Download the Arduino IDE v1.8.X (Legacy IDE) from the link given below:

https://www.arduino.cc/en/software

After downloading, connect the Arduino Uno/Maker Uno to your laptop using the USB Cable

Note: Make sure your laptop is not connected to a charger and sit 5m away from any AC appliances for best signal acquisition.

Now unzip the folder you downloaded from the GitHub Repository and open it.

Go to the "signal" folder (BCI-main<BCI-main<signal) where you will find an Arduino Sketch named signal.

Open the Arduino Sketch in Arduino IDE v1.8.X and upload it on your Arduino Uno/Maker Uno.

Download the following software using the link provided and install it in your laptop:

1. Visual Studio Code: https://code.visualstudio.com/Download
2. Python v3.11.X: https://www.python.org/downloads/

Open Visual Studio code, click on File<Open folder to open the folder you downloaded from the GitHub Repository.

Open the terminal, and ensure that the path is configured to the location of the requirement.txt file.

Example:

PS C:\Users\bhawna\Github\BCI-main\BCI-main>

To install all the modules that are required to run the Python script, write the command "pip install -r requirements.txt" in the terminal:

PS C:\Users\bhawna\Github\BCI-main\BCI-main> pip install -r requirements.txt

Open prediction.py and change the COM Port in the code (line 58) as per the COM Port you selected in Arduino IDE (for us it is COM9). Save the file by clicking CTRL + S.

Run the Python script prediction.py by writing the command "python prediction.py" in the terminal.

PS C:\Users\bhawna\Github\BCI-main\BCI-main> python prediction.py

Open the EVO F4 game by clicking the link below:

EVO F4: https://gamaverse.com/evo-f4-game/

The EEG signals recorded are classified into 2 states:

• Attentive
• Relaxed

So you will have to stay attentive to move the car and relax your mind in order to apply brakes.

Note: This may not work for you as the sample data provided to train the model is limited. Umang Bansal only took his EEG recordings to train the model. But you can collect more data and train the model accordingly using the steps given below.

This is the most crucial step so please make sure you are doing everything right and collecting good quality data.

Data collection process:

First, you should collect 20 minutes of EEG data when you are attentive and then 20 minutes of EEG data when you are relaxed.

To be attentive you can play video games which makes you focused and to be relaxed you can listen to soothing music which calms your mind.

To collect the data, follow the steps given below:

• In Visual Studio code, open collect.py and change the COM Port in the code (line 6) as per the COM Port you selected in Arduino IDE.
• Save the file by clicking CTRL + S.

Collecting data in attentive state:

• Start playing any video game that will help you focus like magic tiles, subway surfers, etc.
• Run the Python script collect.py by writing the command "python collect.py" in the terminal.

PS C:\Users\bhawna\Github\BCI-main\BCI-main> python collect.py

• It will start saving the data in a CSV file named "signal" which will be automatically created in the GitHub folder you downloaded earlier. After 5 minutes (300 seconds), the data recording will stop.
• Again you can write the same command to collect another 5 minutes of data in the same CSV file which was created earlier. Repeat this step 2 more times to get 20 minutes of EEG data when you are focused.
• You can check the CSV file to see how many data points are recorded. 512 data points are recorded in 1 second. So in total, there should be around 6 lakh data points recorded when you are in the attentive state.

Collecting data in relaxed state:

• Plug in the earphones and start playing soothing music that calms your mind.
• Again write the same command "python collect.py" in the terminal to collect 5 minutes of data when you are in relaxed state. Repeat this step for a total of 4 times to get 20 minutes of EEG data.
• By now there should be a around 12 lakhs data points recorded in the CSV file.

We're collecting data in 5-minute intervals because we believe that individuals can't remain in the same state for extended periods.

Some things to keep in mind:

1. If you're capturing data for person 1, the demo should be done using the same individual. It may not work if tested on someone else. To make this experiment work on anyone, you would need to record lots and lots of data to make the model more accurate.
2. If you're collecting data for the attentive state, collect the EEG data for that state consecutively before moving on to recording data for the relaxed state.

1. Click the link below to open the files on Google Colab:

https://colab.research.google.com/drive/10P_hVLXSnEZ7HvoyYIsneu71umhhQWke?usp=sharing

2. Wait for some seconds as the Google Colab will be connecting to the Python 3 Google Compute Engine backend.

3. Click on the "file" icon in the vertical bar on the left side and upload the CSV file (signal.csv) where you have collected the data. It may take some time depending on the amount of data you are uploading.

Some changes to be made in Google Colab:

1. In block 2, change sample.csv to the name of the file where you have collected the EEG data (signal.csv).
2. In block 3, you must change the values as per the amount of data collected. Label 1 denotes data points representing one specific state, while label 0 denotes data points representing another state. Therefore, first, assess the total number of data points in the CSV file, then allocate half for Label 0 and the remaining half for Label 1.

For example: If you are collecting 20 minutes of data when you are attentive and 20 minutes of data when you are relaxed, then

Data points collected in 1 sec = 512

Data points collected in 1200 sec (20 minutes) = 6,14,400

So we can add the values 6,14,400 on label 0 and 6,14,400 on label 1

4. Now go to the top menu, and click on runtime<run all.

5. After successful execution (it may take 3-4 minutes), you will get 2 files model.pkl and scalar.pkl (visible in the "files" section on the left)

6. Download these files and save them in your project folder.

7. Finally, the data is recorded and you have trained the model accordingly.

Now, run the Python script prediction.py again by writing the command "python prediction.py" in the terminal.

PS C:\Users\bhawna\Github\BCI-main\BCI-main> python prediction.py

You are now ready to play the game again just by thinking :)

Complete YouTube Video: https://www.youtube.com/watch?v=EWsdFdE0LCw

Let us know what you think about this tutorial in the comments section below. In case of any queries, you can reach out to us at support@upsidedownlabs.tech. We would love to help you out.