Knee Rehab and Activity Monitor with Bluetooth

My project is a knee brace that uses sensors to track the user’s movements and alerts them of incorrect form when doing exercise that could impact their knee. More specifically, when the user is doing squats, the brace should alert the user if their knee bends inwards through a buzzer and through data sent to their computer. This knee brace specifically monitors squats by using an accelerometer and flex sensor that measures the angle of bend, acceleration, and angular velocity of the knee. The brace will send data to the user’s laptop through Bluetooth for the user to view information about their knee.

Engineer	School	Area of Interest	Grade
Shefali G	Lynbrook High School	Electrical Engineering	Rising Junior

Headstone Image

Demo Night Presentation

Final Milestone (Modifications)

My addition to the working knee brace is a web dashboard. This dashboard uses the Adafruit IO website that automatically formats data that it receives from the Arduino based on the widgets I added. The data is sent using the MQTT protocol.

The first step of this modification was to find a way to create a TCP/IP connection. Since the connection between the HC-05 module and laptop is a serial connection, there needed to be a TCP/IP connection created over the serial connection. Luckily, I found a library for the Arduino that would create the connection. This was the Arduino TCP over Serial Client library https://github.com/RoanBrand/ArduinoSerialToTCPBridgeClient.

This library requires a gateway protocol to be run at the same time as the serial connection, so I downloaded it and changed the serial port to the port of my Bluetooth module https://github.com/RoanBrand/SerialToTCPBridgeProtocol.

Next, I configured my Bluetooth module to be able to work with the connection. Using AT mode (Attention mode), I was able to change the settings of the module. I set the default baud rate of the module to 115200 bits/second and reversed the polarity of the module. Once these steps were completed, I uploaded the library example code to the Arduino and modified it to connect to the adafruit.io server. After this, the Arduino uploaded basic test data to an Adafruit feed while the gateway protocol was running on my computer (through the terminal).

Now, I needed to publish my accelerometer data and status to the feed. I combined my accelerometer code and publishing code in my first attempt at this. However, I ran into two main problems. First, the methods of the MQTT publishing library only took char pointers and integers as arguments to publish. To accommodate this, I converted my float integer values into strings, and then into the char array. The other problem was that there were conflicting libraries in the accelerometer library and the Serial-to-TCP library (HardwareSerial vs. NeoHWSerial). This was hard to identify, but once I did, I commented out the instances of the conflicting library in the source code of the accelerometer library. These fixes allowed my code to successfully compile!

To display feed data in a dashboard, I used the adafruit.io website. I sent data to separate feeds and created a dashboard that formatted each based on widgets that I added. There is one issue with the way that the server works that reduces the precision of my program. The Adafruit IO website only allows 30 data points to be sent per minute for a free account. Since I have three different values to send (Y acceleration, Z acceleration, and current status), I can only publish every six seconds. That means my code only reads data every six seconds. Because of this, I lost the accuracy from when it was outputted to the serial monitor/plotter, and it is difficult to differentiate between “knee bending inwards” and “bend backward, not just down”.

My next steps will be to find a way to publish my data in a way that keeps the code running quickly, as well as continue making my knee brace more reliable and consistent.

Dashboard

Headstone Image

Third Milestone

The final part of my project was to program the Arduino to distinguish between good and bad squat forms. My first step was to learn what mistakes I should be looking for while squatting. After searching on the internet, I decided to focus on detecting when the user’s knees were bending inwards and when the user squatted down instead of backward. These increase force on the knee, which is unsafe.

Next, I formatted my code to print out accelerometer values in a way that would be easy to convert into a spreadsheet- brackets around every six values and commas in between each value (x acceleration, y acceleration, z acceleration, x rotational velocity, y rotational velocity, z rotational velocity). Once printed onto the serial monitor, I found a Python program that would automatically format the data into a spreadsheet (.xlsx file) in a way that would be easy to graph. This step required me to learn how to work with Python in the terminal to install libraries and use proper syntaxes for Python code. The version of syntax for the program depends on the version of the Python interpreter (the program that executes the code line-by-line).

Once able to graph the values using both a spreadsheet and through the Arduino IDE Serial Plotter, I looked for ways that the values varied based on different forms. I performed squats with correct form, with knees bending inwards, and bending down without bending back. After graphing the data, I was able to notice the patterns for each form.

After this, I added code to my Arduino program to print out a statement for each state. Since the Arduino IDE uses C++, I needed to learn more about the language in order to do this. So, I learned how to create functions, arrays, and for-loops, as well as syntaxes specific to C++. For example, arguments in C++ are passed by the value inside the variable. To change the value of the variable, I had to pass by reference (the location in memory) by adding an ‘&’ after the variable’s data type. I mostly used the Y acceleration value to distinguish the forms because it varied the most and was clearly different for each motion.

After this, my knee brace printed “good form”, “knee bending inwards”, “bend backward, not down”, or “no activity detected” to the serial monitor. When the bad form is detected, the buzzer beeps.

The challenges that I faced during this milestone were using the Bluetooth module, calibrating the sensor, and attempting to use a Serial-to-TCP library to publish data to a web server. Although my Bluetooth module worked well in the first part of my project, it began having connectivity issues. I attempted to fix the issues by consulting online forums, trying a new Arduino board and Bluetooth module, and changing the layout of my circuit. Eventually, it began working again when I gave it time to reconnect upon initially disconnecting. I was also challenged during identifying patterns in my accelerometer data. The patterns were not clear at first, and it required many tests for the distinctions between forms to become obvious to me. Lastly, the Serial-to-TCP connection was completely new to me, and I struggled to find the correct order of steps. It required me to run a gateway protocol on my computer, then connect the Arduino through a serial connection to be able to publish data with HTTP or MQTT. However, this only worked when the Arduino was connected to the computer with a serial cable, and not over Bluetooth serial connection.

My main takeaway from Bluestamp is that even if I don’t know something at first, I can learn it along the way. I learned to learn on my own. Previously, I was often reluctant to start something new because I believed I wouldn’t know how to go about it. Now, I realize that even if I don’t know now, I can definitely learn. The internet is full of resources, and my experience at Bluestamp has taught me how to find them. I felt the most pride when I solved a difficult problem with the resources available to me. After Bluestamp, I want to keep building devices and circuits with new components and concepts and keep learning along the way. I’ve become so excited to see what I have the ability to build!

Milestone 3 Code

//Reference: https://lastminuteengineers.com/mpu6050-accel-gyro-arduino-tutorial/
//works better with 115200 baud but also works with 9600 
//for bluetooth module
#include <Adafruit_MPU6050.h>
#include <Adafruit_Sensor.h>
#include <Wire.h>
const int buzzer = 9; //buzzer to arduino pin 9

double accelX;
double accelY;
double accelZ;
double rotateX;
double rotateY;
double rotateZ;
double temp;

float accelYRecord [10];
float accelZRecord [10];

String currentStatus = "";

Adafruit_MPU6050 mpu;

void setup(void) {
  Serial.begin(9600);
  pinMode(buzzer, OUTPUT); // Set buzzer - pin 9 as an output

  // Try to initialize!
  if (!mpu.begin()) {
    //Serial.println("Failed to find MPU6050 chip");
    while (1) {
      delay(10);
    }
  }

  // set accelerometer range to +-8G
  mpu.setAccelerometerRange(MPU6050_RANGE_8_G);

  // set gyro range to +- 500 deg/s
  mpu.setGyroRange(MPU6050_RANGE_500_DEG);

  // set filter bandwidth to 21 Hz
  mpu.setFilterBandwidth(MPU6050_BAND_21_HZ);

  delay(100);
}

void evalForm (float arrY[], float arrZ[], String& stat){
  sensors_event_t a, g, temp;
  mpu.getEvent(&a, &g, &temp);
  for (int i = 0; i < 10; i++) {
        // Append sensor value to the array
        arrY[i] = a.acceleration.y;
        arrZ[i] = a.acceleration.z;
        delay(500);
  }
  
  float maxY = arrY[0];
  float maxZ = arrZ[0];

  for (int i = 0; i < 10; i++) {
        // Append sensor value to the array
        if (arrY[i]>maxY){
          maxY = arrY[i];
        }
        if (arrY[i]>maxY){
          maxY = arrY[i];
        }
  }

 if(maxY>2.5){
  stat = "Good form!";
 }
 else if(maxY<1.5){
  stat = "Knee bending inwards";
 }
 else if(maxY<2.5){
  stat = "Bend backwards, not just down";
 }
}

void loop() {
  /* Get new sensor events with the readings */
  sensors_event_t a, g, temp;
  mpu.getEvent(&a, &g, &temp);

  if((a.acceleration.z>0.0) && (a.acceleration.y>0.0)){
    evalForm(accelYRecord, accelZRecord, currentStatus);
    Serial.println(currentStatus);
    if((currentStatus == "Knee bending inwards")||(currentStatus == "Bend backwards, not just down")){
      tone(buzzer, 1000); // Send 1KHz sound signal...
    }
    else if (currentStatus == "Good form!"){
      noTone(buzzer);
    }
    Serial.println("");
  }
  else{
    Serial.println("No activity detected");
    Serial.println("");
    noTone(buzzer);     // Stop sound...
    delay(500);
  }
  delay(500);
}

Graph of Accelerometer values

Headstone Image

Second Milestone

In this milestone, I connected my accelerometer to the Arduino, set up the Bluetooth module, and attached all the parts to the knee sleeve.

First, I wired the MPU-6050 accelerometer and gyroscope with the Arduino. The VCC pins and GND pins are connected to 5V and ground, respectively, and the SCL and SDA pins of the accelerometer are connected to the SCL and SDA pins of the Arduino. I found code online that allowed me to print out all the six values that the module measures. These values are acceleration in the X, Y, and Z axes and rotational velocity along the X, Y, and Z axes. I struggled with interpreting the values, but I was able to understand the significance of each by seeing how they changed when the module was moved. This will allow me to determine proper and improper knee movements in the next milestone.

My next step was to connect the Bluetooth module to the Arduino and the laptop to allow wireless viewing of sensor data. The VCC and GND pins are connected to 5V and ground, the TX pin of the Bluetooth module is connected to the RX pin (pin 0) of the Arduino, and the RX pin is connected to the TX pin (pin 1) of the Arduino. No code is necessarily needed to use the module. After connecting the Arduino to power, the module should appear in the list of Bluetooth devices for a Mac laptop. I entered the password “1234”, selected the port of the module in the Arduino application, and was able to view messages in the serial monitor and plotter without physically attaching the USB! At first, I thought that an Android phone was required for connection, but a laptop will do the same job. Online instructions utilize the Bluetooth module in different ways, so it required research to find the instructions suitable for my purposes. I considered using an ESP8266 WiFi module, but it required extra components and was much more complex to use than the Bluetooth module, so I decided I would come back to it during modifications.

After connecting the accelerometer and Bluetooth module, I soldered the components onto a perf board and sewed all the components onto the knee sleeve. I faced some difficulty during soldering because the circuit layout has to be adjusted for the perf board. There are no connections between rows like a breadboard, so I connected the components and connected by row using solder and pieces of wire. Then, I attached all the parts to the knee sleeve: the Arduino, perf board, accelerometer (separately), flex sensor (separately), Bluetooth module, and a pocket to hold a power source.

Now that all the components are attached to the knee sleeve, I can calibrate the flex sensor and accelerometer with actual values from the knee bending. Additionally, I will continue working on a dashboard.

Milestone 2 Schematic

Headstone Image

First Milestone

My first step was to create my bend sensor by attaching a piece of velostat (pressure-resistance material) in between two pieces of neoprene (fabric) sewn with conductive thread to create a pressure sensor. The more pressure was applied to the sensor, the lower the resistance became. However, the range of resistance changed often, so calibrating the resistance to match the bend angle was too time-consuming. Because of this, I changed the sensor to a flex sensor where its resistance increases as it bends.

To measure the resistance of the bend sensor, I created a circuit with a static resistor and the flex sensor, which is a variable resistor. By measuring the voltage between a known resistance and the unknown resistance, I was able to find the resistance of the flex sensor. Next, I translated the amount of resistance to the angle of bend of the resistor. I recorded how the resistance varied when bending, and I was able to determine how much change in the resistance correlated with the bend of the sensor. This step was relatively simple due to how the resistance remained stable and consistent when compared to the sensor that I previously made. My next step was to add a buzzer to go off when the bend of the resistor went past a certain angle. After doing some research, it seems like the bend of the knee should not go past 90, so I set my buzzer to alert the user of that. However, I am planning to do more research on the proper form of the knee to adjust my program.

I also was able to send information to the Arduino from a smartphone using a Bluetooth module, but I am planning to expand on this with my next milestone. My next steps will be connecting the Arduino with a smartphone or computer through a Bluetooth module to send information and learning how to integrate an accelerometer into the circuit. I am planning to try to calibrate my self-made bend sensor again at the end of the project. My main challenges for this milestone were calibrating the bend sensor and creating the pressure (bend) sensor.

Starter Project

For my starter project, I built the TV-B-Gone remote that has the ability to turn most TVs on and off. This happens through the wavelengths emitted by the four infrared LEDs on the front of the remote. Their wavelength is 940nm, and their varying range and output allow them to match the wavelengths needed to turn off most TVs. The button on the remote resets the microcontroller, meaning that all the code stored in the microcontroller is run when the button is pressed. To control the timings of the release of wavelengths from the IR LEDs, an 8.0MHz resonator is used. This is important because a specific sequence of pulses sends the “on” or “off” signal to the TV. Transistors are used to amplify the current supplied by the microcontroller to the amount needed by the LEDs. This remote is powered by two AA batteries that provide three volts together. The challenges that I faced while building this project were being unfamiliar with the tools needed, such as a soldering iron, a desoldering pump, and a tool needed to remove insulation for the wire. However, this project allowed me to become comfortable using these tools, and my remote now successfully turns on and off my home TV!

Bill of Materials

Part	Note	Price	Link
Arduino Uno Rev3	Microcontroller that uses sensor data to make decisions	$27.60	Link
HC-05 Bluetooth Module	Wireless communication with computer	$10.39	Link
MPU-6050 Accelerometer and Gyroscope	Measuring specific movement of the knee	$3.33	Link
Flex Sensor(4.5”)	Measuring bend of the knee	$17.95	Link
Knee Compression Sleeve	Holding all the components on the knee	$7.99	Link
5000 mAh Power Bank (10cm x 3 cm)	Powering components	$17.99	Link
Neoprene fabric (2” x 6”)	Securing flex sensor onto knee sleeve	$8.49	Link
Sewing Kit	Sewing all the components onto the knee sleeve	$6.99	Link
Jumper Wires	Wiring together electrical components	$2.33	Link
USB cable for Arduino	Connect power bank and Arduino	$7.00	Link
PCB board	Soldering on some components securely	$3.43	Link
Piezo Buzzer	Alerting user of improper form	$1.33	Link
10K Ohm Resistor	Helps to measure flex sensor resistance	$0.55	Link
100 Ohm Resistor	Regulating voltage to buzzer	$0.55	Link