Projects 2024

PosturePerfect

Real-time posture correction with AI and geometry

Introduction

With the recent rise in remote work and social media use, prolonged usage of mobile devices has led to an alarming increase in poor posture (fig. 1), exacerbating the risk of musculoskeletal discomfort. Research shows that a significant portion of individuals report discomfort in areas like the neck, lower back, and shoulders. According to the CDC, back pain is the most common form of pain among adults. It is important to have good posture to reduce the risk of musculoskeletal symptoms. My research aims to use artificial intelligence and mathematical models to help individuals rectify their posture habits. I use a state-of-the-art machine learning model called PoseNet to estimate a user’s pose from a camera image. I then use a mathematical model I created to detect a user’s bad posture efficiently and accurately from their pose. Putting these components together, the PosturePerfect application provides real-time feedback and guidance to users, enabling them to cultivate and sustain good posture habits and reduce the risk of discomfort.

Illustration comparing good sitting posture with slouching

Fig. 1: A graphic showing the difference between good and bad posture. Source: Health Direct Australia

Engineering Goal

The initial goal for this project was to create a robust and user-friendly desktop application that could perform accurate real-time detection and correction of a user’s posture using a laptop camera. By alerting users of bad posture, PosturePerfect aims to promote healthy habits and reduce the incidence of musculoskeletal discomfort among users working remotely or spending extended periods of time in front of a computer. To achieve this, I planned to use the PoseNet neural network to get key information about the user’s pose and create a formula that would allow me to accurately and quickly determine if a user is slouching from the pose information generated by PoseNet. Using these components, I planned to create a desktop application using Python and OpenCV that would detect the user’s posture in real time and provide feedback & guidance to help them correct it. I wanted to make sure that the application was accurate but also lightweight and wouldn’t slow the computer down much.

Creation of Formulas to Detect Slouching

The first formula looked at the angle of the line from the head to the shoulders compared to a horizontal line. If the angle was outside of a threshold, then the user was slouching. This worked for detecting sideways slouching, but it didn’t work well for detecting if the user was slouching forward or backward because in this case, the angle didn’t change even though the user was slouching. Also, the results were different for each camera position The second formula looked at the angle of the shoulders compared to a horizontal line. If this angle was above a certain threshold, the user was slouching. Like formula 1, this worked well for side-to-side slouching but not well for other types. It was also dependent on the camera position being straight in front of the user and well-oriented. The third formula I created used the ratio of the distance from the head to the shoulders to the distance between the shoulders themselves and compared it to a threshold value. This was able to detect forwards slouching well, but not side-to-side slouching. For the fourth formula, I put the previous two formulas together, looking at both the angle of the shoulders and the ratio of the head-to-shoulders distance to the distance between the shoulders. This was able to accurately detect forward, backward, and sideto-side slouching. However, it had the drawback of not working well with some positions of the camera. I fixed the previous issue in the fifth formula by taking a baseline image of the user to begin with. This allowed me to compare the angle and ratio values to the baseline image instead of using absolute thresholds. This corrected for the camera being in different positions and could accurately detect slouching.

Testing and Tweaking

To make sure the final formula was accurate for a wide range of test cases, I extensively tested it. First, I tested it in a straight camera position in front of me and did many trials where I slouched left, right, forward, and backward. Then, I moved the camera farther away from me and redid the tests. I repeated this with the camera closer to me, to my left, to my right, above me, and below me. Another idea I tested was if it could detect slouching while standing up. I stood up and slouched in various ways in front of the camera. I repeated this for various camera positions. Finally, I tested how accurate it was each frame by slouching for a long period of time. Using the information I got from doing these test repeatedly, I adjusted the threshold values in the formula until I achieved maximum accuracy. I then did one final test with the final formula to determine its accuracy and precision.

Results

Overall, the formula did really well when testing it on myself with varying camera angles. Here are some metrics I calculated from the trials:

  • Left slouching accuracy: 98%
  • Right slouching accuracy: 95%
  • Forward slouching accuracy: 97%
  • Backward slouching accuracy: 99%
  • Total accuracy: 97.25% As you can see, the formula performed best on backward slouching. However, all of the results are very close to each other, so this could just be due to random error. The overall accuracy is very high, meaning that in almost all cases of slouching the formula will detect it. The formula is also very consistent and it doesn’t waver.

Desktop App

I created a PosturePerfect desktop app using Python, NumPy, PyTorch, and OpenCV. I used Python to code the app, NumPy to perform the mathematical calculations necessary for the formula, PyTorch to run PoseNet, and OpenCV to get data from the camera. The app runs by first taking a baseline image of the user, then checking if the user is slouching using the formula each frame. If the user is slouching for more than a certain time limit, the app alerts them that they are slouching by beeping.It also displays a real-time preview of the user, the key points that PoseNet generates, and whether they are slouching or not. The application runs on the machine’s hardware, meaning that it can be run locally without needing an internet connection. Furthermore, the application is lightweight, meaning that it is fast and doesn’t take up battery life.

Conclusion

To achieve my engineering goal, I used a state-of-the-art computer vision model called PoseNet and the mathematical formula I created to accurately and reliable detect bad posture. Using this technique, I developed a desktop app in Python that anyone can use to help correct their posture, leading to decreased risk of musculoskeletal discomfort. I hope that my work can help lower the incidence of discomfort due to bad posture and foster overall spinal health in the digital era.

Future Work

In the future, I want to add more features to the desktop app. Firstly, I want to port it to a web application so that users can monitor their posture directly from the browser. I also want to make a smartphone app out of PosturePerfect. This would help users maintain good posture while using smartphones in addition to computers, which would further lessen the risk of musculoskeletal discomfort. Thirdly, I want to add a feature where you can view your history of posture. You would be able to see how your posture habits have improved over time and view insights on your progress.