Bio

From the ornate city of Lahore, Nabeel, a rising senior at PakTurk Maarif foundation, is the most wanted person in his school and town when it comes to tech. He is often found in his PJs out on a random adventure or just going for a walk. Up in his lab, he is found building or opening machines, programming applications and games, or doing penetration testing. In the future, he hopes to get his Bachelor's in Computer Engineering and pursue his dream of creating a new computer processing architecture. He is the founder and the active President of the Al-Jazari Robotics and Engineering Society (JRES), the hottest society on campus. Along with that, he is found teaching his fellow school students Python and C++ (CUDA) or helping them with their projects on weekends.

Project: Parse What Is Happening in a Videogame from Images

Goal

The goal of my project was to annotate Pac-Man video game footage after the game is played. The details of what is going on in the game at a specific moment are parsed and spoken out loud. The steps included making logs of the activities in the game footage of the score, the status of the sprites, the difficulty of the level a player is currently on, the positions of the Pac-Man and enemies, and the position of the closest enemy relative to the Pac-Man.

Summary of Results

At this time, my project can analyze all the frames of the video provided and can narrate the status of the game. Since I had to make a computer understand what's going on in the captured footage, I first thought of using external parameters passing methods to get data from the game itself, but it was not a practical idea since many old classic games were already compiled while the newer ones wouldn't allow the release of the parameters due to security reasons. I decided to apply computer vision on the footage and developed my LegacyTracker function, which tracked a defined object with a set color. It could release either the rendered image or the coordinates of the item tracked with respect to the image dimensions. After that, I calculated the vector angles of the ghosts in relation to Pac-Man and used vector distance to see if the ghost could be a potential threat to the player sprite. In parallel to that, I also made a small masking function that allowed us to read the text contents of the image without confusing anything else. Lastly, I used the Speak function to narrate any possible threat to the main sprite.

Future Work

As for my future work, I would like to shift my way toward accessibility computing and would take this project to a more-advanced level where it performs in real time on multiple games. This program can make the game accessible to visually-impaired people by speaking what is happening. Possible future developments include real-time parallel processing and music control as the difficulty of the game increases, implementation of machine learning on speech synthesis and 8D audio previews of everything going on in the game to simulate the whole environment of the game for the player.