Categories
Grant Stories

Team Vibe Vision and Team Spider Sense Connecting Technology and Societal Good

Building a good team is often cited as the key to success in a variety of spheres — from business to the arts, from sports to science. For Mason Mechanical Engineering professor Jeffrey Moran, however, the ability to hand-pick a team for the second phase of the 4-VA Collaborative Research project “Toward T-Shaped Graduates: A Joint Capstone Program at the Nexus of Mechanical Engineering and Science and Technology Policy”, was completely out of his hands. Fortunately, though, the eight students who signed up for the Capstone project last summer — indicating an interest in building on the continuing project — turned out to be Moran’s Dream Team.

As luck would have it for Moran, when surveys went out to rising mechanical engineering seniors outlining opportunities for capstone projects last summer, the ‘just right’ eight students chose the option of working on the vehicle alert system.

The project, which tackles barriers to operating a vehicle for the hard of hearing and deaf communities by building assistive technologies to alert drivers and passengers to sounds around the vehicle, was launched by Moran’s capstone students in collaboration with a counterpart team at James Madison University, in the 2020-21 academic year. Yet, in part because of pandemic-related restrictions that kept the teams from working together, the tools and technologies that were developed on the project needed some advancements to improve the final product. Refinements and enhancements were necessary.

The reasons the eight students selected the project were as diverse as their backgrounds. A sampling:

  • Javeria Jawad, who acted as one of the two team leads, was drawn to the hands-on aspect of the task, explained, “I liked the practical implications of the project, it wasn’t just theory, it actually built something.”
  • Wadeed Fakhoury felt that the project fulfilled two of his interests, “I like to work on cars, and I love to help people. This opportunity to benefit the hard of hearing operating a vehicle was just what I wanted.”
  • Michael Mullins enjoys developing electronic games and has a computer science background. He saw the project as an opportunity to add to his coding skills. In fact, he led the coding efforts on the project, making crucial improvements to the machine learning-based sound detection platform developed by last year’s team.
  • Faiza Al-Bahrani had perhaps the most compelling interest in joining the mechanical engineering team with its sights set on helping hard of hearing communities. Al-Bahrani has a cochlear implant, without which she is clinically deaf. Removing her implant during work sessions, Al-Bahrani served as the team’s resident Chief of Quality Assurance.

Moving Forward.

This second-year team met Moran’s hopes and expectations for the phase two of the project, combining their skills and interests to build on the existing project.

That initial effort began in 2019 when Moran reached out to 4-VA@Mason with an interest in creating an interdisciplinary capstone program in collaboration with colleagues in the School of Integrated Sciences at James Madison University. The faculty members’ backgrounds ranged from mechanical engineering to political science, Moran wanted to tackle problems that do not fall neatly into one disciplinary category, including the development of renewable energy technologies, autonomous vehicles (often called self-driving cars), the use of robotics in medicine, and more. Moran sought to task students with the goal of addressing public needs; undertaking problems that straddle boundaries between disciplines. “The overarching goal is to create T-shaped graduates who have a general level of knowledge about a broad span of subjects, forming the horizontal part of the T, while cultivating deep knowledge in their own specific area, representing the vertical part,” Moran says.

The first cohort of students, working in the pandemic-affected 2020-21 capstone year, sought to focus on making automobiles easier to use by deaf and hard of hearing populations, with an eye toward self-driving cars, which are expected to number in the hundreds of millions by 2030 and will be used by individuals with varying needs. Armed with data indicating the challenges that deaf and hard of hearing populations face when driving, the students set about outfitting a golf cart with a microphone to detect noises near the vehicle, using machine learning to identify the sound, and creating a seat cushion outfitted with a haptic sensor which vibrates to let the driver know that a hazard is nearby. The driver is then prompted to read a tablet screen mounted on the dashboard that identifies the noise.

Although the 2020 students got a good start, Moran knew there could be more to the project. Much of the in-person work was not possible because of the COVID-19 pandemic, which limited lab work. However, with funding still left in the budget, Moran opted to create a phase two of the effort and asked the next group of students to take up the project.

Take it up, they did. As Hoa “Andy” Huynh, another member of the team leads exclaimed, “We improved on it in every way!”

When the two team leads, Huynh and Jawad, got the project last September, they discussed a plan of activity and started preparing schedules. Jawad explains that the teams did theoretical work over Zoom meetings in the fall. Jawad’s team was tasked with input — working on microphone processing sounds to the laptop, while Huynh’s team focused on output — from the battery to the haptic feedback system. Huynh notes that the two teams initially worked separately on their efforts from September to December of 2021, and then they worked as a group every Tuesday and Thursday in a lab on the Sci Tech campus from January through May 2022. Huynh adds, “Professor Moran attended the Tuesday meetings to check in on our progress, give us feedback, and help us with questions.”

Hamzeh Amin plays an audio siren to prompt the system to identify the sound which is displayed on the laptop.

The teams did all the work in-house except for the original code which was written at James Madison University. Mullen accessed that code and built on it. He then went on to convert Spectron graph images to recognize sounds – a dog bark, gun shot, car horn, and siren — through machine learning.

Indeed, the new teams examined every element of the system and made improvements. Huynh says, “First, their system wasn’t integrated, there was only one microphone, the existing Raspberry Pi was not strong enough, we upgraded to a Windows laptop which is much more powerful.” Huynh adds that while the original cart had only one microphone, the new team installed four. “Based on the feedback from the deaf community surveys, we understood that it was important to indicate which direction the sound was coming from,” he says. “Michael was able to develop a system that identifies the sounds in 2.5 seconds and then it appears on the laptop monitor and also indicates the direction from which the sound is emanating.” Jawad adds that the seat cushion was also expanded to include four vibrating haptics which reveals to the driver the direction of the sound.

Mounted laptop identifying sound and the direction of the sound.

With the newly enhanced golf cart, the 2021-22 group was ready to share the results on the lawn in front of the Nguyen Engineering Building on the Mason Fairfax campus on May 5 for Capstone Day. Nathan M. Kathir, Associate Professor & Director of Senior Projects, says it was an opportunity to, “See their creativity in-person.”

Cars of the Future.

Moran reflects on the progress made by the second phase team and indicates that, with funding still left in the budget, he’d like to return to another collaboration with Integrated Science and Technology group at JMU next year. “Tremendous possibilities remain with this project; we can take this to the next level by making the sound detection system even faster, training it to recognize a wider array of sounds, or filtering the input noise to pick out the hazard from a noisy background,” says Moran. “We’re also interested in making the intensity of the haptic feedback depend on the distance between the sound source and the vehicle.”

Because it is expected that the coding requirements for the project will be expanded, Moran anticipates adding at least one Computer Science major in the team to take ownership of the increasing demands. Moran also hopes to continue to address the policy-related issues associated with the use of conventional and autonomous vehicles by deaf and hard of hearing communities, explaining, “In keeping with the original vision of this project, I would also like to see next year’s students look at the policy implications of this work, particularly the updates that are needed to the Americans with Disabilities Act of 1990 to enable individuals with various needs to use autonomous vehicles more effectively, since AVs are only going to become more numerous on our roads.”

Kyung Min (left) reviews the project with attendees at the Capstone Day project presentations.

“After the Capstone Day event, we had several people say to us, ‘I want this for my car’ – even people who have full use of both ears!” Moran adds. “We’re grateful for 4-VA’s continued support and flexibility as we’ve steered this project through two — and soon to be three — academic years, not to mention a global pandemic. We’re excited to see where it goes next.”

Team Vibe Vision: Hoa “Andy” Huynh (team lead), Michael Mullins, Wadeed Fakhoury, Faiza Al-Bahrani
Team Spider Sense: Javeria Jawad (team lead), Jimmy Torrico, Hamzeh Amin, Kyung Min

The Dream Team (Featured photo) Left to right: Wadeed Fakhoury, Kyung Min, Jimmy Torrico, Professor Moran (in cart), Hamzeh Amin, Faiza Al-Bahrani, Javeria Jawad, Hoa “Andy” Huynh, Michael Mullins (kneeling).