Project DetailsThis project focuses on developing and implementing an adaptive beamforming algorithm for Meta-Aria smart glasses, aiming to enhance the audio quality of recorded conversations. The system was designed to identify and mute the wearer's voice and other background noises, while preserving the voice of the target speaker standing opposite them. System evaluation and performance assessment were conducted using multi-channel recordings of natural conversations from the CHiME-8 dataset.
Throughout the project, several audio processing methods were examined, ranging from basic beamforming algorithms (such as TDBF and FDBF) to the use of the MVDR algorithm, which enables directional noise attenuation without distorting the target signal. To overcome the limitations of theoretical models that do not account for room reverberation, a data-driven approach was applied. In this approach, the system "learns" the unique acoustic signature of the speaker directly from the recording, allowing for more natural, robust, and accurate voice tracking in a real-world environment.
Additionally, the system was adapted for continuous processing, enabling it to adapt to changes in real-time. As part of the transition to a fully autonomous system that does not require prior human annotation, an automated speaker classification mechanism based on machine learning (K-Means) was integrated. This mechanism distinguishes when each speaker is active based on physical characteristics, stemming directly from each speaker's distance from the glasses. The results on real data demonstrated success in drastically attenuating interference segments from the glasses wearer, alongside preserving the external speaker, creating a continuous processing chain that serves as a foundation for real-time processing on wearable hardware.
