Project DetailsIn this project, we developed a system for classifying musical instruments from audio recordings, which successfully met all its primary objectives. The main goal was to build an accurate CNN-based classification model that leverages the spatial dependencies inherent in audio data, while demonstrating the feasibility of using a novel transform called Joint Time-Frequency Scattering (JTFS) for feature extraction. This transform is designed to capture additional information that is typically lost in standard audio recognition methods. Based on a reference project implemented in MATLAB, it appeared that this additional information could enhance classification efficiency. As part of the development process, we performed a comparison against the reference project and achieved significant improvements in both accuracy and network efficiency, thanks to optimal data utilization and custom-built architectures. Furthermore, we demonstrated the feasibility of successfully using STFT for feature extraction—an approach that had been dismissed in the reference project. To evaluate performance, we compared our networks to well-known CNN and ViT architectures. The experimental results highlighted the necessity of complex deep learning architectures over classical classification methods, such as kNN combined with PCA, which were found to be less effective for this task and do not serve as a viable substitute for the CNN models we presented.
