SIPL Projects

In this project we developed an automated data preparation system to improve Torah reading recognition with cantillation (te'amim). The main challenge was to adapt existing datasets, which typically consist of long recordings, to the requirements of training models like Whisper, which necessitate short audio segments (under 30 seconds). The developed system includes: VAD-based audio segmentation, initial transcription using Whisper, and a unique iterative text alignment mechanism bridging the gap between modern transcription and biblical text. Additionally, tools were created for automated data collection from YouTube, utilizing an LLM for Parasha/Aliyah identification.

Melody extraction is one task from a variety of problems in the field of musical retrieval (MIR), that are designed to develop algorithms and techniques for extracting meaningful information from musical content. This work deals with the problem of extracting the melody from an audio segment, using machine learning techniques. This problem refers to the process of isolating and identifying the melody, the main musical line from a musical piece.   The input to our system is an audio file containing a musical composition, and our approach involves employing a combination of signal processing and machine learning algorithms.

In this work, we created a model based on deep neural networks to classify music genres. During the process, we segmented each song into song excerpts and fed them into the model for training, validation, and testing. Throughout the project, we utilized the classification of songs with a single genre from the MTG-Jamendo song database, which involved working with a database of songs divided into genres in an unbalanced manner (the number of songs from each genre varies significantly). Therefore, we chose to work only with the ten largest genres and used different weighting schemes in hopes of improving the results.

This work aims to create a bass accompaniment track for a solo vocal track. This is achieved using a machine learning model trained on a comprehensive dataset of bass and vocal tracks that sound good together. The system first processes the vocal track and converts it into a spectrogram, a graphical representation of the signal's frequency spectrum over time. A generative diffusion model is then used for producing a corresponding bass track that aligns with the input vocal spectrogram as a conditioning input to the model. Throughout the project, an extensive literature review was conducted to select appropriate models, including mel-gan and HIFI gan.

The goal of the work is exploring & finding smart encoding of music notes, that captures meaningful harmonic relationships in music. The assumption that music is similar to language in the sense that there are also certain grammatical rules, of course with some flexibility in relation to normal language. Thus, related notes & chords will be close to each other in sense of cosine similarity. During the project, we learned tools for storing and representing music files, such as MIDI and others, through which we extracted the database for the project.

The work deals with converting a sound segment of the humming into a sound segment of a selected musical instrument. This conversion relies on established and proven models developed on the basis of deep network algorithms in combination with digital signal processing methods and techniques. The two main models we worked with are DDSP, developed by google, and the other model, that was inspired by it was developed by Dr. Lior Wolf from Tel Aviv University. Later we will expand and detail these models and how they work.

Classifying musical keys is one of music theorys most ancient problems today. In the modern world a real need arises from home composers for a system that can accurately and reliably identify an audio samples musical key, mainly for its incorporation in whole, complex pieces. This problem gets even more complicated when we want to classify the keys of short audio samples, in which unlike full songs, their keys arent as clear. In this work we implemented methods to solve the musical keys classifying problem, while focusing on the audio pre-processing and researching advanced methods for classification using neural networks and improving their results using Semi-Supervised learning.

...

...

...

...

...