Project DetailsThis work explores the classification of plant-emitted ultrasonic sounds using machine learning, adapting speech-processing models to a new biological domain. The dataset, provided by Tel Aviv University, consists of 16,000 samples of 2ms audio recordings (sampled at 500 kHz) from plant species, including tomato, corn, cacti, wheat, lamium, tobacco, and grapevine. These recordings included conditions like “cut” and “dry,” and served as a basis for species and condition classification.
We aim to boost classification performance (evaluated using F1 and accuracy metrics) using advanced model architectures and self-supervised (SS) representations. We start with a baseline CNN trained from scratch, then compare it with speech-based models—HuBERT and Wav2Vec, with focus on Wav2Vec for its superior performance. Both supervised and self-supervised training schemes are evaluated, including experiments where models are pretrained on some species and fine-tuned on unseen classes.
Fine-tuned models, pretrained without their target classes, outperformed those trained from scratch for smaller classes, while larger classes showed similar performance but faster convergence. These results show that speech models can be adapted to classify plant ultrasonic emissions, providing a basis for future domain-specific models. Dataset imbalance and field novelty remain challenges, and expanding the dataset is key for future broader agricultural usage.
