Project DetailsThe goal of this project was to develop a full-song generation system that preserves a stable singer identity while improving melodic and performance-level control. To address this problem, two complementary research tracks were explored. The first track investigated an explicitly factorized conditioning scheme in which melody, lyrics, and vocal identity are represented through separate pathways. Its main contribution was conceptual and architectural: it clarified which control signals are required for controllable singing generation. The second track, which became the primary empirical path of the project, relied on a flow-matching song generation model augmented with lightweight parameter adaptation, an explicit MERT-based voice-conditioning bridge, and later a richer timing and performance-control pathway.
The empirical results support singer personalization in the evaluated setting. The later performance-oriented extension provided a substantially richer control platform, but under the available data regime it did not surpass the strongest timing-corrected identity model. Overall, the project suggests that stable singer identity emerges not from a single architectural addition, but from the joint alignment of data design, conditioning structure, optimization strategy, and evaluation protocol.
