UCSD - University of California - San Diego

06/10/2026 | Press release | Distributed by Public on 06/10/2026 12:26

Beyond the Song Generator: How UC San Diego Students Are Rethinking AI and Music

Published Date

June 10, 2026

Article Content

At the University of California San Diego, researchers are asking what artificial intelligence can do for music - not just whether it can generate a song, but whether it can become a more responsive, controllable and creative partner for musicians.

"We are exploring AI as both a tool and collaborator," said Shlomo Dubnov, a professor in the UC San Diego Departments of Music and Computer Science and Engineering, a principal investigator of the Music Understanding Synthesis and AI Creativity group, and an affiliate of the UC San Diego Qualcomm Institute. "Our projects, which bring together students and faculty across departments, span experimental composition, live performance, music understanding and generative models designed with artists and listeners in mind."

For three graduating students - Tornike Karchkhadze, Mingyang Yao and Zachary Novack - that work has opened different pathways into the future of AI and music.

Building AI to Perform with Musicians

With a background in songwriting, pop, rock and commercial composition and production, Tornike Karchkhadze, a graduating Ph.D. student in the Department of Music, approaches AI from the perspective of a musician.

"My motivation was always to see AI models not from the user angle, but from a musician's angle," says Tornike Karchkhadze, a graduating Ph.D. student who will join Apple.

"My motivation was always to see AI models not from the user angle, but from a musician's angle," said Karchkhadze. "Normally, you write a prompt and get a song back - a fully composed, ready-to-use song. That's fine if you want to make a surprise for your friend on their birthday. But if you are a musician, you need more control. You have your own taste and your own vision of how your music must sound."

Karchkhadze's research has explored that approach. Over the past several years, he has worked on AI systems for accompaniment generation, multi-channel music generation and the interpretation of graphic notation. In one project with Dubnov and UC San Diego math Ph.D. student Keren Shao, Karchkhadze helped use AI to interpret Cornelius Cardew's Treatise, a landmark experimental score made of abstract shapes, lines and symbols rather than conventional musical notation. The team's composition and paper earned runner-up honors at the AI Music Competition held as part of the 2024 IEEE International Conference on Big Data in Washington, DC.

His most recent work moves even closer to live performance. Karchkhadze and colleagues developed a real-time human-AI co-performance system that allows a musician to plug in an instrument, play and receive AI-generated accompaniment in response.

The challenge, Karchkhadze said, is that in real-time musical collaboration the AI system must respond to what a musician is playing while also anticipating where the performance may go. Human musicians do this naturally, but creating an AI system with that kind of responsiveness is difficult. Karchkhadze does not claim to have solved the problem fully. But he sees the system he designed as a step toward a suite of AI tools that can participate in live musical practice, not just produce music offline.

The process also pushed him into new territory. Trained as a musician rather than a computer scientist, Karchkhadze taught himself to code and build AI systems through practice. That dual perspective now shapes his work: he understands both the creative needs of performers and the technical constraints of machine learning.

After graduation, Karchkhadze will join Apple's audio research group, where he will work on video-to-audio generation.

Teaching AI to Understand Musical Style

Mingyang Yao, who recently graduated from UC San Diego with a double major in mathematics-computer science and cognitive science with a specialization in neural computation and machine learning, has focused on how AI models learn musical structure and style.

Mingyang Yao, a UC San Diego alumnus who will attend University of Rochester's Ph.D. program, has discovered he enjoys immersing himself in the research process.

Large music-generation models often rely on massive datasets. But when the goal is to generate music in the style of a specific composer, the available data may be more limited. A researcher may have only a few hundred examples from a composer, genre or musical tradition. Mentored by UC San Diego alumnus Ke Chen, now a research scientist at Adobe Research, and advised by Dubnov and UC San Diego Jacobs School of Engineering Associate Professor Taylor Berg-Kirkpatrick, Yao asked whether an AI model can first learn broad musical knowledge, then adapt that knowledge to a specific style using limited data.

In one project, Yao pre-trained a symbolic music model on a broad collection of classical, folk and popular music, then fine-tuned it on the work of composers such as Bach and Mozart. The goal was to imitate not only surface-level style, but deeper musical features such as melody, rhythm and harmony. The results, later presented at an AI music creativity conference in Belgium, showed strong style adaptation using fewer than 300 pieces from a given composer, comparing favorably with much larger models trained on more data.

Yao later turned to helping AI systems understand harmony in written music. Inspired by how human musicians annotate harmony, Yao created a model that makes decisions step by step, beginning with the features it is most confident about. It also detects chord boundaries, identifying where harmonic changes occur before deciding what the chords are. The approach, which was presented at the IEEE International Conference on Acoustics, Speech and Signal Processing, improved accuracy over previous symbolic chord-recognition methods.

Through this research, Yao has realized that he enjoys immersing himself in the process. "The best part is when I'm trying different ideas and exploring, especially when I know the rough directions and am continuing to push to fine-tune a solution," he said. "The few days or weeks before the final solution are the happiest moments for me."

Yao will next attend the University of Rochester as a Ph.D. student, where he plans to continue working on AI and music and hopes to build larger, more impactful projects.

Designing AI Music Tools Around Artists

Zachary Novack, a graduating Ph.D. student in computer science, has focused his doctoral work on how to make generative music models more useful, playful and responsive for artists.

"Nobody is going to beat the gigantic companies with their gigantic song generators, but who wants to?"" says Zachary Novack, a graduating Ph.D. student who will join Spotify.

"The idea of 'press a button and we'll generate the song' is boring to me," he said. "It's a toy, but it's not actually fun. Nobody is going to beat the gigantic companies with their gigantic song generators, but who wants to?"

Novack, advised by Berg-Kirkpatrick and UC San Diego Jacobs School of Engineering Professor Julian McAuley, has been active in helping grow the AI music community at UC San Diego. His research asks how generative music systems can move beyond one-shot song generation and become more like instruments - tools artists can play with, shape, challenge and even creatively misuse.

At the 2025 UC San Diego GenAI Summit, Novack argued that researchers should include musicians and artists throughout the development of AI systems. That philosophy runs through Novack's work. As co-creator of Presto, a model for accelerating music generation, he has explored how to make AI music systems faster and more interactive.

In one recent project, Novack worked on making open-source generative music models responsive enough for real-time interaction. The goal was to shrink and speed up models so they could run locally, respond to controls such as pitch and volume, and become part of a performance setup.

One experimental result was unusual but revealing: a model trained on whale sounds and used in a performance with cello. The system functioned almost like a generative delay effect, responding to the performer in ways that were strange, imperfect and creatively suggestive.

Those collaborations reinforced Novack's conviction that creative AI tools work best when artists are involved from the beginning - not as end users brought in at the finish, but as collaborators who shape what the system should do.

After graduation, Novack will join Spotify, where he expects to continue working on artist-first AI music research.

Recognition in Generative Music Competition

Another UC San Diego student, Anthony Wang, also recently earned recognition for AI music research with Dubnov. Wang, a first-year master's student in computer science, won second place in the efficiency track of the IEEE International Conference on Multimedia Expo's Academic Text-to-Music Generation Grand Challenge.

Mentored by Professor Shlomo Dubnov, master's student Anthony Wang (above) won second place in a text-to-music generation grand challenge.

For Wang, the project was his first experience building and training a generative model from scratch. It taught him how to set up a full machine-learning pipeline without starter code or a template and introduced him to the distinct challenges of audio generation.

The competition asked participants to build text-to-music models using a standardized dataset. Despite using far fewer parameters than some competing systems, Wang's model generated 10-second music samples that were favorably evaluated using both objective metrics and human listening tests.

Together, the projects point toward a broad vision for AI and music at UC San Diego: not replacing human creativity, but expanding the ways musicians can compose, perform, improvise and experiment.

Learn more about research and education at UC San Diego in: Artificial Intelligence

UCSD - University of California - San Diego published this content on June 10, 2026, and is solely responsible for the information contained herein. Distributed via Public Technologies (PUBT), unedited and unaltered, on June 10, 2026 at 18:26 UTC. If you believe the information included in the content is inaccurate or outdated and requires editing or removal, please contact us at [email protected]