How Vocaloid Works: A Beginner’s Guide to the Science Behind Yamaha’s Singing Voice Synthesis Software

How Vocaloid Works explores the technical aspects of Vocaloid, the world’s most widely known and commercially successful singing voice synthesis software. Unlike previous Anglophone academic works on the subject, this book provides readers with a deeper understanding of the technological foundations of Yamaha’s concatenative synthesis approach as well as its updated AI synthesis engine. Vocaloid’s ground-breaking ability to generate a range of both life-like and distinctly artificial timbres continues to demonstrate its lasting impact. However, with the advent of machine learning models capable of both music generation and audio deepfakes, the future of Vocaloid’s position at the forefront of the industry is uncertain. The book examines into how Vocaloid can create human(oid) vocal sound, covering the fundamentals of acoustic voice production, auditory perception, analog audio, digital signal processing and deep neural networks in order to account for the effectiveness and appeal of synthesized singing voices.

Gretchen Jude is Assistant Professor in the Department of Film & Media Arts at the University of Utah. Dr. Jude is a composer and performing artist concerned with presence and embodiment in computer music, language and cultural difference in performance, electroacoustic improvisation, and intermedia artistic collaboration.