Draft:SynthesizerV

• Comment: In addition to the comments above, this article fails to meet the encyclopaedic writing standards and structure required. A possibly related draft (Draft:Synthesizer V) also exhibits similar issues. QEnigma talk 19:17, 4 January 2025 (UTC)

---

Synthesizer V, also referred to as SynthV is a vocal synthesis engine developed by Dreamtonics. A Technical Preview was announced August 8, 2018 and distribution began on August 19, 2018. . The full Production Release officially released on December 24, 2018. Synthesizer V Studio was unveiled June 25, 2020 in a press release alongside voice databases Kotonoha Akane & Aoi and Saki. AI support for Synthesizer V AI was released for Synthesizer V Studio as an update on December 25 alongside an update for Saki known as Saki AI.

Synthesizer V

Original author(s)	Kanru Hua
Developer(s)	Dreamtonics
Initial release	August 19, 2018 (2018-08-19)
Stable release	Synthesizer V Studio 1.11.2 Update / September 12, 2024 (2024-09-12)
Operating system	Microsoft Windows macOS Linux
Available in	Japanese, English, Spanish, Chinese
Type	Voice synthesizer software
License	Proprietary
Website	dreamtonics.com/synthesizerv/

Voice databases that were not recorded with the AI method are known as "Standard voice databases", which are generally recorded at the Dreamtonics studio in Tokyo.

History

Kanru Hua stated on Twitter, the first line of code, the proto-proto libllsm (written in March 2015), eventually became part of Synthesizer V.

Development for Synthesizer V began in 2017. Kanru Hua released a demo using three vocals known tentatively at the time as ENG-F1, JA-F1 & MAN-M1. Synthesizer V made its first official debut on December 1, 2017.

In February 2018, Kanru Hua posted a listening test to recieve feedback on a new singing pitch model for Synthesizer V. In August 2018, Kanru Hua released a "Technical Preview" version of Synthesizer V

On August 20, 2018, Kanru Hua released a survey asking for user feedback on the Technical Preview to be used for future improvements.

On December 24, 2018, Dreamtonics released the Production Release edition of Synthesizer V for sale with substantial improvements over the Technical Preview edition.

In March 2019, Kanru Hua announced an application for licensed purchasers of Synthesizer V for early testing of the macOS edition. This version officially released on March 12, 2019 and all currently released voices were given macOS versions.

On May 31, 2019 Kanru Hua, with DREAMTONICS, announced in a tweet that he was accepting applications for C++ software engineers to work on the next iteration of Synthesizer V which later became known as Synthesizer V Release 2 or "SVR2" . This would later on in 2020 become known as Synthesizer V Studio.

On April 9, 2020 it was announced that the second generation of Synthesizer V would be released soon, and said a demo of Chiyu using the new engine would be coming soon. The demo was released on April 11. Synthesizer V Studio Pro and Synthesizer V Studio Basic were formally announced on June 26 by AH-Software in a press release, as well as the voice databases Kotonoha Akane & Aoi and Saki.

Animen's ANiCUTE store for international customers opened on July 12th and Synthesizer Studio Pro with voice databases Genbu & AiKO available for purchase on July 15. Special discounts were made available for VIP members and purchasers of the 1st generation Synthesizer V editor.

On August 2, Dreamtonics opened a beta-test application for VST and Audio Units versions of Synthesizer V Studio to anyone that purchased the software.

On February 4, 2022, according to AH-Software, the Synthesizer V series had been selling much more than expected within a year ever since it became compatible with AI.

On February 28, 2023, Dreamtonics announced that Synthesizer V Studio would soon add Cantonese Chinese as its fourth supported language. This would allow the engine to support both voice libraries dedicated to the language as well as Cross-lingual Singing Synthesis. The company also announced the future support of rap vocals, showing a demo of a new male vocalist rapping in Mandarin Chinese and English. Support for Japanese rap was expected in the future.

On March 2, Dreamtonics posted a response to fans' concerns with the implementation of Cantonese Chinese and noted that they were checking and fixing the issues with the demonstration clips as reported by the user base. They also noted that Synthesizer V Studio supported the input of lyrics in Jyutping, which was the 1993 version of the Cantonese spelling scheme. It was not equivalent to the X-SAMPA phonetic scheme above the lyric notes on the editor. The X-SAMPA phonetic scheme for a Chinese character was also not equivalent to the Pinyin reading of the character.

On March 15, after receiving feedback in improving the song to be more in line with Cantonese songwriting habits, Dreamtonics replaced the Bilibili version of the debut video, which implemented corrections made to the male vocal's and Feng Yi's demos.

The rap feature for English and Mandarin Chinese, and the implementation of Cantonese Chinese Cross-lingual Singing Synthesis was officially planned to be fully implemented in Version 1.9.0, with a beta version released on April 18. Dreamtonics mentioned that after receiving valuable feedback, they focused on refining pronunciation for an even better user experience. As for how it worked, they said that when the language is set to Cantonese, all Chinese lyrics will be sung with Cantonese pronunciation. If misread lyrics occurred, users can correct them by typing the romanized form in Jyutping directly. Although the phoneme set is largely based on Mandarin Chinese, several phonemes unique to Cantonese were incorporated.

Development of Neural Networks

Following the Synthesizer V Studio 1.2.0 Update on February 19, 2021, Kanru Hua announced a thread on his personal Twitter account about how the optimized neural network inference functions in the recent Synthesizer V updates.

The following day, Kanru Hua elaborated more on the subject. Synthesizer V Studio 1.2 uses JIT-compiled quantized sparse Matrix-vector multiplication (MVM) kernels. In his own words "artificial neural network boils down to a bunch of really simple arithmetic operations, e.g. a + b * x1 + c * x2 + ... But when you (purposefully) compose millions of these together, they can be turned into really complicated machines."

He notes that in order to build a voice, he picks specialized values for the "a, b and c" that best represent the voice, and then plugs the values into millions of equations, These "a, b, c" values are called parameters. Linear algebra is used to help rather than writing each individual equation by hand to make matrices & vectors which are notations that aid with simple math in large batches. Many neural network models are composed of matrix-matrix multiplication. In the case of Synthesizer V, the bottleneck is matrix-vector multiplication, mainly used in a network that generates waveform samples which is known as the “neural vocoder”. One of the challenges he notes is that not only having a large network to manage, the network needs to "run tens of thousands of times per second" to synthesize high quality audio in real time.

Due to this, modern CPUs are needed as they are able to run at several "gigacycles" per second ("GHz"). This is on a similar order of magnitude as the number of operations per second above however, he notes that the margin is very tight. He states that because of this, not all CPU cycles can do "useful work" and presents additional challenges. The goal here is to make this MVM operation perform as fast as possible on modern CPU systems.

The following day, he elaborated further into the usage of Sparse Matrix-Vector Multiplication (SpMVM) for Synthesizer V's neural networks. Kanru Hua states that out of the millions of parameters, many are redundant and thrown out without hurting sound quality which results in what is called a Sparse matrix. Some parameters that are truly important can't be thrown away and if too many are removed from too many of the parameters, eventually the quality will drop. "The synthesized voice will sound more and more like from a walkie-talkie until it completely degrades into noise."

The goal here is to remove the less contributing parameters carefully and remove as many as possible without hurting the quality. Typically over three-fourths of the parameters are thrown out if done properly.

When executing the sparse neural network, the program needs to skip the parameters that were removed. This skipping process adds a sometimes expensive overhead. This aids in boosting the speed up to four-times the initial speed. "Going sparse is an effective way to compress a neural network. If done right, it can still speed up execution by a few times, although this would require highly optimized code for SpMVM."

Over the course of the following three days, Kanru Hua posted three additional threads further elaborating on the intricises of developing the neural networks. After making matrices sparse, integers are quantized to scale the values down before doing MVM to make sure the result will be in the range, it was noted that if the addition or multiplication programming goes out of range, they could be wrapped back to the lower end of the range resulting in an overflow and the synthesized voice could potentially sound akin to that of a mistuned radio or be complete noise.

He states that neural networks used in Synthesizer V AI come in many different sizes, some of which can be made sparse, some can not. The software is able to run on every x86 CPU since the Pentium 4 processor that was developed in 2004.

External links

• Official website
• [1] SynthesizerV Wiki

Category:2018 software Category:Music production software Category:Japanese inventions