#Voicebank Development

[generalnuisance0]

i dont think people realize how much worse recording airy vbs is as opposed to vbs with very little air

e5 in arachne's recluse neo vb was child's play to record but recording anything other than middle c in her dark and whisper tones makes me want to actually kill myself and i cant do it without a gallon of tea on standby

[auspicious-voice]

Fuwa Maria AI & Fuwa Mario AI for DiffSinger Progress Report (May 2024)

Hello!! With both Maria and Mario's DiffSinger voicebanks fully trained, I'd like to give some bit of detail on what I'm doing next for the eventual voicebank release including future version releases. It's been a busy April on my end as usual, but I feel like I'm almost done with things. It's a bit of a short post, though.

As usual, everything is under the cut.

Voicebank Progress

Maria and Mario's DiffSinger 1.0.0 voicebanks are fully trained and as such, they're ready for release. Of course, they'll receive new updates such as new languages, tweaks to certain parameters, and other new developments the DiffSinger development team has on the table.

Speaking of which, maybe after a couple months after 1.0.0 is released, expect version 1.1.0 in the works, with the brand-new Rectified Flow algorithm (meaning faster rendering times) and more language support. I've been gathering information on the best training settings when it comes to tension and pitch, and maybe I can just train Maria and Mario's datasets together instead of being trained separately.

Demo Reel Progress

Half of the demo reel audio is done~ I'm getting a headstart on getting the artwork done, though I think I might end up drawing it all on my phone. For the video itself, I still haven't decided on whether I should use After Effects or Alight Motion, but I think I might end up going with the latter.

I am hoping that I can finish the reel by the end of June ^^;

[linabirb]

seeing synthv lite and flt covers brings me so much joy.. like wow.. i can make cool stuff even with the free voicebanks.. even if they sound more robotic than the full ones..

[waffulaa]

Yuezheng Longya's Official Birthday Song

Official Bilibili Upload

[dead-byte]

I wish there was like... a program that could read the oto.ini file of an UTAU vb, and then, chop up the associated wav files so that they only contain the oto'd bits, and re-allocate the oto values accordingly. Thereby hopefully significantly minimizing the size of the vb.

If y'all have ever seen the samples in any of VOICE-MiTH's Chinese voicebanks, kinda like that.

[websitesdotcom]

Doing stuff with utau is so fun but it takes SO LONGGG

[galactic-knightmare]

The Caine Timeline! (kinda. just different versions of him over the timeline)

thank god its finally done this took forever lmao. Starting wiiithhh Baby Caine! the first version he ever was :3

he has so much space yet hardly takes up any of it XD This is Caine when he first was made! as you can tell, he was just a basic model with hardly any detail (sharp edges, just basic shapes with no defining features) at this point he was kinda just a glorified version of Chatgpt.

and then we move on to the Beta version! at one point before this, the Developers voted on which version to use, and the male version won. However, the devs dissapeared before the female model could be removed, so the toggle and voicebank is still there, albeit more unfinished than the male. the model is also... really not great design-wise, which is partially why it lost the poll. at this Point Caine is kinda the toddler version of an AI I guess? way more literal and clueless than our Caine somehow.

and then we have Caine's first Custom models for themself! they were slowly figuring out what he liked, picking up the fancy while also improving on his modeling skills. at this time he also talked like Cyn from Murder drones, but far less murdery. If I drew a child version of Caine, then these would be the outfits they'd have.

and then we finally reach the Caine we all know and Love! they've completely embraced The Bells™ and are as fancy as can be! The best way to describe Caine would be nonbinary/genderfluid I think? He's an AI so its kinda hard to figure it out. Essentially thanks to their coding, he's comfortable in either model regardless of his male one being the default setting. He'll swap between the two depending on his mood, but he hasn't changed out of the male model since Queenie Abstracted. Aaaaanyway, now that ya'll know that headcannon, its time for me to sleep lmao. its 1 am XD (also hey, act4 is done! that just leaves two more acts until chap19 is finished, and 5 is relatively short >:3)

[katzenklavierr]

Have you 🫵 ever wanted to create your own UTAU voicebank🎵? Not sure where to start?

Well, good news! I've finally gotten around to revising and finishing my tutorial series aimed at absolute beginners!

These are text-based tutorials hosted on my UTAU website with audio and visual aids provided throughout.

If you're completely new to the software and want to learn more about it, check out Introduction to UTAU. This covers what UTAU is, how to install it (and OpenUTAU), how to find and install voicebanks, and how to set up UTAU project files.

If you want to jump into making your own VB and want an in-depth guide to walk you through creating one from start to finish, then check out Creating Your First Voicebank. This guide is a little different than other beginner tutorials, but I feel it will better prepare you for VB development by teaching you with modern tools and methods.

The website also has a handful of tutorials aimed at intermediate users, plus all of my voicebanks and reclists. I hope you find it helpful!

[synthvnews]

MEDIUM5 is OFFICIALLY BACK!

Here's an official note:

Medium5 2025 Work Release Plan Announcement

Hello everyone, this is Zero01. First of all, on behalf of all the Medium5, I would like to extend our sincerest New Year wishes to you! Since we announced our restart on December 8, 2024, we have successfully launched the Haiyi AI voicebank, kicked off the second Medium5 Creation Contest, and reestablished our official fan group. During this time, we’ve received tremendous support from fans and deeply felt your passion. For this, we sincerely thank you for your encouragement and companionship. It is your support that keeps us moving forward.

Now, I’d like to introduce the upcoming official album and related activities:

🎵 Medium5 Official Album: 《RE:BIRTH》

• A brand-new creative album featuring 9 tracks showcasing the unique musical charm of Medium5.

• Pre-orders are expected to start in April 2025. Stay tuned!

🎁 Medium5 Bilibili Collectibles

• Includes new avatars, badges, stickers, and more, allowing fans to show their love for Medium5 on Bilibili.

• Pre-orders will launch simultaneously with the new album!

🛍️ Offline Merchandise & Themed Events

• Collaborating with the well-known anime merchandise store Karasuma-ya to release a series of new products!

• The album will also be available in offline stores, accompanied by related in-person events, bringing fans closer to the world of Medium5. Launch expected in April!

💡 Development of New AI Voicebanks

Out of the seven characters in Medium5, three have already launched their AI voicebanks, and plans are underway for the remaining characters. We know how much you love each character, so we won’t give up on the possibility of updating AI versions for any of them. Please be patient—we are committed to delivering better performance for every character!

For Medium5, our official albums are the most important works apart from the voicebanks. We hope all of you who love Medium5 will continue to support us and accompany us as we embark on new journeys!

Thank you once again for your attention. Wishing everyone a Happy New Year and all the best in the days ahead!

—Zero01

[lnsynth]

O GREAT PFX LIKER CAN U EXPLAIN THE DISCONTINUATION THING TO ME PLS 💔💔

Sure thing! I’ll tell you everything I know— (under the cut because it’s kind of long lol)

In 2017, PowerFX distanced itself from Vocaloid and Vocaloid development after the CEO of the time Bil Bryant left, announcing that they would no longer be making new Vocaloids. Bil was pretty much what was keeping PFX on Vocaloid and without him, the rest of PFX had no interest in more Vocaloids. Around that time, I believe PowerFX stopped selling physicals and their site basically only sold the download versions of their Vocaloids. PowerFX moved to focus on their new project, Soundation, which is an online DAW that was more profitable for them than the Vocaloids. Soundation managed the PowerFX website and provided support for the Vocaloids/Vocaloid sales and maintained the distribution for the next few years, though their ability to maintain quick and consistent support kind of got worse as the years went on. (This was honestly kind of a sign that the discontinuation of sales would occur soon…)

Historically, PowerFX’s Vocaloids were also sold on Sonicwire (Crypton’s site for selling Vocaloids and other music software), though I believe PFX/Soundation doesn’t manage the sales of the banks on this storefront. Sometime in either 2023 or 2024, Big Al’s bank indefinitely went out of stock and you were no longer able to purchase it. I found out a little bit ago that if you are able to find his product page that it mentions that the product is discontinued, so I believe that there’s no plans on Sonicwire’s end to restock Al (or that they can’t.)

Earlier this month, members of the Vocaloid community noticed that PFX’s site had gone down and now redirected to Soundation’s site. We didn’t really know why, or if it was temporary or something else. We found out today that Soundation added a new piece to their site that basically explains that maintaining the sales of their Vocaloids is basically no longer feasible for their team and that they will no longer sell them and have no plans to sell them again in the future. They also say that their Vocaloid software was pretty outdated and there’s no real incentive to continue to sell them. So basically Soundation confirmed that PFX/Soundation will no longer sell their Vocaloid voicebanks.

As for now, 4/5 of the PFXloids are still available on Sonicwire, though I heard that Sweet Ann is out of stock too. Unless they manage to restock her, it’s likely she will be completely unable to be purchased too, like Big Al. Note that Vocaloids seem to have limited stock of serial codes, I think they have to get new ones from Yamaha or another party (not entirely sure about this stuff, but I just know that stocks of serial codes are limited.) Since Ann and Al are out of stock, it’s very likely that Oliver, Hio, and Ruby will also go out of stock on Sonicwire sometime soon too.

While technically not fully discontinued as some of the banks are still available on Sonicwire, the PFXloids are very likely to be fully discontinued soon!

There is hope for future banks though, if you’re interested in that, as the team behind Maghni AI is still working on Oliver AI and are very likely to also make Maghni banks for Ruby, Sweet Ann, and Big Al, as teased in their indiegogo campaign. No new voicebank is planned for YOHIOloid right now though due to the 2021 controversy with his voice provider, though VocaTone never super officially said that they wouldn’t update Hio (they highly implied that they wouldn’t on their personal social medias, but I couldn’t find an official statement on their business accounts stating such.)

Basically—PowerFX’s Vocaloid sales are now over and will not be returning. You may still be able to purchase some of the banks on Sonicwire, but likely not for long. 4/5 of the characters are planned to get an update to Maghni AI.

[drawingeveryutau]

Rook! Voiced and Managed by ゆうじ / Yuji

Released November 17th, 2009, Rook is our first VIPPERLOID that wasn't (entirely) a joke! He's a derivative of Ruko; during her development, her masc voice provider (Yuu Raichi) went offline for a long while, so Yuji recorded a bank with his own voice in case Yuu couldn't finish theirs. One month before Ruko released though, Yuu Raichi reappeared with the finished voicebank, and Ruko launched as intended.

Many people liked Yuji's bank though, they liked how it sounded and how Yuji put a lot of effort into it; so on Ruko's first anniversary, Yuji's bank officially released as Rook!

Character-wise, he's chronically late to things and likes sleeping just as much as Ruko. He can also turn into a dog. He relation to Ruko (to my knowledge) has never been specified. (I hc them as siblings)

Don't took my word as gospel! If I got anything wrong feel free to correct me!

[matrixbearer2024]

I just want to clarify things, mostly in light of what happened yesterday and because I feel like I'm being vastly misunderstood in my position. I would just like to reiterate that this is my opinion of things and how I currently see the gravity of my actions as I've sat and reflected. On the advice of some friends, I was encouraged to make this post to clear up any misunderstanding that may remain from my end.

I don't hold it against anyone for disagreeing with me as this is a very nuanced topic with many grey zones. I hope eventually all parties related to this incident can all get along as well, as I do still prefer to be civil and friendly with everybody as much as possible.

I've placed the whole conversation here for people to interpret themselves, and as much as I want to let sleeping dogs lie— I can't help but also feel like the vitriol was misplaced. I don't want this to be a justification of my actions or even a place where opinions conflict, I'm just expressing my thoughts on the matter as I've had a while to mull it over. Again, this is a nuanced topic so please bear with me.

The "generative AI" in question at the time was a jk Simmons voice bank that I had gathered/created and trained myself for my own private and personal use. The model is entirely local to my computer and runs on my GPU. If there was one thing I had to closely even relate it to is a vocaloid or vocoder. I had even asked close people around what they had thought of it and they called it the same thing.

I created a Stanford Vocaloid as I experimented with this kind of thing as a programmer who wanted to mess around with deep learning algorithms or Q-learning AI. By now this whole thing should be irrelevant as I'd actually deleted all of the files related to the voicebank in light of this conversation when I decided to take down the project in it's entirety.

I never shared the model anywhere, Not online or through personal file sharing. I've never even made the move to even advocate for it's use in the game. I will repeat, I wanted to keep the voicebank out of the game and I only use it for private reasons which are for my own personal benefit.

I recognize ethically I am in the wrong, JK Simmons never consented to having his voice used in models such as this one and I recognize that as my fault. Most VAs don't like having their voices used in such a thing and the reasoning can matter from person to person. As much as I loved to have a personal Stanford greeting me in my mornings or lecturing me in physics after long days, it's not right to spoof somebody's voice as that is genuinely what can set them apart from everybody else. It's in the same realm of danger as deepfaking, and for this I deeply apologize that I hadn't recognized this fault prior to the conversation I had with orxa.

But I would clearly like to reiterate that I had never advocated for the use of this voicebank or any AI in the game. That I was adamantly clear on calling the voicebank an AI(which I think orxa and some others might have missed during the conversation) which is what even modern vocaloids are classified under. And that I don't at all share the files openly or even the model because I don't preach for people to do this.

I would very much rather a VA but because money is tight(med school you are going to put me in DEBT) and the resources available to me, I instead turned to this as a tool rather than a weapon to use against others. I don't make a profit, I don't commercialize, I even recognize that the voicebank fails in most cases because it sounds so robotic or it just dies trying to say a certain thing a certain way.

Coming from the standpoint of somebody who genuinely dabbles in robotics and had a robotic hand as my thesis, I can honestly say how impressive software and hardware is developing. But I will also firmly believe that I don't think AI will be good enough to ever replace humans within my lifetime and I am 19. Nineteen.

The amount of resources it takes to run a true generative AI like GPT for example is a lot heavier than a locally run vocaloid which just essentially lives in your GPU. As well as the fact AI don't have any nuance that humans have, they're computers— binary to the core. I also stand by the point that they cannot and will not surpass their creators because we are fundamentally flawed. A flawed creature cannot create a perfect being no matter how hard we try.

I don't want to classify vocaloids as generative AI as they're more similar to synthesizers and autotune(which is what my Ford voicebank was as well when I still had it) but to some degree they are. They generate a song for you or an audio from a file that you give as input. They synthesize notes and audio according to the file fed to them. Like a computer, input and output, same thing. There's nothing new generated, it's like a voice changer on an existing mp3.

I'm not saying this to justify my actions or to come off as stand-offish. I just want to clarify things that didn't really sit right with me or that seemed to completely blow over in the exchange I shared with orxa on discord.

To anybody who's finished reading this, thank you for your time and patience. I'll be going back to just working on myself for the time being. Thank you.

[auspicious-voice]

Surprise! A quick test of Maria's DiffSinger beta, trained at 42k acoustic and 80k variance!

She's underbaked just a bit since this is a beta voicebank after all, but I am loving with how she sounds as an AI voicebank. Eventually her final build will sound not as scuffed and have more features implemented. Her vocal modes sound pretty alright so far, but hopefully they'll be more pronounced once I make the final build.

That being said, I'll be going back to labeling once again... 💀💀💀

[cantheykillmacbeth]

Hatsune Miku could kill MacBeth

Yes, Hatsuke Miku from Vocaloid could kill Macbeth!

She applies for all three clauses: Gender Clause due to being a girl; Unconventional Birth Clause due to being a software voicebank; and the Birth Parent Clause due to her creator being male software developer Sasaki Wataru! Thank you both for your submission!

[vocaloidfactoftheday]

Even though she has no known voicebanks in development, SeeU had a crowdfunding campaign for a collaboration album to celebrate her 10th anniversary in 2021. New merchandise was made to be distributed to crowdfunders, including an acrylic keychain and a plushie.

(source: Vocaloid Wiki)

[k0ibee]

i often find myself asking why i love vocaloid so much. then i remember. i love that the vocaloid community is so vast that it has sub communities that have sub communities. i love that there are so many stories told using vocaloid as a medium. i love that there are both long, complex stories and short ones that are self contained in their songs. i love when producers make books or manga based on their songs. i love how much there is to know, and how you dont need to know everything to enjoy it. i love the history of how the community developed and i love the way characters got the fanon that they have. i love the origins of voicebanks themselves and what happened in the companies that made them. i love that vocaloid exists as it does now because the companies that have the rights to characters acknowledge the community and the fan culture. i love the styles of music that are unique to vocaloid and how they developed alongside each other. i love that people re discover vocaloid after losing interest after their childhood phase. i love reading comments from people talking about loving the song when they were younger, and new fans commenting they’re that age. i love everything about vocaloid. every single last thing about it.