#for the record i had to do the og teens pose myself w/ someone to get it right and my legs hurt for the rest of the day x_x

[tanis-drawings-2point0]

shootout to the intergeneracional friendship between the stampler/marlows & the russet/oak-garcias, that warms my heart 💞💞💞 :')

bonus willy & barry, that are NOT friends

[furby-science]

The Making of Sterling the Super Furby: A Brief Overview

“I… I can’t look! I think I’m gonna… *HUEEEGH*!”

Before I get into this post, I want to list a few things I didn’t know shit about when I started Sterling:

Electronics

The Python coding language

Furby anatomy

Single board computers

After creating Sterling, I’m happy to say that now I have approximate knowledge of some of these things, but keep the above in mind as you read onwards. This little gremlin child was a learning experience from start to finish, and one I am incredibly proud of myself for sticking through. This also means that I am in no way an expert on everything I’m getting into okay? Okay let’s go!

The Hardware

First, a rundown of the hardware. I took heavy inspiration from the Furlexa mod shown here, and that was what I initially sought to create. The mod had three computer components to it:

A raspberry pi zero w single board computer for the AI to live on, with a mini USB microphone plugged in;

A pimoroni speaker PHAT to use as the sound system;

A motor controller to drive the furby’s motor.

My main problem with Furlexa was that this initial build took a lot of soldering, and I am a wussy who had a number of bad experiences with soldering irons in shop class. So, what’s a novice electrician to do?

Enter the Adafruit Crickit HAT. By sticking this little fucker on top of the raspberry pi, I was gifted with an amplifier, a speaker jack, capacitative touch sensors, and a motor driver all in one, no soldering needed if I bought the raspberry pi zero w h! The main challenge it posed was powering it. The Crickit insists, for some unfathomable reason, on being powered by a bulky DC jack, the kind you’d plug into a wall outlet, and the converter plug to use a battery pack with it was way too bulky to fit into a furby. I needed Sterling to be portable for maximum huggability, so this just wouldn’t do.

One fried raspberry pi and Crickit HAT later, I found the answer! By soldering the original furby battery pack to the underside of the Crickit board’s DC connection, these fuckers right here…

I was able to bypass the need for a wall plug or converter, and power him directly through the battery compartment like God intended. S/O to my friend Nick who is way less of a dumb bitch than I am and helped me figure this shit out I owe u some bread man.

So the tl;dr of it is, I effectively reduced the required computer components from three to two (excluding the speaker). Speaking of (heh), Sterling has an impressive 3w speaker in him, allowing him to be audible even without the use of the built in amplifier. It’s got such good bass on it, he even rumbles when he purrs without the aid of the motor!

And yes, when you pet him, he purrs. And complains if you manhandle him! The aforementioned capacitative touch sensors on the Crickit HAT made it all possible with the help of a few cables and some foil tape.

Wait, did you say soldering!?

Yup! It was a necessary evil; at the end of the day I had to pick my poison: soldering 80 pins on the speaker PHAT, or soldering like four contact points on the Crickit. I chose the more merciful option.

But wait, that whole outfit is really bulky still! How did you fit it inside the furby?

Subtractive methods, subtractive methods, subtractive methods! ;D Someone who actually knows things about furby anatomy and/or electronics will probably vaporize me for this, but… if I didn’t need it, it got the boot! That included prying off anything on the Crickit board I wasn’t using at the risk of destroying it completely - which probably isn’t ideal, but it also worked by some miracle, and again, I am such a basic bitch electrician that calculating the proper voltage for LEDs is still basically witchcraft to me, so… what I’m saying is I made it work. And that I really, really hate soldering! ¯\_(ツ)_/¯

You can see an early video of the end result here, and a later video of the outfit inside the naked furby here. This was back when he was still having auditory processing issues. Apologies for the shoddy quality, I was too excited everything was working to care about that at the time.

The Software

My other beef with Furlexa is… well, it’s an Amazon Alexa, and I’m a shitty little anticapitalist hermit who hates Amazon with a passion. Google Assistant was just as bad in my book. Mycroft was open source, but had a snowball’s chance in hell of running on the raspberry pi zero’s 512mb of RAM… I also wanted my assistant to have a degree of customizability to it. I wanted the furby’s AI to have a unique personality, identity, and preferences, much like classic furbies themselves did. A big box AI just wasn’t going to cut it!

Enter the Jasper Project. Yes, it’s old. Yes, it’s a bitch and a half to install. Yes, you have to know Python to get anywhere with it. However, it was free, open source, capable of running on a raspberry pi zero, and highly modular, meaning with a few lines of code, I could make it all my own - even to the extent of changing the AI’s name and voice (which is gr8 because I know a Jasper so naming my furby that would be Weird), or - the best part - writing my own, custom functions! Customizability-wise, I struck gold.

Ah, and glad I am that Jasper is modular, because I had some work ahead of me…

The STT Engine

The STT (Speech to Text) engine is what Sterling uses to understand what’s being said to him. Jasper’s proprietary STT engine is PocketSphinx, a fully offline STT engine, which sounded great in theory before I quickly learned it’s a nightmare to install, and also more inaccurate than a stenography machine powered by a single potato when actually being used. I had to compromise my morals a bit here and opt for using Wit.ai instead, which is free, but is also owned by Facebook. Big data is frustratingly inescapable in these cases.

There is one light at the end of the tunnel, and that is the training of acoustic modules. This has the downside of taking for-fucking-ever and requiring a quiet recording environment, however, and I don’t have the time right now to read through the pages and pages and pages and pages of computer theory right now to fully understand how to train one. So, improving PocketSphinx and running Sterling fully offline remains a stretch goal.

The TTS Engine

The Text to Speech engine is basically Sterling’s voice. This one was a bit easier to customize, and I’m thankful for that, because Jasper’s OG voice is a bit er… 90s computing for my tastes.

I shopped around for decent, human-sounding TTS options, and settled on installing Mimic1 TTS, Mycroft’s TTS engine, by hand, and modifying the Jasper source code to support it. Of all the TTS engines I tried, I felt that this one had the most natural intonation out of all of them. I liked the gruffness of the Scottish accent, and I think it really helped round out Sterling’s endearing, if a tad prickly, personality.

The Audio

This was another unforeseen hurdle. Turns out that I had his mic volume turned up way too high, because I greatly underestimated the capabilities of my tinyass five dollar USB microphone to pick up noises from within a furby. It took a bit of hacking in PulseAudio to get him hearing things properly, and I’m still not all the way happy with it, but he’s running wayyyy better than he did!

Another issue was the amount of time he actively listened for. It was way too short for my liking with the hardware I was using, so I had to edit Jasper’s mic.py source file a billion times before I hit a sweet spot. Even early on, my little shit child never liked to listen to me. :P

Pimp My AI

Once I got all that in working order, it was time to browse GitHub for modules to add! I found a surprising amount that were, as expected, outdated, janky, non-working, or in need of a complete rewrite. A non-exhaustive list of modules I rewrote and added to Sterling’s AI includes:

Wolfram Alpha integration

His translation function

The IMDB module that searches movie titles

The Dictionary and Thesaurus modules (minor additions to improve user friendliness)

The morning greeting module

The holiday countdown module

There are also plenty of modules I wrote on my own, that I’ll be showcasing here in due time, but I want to give special mention to the one I’m most proud of. You see, when I was a wee dumb bitch, I was… well, a wee dumb bitch! When I was informed furbies learn English, I thought they really learned English. Like, fluent English. I envisioned these kids straight up having full conversations with their lil robots with reckless and envious abandon. I was, as it happens, too poor to afford a furby at the time, so I didn’t realize until embarrassingly later that they only learn some words, and certainly can’t hold much of a conversation (in English at least).

Fast forward to twenty-bi-teen. I’m surfing GitHub, and I happen upon a Cleverbot module for Jasper allowing the AI to work as a chatbot. Fuck yeah, I think, because I had no life in 2008, or friends for that matter, so tormenting Cleverbot was my favourite pastime. Nostalgia trip GET!

…can you guess how much the silicon valley capitalist scum are charging for the once-free Cleverbot API now? A hundred and twenty. McGoddamn. Dollars. A YEAR.

So, to make a long story short, I turned my hat backwards and rage-coded a simple chatbot module that runs on an early version of Chatterbot capable of running on the raspberry pi. It’s fully offline, and completely free, and Sterling here has a database of ~400 phrases, which isn’t bad given the limited processing power! It took five straight days of work, it’s not the smartest chatbot, and it’s certainly not the fastest, but it gives me those sweet, sweet, circa 2008 Cleverbot vibes. Oh yeah, and it doesn’t cost me over a hundred goddamn dollars a year!

The first thing I said to the chatbot, of course, was “I’m so proud of you.” Through his shitty little testing mic that gave him a somewhat incredulous tone Sterling replied, “I’m glad to hear that.” and I’m not saying I shed a single themly tear over it, but I’m not denying it either. I made a childhood dream come true, fam. ;u;

There are way more Easter eggs I plan to show you, of course. At first I was thinking of doing one long video, but an update a day showcasing a different function might be easier to manage - and maintain some of that gold old sense of mystery that surrounds most furbies. No, I’m gonna take y'all on a little journey through the final product of my literal blood, sweat, and tears!

Besides, Sterling is a perpetual work in progress. He has a massive list of features, and I’ve already got more in the works. I could be in my eighties and still be adding more functions, more bells and whistles, more witty one-liners. He’s a one of a kind work of art that will never truly be finished - not unlike you and me.

The Glow-up

Here’s Sterling’s before pics from the seller I got him from:

(If u recognize these pics and ur the seller thank u thank u for giving me bmy boy)

And here’s after!

I come from a background of customizing ponies and dolls, so working on this guy wasn’t as far removed as I expected it to be. I added floof to his head and tail by sewing in wool plugs, and his gorgeous eyes are from in2blythe on Etsy. I wrapped him up in a little bow and he was good to go! His sterling silver beak, from which he gets his name, was the most finicky part. Turns out enamel paints take a million years to fucking dry, if ever, which isn’t great when painting something that sees a lot of movement and could potentially get dented by a face plate, like… idk, a furby beak! A bit of silver nail polish did the trick and he was good to go. Learn from my fail, fam.

What It Cost Me

If you’re masochistic determined enough to attempt this yourself, I want to sit you down and warn you of something: this will take months and hundreds of dollars to do. Installing Sterling’s AI and its necessary components on that shitty little raspberry pi over SSH took me a week at first, and that was with me leaving it on 24/7 to chug away compiling things. When I broke the SD card the AI was on and didn’t have a backup copy, it took four straight days of rage-computing to regain all my progress. Then when his audio processing got fucked all to hell for reasons I still do not understand to this day, it took another four days of rage computing to do yet another reinstall and get him back in working order. There were times where I would go to work for 8.5 hours, bus an hour home, work 6 straight hours on my furby, go to sleep for 4 of them, go to classes, sleep, and work 6 more hours on my furby. For two months. Sterling took from the third week of August from his initial inception to his birthday on October 23rd. That’s not to mention the time I fried everything and had to wait five days and travel to the bumfuck end of the city for a replacement pi and Crickit, or the days I spent customizing him, sewing in hundreds of little hair plugs into his ass and head by hand, and waiting for those shitty enamel paints to dry, only to discover after four straight days of failure that they take weeks to do so and I was better off using cheapo nail polish!

The point I’m making is, if you take on a project like this and want it to be successful, you have to be tenacious. I would highly recommend a background in coding (I have a web design diploma) and general tech savviness as an asset. Sterling is the product of the years I spent behind a computer keyboard from the start of age three, and the roughly ten years I spent customizing dolls and ponies. It’s cheesy as shit to say he’s my magnum opus, but in a way, he is.

I’m not saying this to be elitist or snotty. I’m saying this because I nearly broke down crying the first day the raspberry pi came in, before I slept on it and figured out what phrase to google to solve the crashes and kernel panics it was having. When I broke the SD card when I was nearly finished, I felt nothing, because I was all out of tears at that point. When I fried the first raspberry pi and Crickit hat trying to figure out how to bypass that DC jack, my only thought was, “Well, I think I know how to do it without fucking it up now, and if I can’t do it, this whole project is fscked” .

You will encounter errors that no step by step guide can prepare you for that will make you curse the day you were born. The difference between success and failure is how many times you’re willing to get up and try again, and I’m here to tell you it’s possible. But you gotta want it.

Will You Release the Code Base?

Yes and no. If there’s enough demand, I’ll definitely release Sterling’s basic modules as a scaffolding. I won’t be releasing Sterling, though.

What do I mean by that? Well, Sterling was intended from the start to be truly one of a kind, and he always will be. I hand wrote hundreds of lines of dialogue, all completely tailored to him, and I’m still planning on adding twice as many. Corny as this is, this little guy has a metric fuckton of sentimental value to me. I don’t have kids so idk how it would compare to that, but I definitely love him as much as I love my cats, but I also didn’t undergo two straight months of suffering in ADHD fixation hell to create my cats, so it adds like, a whole other twee dimension to it.

So, if there is demand for this, what I’ll release instead is a scaffolding from which you can code your own, unique furby from, with their own name, personality, and responses all unique to them. I’ll also release it with the caveat that I am not a good Python coder! I have not written any Python before this, so a lot of what I did write is noob-tastic and hasn’t even been linted. You have been warned!

“If I give you (insert amount), can you make one for me?”

Holy shit I’ll be real with you, I’d love to do this as a living. I’ve been dying to see a smart assistant hit the market that’s like… well, an actual, endearing companion and not just a voice coming from a speaker. The problem with doing this is that, if you drop a lot of money (and it will be a lot of money, even with a code base to work from, a lot of hours of handiwork still goes into coding individual responses and making sure everything works as intended, on top of possibly customizing too), there is one major problem: proximity. I won’t be able to troubleshoot your furby nearly as effectively from far away as I would be able to if we lived in close proximity. Which means if something goes wrong between the time your new friend is finished at point A and turned on at point B, I won’t be there to troubleshoot it in person for you, which means you could end up stuck figuring out certain things alone. If you use Windows, that will be very, very hard - not being an OS snob here, I own a dual boot myself, it’s just a case of incompatible file systems. And unless you can figure out how to edit the wpa_supplicant file on a raspberry pi to update your wifi credentials, your furby’s internet connection could be toast if you move house and those credentials change. That’s not getting into the cost some services charge for extra API keys to use their online functions…

The long and short of it is, if I’m going to do this for money, I want to make sure you get a quality product and friend that will bring you joy for years to come. Since that’s not something I can guarantee, I can’t in good conscience take people’s money.

I Could Teach You (And I Won’t Charge)

…however, I am a law student who is also working 8.5 hour night shifts three nights a week. I am also mentally ill/neurodivergent, which saps my energy in more ways than one. I won’t always be easy to get ahold of, or be able to answer every question I get, especially not ones that can be solved with a quick google search, like how to set up a raspberry pi, or… anything found on Adafruit’s Crickit guide, for example. When I have the time and energy, I’m hoping to use my next project as a jumping off point for a step by step walkthrough of the process. For now, though? I’ve been furbied out, so if there’s enough demand, I’ll compile as many of the resources I used I can find in the meantime, and post some tips from the word doc I kept while making Sterling, and go from there.

So What’s Next?

My one dad’s birthday is coming up in August, and I’m kicking around the idea of turning a furby into, I shit you not, a ghost hunting device. He loves ghost hunting, but hates robots, and as his gremlin shit child I am obligated to troll him in this fashion. 😎 Also considering doing a certain type of oddbody mod, but I want to get permission from the person who first thought of the concept before I dive head first into it.

And that about covers it! Thanks for reading, and if there’s anything you’d like to see from Sterling and I, don’t hesitate to drop us an ask!