forastic

[forastic]

Open Studios interactive project

UCSC DANM grad school winter 2023 - week 9-10

The final prompt for my interactive art class involved a skill share. From one other student I learned how to use a RFID reader with a raspberry pi. From the other I learned how to do needle felting. We each created small figures out of felt and decided to put them in a space together, with different zones that they could interact with. Here’s the felt figure I created.

This was also the first time I ever sewed anything! I sewed the wings. 

Here’s the brainstorming board the three of us put together to create a weird version of a petting zoo:

We planned on showing it off at the art department’s open studio (despite the fact that we’re not part of the art department). While we were able to show stuff at the open studios, other projects ended up taking priority and this didn’t come to fruition the way that we had originally planned.

I wanted to make a reference to Nam June Paik’s TV buddhas, but I wanted to make it softer and kind of silly. I crafted a cute TV out of felt and replaced the dragon’s head. 

I intended on having the dragon (now a TV dragon) watch himself on a TV while it streamed to Twitch. I wanted to install COZYSPACE as well, near where this was. When you place the TV dragon in the right place, I wanted a little bit of music to play. This idea got a little out of hand, but I managed to set everything up that I planned, and it worked fine.

I began 3D printing the shapes in COZYSPACE at various sizes with the intention of hanging them near the TV and the dragon. 

I also wanted to have the controllers for COZYSPACE be closer to my original vision of regular Super Nintendo controllers, but wireless. I got a pair and began painting them to make them cuter.

I used regular spraypaint from the hardware store, using the splatter technique I learned from making stuff for BEARPAD. I chose a purple and teal color scheme with pastel accents, as COZYSPACE was originally intended to chill me out personally, and I like that early 90s party cup/taco bell aesthetic. The spray paint took a long time to cure - the controllers were still a little sticky for up to a week after I painted them.

I rigged a raspberry pi and RFID reader to activate when placing a chip on it, combining a couple tutorials online to get it to also play music using the pygame library. I have only minimal python coding experience and it took a bit longer than I anticipated. I was amused that people recommended playing .ogg files for audio, as I haven’t really heard of those being used in over 15 years. I wrote a piece of music really quickly to play when the TV dragon rests on the cozy spot. I made a synth sound, threw a bunch of effects on it, and improvised in g flat major pentatonic using my computer keyboard in ableton.

https://on.soundcloud.com/7ALpC

Patrick designed a new shape that would look good alongside the shapes in COZYSPACE, but was large enough to hold a raspberry pi and an RFID reader. We grabbed a raspberry pi case design from thingaverse and dropped it into Blender to make sure we had the size right, built a chamber for it, then he built a shape out around the chamber. Here’s some pics of it hanging in action. 

I tried it out streaming on twitch, but it immediately brought up concerns of privacy, so I ditched it and left it on the OBS screen, as though he was about to start streaming on twitch. I felted the RFID sensor inside of the bottom of his head so that his head would activate the sensor. I threaded a wire through him and attached an alligator clip to fishing line from above, so that people could attach it to that and have him rest on the cozy shape, watching himself on the TV. The sensor range was very small, however, making it a little finicky and not the experience I had hoped.

Here’s my todo/sketch board I used for a lot of the install:

I added the LED strips late in the install. There are kits you can buy for ambient lighting that sits behind a flatscreen TV. You plug in your HDMI cable and it takes the video signal and translates some of the colors to the LED strips. I know there are ways to make them with raspberry pis or with arduinos, but i was running out of time and wanted the effect of the colors in the COZYSPACE also changing the color of the 3D printed shapes, so I bought the kit from amazon. It plugged in and worked well enough right away for my purposes. I draped the LEDs a little haphazardly around the install in a way that I thought looked good for my purposes.

Setting up took MUCH longer than I had anticipated. I spent about 2 and a half days of work just blacking out the space, setting everything up, and hanging the cozy shapes. 

I was a little worried I wouldn’t have enough time for everything to get done, but I ended up setting up everything I wanted. I left programming the controllers for last because I didn’t want to spend more time in Unity after not using it for so long. I found a way to address the joystick buttons directly in the code rather than using Unity’s input manager, which was a relief. 

I also made some tweaks to the sound of cozyspace. Each of the 4 channels is playing a sub bass frequency that is slightly out of sync with the others, giving the sensation of something moving around in space around people in the center of the installation. I made the frequencies closer, which made the movement feel much much slower. I also made the looping track twice as long, adding some slight movement to the tracks, and reset everything to G flat major.

The actual open studios was a very educational experience, hah. I learned that people need to be told when you can touch the art. I assumed (incorrectly) that bean bags and video game controllers would be enough, but I had to invite people to interact with it. In the future I’ll have signage with my name and info about the work, even for an open studios.

https://youtube.com/shorts/PYVX6z-7OGc

https://youtu.be/XffYEqUQ8qc

https://youtu.be/y0zdaRWFfSc

[forastic]

Jam session project

UCSC DANM grad school winter 2023 - week 6 (feb19 - feb25)

The prompt for this project was to create a gift for someone that you could not deliver. After deliberating about making something really sad or angsty, I decided to make a project I’ve had in the back of my mind for a while. 

I’ve had dreams where I can pick up a guitar and play it effortlessly - my every pluck and strum sounding beautiful. I’m not a guitar player. I decided to make this interactive project for myself when I’m dreaming - a gift I cannot give.

I was unable to get a guitar, unfortunately, so I settled on two keyboards and a microphone. I created an ableton live project with 3 instruments. One keyboard controlled a pad sound, mostly made of slightly detuned square waves with a slow attack and decay. The other keyboard controlled a lead sound, with a little bit of portamento and heavy delay effects. Both sounded a little sad and video gamey (they were influenced by disasterpeace’s work on Fez) and were made in KiloHearts Phase Plant soft synth. The microphone input had a series of compressors and EQ, as well as a little distortion and delay. It also had auto-tune on it, set to G major.

The keyboards both had midi effects on them that corrected every note press to the G major pentatonic scale.

I’ve been thinking a lot about fail states in games and player agency. Is a fail state necessary to feel like a player has agency in games? No, of course not. But how about music? Guitar hero is fun, but it’s basically advanced simon says. There’s very little room for improvisation in it - you’re doing it right or wrong. Picking up a musical instrument can be incredibly intimidating as the learning curve is steep. How much failure in music performance is necessary for a feeling of agency? I wanted to make it impossible to sound bad, but possible to sound better. I wanted the person interacting with the jam session to feel free to explore and improvise.

I set up the two keyboards and the microphone deliberately, so the participants would be required to look at each other. In jam sessions a lot of what is conveyed is through eye contact and subtle body motion. I wanted my amateur musicians to be able to intuit this just from deliberate positioning of instruments.

I also wrote on post-it notes near the two keyboards, encouraging people to play with the octave buttons built in. There were also instructions - the keyboard with the lead had a note saying “Play one note at a time! Play a cool melody!” (the instrument was set up to be monophonic). The pad keys said “Hold down 2-3 notes at a time!”

I had members of my class (and patrick) try it out. They played for a couple minutes, then switched stations.

I received some good feedback about what it felt like and how to improve. The vocals needed better monitoring - without hearing the autotune clearly the singers were still feeling as though they could fail. I really want to try this with an electric guitar, so I’m going to prioritize that next time.

Here are some videos I captured of the test:

https://youtu.be/oVpKE83XiQY

https://youtu.be/ECR9qt01Rv8

https://youtu.be/to_qYGYHo4E

[forastic]

Pure Data sound puzzle prototyping

UCSC DANM grad school winter 2023 - week 4 (jan30 - feb5)

A class has a ‘skill share’ component, where we teach a skill to one or more students. I’m teaching Pure Data and Raspberry Pis. One student in my cohort is working on escape rooms, and I started thinking about what kind of sound puzzles an escape room could have. Pure Data can activate python scripts, which can then turn on pins on Raspberry Pi boards, which means it can activate LEDs and magnetic locks. I came up with 4 ideas.

Play the right thing on the piano

A midi piano is hooked up to pd. Mtof from midi keyboard to osc. Put an object between keyboard and mtof that sorts the right notes. If wrong note play sound file. Create tab array that reads incoming notes. If in right order keep reading them, if not discard and start again. If note order is hit, play a victory sound. BONUS - and unlock maglock 1 - edit - set it to any incoming audio using pitch detection

Bird puzzle

There are 4 audio sources and 4 microphones. Each audio source (set up around a room, ideally) is playing ambient jungle noises, slightly different (idea - start at random point in audio file, OR have a huge number of ambient sounds that are triggered randomly). There is also audio of one bird that is out of place - a loon. The loon only plays out of one speaker at a time. Microphones are set up near the speakers. There is one button set up near each speaker as well. Check audio in for each of those speakers. If volume level hits a certain point AND the loon is playing out of that speaker, trigger a wing flapping noise and a few moments later start the sound at another speaker. If the loon is playing out of the speaker and the player is able to hit the button, they ‘capture’ the bird and a victory sound is played (and a maglock unlocks?)

Chu chu chu

A rhythm game type puzzle - simple beat to metronome with clapping at certain times - player has to hit it near those times in second measure

Telephone bros

There are 4 sets of headphones with single ear pieces. At each of these stations is a series of buttons. You can hear a bunch of voices coming out of each ear piece. Press and hold one button and you’ll hear one get louder and the rest get quieter. Each earpiece has a separate part of a short sentence from each voice, IE the sentence: “You need to pick up the red pot and put it in the bottom shelf of the refrigerator” - speaker 1 will be playing “You need ___ red_____it in____shelf___”. The gaps are filled in with the other speakers. The intention is that a group of players has to speak it aloud as they hear it, or that one player has a notebook and pieces it together themselves.

These make sense to me, at least. I started working on the first 2.

PIANO PUZZLE

For the piano puzzle, I chose Zelda’s Lullaby from Ocarina of Time because I played that game a bunch when I was younger. I connected a midi keyboard to Pure Data and grabbed the incoming midi values, converting them to sound using a simple oscillator and basic note start and end using delay values into the volume multiplier. I chose to make this an ‘easy mode’ puzzle, where the only notes that play audio are the 4 notes necessary to play the song. The rest of the notes play an ‘error’ sound. This was achieved using the select object. (MIDI sends the note value twice - once when pressing the note down, once when releasing. The spigot button prevented it from sending the note during the release, meaning that it only played the error sound once.)

To get the patch to recognize when I played the melody properly, I initially tried to get it to record incoming values to an array. I wasn’t sure how to properly reset the array, or how to compare it against an array with the proper values, so I abandoned the idea. I still believe that this would be the best and most accurate way to work the puzzle, but I just don’t have enough knowledge and experience with Pure Data to get it to work.

The messier way to get it to work was through a series of gates. If you played one note correctly, it ‘opened the door’ to the next gate. If you played that note correctly, the next gate opened, etc. I achieved this using a series of select and spigot objects.

Unfortunately, I couldn’t check the ‘wrong’ number at every gate (select object), because it was still checking every number that passed through. My workaround was to only have it turn off every gate if the FINAL note press was incorrect. This worked, mostly. It still occasionally sends a reset value when I don’t intend it to, and I’m not sure why. The patch mostly works though.

pianopuzzle.pd

BIRD PUZZLE

The bird puzzle is still in progress. This version is not working. 

I grabbed a (cc0) field recording of a rainforest from freesounds.org.  I edited the file 3 times, changing the start and end time and making them loop. I like working with readsf~ because it’s so straightforward compared to a tabread~, so if there’s work I can do in an audio editor first instead of Pure Data I’ll do that. I set all of them playing simultaneously from different speakers.

I chose a loon for the bird, as loons are from North America and wouldn’t normally be heard in a jungle (even though they’re associated with a jungle in public consciousness). I figured the easiest way to make it heard in one speaker rather than another was to have it playing out of all the speakers and just control the volume.

You can see that at load it should only come out of the first speaker.

I monitored the input volume of a microphone using the env~ object. I placed a spigot after that to be able to control when the microphone was actually affecting anything. If the volume goes over a certain amount, it triggers a flapping noise in that speaker, and also changes the active bird speaker to one of the other 3.

For some reason, the mic will still create a bang when the env~ value is over the desired input, even if the spigot is ‘turned off’. I’m not sure why. There are a few other quirks with how it’s working, and there aren’t ‘capture’ buttons set up yet.

If I’m going to teach this, I’ll have to work out the errors and get it functioning.

birdpuzzle.pd

[forastic]

HOTMILK card game

UCSC DANM Grad school winter 2023 - week 3 jan 23-29

In the winter of 2020 I was taking an undergrad class about making card games. I came up with an idea for a game that involved shopping at the Grocery Outlet. I was really happy with the concept. Here’s the description that I wrote at the time:

Weird Food Shit - a game about being poor and trying to cook a decent meal with limited food and resources. Each player has a limited amount of money and goes to a Grocery Outlet type store, where the food available is randomized. In the first phase they buy what they think will make a good meal with their funds, but when they get home they find that some of their cooking appliances/utensils are not working or available because of how awful their landlord is. The second phase is a trading/bartering phase, where the players can trade food items amongst themselves. The next phase involves the players describing at length what they would cook with the food they have and how they would do it with limited cooking ability. The final phase is a vote, where everyone votes for their favorite meal. The cook with the most votes wins. (Likely resources: Money, Food, Working Appliances.)

The class was more focused on systems interaction and the professor nixed the game as the final outcome was decided by performance and vote. It sat in my google docs for the entirety of a pandemic.

I was given an assignment to make a ‘recombinant’ game in a short amount of time for a class - a series of decisions that combined things together to create an unexpected outcome. My two classmates and I decided to make a game together. We spent an hour or so brainstorming, when I brought up this game that I came up with, and we agreed to work on it together.

The green on the top is when we were discussing concepts, the blue is a rough version of the game loop.

Here’s the first version of the rules:

How to play v1: Shopping Each player starts with $20 ($15? $25?). A set number of FOOD cards are dealt face up (10? 20?). The players take turns picking up items one at a time, noting the cost of each item. This number reduces their $$ amount. The players must pick up a food item as long as they have the money to buy an item on the board. This phase ends when the players can no longer buy any item on the board. Appliances 2 (or 1 or 3?) APPLIANCE cards per player are placed on the table (faceup? facedown?). Each player grabs two of the APPLIANCE cards. The first player to each card gets it. Bargaining The players have 2 minutes (1 minute?) to bargain with each other, to trade or give away FOOD cards with each other. Recipe Pitch One at a time, the players must name their recipe and describe how they plan to prepare it. They must use all of the FOOD ingredients that they have in their possession and they must use every APPLIANCE card in their ‘preparation’. In addition to the APPLIANCE cards, in their recipe each player may use one pot, one pan, utensils, cold water, plates, bowls, and a refrigerator. They may use none or all of them. Voting The players must vote on one recipe other than theirs, saying the name of the recipe aloud. The person with the most votes keeps their FOOD cards and writes down the name of their recipe. The other players discard. All players return their appliance cards to the appliance card deck, which is shuffled.

The player with the most recipes after a set number of turns must read aloud their recipe names. They are declared the winner.

BONUS: they are declared the extra-super-ultra chef if they immediately go and buy the ingredients in one of their recipes and prepare it for everyone.

to be tested! -cost of food -amount of money each turn -food cards placed during shopping phase -number of appliances placed down per player -are the appliance cards face up or face down in phase 2? -how long the bargaining phase goes -are ingredient cards discarded after every round or shuffled back in the deck -appliance cards - how many wacky vs normal? 50/50? -how many turns? -purpose of meal? Include that ppl need to eat it? -leave out price and just have set number of food cards per player?

I was initially really wanting to make the game commentary about poverty - the experience of shopping at grocery outlet with only a little bit of money is really common to me. The food cost money, but then when you take it home and find out what appliances work the value shifts. If you can’t cook what you took home it might suddenly be much less valuable to you. I wanted to play with the player’s sunk cost fallacy and how that informs decisions. When a person is really poor, it can be harder for them to give up things that they spent more money on, even when they don’t have any use for them anymore. I wanted the game to point to that.

That, however, was difficult to test and wasn’t very fun. We removed the money altogether and changed the requirements for the players to have 4 cards each, and require them to use all ingredients and all appliances. This made for a more fun experience, and required much less testing.

We made plenty of cards. The quantity of cards was very necessary for the experience. Here are the current food cards. Elliot did most of the art.

And here are the current appliance cards.

Here are the current rules.

How to play HOTMILK - a party card game for 3-5 players

You and your neighbors all have to go shopping and prepare a meal! The nearest grocery store only has a few items in stock, and you're never sure what you're going to get. And when you get home with your groceries, most of your appliances aren't working! You only have two to prepare your dish, and you're never sure which two. Can you make the best meal out of all of your friends?

1) Shopping

Place a number of FOOD cards face-up on the table (4 per player). Choose a player to go first. The players take turns picking up FOOD items one at a time, clockwise. This phase ends when there are no longer any cards on the table. Each player should have 4 cards.

2) Appliances All APPLIANCE cards are placed face down on the table in a loose pile. Each player takes two cards from the pile without looking at them until they are in their hand.

3) Bargaining

The players have 2 minutes to trade or give away FOOD cards with each other.

4) Recipe Pitch Take a moment to consider what recipe you would like to create. One at a time, the players must name their recipe and describe how they plan to prepare it. They must use all of the FOOD ingredients that they have in their possession and they must use every APPLIANCE card in their ‘preparation’. In addition to the APPLIANCE cards, in their recipe each player may use one pot, one pan, utensils, cold water, plates, bowls, glasses, and a refrigerator.

5) Voting Each player must vote on the recipe that they would most like to try, saying the name of the recipe aloud. Each player cannot vote on their own recipe. The person with the most votes keeps their FOOD cards and writes down the name of their recipe. The other players discard. All players discard their appliance cards. The winner goes first in the next Shopping round. The discarded FOOD and APPLIANCE cards are not used again until the deck runs out or the game ends.The player with the most recipes after 3 turns must read aloud every winning recipe as courses for dinner. They are declared the winner. BONUS: they are declared the extra-super-ultra chef if they immediately go and buy the ingredients in one of their recipes and prepare it for everyone.

This delivered the cleanest experience for 3 players. 

For the future: We did not test 4 or 5, so if we were to continue that is one thing we would have to test. Here’s a short list of notes for potential changes:

—notes— -do we live together? Are we sharing the dish? -does refrigerator include freezer? -specify utensils? -use of the word dinner -marker that indicates first player that rotates - spatula?? -give something to players that lost? -option to replace appliance cards?

-food allergies? Vegetarians?   -frame as making food for someone else?   -‘what recipe you would most like to try’

-change language of food usage to ‘must use some or all of food - don’t need to use all

-money on cards - bonus if you make the ‘thriftiest’ meal -bonus points for wacky appliances

-make it so everyone can swap out 1 appliance

-person who goes last is stuck with last card - how to cycle that -tiebreakers? —

Expanding to more people: after a certain number of players - vote for 2 per round?

Gameshow - cook at all costs

Limit number of people - 5 ppl max??

Themed rounds? Where is the food being eaten? With whom? - impress the boss, funeral, meet the parents, 3 year old birthday party

People make their own food cards?

Be mindful about poverty simulator

Money - abstract it for longevity?

Interest in bargaining round?? -remove as step, indicate it as additional thing if ppl want

If more people - Ppl work in pairs? Each player makes their own dish?

Where to take next?

-card design and polish if going for commercial

-polish game design

We don’t currently plan on working on it any more - we all started work on other projects. It’s really fun and a pretty solid game, so if we were to polish it I believe it could get popular.

[forastic]

COZYSPACE

Patrick and I made COZYSPACE.

It’s my first year of grad school and I was confused about what I was supposed to be working on, how I was supposed to be working on it...

The digital arts and new media mfa that I’m in is like an understaffed makerspace inside a university.

I had an idea for a much edgier, weirder thing to make for school. I realized after working through it and talking about it that I was making it for the program, for the university, for other people. Undergrad basically ruined me psychologically and what I really needed was to chill out. I came up with an art project about me chilling out: a 4 channel ambient music with 3D shapes bouncing around, and a Super Nintendo controller that can affect them. I started building it in Unity as a proof of concept.

There’s a church in Santa Cruz at the center of a series of roads in concentric circles.

In the center of those circles is a church, and that church is getting torn down (to make space for housing). The owner of the church gave access to the building to a local arts collective until it gets destroyed. That collective, Liminal Space, put a call out to artists for art, installations, and murals, for a one-night party called Northern Lights that doubles as a send-off for the church. Patrick and I heard about it and decided to bring COZYSPACE to life there, first.

I was planning on building a final version in Unreal 5, but learning the software was too much of a learning curve for me in the time constraints, so I continued to build it in Unity.

My first build I used a package of ‘low poly clouds’ as the shapes. Here’s a video of the first build of COZYSPACE.

The shapes weren’t cozy though. I actually really liked the flat surfaces and hard edges, but it didn’t fit the theme. I asked Patrick to design the shapes, so he built them in VR using a sculpting program.

I changed the motion slightly and mapped the controls to a Super Nintendo controller. After adding controls and a ridiculous amount of bloom in the high definition render pipeline, it looked like this.

It was projected onto the ceiling with 2 projectors playing the same visuals, slightly overlapping. The vignette removed the hard edge of the projectors, giving the installation a more organic feeling. The person interacting is able to change the speed of the objects and the rate of rotation. They are also able to add and remove objects, both large and small. When removing large objects there is a particle effect that appears as either slow motion fireworks or a weird fuzzy phasing jittery thing, which just happens when you change some settings in the trails. The lights are also changeable. There are 3 lights at different points in the scene. While holding a button they all rotate around the center of the scene. Another button shifts the hues of all 3 lights.

My desire was to give the user control over both elements of sound and game at the same time, but it turns out getting quadrophonic sound to work in Unity is a nightmare on Windows. Unity defaults to the Windows audio driver, which didn’t work great with the Focusrite Scarlett 8i6 audio interface that I was using. In other software I was able to process the audio through ASIO, and that worked great with the audio interface. There were plugins to get Unity working with ASIO, but they were old and kind of confusing and expensive.

I ended up running the audio through Ableton Live for the first installation, mixing the tracks down to wav files to make it easier on my processor. They were about a 2 minute loop, but they were simple and background-y enough that they didn’t get old being in the space. Here’s a link to the ones that I used for the first installation. A nearby installation had ambient music that was ‘all black keys’ so I retuned my ambient stuff to g flat major. I actually have a much better more seamless method for looping, but I didn’t implement it this time because I ran out of time and kinda forgot about it. This means that the beginning of each loop gave a little bit of unintended punctuation.

Patrick built projector stands so that they would shoot out at angles that worked best with the space and projectors.

We worked together on painting the space, using Patrick’s designs from BEARPAD. He freehanded a couple of bubs on the walls and I added some blobby colors around them. 

We also projected the classic HANG 10 design onto one end of the room, painting the lines where they fell to give it an optical illusion that only looked right from one spot. Otherwise the lines were skewed.

We bought some rugs from my favorite thrift store, Grey Bears, and cut out fun blobby shapes. The remnants of the rugs were nailed down to the edges of the room. We took one of the doors outside and Patrick spray painted on them, experimenting with freehand lines with spray paint for the first time. There was a couch and a chair in one room downstairs that we hauled up to the room. We also sourced a couple of oversized bean bags from a friend up in San Francisco. They’re honestly enormous and almost didn’t fit in my car.

The event went really well! Here’s some pictures and a video compilation.

The projectors provided almost all of the light for the space, so the person controlling the game environment also ended up controlling the lighting in the room as well. I have a list of notes and potential changes from that night that I hope to get to someday. Since I’m considering doing this for my thesis I’ll have a lot to work with.