#application extension

[comradekarin]

when you’re in a “shipping random white m|m/w|m ships to spite the bwwm ship” and your competition is every fandom ever like you guys fucking suck lmaoooo suddenly it’s no chemistry! heteronormativity! he’s gay and she’s a lesbian! platonic only! poor acting! she is an independent woman! even the elite ships aren’t safe for as soon as the girl looks physically different than their (usually book) canon counterpart, the people are up in arms. like suck my dick

[mearcatsreturns]

why is applying for jobs the worst thing on the planet

[teetle-time]

from a doylist/logistical standpoint, i get why a lot of the fiddling with dimensions we get tends to stay within the set boundaries of a given iteration. just look how they massacred the 87!gang, especially in the 2012 xovers. heck, they just slapped together some new lore for 87!krang all retcon style!

but the idea of having a big reveal where a character already baked into a modern iteration turns out to be not just a counterpart of a previous version of the character, but that actual character themself, having traveled through dimensions to end up in the new iteration…you gotta admit, if it got done properly, it'd be so cool.

ope! this villain swore vengeance as their fancy schmancy machinery malfunctioned and seemingly vaporized everything? huh. well they look to be gone but we'll keep our eyes peeled for their return…except they never come back. maybe they were planned to return, maybe not. maybe there was time in the show/comic's run to address it, maybe it got cancelled before they got a chance. it just seems like a plot thread that never got properly taken care of.

smash cut to another iteration. the local version of the villain in question has been fairly close to the source material thus far, but they've been working on something In Secret™, eh? whoopsie-daisies, it's straight-up just the dude from before, slowly building their power back up! their new and improved evil plan is revealed: they intend on taking out the local turtles before plunging into the multiverse to return home and eliminate their turtles! MWAHAHAHAHA!

yes, it would probably be a nightmare to coordinate any returning VAs or to figure out how to account for VAs who maybe can't reprise their roles. yes, it would probably suck balls to try and do multiple artstyles justice at the same time, be they animated, comic book, or being brought from one to the other. yes, trying to balance two different turtleverses' "vibes" by striking a happy medium between the two without completely wrecking anybody's characterization would be (and clearly has been) notoriously difficult.

but man, that would be such a fun watch. and potentially even a good way for canon catharsis for the older iteration, if the villain in question was one the fans already liked.

[jcmarchi]

Virtual Personas for Language Models via an Anthology of Backstories

New Post has been published on https://thedigitalinsider.com/virtual-personas-for-language-models-via-an-anthology-of-backstories/

Virtual Personas for Language Models via an Anthology of Backstories

<!–

We introduce Anthology, a method for conditioning LLMs to representative, consistent, and diverse virtual personas by generating and utilizing naturalistic backstories with rich details of individual values and experience.

–>

We introduce Anthology, a method for conditioning LLMs to representative, consistent, and diverse virtual personas by generating and utilizing naturalistic backstories with rich details of individual values and experience.

What does it mean for large language models (LLMs) to be trained on massive text corpora, collectively produced by millions and billions of distinctive human authors?

In “Language Models as Agent Models”, compelling evidence suggests that recent language models could be considered models of agents: provided with a textual context, LLMs are capable of generating conditional text that represents the characteristics of an agent likely to have produced that context. This suggests that, with appropriate conditioning, LLMs could be guided to approximate the responses of a particular human voice, rather than the mixture of voices that otherwise emerges. If realized, this capability of LLMs would have significant implications for user research and social sciences—conditioned language models as virtual personas of human subjects could serve as cost-effective pilot studies and supporting best practices in human studies, e.g. the Belmont principles of justice and beneficence.

In this work, we introduce Anthology, an approach for steering LLMs to representative, consistent, and diverse virtual personas by providing richly detailed life narratives of individuals as conditioning context to models. In doing so, we also present methods to generate backstories from LLMs themselves as a means to efficiently produce massive sets covering a wide range of human demographics. By grounding language models in naturalistic backstories, Anthology allows LLMs to simulate individual human samples with increased fidelity, measured in terms of matching the distributions and consistencies of human responses.

Our Approach: Anthology

Conditioning Language Model Generation with Individual Life Narratives

A significant limitation of earlier methods in steering LLMs to virtual personas has been the inability to reliably approximate individual human samples. Prior approaches prompt LLMs with broad demographic information, e.g., “I am a 25-year-old from California. My highest level of education is less than high school,” which are essentially bodies of text generated from a tuple of demographic variables. With these methods, we are only able to approximate human samples at a population level, not at the individual level, which results in:

Responses prone to LLMs defaulting to stereotypical and/or prototypical portrayals, as they are only conditioned on demographic variables (e.g., race and gender)

Inability to provide important metrics of interest such as covariance and statistical significance, as individual responses are required for such compuatations

Anthology enables the approximation of individual subjects by conditioning with richly detailed backstories. Through these backstories, the model captures implicit and explicit markers of personal identity, including demographic traits and spontaneous references to cultural, socioeconomic backgrounds, and life philosophies. Our approach involves generating a vast set of backstories representing a wide range of demographic attributes via language models queried with unrestricted, open-ended prompts such as, “Tell me about yourself.” We then match virtual personas conditioned by each backstory to real-world survey samples.

Results: Closer Approximation of Public Opinion Polls

For evaluation, we compare the effectiveness of different methods for conditioning virtual personas in the context of approximating three Pew Research Center ATP surveys: Waves 34, 92, and 99.

Results on approximating human responses for Pew Research Center ATP surveys. Boldface and underlined results indicate values closest and the second closest to those of humans, respectively.

As measures of success in approximating human samples with virtual personas, we consider the following metrics:

Average Wasserstein distance (WD) between response distributions as a measure of representativeness

Frobenius norm (Fro.) between correlation matrices as a measure of consistency

Cronbach’s alpha as an additional measure of internal consistency

Prior to analyzing virtual subjects, we estimate the lower bounds of each evaluation metric by repeatedly dividing the human population into two equal-sized groups at random and calculating these metrics between the subgroups. We take averaged values from 100 iterations to represent the lower-bound estimates.

We consistently observe that Anthology outperforms other conditioning methods with respect to all metrics, for both the Llama-3-70B and the Mixtral-8x22B. When comparing two matching methods, the greedy matching method tends to show better performance on the average Wasserstein distance across all Waves. We attribute differences in matching methods to the one-to-one correspondence condition of maximum weight matching and the limited number of virtual users available. Specifically, the weights assigned to matched virtual subjects in maximum weight matching are inevitably lower than those in greedy matching, as the latter relaxes the constraints on one-to-one correspondence. This discrepancy can result in a lower demographic similarity between matched human and virtual users compared to the counterpart from greedy matching. These results suggest that the richness of the generated backstories in our approach elicits more nuanced responses compared to baselines.

Final Thoughts

Anthology marks a promising new direction in conditioning virtual personas in LLMs that could potentially reshape how we conduct user research, public opinion surveys, and other social science applications by offering a scalable, and at times, ethical alternative to traditional human surveys. However, the use of Anthology, as in any other application of language models in the social sciences, also brings several considerations to the forefront: although the generated backstories help create more representative personas, there remains a risk of perpetuating biases or infringing on privacy, so results should be used and interpreted with caution.

In terms of future steps, we envision our approach benefiting from a more expansive and diverse set of backstories, each representing a consistent life narrative of individuals. Additionally, a valuable extension of the work would be to consider free-form response generation, enabling more natural and nuanced persona simulations beyond structured survey formats such as multiple-choice. Finally, an exciting next dimension in applying LLMs in behavioral studies would involve simulating longer-term effects, allowing virtual personas to model and retrospectively examine changes over time.

All of these directions present multitudes of technical challenges; please let us know if you are interested in collaborating or want to discuss our work further!

Learn more about our work: link to full paper

@articlemoon2024virtual, title=Virtual personas for language models via an anthology of backstories, author=Moon, Suhong and Abdulhai, Marwa and Kang, Minwoo and Suh, Joseph and Soedarmadji, Widyadewi and Behar, Eran Kohen and Chan, David M, journal=arXiv preprint arXiv:2407.06576, year=2024

[arodrwho]

uses of generative ai i think it makes sense to get mad about or be appalled by:

creating any kind of art, especially if then trying to build an audience using it and/or trying to make money off it

completing homework assignments

completing work tasks that significantly impact other people

completing work tasks that could significantly impact your professional reputation & career

any kind of research / treating it like a search engine

plugging unfinished fics into it to get an ending

editing writing of any kind

uses i think it doesnt make any sense to get all up in arms about:

writing emails

[pochapal]

part of me wonders if there was an intentional seeking out of beatrice (ie searching for The Witch and not The Person Behind The Witch) that she might actually let herself be found

[smartratework]

pvc electric red and blue wire #smartratework#tumblr

[kamil-a]

probably the first like, 2 or 3 aerolith ais were unfortunately bigoted in various ways by human standards because if youre a human being programming computers to Advance The Human Race your biases are gonna come thru loud and clear right. but by the time youre up to the like agent 8-10 series thats very like autonomous of personality. i think the current crop of computers is like im NOT SEXIST!!!!!!!!!! OKAY!!!!!! but i have compartmentalized humans in ways you didnt know existed. i can show you biases your feeble brain wouldnt be able to comprehend

[rainia]

HATE emailing people but even more hate emailing known unhelpful people like girllll this is such a lost cause frfr

[thelordofshrimp]

the weird noises means it's working

[softskiesahead]

this song and this song have the most intense angsty chiaya energy possible I think

[adriles]

when a soldier enters my tent it means either the commanders are trying to get me to rejoin the war effort for the 10000th time or the worst possible impediment to my returning to phthia has occured

[the-travelling-witch]

do i really have to buy skin care, hair care and new jeans all at once bc i procrastinated on it? okay, i guess

that high pitched noise you’re hearing is my wallet crying

[vulpinesaint]

what's the desktop site you use for sticky notes? i randomly stumbled across your post and now im curious

k so i used to use this one for chrome (it worked pretty separately from the browser though i hate sticky note extensions that are stuck to the page or only operational through your bookmarks. killing and biting i want to be able to have them open over other apps) and this morning i downloaded this one off the app store for my macbook (works the same except like. kind of better. tbh). so glad you could be with me on this journey hope this is applicable to your life at all <3

[seilon]

just wrote like three paragraphs ranting about my living situation and deleted it just know I am going insane and i hate it here and I need to live by my fucking self or I am going to absolutely fucking lose it

[benevolentbucky]

job 😎