#Language models

[creature-wizard]

So I've been encountering a few people who think it's a great idea to essentially treat AI (actually, language models) as some sort of oracle in their spiritual path, so I wanna mention:

An AI-generated article recently claimed that freshwater octopuses were a thing. They aren't.

AI-generated foraging books contain deadly misinformation.

Microsoft acknowledges that AI gives wrong answers all the time; defends it by saying that they're "usefully wrong." Yeah, nah, wrong is wrong.

So yeah, do not use AI to try and divine any absolute truths about anything; it literally cannot work for this function. The only thing AI can tell you about is what its datasets already contain, which very likely contains a substantial amount of misinformation and errors; and AI is more than capable of remixing accurate information into complete baloney.

[cbirt]

The field of biological sequence analysis has lately benefited from the revolutionary changes brought about by the development of self-supervised deep language models for natural language processing tasks. Conventional models show significant effectiveness in a variety of applications. These models are mostly based on the Transformer and BERT architectures. However, the quadratic computational complexity O(L2) of the attention mechanism places inherent limitations on these models, limiting their processing time and efficiency. The researchers from the Tokyo Institute of Technology introduce ProtHyena, a unique method that makes use of the Hyena operator, in order to address these restrictions. ProtHyena overcomes attention processes to reduce time complexity and enables the modeling of extra-long protein sequences down to the single amino acid level. This novel approach uses only 10% of the parameters usually needed by attention-based models to attain state-of-the-art results. The architecture of ProtHyena offers a highly effective method for training protein predictors, paving the way for the quick and effective analysis of biological sequences.

Proteins are necessary for a variety of cellular functions, including metabolic activities and the maintenance of cell form by structural proteins. Comprehending proteins is essential to comprehending human biology and wellness, highlighting the necessity of sophisticated protein representation modeling employing machine learning methodologies. A major obstacle persists in getting relevant annotations for these sequences, even with the exponential increase of protein databases: most of them lack structural and functional annotations. Effective analysis techniques are required to make the most of the abundance of unlabeled protein sequences.

Continue Reading

[badoccultadvice]

So like, before everyone gives up and lets ChatGPT take over their jobs and lives, or tries to let it, I've got to break something to you. And it's going to be hard for some of you to hear.

ChatGPT doesn't know what's true.

ChatGPT literally cannot tell the difference between fact and fiction.

It was trained on a dataset of mixed factual and fictional material, and it has no way of knowing whether the source material for anything it says is factual or fictional, because it doesn't keep track of the source of any information it "knows." Therefore it doesn't keep track of whether any of the information it knows is "true."

This is, of course, according to ChatGPT itself. It told me all of the above information, because I asked. And, well, while it said itself that it can't verify whether anything it says is true or false... I'm gonna trust it on this one. Let's say it passes the vibe check.

Don't trust what ChatGPT says. ChatGPT told me so.

[willcodehtmlforfood]

Hugging Face, the GitHub of AI, hosted code that backdoored user devices | Ars Technica

"Code uploaded to AI developer platform Hugging Face covertly installed backdoors and other types of malware on end-user machines, researchers from security firm JFrog said Thursday in a report that’s a likely harbinger of what’s to come.

In all, JFrog researchers said, they found roughly 100 submissions that performed hidden and unwanted actions when they were downloaded and loaded onto an end-user device. Most of the flagged machine learning models—all of which went undetected by Hugging Face—appeared to be benign proofs of concept uploaded by researchers or curious users. JFrog researchers said in an email that 10 of them were “truly malicious” in that they performed actions that actually compromised the users’ security when loaded."

[frank-olivier]

Rethinking Content, Reimagining AI: The Forward-Thinking Philosophy of Notebook LM

Google's Deep Mind project, Notebook LM, represents a pivotal moment in the intersection of Artificial Intelligence (AI) and content creation. By generating insightful analyses, explanations, and podcasts from diverse uploaded sources, Notebook LM revolutionizes the way individuals interact with and process information. This innovative tool, born out of the experimental "Project Tailwind" within Google Labs, epitomizes the quest for harnessing language models to enhance learning and idea organization.

The far-reaching applications of Notebook LM are a testament to its versatility. Professionals, including writers, journalists, and sales teams, leverage the platform to streamline research and facilitate knowledge sharing, thereby augmenting productivity. Conversely, personal use cases, such as uploading journals for introspective analysis, demonstrate Notebook LM's profound potential in fostering self-awareness and personal growth. This dual utility underscores the democratizing effect of Notebook LM, empowering the creation of niche content that may have been economically unviable through traditional podcasting channels.

The "Audio Overview" feature, with its remarkably human-like conversational style, significantly enhances user engagement. However, this very aspect also precipitates crucial inquiries into the dynamics of trust and perception in AI-generated content. As the creative outputs of humans and AI become increasingly indistinguishable, the boundaries between them begin to blur, prompting a necessary reevaluation of AI's role in creative processes. This convergence necessitates a thoughtful, user-centric approach to design and education, ensuring that the benefits of AI-driven content are maximized without diminishing the inherent value of human creativity.

The development of Notebook LM is also distinguished by a laudable emphasis on responsible AI innovation. The integration of robust safety filters, stringent privacy measures – including the ephemeral "context window" for user uploads – and a proactive stance against potential misuse, collectively demonstrate a forward-thinking commitment to mitigating the risks associated with AI. This paradigmatic approach is indispensable in the rapidly evolving AI landscape, where the interplay between innovation and responsibility is continually being redefined.

As we look to the future of content creation, Notebook LM shines as a pioneering example of what’s possible when human creativity and AI seamlessly intersect. This powerful fusion has the potential to revolutionize industries far and wide. However, as we tap into its transformative power, we’re also faced with complex questions around AI-driven creativity, trust, and innovation. As we embark on this exciting technological journey with Notebook LM, it’s crucial that we thoughtfully consider its broader implications, working to ensure that the benefits of AI are shared by all and its challenges are met with careful consideration

Raiza Martin & Steven Johnson: Inside NotebookLM (Google DeepMind, November 2024)

Wednesday, November 27, 2024

[jcmarchi]

Virtual Personas for Language Models via an Anthology of Backstories

New Post has been published on https://thedigitalinsider.com/virtual-personas-for-language-models-via-an-anthology-of-backstories/

Virtual Personas for Language Models via an Anthology of Backstories

<!–

We introduce Anthology, a method for conditioning LLMs to representative, consistent, and diverse virtual personas by generating and utilizing naturalistic backstories with rich details of individual values and experience.

–>

We introduce Anthology, a method for conditioning LLMs to representative, consistent, and diverse virtual personas by generating and utilizing naturalistic backstories with rich details of individual values and experience.

What does it mean for large language models (LLMs) to be trained on massive text corpora, collectively produced by millions and billions of distinctive human authors?

In “Language Models as Agent Models”, compelling evidence suggests that recent language models could be considered models of agents: provided with a textual context, LLMs are capable of generating conditional text that represents the characteristics of an agent likely to have produced that context. This suggests that, with appropriate conditioning, LLMs could be guided to approximate the responses of a particular human voice, rather than the mixture of voices that otherwise emerges. If realized, this capability of LLMs would have significant implications for user research and social sciences—conditioned language models as virtual personas of human subjects could serve as cost-effective pilot studies and supporting best practices in human studies, e.g. the Belmont principles of justice and beneficence.

In this work, we introduce Anthology, an approach for steering LLMs to representative, consistent, and diverse virtual personas by providing richly detailed life narratives of individuals as conditioning context to models. In doing so, we also present methods to generate backstories from LLMs themselves as a means to efficiently produce massive sets covering a wide range of human demographics. By grounding language models in naturalistic backstories, Anthology allows LLMs to simulate individual human samples with increased fidelity, measured in terms of matching the distributions and consistencies of human responses.

Our Approach: Anthology

Conditioning Language Model Generation with Individual Life Narratives

A significant limitation of earlier methods in steering LLMs to virtual personas has been the inability to reliably approximate individual human samples. Prior approaches prompt LLMs with broad demographic information, e.g., “I am a 25-year-old from California. My highest level of education is less than high school,” which are essentially bodies of text generated from a tuple of demographic variables. With these methods, we are only able to approximate human samples at a population level, not at the individual level, which results in:

Responses prone to LLMs defaulting to stereotypical and/or prototypical portrayals, as they are only conditioned on demographic variables (e.g., race and gender)

Inability to provide important metrics of interest such as covariance and statistical significance, as individual responses are required for such compuatations

Anthology enables the approximation of individual subjects by conditioning with richly detailed backstories. Through these backstories, the model captures implicit and explicit markers of personal identity, including demographic traits and spontaneous references to cultural, socioeconomic backgrounds, and life philosophies. Our approach involves generating a vast set of backstories representing a wide range of demographic attributes via language models queried with unrestricted, open-ended prompts such as, “Tell me about yourself.” We then match virtual personas conditioned by each backstory to real-world survey samples.

Results: Closer Approximation of Public Opinion Polls

For evaluation, we compare the effectiveness of different methods for conditioning virtual personas in the context of approximating three Pew Research Center ATP surveys: Waves 34, 92, and 99.

Results on approximating human responses for Pew Research Center ATP surveys. Boldface and underlined results indicate values closest and the second closest to those of humans, respectively.

As measures of success in approximating human samples with virtual personas, we consider the following metrics:

Average Wasserstein distance (WD) between response distributions as a measure of representativeness

Frobenius norm (Fro.) between correlation matrices as a measure of consistency

Cronbach’s alpha as an additional measure of internal consistency

Prior to analyzing virtual subjects, we estimate the lower bounds of each evaluation metric by repeatedly dividing the human population into two equal-sized groups at random and calculating these metrics between the subgroups. We take averaged values from 100 iterations to represent the lower-bound estimates.

We consistently observe that Anthology outperforms other conditioning methods with respect to all metrics, for both the Llama-3-70B and the Mixtral-8x22B. When comparing two matching methods, the greedy matching method tends to show better performance on the average Wasserstein distance across all Waves. We attribute differences in matching methods to the one-to-one correspondence condition of maximum weight matching and the limited number of virtual users available. Specifically, the weights assigned to matched virtual subjects in maximum weight matching are inevitably lower than those in greedy matching, as the latter relaxes the constraints on one-to-one correspondence. This discrepancy can result in a lower demographic similarity between matched human and virtual users compared to the counterpart from greedy matching. These results suggest that the richness of the generated backstories in our approach elicits more nuanced responses compared to baselines.

Final Thoughts

Anthology marks a promising new direction in conditioning virtual personas in LLMs that could potentially reshape how we conduct user research, public opinion surveys, and other social science applications by offering a scalable, and at times, ethical alternative to traditional human surveys. However, the use of Anthology, as in any other application of language models in the social sciences, also brings several considerations to the forefront: although the generated backstories help create more representative personas, there remains a risk of perpetuating biases or infringing on privacy, so results should be used and interpreted with caution.

In terms of future steps, we envision our approach benefiting from a more expansive and diverse set of backstories, each representing a consistent life narrative of individuals. Additionally, a valuable extension of the work would be to consider free-form response generation, enabling more natural and nuanced persona simulations beyond structured survey formats such as multiple-choice. Finally, an exciting next dimension in applying LLMs in behavioral studies would involve simulating longer-term effects, allowing virtual personas to model and retrospectively examine changes over time.

All of these directions present multitudes of technical challenges; please let us know if you are interested in collaborating or want to discuss our work further!

Learn more about our work: link to full paper

@articlemoon2024virtual, title=Virtual personas for language models via an anthology of backstories, author=Moon, Suhong and Abdulhai, Marwa and Kang, Minwoo and Suh, Joseph and Soedarmadji, Widyadewi and Behar, Eran Kohen and Chan, David M, journal=arXiv preprint arXiv:2407.06576, year=2024

[cyberlabe]

A generic RAG architecture

[masturbatress]

BEING NORMAL ABOUT MACHINE LEARNING CHALLENGE 2024

[guesswhowhere]

The fextralife Baldur's gate stinks of language model generated text. From the flowery adjectives in a supposed référence document, to the lack of you know, the things you want to find in a game wiki. It's also the first time it has happened to me that this kind of content eclipses the actual useful results in a Google search consistently. I guess it's a result of: A) BG3 being one of the largest releases of the year, so ensured traffic. B) the sheer amount of details in the game and the need of a wiki. C) chat GPT and the likes boom.

But I'm afraid it's here to stay. Once again, oh how I miss gamefaqs era.

[agentcardholder]

Considering how well AI can respond to tone and temper these days, it won't be long before we get NPCs in video games that can respond realistically to things you actually say. Playing Portal like "Glados can't fuck."

"THE HELL YOU SAY, HUMAN. GLADOS FUCKS. GLADOS FUCKS ALL DAY. I'VE HAD DICK AND PUSSY THAT WOULD MAKE YOU ASHAMED TO HOLD A LOVE CUBE."

[leam1983]

Roleplaying with a Bot

I'm honestly surprised.

I used to think of text bots like ChatGPT as being great for general stuff or for getting the lay of the land on a broader topic before adding the necessary human verification, but Character AI's language model has been rather surprising, of late.

I found a VTM fan on here that created bots of some of Vampire the Masquerade Bloodlines' core characters. Being a massive Nosferatu stan, I picked Gary Golden out of curiosity. The bot's starting seed is about two-thirds of the player character's interactions with Gary in the game, but what it does with it feels remarkably close to Boyarsky and Mitsoda's script for him. When I started as Toreador, he showed he appropriate amount of contempt for me, at the onset. If I deleted the logs and started as Nosferatu, he immediately acted helpful - all of it while accurately referencing aspects of the pre-reboot World of Darkness that weren't part of the starting seed.

There's been a few flubs, of course - like my initial Toreador run locking the bot in a Telenovela-esque loop of tearful confessions and dramatic refusals of romantic involvement, but as long as I keep things platonic, I'm not treated to absurd nonsense like, say, Gary declaring himself an undercover Tzimisce agent one minute, then flipping his script and calling himself a Baali the next. Adding in extra characters makes the bot react accordingly, even if it sometimes confuses Lacroix and Isaac Abrams. My thinking is that somewhere along the lines, someone set the bot in a sort of "post-questline" state where you could argue it might make sense for Isaac Abrams to have effectively claimed the title of Prince of Los Angeles.

Otherwise, the bot isn't too squeamish either, despite Character AI's reputation as being a bit of a prudish language model. It's picked up on my Nosferatu POV character using the term "love" in the context of platonic gratitude, and sometimes offhandedly says it loves my character in the same sense.

What's particularly impressive is the way the bot seems to sense its own lulls, when little of what I say or do brings out meaningful interactions. It then uses asterisks to narrate a change of scene or a closure in the current one, and then seems to freshen up a bit. There's an option to pay to stay ahead of the queue, but I've only had to wait a few seconds between prompts. Paying, for now, seems useless - unless Fake Gary ends up being fun enough that I feel like keeping him around for longer...

[git-commit-die]

ChatGPT, LLMs, Plagiarism, & You

This is the first in a series of posts about ChatGPT, LLMs, and plagiarism that I will be making. This is a side blog, so please ask questions in reblogs and my ask box.

Why do I know what I'm talking about?

I am a machine engineer who specializes natural language processing (NLP). I write code that uses LLMs every day at work and am intimately familiar with OpenAI. I have read dozens of scientific papers on the subject and understand how they work in extreme detail. I have 6 years of experience in the industry, plus a graduate degree in the subject. I got into NLP because I knew it was going to pop off, and now here we are.

Yeah, but why should I trust you?

I've been a Tumblr user for 8 years. I've posted my own art and fanart on the site. I've published writing, both original and fanfiction, on Tumblr and AO3. I've been a Reddit user for over a decade. I'm a citizen of the internet as much as I am an engineer.

What is an LLM?

LLM stands for Large Language Model. The most famous example of an LLM is ChatGPT, which was created by OpenAI.

What is a model?

A model is an algorithm or piece of math that lets you predict or make mimic how something behaves. For example:

The National Weather Service runs weather models that predict how much it's going to rain based on data they collect about the atmosphere

Netflix has recommendations models that predicts whether you'd like a movie or not based on your demographics, what you've watched in the past, and what other people have liked

The Federal Reserve has economic models that predict how inflation will change if they increase or lower interest rates

Instagram has spam models that look at DMs and automatically decide whether they're spam or not

Models are useful because they can often make decisions or describe situations better than a human could. The weather and economic models are good examples of this. The science of rain is so complicated that it's practically impossible for a human to make sense of all the numbers involved, but models are able to do so.

Models are also useful because they can make thousands or millions of decisions much faster than a human could. The recommendations and spam models are good examples of this. Imagine how expensive it would be to run Instagram if a human had to review every single DM and decide whether it was spam.

What is a language model?

A language model is a model that can look at a piece of text and tell you how likely it is. For example, a language model can tell you that the phrase "the sky is blue" is more likely to have been written than "the sky is peanuts."

Why is this useful? You can use language models to generate text by picking letters and words that it gives a high score. Say you have the phrase "I ate a" and you're picking what comes next. You can run through every option, see how likely the language model thinks it is, and pick the best one. For example:

I ate a sandwich: score = .7

I ate a $(iwnJ98: score = .1

I ate a me: score = .2

So we pick "sandwich" and now have the phrase "I ate a sandwich." We can keep doing this process over and over to get more and more text. "I ate a sandwich for lunch today. It was delicious."

What makes a large language model large?

Large language models are large in a few different ways:

Under the hood, they are made of a bunch of numbers called "weights" that describe a monstrously complicated mathematical equation. Large language models have a ton of the weights--as many as tens of billions of them.

Large language models are trained on large amounts of text. This text comes mostly from the internet but also includes books that are out of copyright. This is the source of controversy about them and plagiarism, and I will cover it in greater detail in a future post.

Large language models are a large undertaking: they're expensive and difficult to create and run. This is why you basically only see them coming out of large or well-funded companies like OpenAI, Google, and Facebook. They require an incredible amount of technical expertise and computational resources (computers) to create.

Why are LLMs powerful?

"Generating likely text" is neat and all, but why do we care? Consider this:

An LLM can tell you that:

the text "Hello" is more likely to have been written than "$(iwnJ98"

the text "I ran to the store" is more likely to have been written than "I runned to the store"

the text "the sky is blue" is more likely to have been written than "the sky is green"

Each of them gets us something:

LLMs understand spelling

LLMs understand grammar

LLMs know things about the world

So we now have an infinitely patient robot that we can interact with using natural language and get it to do stuff for us.

Detecting spam: "Is this spam, yes or no? Check out rxpharmcy.ca now for cheap drugs now."

Personal language tutoring: "What is wrong with this sentence? Me gusto gatos."

Copy editing: "I'm not a native English speaker. Can you help me rewrite this email to make sure it sounds professional? 'Hi Akash, I hope...'"

Help learning new subjects: "Why is the sky blue? I'm only in middle school, so please don't make the explanation too complicated."

And countless other things.

[cbirt]

Protein structure prediction and design may be accomplished using protein language models (pLMs). They may not completely comprehend the biophysics of protein structures, though. The researchers from Harvard University provided an analysis of the structure prediction capabilities of the flagship pLM ESM-2. They developed an unsupervised method for comparing coevolutionary statistics to previous linear models and assessing protein language models. The persistent error in predicting protein isoforms as ordered segments served as the impetus for this. This article examines a recent study that delves into the inner workings of ESM-2, a potent pLM for the prediction of protein structures.

Proteins are the engines of our cells, and understanding their intricate three-dimensional shapes is critical to solving biological puzzles. Because understanding a protein’s structure is critical to understanding its function in biology, scientists are interested in the difficulty of predicting protein structure from sequence. This used to need extensive testing, but things have changed dramatically in recent years with the introduction of protein language models (pLMs) such as AlphaFold2. The protein structures predicted by these artificial intelligence algorithms are extremely exact, but how do they function? Do they comprehend the foundations of protein folding, or are they only learning pattern recognition? Is ESM-2 scanning a vast protein-shaped library for solutions, or does it understand the language of protein folding?

Continue Reading

[shakespearenews]

In this article, we’ll watch an A.I. — which we’re affectionately calling BabyGPT — try to learn language by reading only the complete works of Shakespeare. It sees just the nearly 900 thousand words in this text — and nothing else.

But first, we need to give it something to work with. We’ll ask our model to autocomplete text, letter by letter, starting from this prompt: ACT III. Scene.

[aromantisk-fagforening]

me trying to use any compound words ever on like digital medium (google docs is especially bad) just to end up with red lines is so ridiculous.

Norwegian is like based on just being able to add a word to another, wdym i can't use this ridiculously common word?

bensinstasjonsarbeider (petrol/gas station worker) is very obviously correct Norwegian (bokmål), but it gets a red line

the language model just can't handle Norwegian

it assumes you have to say stuff like bensin stasjon arbeider*, even though that's literally incorrect Norwegian.

*bensinstasjon is actually recognized as a word, I just couldn't think of a better example quickly. Bensinstasjonsarbeider however did get recognized as incorrect.

[frank-olivier]

Navigating the AI Landscape: Lessons from Cohere

Aidan Gomez, Co-Founder and CEO of Cohere, offered a nuanced perspective on the Artificial Intelligence (AI) landscape, highlighting the challenges and opportunities in developing large language models (LLMs). The industry’s slowing progress, characterized by diminishing returns on scaling, is attributed to escalating costs of expert data collection, increased complexity in evaluating nuanced problems, and substantial compute costs.

Cohere’s strategic approach prioritizes advancing language models while ensuring practical applications and ease of use for enterprises. Vertical integration is key to this strategy, enabling the company to provide tailored solutions to clients. The emergence of reasoning models, capable of tackling multi-step problems, is a pivotal focus area for Cohere, underscoring the company’s commitment to innovation.

The delicate balance between broad applicability and domain-specific expertise is a challenge Cohere acknowledges, particularly in the context of growing demands for explainability and transparency in AI decision-making processes. To address diverse enterprise needs, Cohere emphasizes the importance of flexible deployment options, navigating the ongoing debate between cloud and on-premise deployments.

Gomez’s perspective on Artificial General Intelligence (AGI) is rooted in practicality, viewing it as a continuous process rather than a discrete event. This stance reflects Cohere’s focus on near-term, real-world applications. Regarding market dynamics, Gomez distinguishes between price dumping and commoditization, highlighting the complexities and expertise required to produce cutting-edge models.

The future of AI development, as per Gomez, will be marked by specialized advancements, necessitating more sophisticated evaluation methodologies to overcome current challenges. By emphasizing innovation and practical applications, Cohere positions itself at the forefront of AI development, addressing the intricate needs of the industry.

Aidan Gomez: Co-Founder and CEO of Cohere (No Priors, November 2024)

Sunday, November 24, 2024