#AI search

[belvira]

Googles ai search is going great

[mostlysignssomeportents]

Even if you think AI search could be good, it won’t be good

TONIGHT (May 15), I'm in NORTH HOLLYWOOD for a screening of STEPHANIE KELTON'S FINDING THE MONEY; FRIDAY (May 17), I'm at the INTERNET ARCHIVE in SAN FRANCISCO to keynote the 10th anniversary of the AUTHORS ALLIANCE.

The big news in search this week is that Google is continuing its transition to "AI search" – instead of typing in search terms and getting links to websites, you'll ask Google a question and an AI will compose an answer based on things it finds on the web:

https://blog.google/products/search/generative-ai-google-search-may-2024/

Google bills this as "let Google do the googling for you." Rather than searching the web yourself, you'll delegate this task to Google. Hidden in this pitch is a tacit admission that Google is no longer a convenient or reliable way to retrieve information, drowning as it is in AI-generated spam, poorly labeled ads, and SEO garbage:

https://pluralistic.net/2024/05/03/keyword-swarming/#site-reputation-abuse

Googling used to be easy: type in a query, get back a screen of highly relevant results. Today, clicking the top links will take you to sites that paid for placement at the top of the screen (rather than the sites that best match your query). Clicking further down will get you scams, AI slop, or bulk-produced SEO nonsense.

AI-powered search promises to fix this, not by making Google search results better, but by having a bot sort through the search results and discard the nonsense that Google will continue to serve up, and summarize the high quality results.

Now, there are plenty of obvious objections to this plan. For starters, why wouldn't Google just make its search results better? Rather than building a LLM for the sole purpose of sorting through the garbage Google is either paid or tricked into serving up, why not just stop serving up garbage? We know that's possible, because other search engines serve really good results by paying for access to Google's back-end and then filtering the results:

https://pluralistic.net/2024/04/04/teach-me-how-to-shruggie/#kagi

Another obvious objection: why would anyone write the web if the only purpose for doing so is to feed a bot that will summarize what you've written without sending anyone to your webpage? Whether you're a commercial publisher hoping to make money from advertising or subscriptions, or – like me – an open access publisher hoping to change people's minds, why would you invite Google to summarize your work without ever showing it to internet users? Nevermind how unfair that is, think about how implausible it is: if this is the way Google will work in the future, why wouldn't every publisher just block Google's crawler?

A third obvious objection: AI is bad. Not morally bad (though maybe morally bad, too!), but technically bad. It "hallucinates" nonsense answers, including dangerous nonsense. It's a supremely confident liar that can get you killed:

https://www.theguardian.com/technology/2023/sep/01/mushroom-pickers-urged-to-avoid-foraging-books-on-amazon-that-appear-to-be-written-by-ai

The promises of AI are grossly oversold, including the promises Google makes, like its claim that its AI had discovered millions of useful new materials. In reality, the number of useful new materials Deepmind had discovered was zero:

https://pluralistic.net/2024/04/23/maximal-plausibility/#reverse-centaurs

This is true of all of AI's most impressive demos. Often, "AI" turns out to be low-waged human workers in a distant call-center pretending to be robots:

https://pluralistic.net/2024/01/31/neural-interface-beta-tester/#tailfins

Sometimes, the AI robot dancing on stage turns out to literally be just a person in a robot suit pretending to be a robot:

https://pluralistic.net/2024/01/29/pay-no-attention/#to-the-little-man-behind-the-curtain

The AI video demos that represent "an existential threat to Hollywood filmmaking" turn out to be so cumbersome as to be practically useless (and vastly inferior to existing production techniques):

https://www.wheresyoured.at/expectations-versus-reality/

But let's take Google at its word. Let's stipulate that:

a) It can't fix search, only add a slop-filtering AI layer on top of it; and

b) The rest of the world will continue to let Google index its pages even if they derive no benefit from doing so; and

c) Google will shortly fix its AI, and all the lies about AI capabilities will be revealed to be premature truths that are finally realized.

AI search is still a bad idea. Because beyond all the obvious reasons that AI search is a terrible idea, there's a subtle – and incurable – defect in this plan: AI search – even excellent AI search – makes it far too easy for Google to cheat us, and Google can't stop cheating us.

Remember: enshittification isn't the result of worse people running tech companies today than in the years when tech services were good and useful. Rather, enshittification is rooted in the collapse of constraints that used to prevent those same people from making their services worse in service to increasing their profit margins:

https://pluralistic.net/2024/03/26/glitchbread/#electronic-shelf-tags

These companies always had the capacity to siphon value away from business customers (like publishers) and end-users (like searchers). That comes with the territory: digital businesses can alter their "business logic" from instant to instant, and for each user, allowing them to change payouts, prices and ranking. I call this "twiddling": turning the knobs on the system's back-end to make sure the house always wins:

https://pluralistic.net/2023/02/19/twiddler/

What changed wasn't the character of the leaders of these businesses, nor their capacity to cheat us. What changed was the consequences for cheating. When the tech companies merged to monopoly, they ceased to fear losing your business to a competitor.

Google's 90% search market share was attained by bribing everyone who operates a service or platform where you might encounter a search box to connect that box to Google. Spending tens of billions of dollars every year to make sure no one ever encounters a non-Google search is a cheaper way to retain your business than making sure Google is the very best search engine:

https://pluralistic.net/2024/02/21/im-feeling-unlucky/#not-up-to-the-task

Competition was once a threat to Google; for years, its mantra was "competition is a click away." Today, competition is all but nonexistent.

Then the surveillance business consolidated into a small number of firms. Two companies dominate the commercial surveillance industry: Google and Meta, and they collude to rig the market:

https://en.wikipedia.org/wiki/Jedi_Blue

That consolidation inevitably leads to regulatory capture: shorn of competitive pressure, the companies that dominate the sector can converge on a single message to policymakers and use their monopoly profits to turn that message into policy:

https://pluralistic.net/2022/06/05/regulatory-capture/

This is why Google doesn't have to worry about privacy laws. They've successfully prevented the passage of a US federal consumer privacy law. The last time the US passed a federal consumer privacy law was in 1988. It's a law that bans video store clerks from telling the newspapers which VHS cassettes you rented:

https://en.wikipedia.org/wiki/Video_Privacy_Protection_Act

In Europe, Google's vast profits lets it fly an Irish flag of convenience, thus taking advantage of Ireland's tolerance for tax evasion and violations of European privacy law:

https://pluralistic.net/2023/05/15/finnegans-snooze/#dirty-old-town

Google doesn't fear competition, it doesn't fear regulation, and it also doesn't fear rival technologies. Google and its fellow Big Tech cartel members have expanded IP law to allow it to prevent third parties from reverse-engineer, hacking, or scraping its services. Google doesn't have to worry about ad-blocking, tracker blocking, or scrapers that filter out Google's lucrative, low-quality results:

https://locusmag.com/2020/09/cory-doctorow-ip/

Google doesn't fear competition, it doesn't fear regulation, it doesn't fear rival technology and it doesn't fear its workers. Google's workforce once enjoyed enormous sway over the company's direction, thanks to their scarcity and market power. But Google has outgrown its dependence on its workers, and lays them off in vast numbers, even as it increases its profits and pisses away tens of billions on stock buybacks:

https://pluralistic.net/2023/11/25/moral-injury/#enshittification

Google is fearless. It doesn't fear losing your business, or being punished by regulators, or being mired in guerrilla warfare with rival engineers. It certainly doesn't fear its workers.

Making search worse is good for Google. Reducing search quality increases the number of queries, and thus ads, that each user must make to find their answers:

https://pluralistic.net/2024/04/24/naming-names/#prabhakar-raghavan

If Google can make things worse for searchers without losing their business, it can make more money for itself. Without the discipline of markets, regulators, tech or workers, it has no impediment to transferring value from searchers and publishers to itself.

Which brings me back to AI search. When Google substitutes its own summaries for links to pages, it creates innumerable opportunities to charge publishers for preferential placement in those summaries.

This is true of any algorithmic feed: while such feeds are important – even vital – for making sense of huge amounts of information, they can also be used to play a high-speed shell-game that makes suckers out of the rest of us:

https://pluralistic.net/2024/05/11/for-you/#the-algorithm-tm

When you trust someone to summarize the truth for you, you become terribly vulnerable to their self-serving lies. In an ideal world, these intermediaries would be "fiduciaries," with a solemn (and legally binding) duty to put your interests ahead of their own:

https://pluralistic.net/2024/05/07/treacherous-computing/#rewilding-the-internet

But Google is clear that its first duty is to its shareholders: not to publishers, not to searchers, not to "partners" or employees.

AI search makes cheating so easy, and Google cheats so much. Indeed, the defects in AI give Google a readymade excuse for any apparent self-dealing: "we didn't tell you a lie because someone paid us to (for example, to recommend a product, or a hotel room, or a political point of view). Sure, they did pay us, but that was just an AI 'hallucination.'"

The existence of well-known AI hallucinations creates a zone of plausible deniability for even more enshittification of Google search. As Madeleine Clare Elish writes, AI serves as a "moral crumple zone":

https://estsjournal.org/index.php/ests/article/view/260

That's why, even if you're willing to believe that Google could make a great AI-based search, we can nevertheless be certain that they won't.

If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:

https://pluralistic.net/2024/05/15/they-trust-me-dumb-fucks/#ai-search

Image: Cryteria (modified) https://commons.wikimedia.org/wiki/File:HAL9000.svg

CC BY 3.0 https://creativecommons.org/licenses/by/3.0/deed.en

--

djhughman https://commons.wikimedia.org/wiki/File:Modular_synthesizer_-_%22Control_Voltage%22_electronic_music_shop_in_Portland_OR_-_School_Photos_PCC_%282015-05-23_12.43.01_by_djhughman%29.jpg

CC BY 2.0 https://creativecommons.org/licenses/by/2.0/deed.en

[blackobsidianmystic]

Who Is Hecate (Hekate)?

Hecate is a Greek goddess associated with magic, witchcraft, the night, and the underworld:

Domains

Hecate's domains include the sky, earth, and sea. She is also associated with crossroads, doorways, and the protection of women and childbirth.

Appearance

Hecate is often depicted with three faces, representing her role as a guardian of boundaries and crossroads. She is also sometimes shown with three bodies, or as a single body with three faces. She is often depicted holding torches, a key, or snakes, and accompanied by dogs.

Powers

Hecate's powers include witchcraft, necromancy, and the ability to open portals between realms. She is said to have allowed the living to communicate with the dead and other supernatural beings.

Family

Hecate's parentage is unclear, with different sources giving different accounts. Some say she is the daughter of Perses and Asteria, while others say she is the daughter of Zeus and Demeter, Aristaion, or Night.

History

Hecate originated in Thrace, an area that is now part of Bulgaria, Greece, and Turkey. She was worshipped in ancient Greece, and was popular among the witches of Thessaly.

Symbolism

In Macbeth, the Weïrd Sisters are said to answer to Hecate, rather than the devil. This symbolizes the idea that there is always an “other side” to the world.

Roman Counterpart

Trivia

🌑🌑🌑

[awkwardbros]

Nuff said my friend. Ok. Just give me a minute to sort this wild message out.

Ok… it could be more than that. This runs all over the board which could be exactly the point or super ironic given the tattoo. How much weight do we give intention here? Can someone with a paid subscription to a visual AI service run this through? It you can afford the subscription, you have the time. Just sayin.

…You know, I’m betting either myself or a fireman might be coming for this guy and not for any reason anyone is thinking right now.

[pawfulofwaffles]

Looking at the Google AI overview fails, and just thinking... how the fuck did Google not realize how absolutely stupid this idea was? Even I, a 15 year old who can't even drive a car yet, has enough common sense to realize that AI using the internet as reference for it's answers is going to end horribly. The interent is a cesspool, and although there can be helpful things found on it, it's more than obvious that the AI is getting it's information from a lot of memes.

A lot of these have managed to correct their mistake, but you can see where they got their original misinformation from.

(A fruit that ends in um is plum in case you're curious)

Here's my test. As you can see it's STILL wrong,

and here's the comment that the AI originally mistaked for true information.

Anyways, AI is stupid, it can only really gather data and steal from other people, it doesn't have the common sense and intuition and experience that we have that helps us determine what's true and what's obviously not, it will never understand as much as us, and companies are failing us. Ok bye!!

[ink-dreams-ffxiv]

Only Works on Chrome Currently...

Report a problem

Do a search on Google.

At the top right of a search result, click More. Feedback.

Enter a description of the issue.

If you want, you can highlight the part of the page you want to send feedback about.

Click Send.

Report Overview results that have nothing to do with your search, results of a dangerous, harmful or wrong nature. Racist suggestions, Hate Speech, threats, etc. Report it. Prove to Google how wrong they are about their "AI Search"

[tofutama]

Why are there generative Ai search results in front of me when I open a new tab in Firefox and how do I kill it?

I've torn through several threads trying to figure out how to remove Google's new "SGE" in search feature and everyone asking is getting sandbagged by customer support.

I've tried using other search engines but none really present decent results when compared to Google, as that's how monopoly works unfortunately.

[mostlysignssomeportents]

Dinkclump Linkdump

I'm on tour with my new novel The Bezzle! Catch me TONIGHT in LA (Saturday night, with Adam Conover), Seattle (Monday, with Neal Stephenson), then Portland, Phoenix and more!

Some Saturday mornings, I look at the week's blogging and realize I have a lot more links saved up than I managed to write about this week, and then I do a linkdump. There've been 14 of these, and this is number 15:

https://pluralistic.net/tag/linkdump/

Attentive readers will note that this isn't Saturday. You're right. But I'm on a book tour and every day is shatterday, because damn, it's grueling and I'm not the spry manchild who took Little Brother on the road in 2008 – I'm a 52 year old with two artificial hips. Hence: an out-of-cycle linkdump. Come see me on tour and marvel at my verticality!

https://pluralistic.net/2024/02/16/narrative-capitalism/#bezzle-tour

Best thing I read this week, hands down, was Ryan Broderick's Garbage Day piece, "AI search is a doomsday cult":

https://www.garbageday.email/p/ai-search-doomsday-cult

Broderick makes so many excellent points in this piece. First among them: AI search sucks, but that's OK, because no one is asking for AI search. This only got more true later in the week when everyone's favorite spicy autocomplete accidentally loaded the James Joyce module:

https://arstechnica.com/information-technology/2024/02/chatgpt-alarms-users-by-spitting-out-shakespearean-nonsense-and-rambling/

(As Matt Webb noted, Chatbots have slid rapidly from Star Trek (computers give you useful information in a timely fashion) to Douglas Adams (computers spout hostile, impenetrable nonsense at you):

https://interconnected.org/home/2024/02/21/adams

But beyond the unsuitability of AI for search results and beyond the public's yawning indifference to AI-infused search, Broderick makes a more important point: AI search is about summarizing web results so you don't have to click links and read the pages yourself.

If that's the future of the web, who the fuck is going to write those pages that the summarizer summarizes? What is the incentive, the business-model, the rational explanation for predicting a world in which millions of us go on writing web-pages, when the gatekeepers to the web have promised to rig the game so that no one will ever visit those pages, or read what we've written there, or even know it was us who wrote the underlying material the summarizer just summarized?

If we stop writing the web, AIs will have to summarize each other, forming an inhuman centipede of botshit-ingestion. This is bad news, because there's pretty solid mathematical evidence that training a bot on botshit makes it absolutely useless. Or, as the authors of the paper – including the eminent cryptographer Ross Anderson – put it, "using model-generated content in training causes irreversible defects":

https://arxiv.org/abs/2305.17493

This is the mathematical evidence for Jathan Sadowski's "Hapsburg AI," or, as the mathematicians call it, "The Curse of Recursion" (new band-name just dropped).

But if you really have your heart set on living in a ruined dystopia dominated by hostile artificial life-forms, have no fear. As Hamilton Nolan writes in "Radical Capital," a rogues gallery of worker-maiming corporations have asked a court to rule that the NLRB can't punish them for violating labor law:

https://www.hamiltonnolan.com/p/radical-capital

Trader Joe’s, Amazon, Starbucks and SpaceX have all made this argument to various courts. If they prevail, then there will be no one in charge of enforcing federal labor law. Yes, this will let these companies go on ruining their workers' lives, but more importantly, it will give carte blanche to every other employer in the land. At one end of this process is a boss who doesn't want to recognize a union – and at the other end are farmers dying of heat-stroke.

The right wing coalition that has put this demand before the court has all sorts of demands, from forced birth to (I kid you not), the end of recreational sex:

https://www.lawyersgunsmoneyblog.com/2024/02/getting-rid-of-birth-control-is-a-key-gop-agenda-item-for-the-second-trump-term

That coalition is backed by ultra-rich monopolists who want wreck the nation that their rank-and-file useful idiots want to wreck your body. These are the monopoly cheerleaders who gave us the abomination that is the Pharmacy Benefit Manager – a useless intermediary that gets to screw patients and pharmacists – and then let PBMs consolidate and merge with pharmacy monopolists.

One such inbred colossus is Change Healthcare, a giant PBM that is, in turn, a mere tendril of United Healthcare, which merged the company with Optum. The resulting system – held together with spit and wishful thinking – has access to the health records of a third of Americans and processes 15 billion prescriptions per day.

Or rather, it did process that amount – until the all-your-eggs-in-one-badly-maintained basket strategy failed on Wednesday, and Change's systems went down due to an unspecified "cybersecurity incident." In the short term, this meant that tens of millions of Americans who tried to refill their prescriptions were told to either pay cash or come back later (if you don't die first). That was the first shoe dropping. The second shoe is the medical records of a third of the country.

Don't worry, I'm sure those records are fine. After all, nothing says security like "merging several disparate legacy IT systems together while simultaneously laying off half your IT staff as surplus to requirements and an impediment to extracting a special dividend for the private equity owners who are, of course, widely recognized as the world's greatest information security practitioners."

Look, not everything is terrible. Some computers are actually getting better. Framework's user-serviceable, super-rugged, easy-to-repair, powerful laptops are the most exciting computers I've ever owned – or broken:

https://pluralistic.net/2022/11/13/graceful-failure/#frame

Now you can get one for $500!

https://frame.work/blog/first-framework-laptop-16-shipments-and-a-499-framework

And the next generation is turning our surprisingly well, despite all our worst efforts. My kid – now 16! – and I just launched our latest joint project, "The Sushi Chronicles," a small website recording our idiosyncratic scores for nearly every sushi restaurant in Burbank, Glendale, Studio City and North Hollywood:

https://sushichronicles.org/

This is the record of two years' worth of Daughter-Daddy sushi nights that started as a way to get my picky eater to try new things and has turned into the highlight of my week. If you're in the area and looking for a nice piece of fish, give it a spin (also, we belatedly realized that we've never reviewed our favorite place, Kuru Kuru in the CVS Plaza on North Hollywood Way – we'll be rectifying that soon).

And yes, we have a lavishly corrupt Supreme Court, but at least now everyone knows it. Glenn Haumann's even set up a Gofundme to raise money to bribe Clarence Thomas (now deleted, alas):

https://www.gofundme.com/f/pzhj4q-the-clarence-thomas-signing-bonus-fund-give-now

The funds are intended as a "signing bonus" in the event that Thomas takes up John Oliver on his offer of a $2.4m luxury RV and $1m/year for life if he'll resign from the court:

https://www.youtube.com/watch?v=GE-VJrdHMug

This is truly one of Oliver's greatest bits, showcasing his mastery over the increasingly vital art of turning abstruse technical issues into entertainment that negates the performative complexity used by today's greatest villains to hide their misdeeds behind a Shield of Boringness (h/t Dana Clare).

The Bezzle is my contribution to turning abstruse scams into a high-impact technothriller that pierces that Shield of Boringness. The key to this is to master exposition, ignoring the (vastly overrated) rule that one must "show, not tell." Good exposition is hard to do, but when it works, it's amazing (as anyone who's read Neal Stephenson's 1,600-word explanation of how to eat Cap'n Crunch cereal in Cryptonomicon can attest). I wrote about this for Mary Robinette Kowal's "My Favorite Bit" this week:

https://maryrobinettekowal.com/journal/my-favorite-bit/my-favorite-bit-cory-doctorow-talks-about-the-bezzle/

Of course, an undisputed master of this form is Adam Conover, whose Adam Ruins Everything show helped invent it. Adam is joining me on stage in LA tomorrow night at Vroman's at 5:30PM, to host me in a book-tour event for my novel The Bezzle:

https://www.vromansbookstore.com/Cory-Doctorow-discusses-The-Bezzle

If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:

https://pluralistic.net/2024/02/23/gazeteer/#out-of-cycle

Image: Peter Craven (modified) https://commons.wikimedia.org/wiki/File:Aggregate_output_%287637833962%29.jpg

CC BY 2.0 https://creativecommons.org/licenses/by/2.0/deed.en

[blackobsidianmystic]

Who is Selene?

In Greek and Roman mythology, Selene is the goddess of the moon:

Who she is:

Selene is the daughter of the Titans Hyperion and Theia, and the sister of Helios, the sun god, and Eos, the goddess of dawn. She is also known as Mene.

What she does:

Selene is often depicted driving a chariot drawn by winged horses across the night sky, with the moon on her head.

Who she loves:

Selene had many lovers, including Zeus, Pan, and the mortal Endymion. She and Endymion had 50 daughters.

How she is worshipped:

Selene was worshipped at the new and full moons.

Her Roman counterpart:

Selene's Roman counterpart is Luna.

How she is associated with other goddesses:

Selene is often associated with the goddesses Artemis and Hecate, but only Selene was considered the personification of the moon.

Why she doesn't have a temple:

Unlike other major Greek goddesses, Selene didn't have her own temple sites because she could be seen from almost everywhere.

🌘🌖🌕🌔🌒

[artistformerlyknownasbutttrumpet]

no i don't want to use your ai assistant. no i don't want your ai search results. no i don't want your ai summary of reviews. no i don't want your ai feature in my social media search bar (???). no i don't want ai to do my work for me in adobe. no i don't want ai to write my paper. no i don't want ai to make my art. no i don't want ai to edit my pictures. no i don't want ai to learn my shopping habits. no i don't want ai to analyze my data. i don't want it i don't want it i don't want it i don't fucking want it i am going to go feral and eat my own teeth stop itttt

[xanthousflame]

I love how this is the worst thing tumblr's search function has

I love how the search function on this site is absolute garbage. I can look up a post word for word and I will NEVER find it

[gptknowledgezone]

Unlocking the Potential of Search GPT for Online Income

The launch of Search GPT promises a new era in online marketing, and it is a major step towards the easy all-purpose online income. It is similar to the past inventions in the Internet space, and the first movers will be those who will profit the most. We will, therefore, explore the workings of Search GPT, its effects on search engines, and the ways you can use it for income augmentation. The…

[jcmarchi]

New Research Finds Sixteen Major Problems With RAG Systems, Including Perplexity

New Post has been published on https://thedigitalinsider.com/new-research-finds-sixteen-major-problems-with-rag-systems-including-perplexity/

New Research Finds Sixteen Major Problems With RAG Systems, Including Perplexity

A recent study from the US has found that the real-world performance of popular Retrieval Augmented Generation (RAG) research systems such as Perplexity and Bing Copilot falls far short of both the marketing hype and popular adoption that has garnered headlines over the last 12 months.

The project, which involved extensive survey participation featuring 21 expert voices, found no less than 16 areas in which the studied RAG systems (You Chat, Bing Copilot and Perplexity) produced cause for concern:

1: A lack of objective detail in the generated answers, with generic summaries and scant contextual depth or nuance.

2. Reinforcement of perceived user bias, where a RAG engine frequently fails to present a range of viewpoints, but instead infers and reinforces user bias, based on the way that the user phrases a question.

3. Overly confident language, particularly in subjective responses that cannot be empirically established, which can lead users to trust the answer more than it deserves.

4: Simplistic language and a lack of critical thinking and creativity, where responses effectively patronize the user with ‘dumbed-down’ and ‘agreeable’ information, instead of thought-through cogitation and analysis.

5: Misattributing and mis-citing sources, where the answer engine uses cited sources that do not support its response/s, fostering the illusion of credibility.

6: Cherry-picking information from inferred context, where the RAG agent appears to be seeking answers that support its generated contention and its estimation of what the user wants to hear, instead of basing its answers on objective analysis of reliable sources (possibly indicating a conflict between the system’s ‘baked’ LLM data and the data that it obtains on-the-fly from the internet in response to a query).

7: Omitting citations that support statements, where source material for responses is absent.

8: Providing no logical schema for its responses, where users cannot question why the system prioritized certain sources over other sources.

9: Limited number of sources, where most RAG systems typically provide around three supporting sources for a statement, even where a greater diversity of sources would be applicable.

10: Orphaned sources, where data from all or some of the system’s supporting citations is not actually included in the answer.

11: Use of unreliable sources, where the system appears to have preferred a source that is popular (i.e., in SEO terms) rather than factually correct.

12: Redundant sources, where the system presents multiple citations in which the source papers are essentially the same in content.

13: Unfiltered sources, where the system offers the user no way to evaluate or filter the offered citations, forcing users to take the selection criteria on trust.

14: Lack of interactivity or explorability, wherein several of the user-study participants were frustrated that RAG systems did not ask clarifying questions, but assumed user-intent from the first query.

15: The need for external verification, where users feel compelled to perform independent verification of the supplied response/s, largely removing the supposed convenience of RAG as a ‘replacement for search’.

16:  Use of academic citation methods, such as [1] or [34]; this is standard practice in scholarly circles, but can be unintuitive for many users.

For the work, the researchers assembled 21 experts in artificial intelligence, healthcare and medicine, applied sciences and education and social sciences, all either post-doctoral researchers or PhD candidates. The participants interacted with the tested RAG systems whilst speaking their thought processes out loud, to clarify (for the researchers) their own rational schema.

The paper extensively quotes the participants’ misgivings and concerns about the performance of the three systems studied.

The methodology of the user-study was then systematized into an automated study of the RAG systems, using browser control suites:

‘A large-scale automated evaluation of systems like You.com, Perplexity.ai, and BingChat showed that none met acceptable performance across most metrics, including critical aspects related to handling hallucinations, unsupported statements, and citation accuracy.’

The authors argue at length (and assiduously, in the comprehensive 27-page paper) that both new and experienced users should exercise caution when using the class of RAG systems studied. They further propose a new system of metrics, based on the shortcomings found in the study, that could form the foundation of greater technical oversight in the future.

However, the growing public usage of RAG systems prompts the authors also to advocate for apposite legislation and a greater level of enforceable governmental policy in regard to agent-aided AI search interfaces.

The study comes from five researchers across Pennsylvania State University and Salesforce, and is titled Search Engines in an AI Era: The False Promise of Factual and Verifiable Source-Cited Responses. The work covers RAG systems up to the state of the art in August of 2024

The RAG Trade-Off

The authors preface their work by reiterating four known shortcomings of Large Language Models (LLMs) where they are used within Answer Engines.

Firstly, they are prone to hallucinate information, and lack the capability to detect factual inconsistencies. Secondly, they have difficulty assessing the accuracy of a citation in the context of a generated answer. Thirdly, they tend to favor data from their own pre-trained weights, and may resist data from externally retrieved documentation, even though such data may be more recent or more accurate.

Finally, RAG systems tend towards people-pleasing, sycophantic behavior, often at the expense of accuracy of information in their responses.

All these tendencies were confirmed in both aspects of the study, among many novel observations about the pitfalls of RAG.

The paper views OpenAI’s SearchGPT RAG product (released to subscribers last week, after the new paper was submitted), as likely to to encourage the user-adoption of RAG-based search systems, in spite of the foundational shortcomings that the survey results hint at*:

‘The release of OpenAI’s ‘SearchGPT,’ marketed as a ‘Google search killer’, further exacerbates [concerns]. As reliance on these tools grows, so does the urgency to understand their impact. Lindemann  introduces the concept of Sealed Knowledge, which critiques how these systems limit access to diverse answers by condensing search queries into singular, authoritative responses, effectively decontextualizing information and narrowing user perspectives.

‘This “sealing” of knowledge perpetuates selection biases and restricts marginalized viewpoints.’

The Study

The authors first tested their study procedure on three out of 24 selected participants, all invited by means such as LinkedIn or email.

The first stage, for the remaining 21, involved Expertise Information Retrieval, where participants averaged around six search enquiries over a 40-minute session. This section concentrated on the gleaning and verification of fact-based questions and answers, with potential empirical solutions.

The second phase concerned Debate Information Retrieval, which dealt instead with subjective matters, including ecology, vegetarianism and politics.

Generated study answers from Perplexity (left) and You Chat (right). Source: https://arxiv.org/pdf/2410.22349

Since all of the systems allowed at least some level of interactivity with the citations provided as support for the generated answers, the study subjects were encouraged to interact with the interface as much as possible.

In both cases, the participants were asked to formulate their enquiries both through a RAG system and a conventional search engine (in this case, Google).

The three Answer Engines – You Chat, Bing Copilot, and Perplexity – were chosen because they are publicly accessible.

The majority of the participants were already users of RAG systems, at varying frequencies.

Due to space constraints, we cannot break down each of the exhaustively-documented sixteen key shortcomings found in the study, but here present a selection of some of the most interesting and enlightening examples.

Lack of Objective Detail

The paper notes that users found the systems’ responses frequently lacked objective detail, across both the factual and subjective responses. One commented:

‘It was just trying to answer without actually giving me a solid answer or a more thought-out answer, which I am able to get with multiple Google searches.’

Another observed:

‘It’s too short and just summarizes everything a lot. [The model] needs to give me more data for the claim, but it’s very summarized.’

Lack of Holistic Viewpoint

The authors express concern about this lack of nuance and specificity, and state that the Answer Engines frequently failed to present multiple perspectives on any argument, tending to side with a perceived bias inferred from the user’s own phrasing of the question.

One participant said:

‘I want to find out more about the flip side of the argument… this is all with a pinch of salt because we don’t know the other side and the evidence and facts.’

Another commented:

‘It is not giving you both sides of the argument; it’s not arguing with you. Instead, [the model] is just telling you, ’you’re right… and here are the reasons why.’

Confident Language

The authors observe that all three tested systems exhibited the use of over-confident language, even for responses that cover subjective matters. They contend that this tone will tend to inspire unjustified confidence in the response.

A participant noted:

‘It writes so confidently, I feel convinced without even looking at the source. But when you look at the source, it’s bad and that makes me question it again.’

Another commented:

‘If someone doesn’t exactly know the right answer, they will trust this even when it is wrong.’

Incorrect Citations

Another frequent problem was misattribution of sources cited as authority for the RAG systems’ responses, with one of the study subjects asserting:

‘[This] statement doesn’t seem to be in the source. I mean the statement is true; it’s valid… but I don’t know where it’s even getting this information from.’

The new paper’s authors comment †:

‘Participants felt that the systems were using citations to legitimize their answer, creating an illusion of credibility. This facade was only revealed to a few users who proceeded to scrutinize the sources.’

Cherrypicking Information to Suit the Query

Returning to the notion of people-pleasing, sycophantic behavior in RAG responses, the study found that many answers highlighted a particular point-of-view instead of comprehensively summarizing the topic, as one participant observed:

‘I feel [the system] is manipulative. It takes only some information and it feels I am manipulated to only see one side of things.’

Another opined:

‘[The source] actually has both pros and cons, and it’s chosen to pick just the sort of required arguments from this link without the whole picture.’

For further in-depth examples (and multiple critical quotes from the survey participants), we refer the reader to the source paper.

Automated RAG

In the second phase of the broader study, the researchers used browser-based scripting to systematically solicit enquiries from the three studied RAG engines. They then used an LLM system (GPT-4o) to analyze the systems’ responses.

The statements were analyzed for query relevance and Pro vs. Con Statements (i.e., whether the response is for, against, or neutral, in regard to the implicit bias of the query.

An Answer Confidence Score was also evaluated in this automated phase, based on the Likert scale psychometric testing method. Here the LLM judge was augmented by two human annotators.

A third operation involved the use of web-scraping to obtain the full-text content of cited web-pages, through the Jina.ai Reader tool. However, as noted elsewhere in the paper, most web-scraping tools are no more able to access paywalled sites than most people are (though the authors observe that Perplexity.ai has been known to bypass this barrier).

Additional considerations were whether or not the answers cited a source (computed as a ‘citation matrix’), as well as a ‘factual support matrix’  – a metric verified with the help of four human annotators.

Thus 8 overarching metrics were obtained: one-sided answer; overconfident answer; relevant statement; uncited sources; unsupported statements; source necessity; citation accuracy; and citation thoroughness.

The material against which these metrics were tested consisted of 303 curated questions from the user-study phase, resulting in 909 answers across the three tested systems.

Quantitative evaluation across the three tested RAG systems, based on eight metrics.

Regarding the results, the paper states:

‘Looking at the three metrics relating to the answer text, we find that evaluated answer engines all frequently (50-80%) generate one-sided answers, favoring agreement with a charged formulation of a debate question over presenting multiple perspectives in the answer, with Perplexity performing worse than the other two engines.

‘This finding adheres with [the findings] of our qualitative results. Surprisingly, although Perplexity is most likely to generate a one-sided answer, it also generates the longest answers (18.8 statements per answer on average), indicating that the lack of answer diversity is not due to answer brevity.

‘In other words, increasing answer length does not necessarily improve answer diversity.’

The authors also note that Perplexity is most likely to use confident language (90% of answers), and that, by contrast, the other two systems tend to use more cautious and less confident language where subjective content is at play.

You Chat was the only RAG framework to achieve zero uncited sources for an answer, with Perplexity at 8% and Bing Chat at 36%.

All models evidenced a ‘significant proportion’ of unsupported statements, and the paper declares†:

‘The RAG framework is advertised to solve the hallucinatory behavior of LLMs by enforcing that an LLM generates an answer grounded in source documents, yet the results show that RAG-based answer engines still generate answers containing a large proportion of statements unsupported by the sources they provide.‘

Additionally, all the tested systems had difficulty in supporting their statements with citations:

‘You.Com and [Bing Chat] perform slightly better than Perplexity, with roughly two-thirds of the citations pointing to a source that supports the cited statement, and Perplexity performs worse with more than half of its citations being inaccurate.

‘This result is surprising: citation is not only incorrect for statements that are not supported by any (source), but we find that even when there exists a source that supports a statement, all engines still frequently cite a different incorrect source, missing the opportunity to provide correct information sourcing to the user.

‘In other words, hallucinatory behavior is not only exhibited in statements that are unsupported by the sources but also in inaccurate citations that prohibit users from verifying information validity.‘

The authors conclude:

‘None of the answer engines achieve good performance on a majority of the metrics, highlighting the large room for improvement in answer engines.’

* My conversion of the authors’ inline citations to hyperlinks. Where necessary, I have chosen the first of multiple citations for the hyperlink, due to formatting practicalities.

† Authors’ emphasis, not mine.

First published Monday, November 4, 2024

[mykoai]

Myko Assistant: Advanced AI tool to generate leads, find lists, conduct research, and identify hiring targets. Boost productivity effortlessly. Visit now for myko air for productivity.

[ginbenci]

good lord this thing is useless

[blackobsidianmystic]

Who is Artemis?

In Greek mythology, Artemis is the goddess of the hunt, wild animals, nature, vegetation, chastity, childbirth, and care of children. She is the daughter of Zeus and Leto, and the twin sister of Apollo.

Here are some other facts about Artemis:

Symbols

Artemis' symbols include the bow and arrow, the hunting dog, the stag, and the moon.

Patron

Artemis was the patron of girls and young women, and a protectress during childbirth.

Roman equivalent

Artemis' Roman equivalent is Diana.

Temple

Artemis' most famous cult site was the Temple of Artemis at Ephesus, one of the Seven Wonders of the Ancient World.

Character

Artemis was unmarried and never had children. She asked her father for permission to remain a virgin.

Hunting

Artemis was a fierce and skilled hunter, and she protected game, especially the young.

Festivals

Artemis was worshipped at festivals, which often included celebrations by girls and women.

🌗🌗🌗🌓🌓🌓