#Kagi

[mostlysignssomeportents]

Google is (still) losing the spam wars to zombie news-brands

I'm touring my new, nationally bestselling novel The Bezzle! Catch me TONIGHT (May 3) in CALGARY, then TOMORROW (May 4) in VANCOUVER, then onto Tartu, Estonia, and beyond!

Even Google admits – grudgingly – that it is losing the spam wars. The explosive proliferation of botshit has supercharged the sleazy "search engine optimization" business, such that results to common queries are 50% Google ads to spam sites, and 50% links to spam sites that tricked Google into a high rank (without paying for an ad):

https://developers.google.com/search/blog/2024/03/core-update-spam-policies#site-reputation

It's nice that Google has finally stopped gaslighting the rest of us with claims that its search was still the same bedrock utility that so many of us relied upon as a key piece of internet infrastructure. This not only feels wildly wrong, it is empirically, provably false:

https://downloads.webis.de/publications/papers/bevendorff_2024a.pdf

Not only that, but we know why Google search sucks. Memos released as part of the DOJ's antitrust case against Google reveal that the company deliberately chose to worsen search quality to increase the number of queries you'd have to make (and the number of ads you'd have to see) to find a decent result:

https://pluralistic.net/2024/04/24/naming-names/#prabhakar-raghavan

Google's antitrust case turns on the idea that the company bought its way to dominance, spending the some of the billions it extracted from advertisers and publishers to buy the default position on every platform, so that no one ever tried another search engine, which meant that no one would invest in another search engine, either.

Google's tacit defense is that its monopoly billions only incidentally fund these kind of anticompetitive deals. Mostly, Google says, it uses its billions to build the greatest search engine, ad platform, mobile OS, etc that the public could dream of. Only a company as big as Google (says Google) can afford to fund the R&D and security to keep its platform useful for the rest of us.

That's the "monopolistic bargain" – let the monopolist become a dictator, and they will be a benevolent dictator. Shriven of "wasteful competition," the monopolist can split their profits with the public by funding public goods and the public interest.

Google has clearly reneged on that bargain. A company experiencing the dramatic security failures and declining quality should be pouring everything it has to righting the ship. Instead, Google repeatedly blew tens of billions of dollars on stock buybacks while doing mass layoffs:

https://pluralistic.net/2024/02/21/im-feeling-unlucky/#not-up-to-the-task

Those layoffs have now reached the company's "core" teams, even as its core services continue to decay:

https://qz.com/google-is-laying-off-hundreds-as-it-moves-core-jobs-abr-1851449528

(Google's antitrust trial was shrouded in secrecy, thanks to the judge's deference to the company's insistence on confidentiality. The case is moving along though, and warrants your continued attention:)

https://www.thebignewsletter.com/p/the-2-trillion-secret-trial-against

Google wormed its way into so many corners of our lives that its enshittification keeps erupting in odd places, like ordering takeout food:

https://pluralistic.net/2023/02/24/passive-income/#swiss-cheese-security

Back in February, Housefresh – a rigorous review site for home air purifiers – published a viral, damning account of how Google had allowed itself to be overrun by spammers who purport to provide reviews of air purifiers, but who do little to no testing and often employ AI chatbots to write automated garbage:

https://housefresh.com/david-vs-digital-goliaths/

In the months since, Housefresh's Gisele Navarro has continued to fight for the survival of her high-quality air purifier review site, and has received many tips from insiders at the spam-farms and Google, all of which she recounts in a followup essay:

https://housefresh.com/how-google-decimated-housefresh/

One of the worst offenders in spam wars is Dotdash Meredith, a content-farm that "publishes" multiple websites that recycle parts of each others' content in order to climb to the top search slots for lucrative product review spots, which can be monetized via affiliate links.

A Dotdash Meredith insider told Navarro that the company uses a tactic called "keyword swarming" to push high-quality independent sites off the top of Google and replace them with its own garbage reviews. When Dotdash Meredith finds an independent site that occupies the top results for a lucrative Google result, they "swarm a smaller site’s foothold on one or two articles by essentially publishing 10 articles [on the topic] and beefing up [Dotdash Meredith sites’] authority."

Dotdash Meredith has keyword swarmed a large number of topics. from air purifiers to slow cookers to posture correctors for back-pain:

https://housefresh.com/wp-content/uploads/2024/05/keyword-swarming-dotdash.jpg

The company isn't shy about this. Its own shareholder communications boast about it. What's more, it has competition.

Take Forbes, an actual news-site, which has a whole shadow-empire of web-pages reviewing products for puppies, dogs, kittens and cats, all of which link to high affiliate-fee-generating pet insurance products. These reviews are not good, but they are treasured by Google's algorithm, which views them as a part of Forbes's legitimate news-publishing operation and lets them draft on Forbes's authority.

This side-hustle for Forbes comes at a cost for the rest of us, though. The reviewers who actually put in the hard work to figure out which pet products are worth your money (and which ones are bad, defective or dangerous) are crowded off the front page of Google and eventually disappear, leaving behind nothing but semi-automated SEO garbage from Forbes:

https://twitter.com/ichbinGisele/status/1642481590524583936

There's a name for this: "site reputation abuse." That's when a site perverts its current – or past – practice of publishing high-quality materials to trick Google into giving the site a high ranking. Think of how Deadspin's private equity grifter owners turned it into a site full of casino affiliate spam:

https://www.404media.co/who-owns-deadspin-now-lineup-publishing/

The same thing happened to the venerable Money magazine:

https://moneygroup.pr/

Money is one of the many sites whose air purifier reviews Google gives preference to, despite the fact that they do no testing. According to Google, Money is also a reliable source of information on reprogramming your garage-door opener, buying a paint-sprayer, etc:

https://money.com/best-paint-sprayer/

All of this is made ten million times worse by AI, which can spray out superficially plausible botshit in superhuman quantities, letting spammers produce thousands of variations on their shitty reviews, flooding the zone with bullshit in classic Steve Bannon style:

https://escapecollective.com/commerce-content-is-breaking-product-reviews/

As Gizmodo, Sports Illustrated and USA Today have learned the hard way, AI can't write factual news pieces. But it can pump out bullshit written for the express purpose of drafting on the good work human journalists have done and tricking Google – the search engine 90% of us rely on – into upranking bullshit at the expense of high-quality information.

A variety of AI service bureaux have popped up to provide AI botshit as a service to news brands. While Navarro doesn't say so, I'm willing to bet that for news bosses, outsourcing your botshit scams to a third party is considered an excellent way of avoiding your journalists' wrath. The biggest botshit-as-a-service company is ASR Group (which also uses the alias Advon Commerce).

Advon claims that its botshit is, in fact, written by humans. But Advon's employees' Linkedin profiles tell a different story, boasting of their mastery of AI tools in the industrial-scale production of botshit:

https://housefresh.com/wp-content/uploads/2024/05/Advon-AI-LinkedIn.jpg

Now, none of this is particularly sophisticated. It doesn't take much discernment to spot when a site is engaged in "site reputation abuse." Presumably, the 12,000 googlers the company fired last year could have been employed to check the top review keyword results manually every couple of days and permaban any site caught cheating this way.

Instead, Google is has announced a change in policy: starting May 5, the company will downrank any site caught engaged in site reputation abuse. However, the company takes a very narrow view of site reputation abuse, limiting punishments to sites that employ third parties to generate or uprank their botshit. Companies that produce their botshit in-house are seemingly not covered by this policy.

As Navarro writes, some sites – like Forbes – have prepared for May 5 by blocking their botshit sections from Google's crawler. This can't be their permanent strategy, though – either they'll have to kill the section or bring it in-house to comply with Google's rules. Bringing things in house isn't that hard: US News and World Report is advertising for an SEO editor who will publish 70-80 posts per month, doubtless each one a masterpiece of high-quality, carefully researched material of great value to Google's users:

https://twitter.com/dannyashton/status/1777408051357585425

As Navarro points out, Google is palpably reluctant to target the largest, best-funded spammers. Its March 2024 update kicked many garbage AI sites out of the index – but only small bottom-feeders, not large, once-respected publications that have been colonized by private equity spam-farmers.

All of this comes at a price, and it's only incidentally paid by legitimate sites like Housefresh. The real price is borne by all of us, who are funneled by the 90%-market-share search engine into "review" sites that push low quality, high-price products. Housefresh's top budget air purifier costs $79. That's hundreds of dollars cheaper than the "budget" pick at other sites, who largely perform no original research.

Google search has a problem. AI botshit is dominating Google's search results, and it's not just in product reviews. Searches for infrastructure code samples are dominated by botshit code generated by Pulumi AI, whose chatbot hallucinates nonexistence AWS features:

https://www.theregister.com/2024/05/01/pulumi_ai_pollution_of_search/

This is hugely consequential: when these "hallucinations" slip through into production code, they create huge vulnerabilities for widespread malicious exploitation:

https://www.theregister.com/2024/03/28/ai_bots_hallucinate_software_packages/

We've put all our eggs in Google's basket, and Google's dropped the basket – but it doesn't matter because they can spend $20b/year bribing Apple to make sure no one ever tries a rival search engine on Ios or Safari:

https://finance.yahoo.com/news/google-payments-apple-reached-20-220947331.html

Google's response – laying off core developers, outsourcing to low-waged territories with weak labor protections and spending billions on stock buybacks – presents a picture of a company that is too big to care:

https://pluralistic.net/2024/04/04/teach-me-how-to-shruggie/#kagi

Google promised us a quid-pro-quo: let them be the single, authoritative portal ("organize the world’s information and make it universally accessible and useful"), and they will earn that spot by being the best search there is:

https://www.ft.com/content/b9eb3180-2a6e-41eb-91fe-2ab5942d4150

But – like the spammers at the top of its search result pages – Google didn't earn its spot at the center of our digital lives.

It cheated.

If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:

https://pluralistic.net/2024/05/03/keyword-swarming/#site-reputation-abuse

Image: freezelight (modified) https://commons.wikimedia.org/wiki/File:Spam_wall_-_Flickr_-_freezelight.jpg

CC BY-SA 2.0 https://creativecommons.org/licenses/by-sa/2.0/deed.en

[planetpanini]

SRL Musicology here, we’re thrilled to announce the long-awaited return of Wet floor! They’ve brung their ink-fueled tunes from inkopolis to the splatlands.

[whipped-bean]

10 minute doodles from a big random wheel of splatoon lol

also closeups below!!

ALSO I LOVE KAGI i will be drawing him again.

and yes, i changed craig's eyes cause they freak me out

[clickbaitdemo]

Inkoming News!

[riipentry]

wet floor human designs that i forgot to post here... whoops

+ ethnicity hcs

[mechanical-magician]

[thegentlemancollects]

[survivetoread]

Learned about a new translator on the block, called Kagi Translate. Unlike Kagi's search engine, this one is free to use.

It's AI-powered unfortunately, but it does support a massive array of languages and it does a shockingly good job of translating nuance.

For example, I put in part of my recent poetry into both Google and Kagi, and Kagi provided not only more accurate and natural translation, but it retained poetic flair.

Text:

तिचा चेंडू कधी इथल्या गटारात हरवलेला, आता या चौकात तिला थांबायला वेळ नाही या शहरात कुठेही जाण्यासाठी लढावं लागतं, तरीच पावलांचं संगीत ऐकायला वेळ नाही

Google:

Ever lost her ball in the gutter here, now she has no time to wait in this square One has to fight to get anywhere in this city, yet there is no time to listen to the music of footsteps

Kagi:

Her ball once lost in the gutter here, now she has no time to stop at this corner In this city, you have to fight to go anywhere, yet there's no time to listen to the music of footsteps

[justn0t]

Kagiiiiiiiii

[techniktagebuch]

29. September 2024

Endlich nur noch ein einziges Suchfeld (und ich weiß immer, welcher Tag heute ist)

Ungefähr seit 2008 hatten alle meine Handys eine nicht loswerdbares, unbrauchbares Google-Suchfeld auf dem Startbildschirm. Irgendwann nach 2017 habe ich mir zusätzlich ein zweites, besser funktionierendes Suchfeld eingerichtet und das schlechtere der beiden aus meiner Wahrnehmung ausgeblendet. Abgesehen von einem kurzen Zwischenspiel mit anderen Suchmaschinen im Jahr 2021 waren es immer zwei Google-Suchfelder, nur dass eins davon besser funktionierte als das andere. Im März 2024 habe ich Google durch Kagi ersetzt und bin mit dieser Entscheidung weiterhin sehr zufrieden, auch wenn sie mich $10 im Monat kostet. Seitdem gab es also oben eine vorinstallierte und von mir ignorierte Google-Suchzeile und unten eine selbst eingerichtete Kagi-Suchzeile.

Vor ein paar Wochen hat mein Handy angefangen, mich mit einer nicht loswerdbaren Benachrichtigung zu nerven: Ich soll mich für eine Suchmaschine entscheiden. Die App dieser Suchmaschine wird dann installiert. Kagi ist nicht in der Liste der Optionen enthalten, und "ich will keine von denen, lasst mich in Ruhe" auch nicht. Zuerst habe ich die Benachrichtigung ignoriert und gehofft, dass sie irgendwann von allein verschwindet. Heute stört sie mich so, dass ich zuerst recherchiere, ob das Problem vielleicht gelöst ist (ergebnislos) und mich dann eben für eine der vorgeschlagenen Suchmaschinen entscheide. Ich wähle MetaGer, weil ich sie von ganz früher kenne und es eine nichtkommerzielle Option ist, aber vor allem, weil ich gelesen habe, dass die MetaGer-Ära gerade zu Ende geht. Zur Sicherheit lösche ich MetaGer-App außerdem sofort wieder.

Wo früher meine Google-Suchleiste war, ist jetzt immer noch eine Suchleiste, aber sie zeigt ein "hier wird erst noch irgendwas geladen"-Symbol an. Mir egal, ich habe gelernt, diese Stelle auf meinem Handy zu ignorieren.

Als ich das Handy das nächste Mal in die Hand nehme, sieht es anders aus als bisher:

Also nicht so verpixelt, das war ich selbst. Aber oben, wo immer die unbrauchbare Suchzeile war, steht jetzt zum ersten Mal etwas Nützliches, nämlich das Datum. Unten ist meine selbstgemachte Suchzeile zu sehen (Kagi in Firefox). Danke, europäische Gesetzgebung!

(Kathrin Passig)

[dotted-clouds]

very silly thing I thought to draw

[mostlysignssomeportents]

Too big to care

I'm on tour with my new, nationally bestselling novel The Bezzle! Catch me in BOSTON with Randall "XKCD" Munroe (Apr 11), then PROVIDENCE (Apr 12), and beyond!

Remember the first time you used Google search? It was like magic. After years of progressively worsening search quality from Altavista and Yahoo, Google was literally stunning, a gateway to the very best things on the internet.

Today, Google has a 90% search market-share. They got it the hard way: they cheated. Google spends tens of billions of dollars on payola in order to ensure that they are the default search engine behind every search box you encounter on every device, every service and every website:

https://pluralistic.net/2023/10/03/not-feeling-lucky/#fundamental-laws-of-economics

Not coincidentally, Google's search is getting progressively, monotonically worse. It is a cesspool of botshit, spam, scams, and nonsense. Important resources that I never bothered to bookmark because I could find them with a quick Google search no longer show up in the first ten screens of results:

https://pluralistic.net/2024/02/21/im-feeling-unlucky/#not-up-to-the-task

Even after all that payola, Google is still absurdly profitable. They have so much money, they were able to do a $80 billion stock buyback. Just a few months later, Google fired 12,000 skilled technical workers. Essentially, Google is saying that they don't need to spend money on quality, because we're all locked into using Google search. It's cheaper to buy the default search box everywhere in the world than it is to make a product that is so good that even if we tried another search engine, we'd still prefer Google.

This is enshittification. Google is shifting value away from end users (searchers) and business customers (advertisers, publishers and merchants) to itself:

https://pluralistic.net/2024/03/05/the-map-is-not-the-territory/#apor-locksmith

And here's the thing: there are search engines out there that are so good that if you just try them, you'll get that same feeling you got the first time you tried Google.

When I was in Tucson last month on my book-tour for my new novel The Bezzle, I crashed with my pals Patrick and Teresa Nielsen Hayden. I've know them since I was a teenager (Patrick is my editor).

We were sitting in his living room on our laptops – just like old times! – and Patrick asked me if I'd tried Kagi, a new search-engine.

Teresa chimed in, extolling the advanced search features, the "lenses" that surfaced specific kinds of resources on the web.

I hadn't even heard of Kagi, but the Nielsen Haydens are among the most effective researchers I know – both in their professional editorial lives and in their many obsessive hobbies. If it was good enough for them…

I tried it. It was magic.

No, seriously. All those things Google couldn't find anymore? Top of the search pile. Queries that generated pages of spam in Google results? Fucking pristine on Kagi – the right answers, over and over again.

That was before I started playing with Kagi's lenses and other bells and whistles, which elevated the search experience from "magic" to sorcerous.

The catch is that Kagi costs money – after 100 queries, they want you to cough up $10/month ($14 for a couple or $20 for a family with up to six accounts, and some kid-specific features):

https://kagi.com/settings?p=billing_plan&plan=family

I immediately bought a family plan. I've been using it for a month. I've basically stopped using Google search altogether.

Kagi just let me get a lot more done, and I assumed that they were some kind of wildly capitalized startup that was running their own crawl and and their own data-centers. But this morning, I read Jason Koebler's 404 Media report on his own experiences using it:

https://www.404media.co/friendship-ended-with-google-now-kagi-is-my-best-friend/

Koebler's piece contained a key detail that I'd somehow missed:

When you search on Kagi, the service makes a series of “anonymized API calls to traditional search indexes like Google, Yandex, Mojeek, and Brave,” as well as a handful of other specialized search engines, Wikimedia Commons, Flickr, etc. Kagi then combines this with its own web index and news index (for news searches) to build the results pages that you see. So, essentially, you are getting some mix of Google search results combined with results from other indexes.

In other words: Kagi is a heavily customized, anonymized front-end to Google.

The implications of this are stunning. It means that Google's enshittified search-results are a choice. Those ad-strewn, sub-Altavista, spam-drowned search pages are a feature, not a bug. Google prefers those results to Kagi, because Google makes more money out of shit than they would out of delivering a good product:

https://www.theverge.com/2024/4/2/24117976/best-printer-2024-home-use-office-use-labels-school-homework

No wonder Google spends a whole-ass Twitter every year to make sure you never try a rival search engine. Bottom line: they ran the numbers and figured out their most profitable course of action is to enshittify their flagship product and bribe their "competitors" like Apple and Samsung so that you never try another search engine and have another one of those magic moments that sent all those Jeeves-askin' Yahooers to Google a quarter-century ago.

One of my favorite TV comedy bits is Lily Tomlin as Ernestine the AT&T operator; Tomlin would do these pitches for the Bell System and end every ad with "We don't care. We don't have to. We're the phone company":

https://snltranscripts.jt.org/76/76aphonecompany.phtml

Speaking of TV comedy: this week saw FTC chair Lina Khan appear on The Daily Show with Jon Stewart. It was amazing:

https://www.youtube.com/watch?v=oaDTiWaYfcM

The coverage of Khan's appearance has focused on Stewart's revelation that when he was doing a show on Apple TV, the company prohibited him from interviewing her (presumably because of her hostility to tech monopolies):

https://www.thebignewsletter.com/p/apple-got-caught-censoring-its-own

But for me, the big moment came when Khan described tech monopolists as "too big to care."

What a phrase!

Since the subprime crisis, we're all familiar with businesses being "too big to fail" and "too big to jail." But "too big to care?" Oof, that got me right in the feels.

Because that's what it feels like to use enshittified Google. That's what it feels like to discover that Kagi – the good search engine – is mostly Google with the weights adjusted to serve users, not shareholders.

Google used to care. They cared because they were worried about competitors and regulators. They cared because their workers made them care:

https://www.vox.com/future-perfect/2019/4/4/18295933/google-cancels-ai-ethics-board

Google doesn't care anymore. They don't have to. They're the search company.

If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:

https://pluralistic.net/2024/04/04/teach-me-how-to-shruggie/#kagi

[dailyraya]

Raya and her entire family, playing at the Tree of Life's outskirts 🌳

(Art by @frogietcomics)

[nightwalker6200]

Hirano: In another life, I would have really liked doing laundry and taxes with you. Kagiura: ... Kagiura: Okay, but what about this life?

[splatband-headcanons]

i like to think kagi is the one singing those synth-y vocals in inkoming...

[riipentry]

wet floor sleepover !!!!

+ closeups