#notebookLM

[thecurioustale]

I got Google's new AI toy to generate an entire podcast discussing a scene from my book.

I read with much enjoyment and pleasure of nostalebraist's recent experiences with Google's experimental new NotebookLM, which will take a piece of writing that you give it and generate an entire conversation where a pair of AI-generated hosts get into a "deep dive" discussion about it, in the style of a discussion podcast.

I listened to the "The Northern Caves" and "sufficiently advanced" podcasts and was similarly impressed at how far the mimicry of human speech patterns has come.

While it is in the experimental stage, NotebookLM is free to use. All you need is a Google account. That's a pretty low barrier to entry—and I have just enough vanity and masochism synergy to try this out for myself with my own work!

Let's do a deep dive, shall we?! =D

I had Google whip up a "deep dive" of a piece of my own writing that I shared earlier this year for the ATH 25th anniversary, a scene from The Great Galavar, set many years before ATH where a much younger Silence, new to Sele, asks Galavar for a desk and they go to Javelin's furniture shop to see about getting one. (I only fed Google this scene; not the attached scene where Javelin visits Galavar later that night.)

Here it is:

I'll start off with a complaint that spans the entirety of the podcast: I was put off by the disingenuous way that both podcast hosts acted as if they hadn't read the work under discussion and expected the other to explain it to them, because they keep switching these two hats back and forth (proving very conspicuously that they both know the work very closely), and it comes across as patronizing. I noticed, however, that I found this more off-putting with the podcasts nostalgebraist shared than with my own.

And a couple more quick complaints:

First off, there's no recognition of Silence's left-handedness, so frankly I think we can throw these toasters onto the scrap heap for another 20 years of development. 😏

(Actually, stick a pin in that; I'll come back to it later.)

Second off, I forgive these AI hosts for their horrible butchery of the pronunciation of custom names like "Terlais" and "Sele." 😭😭😭 That's understandable and not off-putting (though it does jar).

But mainly:

I am very impressed by the the AI's ability to synthesize an understanding of the material with such high fidelity and general accuracy! I think it's super cool!

While obviously this represents a major stride for AI technology, I think this is first and foremost to my own credit for writing such a coherent scene in the first place (more on that later). With my own work being analyzed, it was a lot easier for me to notice (as opposed to with nostalgebraist's podcasts) just how the AI was essentially digesting and repeating my own text back at me, occasionally literally but more often through a very basic layer of interpretation and recapitulation. For example, at the beginning they talk about how Silence has this sharp intellect and "sees right through you"; this is true and I would even call it insightful, but it's also explicitly there in the text, conveyed in different words but unmistakably the same idea. So really the AI picking up on it is just a validation that I wrote it in the first place. (More on that in a moment.) This makes the AI synthesis somewhat less impressive than it otherwise would've been, i.e. a very elaborate form of mimicry, but the absolute reading on my impress-o-meter is still very high. This is, all things considered, a fantastic level of comprehension, mistakes notwithstanding (more on those in a bit).

It was a very awkward, disorienting experience for me to hear these convincing simulacra of people earnestly discussing my work, especially with the work being a scene about Silence, which is always quite personal for me. I think, echoing nostalgebraist's thoughts, that it hits on a level that would not have been as intense / visceral for me if the exact same conversation were purely in text. But even if it were text, I definitely feel seen (in a rather uncomfortable way, like with a light that's too bright) that the AI easily picked up on the fact that this scene is all about Silence even though she's not the point-of-view character and Galavar actually has a much larger presence in the scene than the AI deep dive would have you think. The AI saw right through that.

The main complaint I gathered from nostalgebraist's reaction to the AI podcasts about his works was that the AI analysis was anodyne, superficial. I partially agree, which I'll get to. But with this criticism in mind, I thought to be on the lookout, when listening to my own AI podcast, for insights about how a more lay audience might engage with and understand my work. Are any hypothetical lay readers picking up on the main themes? Do they understand the basics of what's happening? How does my work come across to the sorts of people who listen to the kinds of podcasts that NotebookLM is simulating? That kind of thing.

I found the answer promising: Like I implied above with the AI's recapitulation of Silence's power to comprehend people's thinking and nature, if this podcast is any indication, I think my work is more readable than I have given it credit for. I don't consider this scene to be a particularly "easy" one to parse (nor particularly "hard"), but the AI did a pretty good job of it, so maybe I was underestimating a hypothetical human lay audience's ability to absorb and enjoy it as well.

I do have a complaint of my own: I was wryly amused at how the AI replicates the common practice of the male podcast host leading the discussion and doing most of the talking, with the female host providing a lot of affirmations and occasionally offering original insights with less egotistical framing. It's very subtle, and I can't rule out that it's just me being oversensitive and construing data out of noise, plus I don't listen to a ton of podcasts (especially the type of podcast that this experiment is most directly imitating), so there's that caveat too...but I still perceive it strongly enough to wonder what it's doing in there, as it's not something I would expect Google to: A) do deliberately; B) fail to notice.

Anyhow, as to the analysis being anodyne or not: Yeah...yeah, I do see it. But I'd probably look at it as an exercise in realistic expectations. What do we realistically want from an AI podcast host? To be challenged both deliberately and serendipitously with insightful perspectives and non-obvious ideas? To what extent?

To approach those questions, consider this: I mentioned earlier that you should stick a pin in the fact that the AI didn't pick up on Silence's left-handedness. That was a joke, obviously, but it touches on a point of interest for me in this experiment: All of today's big-splashy AIs work by virtue of having lots of examples of human-generated data to draw from and emulate (plus the information contained in your prompt). What happens, then, when you invite them to venture into spaces that humans rarely talk about, such as, oh, for instance, a stylish and objectively correct obsession in sinistrality?? Well, the answer seems to be that they just don't pick up on the opportunity. I'd probably have to be more pointed about Silence's handedness in the text, thereby raising left-handedness to the level of "an idea in itself," for the AI to notice it as relevant to the meaning of the text. In essence, I would conclude that uncommon ideas are nearly invisible to the AI when they are present only in the background. (Silence's left-handedness is explicitly mentioned only once, and is alluded to twice more.) This has implications about this kind of AI's ability to compose original philosophy; namely, that the AI is constrained for the most part only to repeating what we already know and connecting different things that we have already said. And I don't know if this is something fixable, because asking someone to interpret something that they don't already have good information on is an exercise in a form of intelligence that I don't know is well-suited to how these AIs work (and would thus be more likely to produce gobbledegook answers and continually infer meaning where none is intended).

So, in this one respect, I am not impressed by this AI performance, but neither am I thus surprised. I would have been shocked if they had picked up on the subject matter that is much more niche or even virtually exclusive to me.

I think perhaps the next big frontier for getting these AI "deep dives" to have more valuable (the first frontier having been developing the ability for the deep dives to be seriously attempted at all) is for AI to begin doing what humans do, plugging in life experience to connect seemingly unrelated ideas together. By my understanding this is emulable (and thus "fixable") under the current paradigm of AI, at least to some extent. I always enjoy it when some character is talking about some straightforward problem, and their mentor / advisor / parental figure / etc. starts telling this seemingly unrelated story that ends up either recapitulating the same idea in a different way (sound familiar?) or else shedding new light on the original idea through context or testimonials. I think this is something the current crop of AI could be taught to do.

This brings me to a comical moment at the end of the podcast where one of the hosts mentions that she gets "chills down [her] spine every time [she] reads it" (lol) when she gets to this point in the scene:

“Fat, eh? Not many of those around here. It’s not really an Ieikili fashion.” “Give it time!” Silence merrily exclaimed. “I will teach this whole society how to grow fat.”

CHILLS !! LOL

And the hosts go on to elaborate that it's "chilling" because with Silence there must be a double meaning to it, i.e. ambition, influence, etc. This is a very interesting take, and it's also the first of several mistakes I'm going to discuss.

The mistake is twofold. First of all, the female host calls Silence's line about teaching society how to get fat the last line of the excerpt. It's not; it's closer to the middle. I find this error fascinating because it implies the absence or failure of some kind of logical error-checking that I would have thought is trivial but which perhaps isn't. Second of all, there's no deeper meaning to Silence's vow. She just likes fat people. If there is a deeper meaning that must be forced out of it, it is an insight into Silence's multifaceted nature, as elsewhere in the scene there is so much focus on her restoring her health and vitality and mobility after a long convalescence from her prior injuries. She's all up for fitness and mobility, but she's also up for curtailing that on her own terms (as opposed to involuntarily), or having a lover do the same, when the context is right. I think this error provides insight into the kinds of calculated risks that the AI takes in order to compose its analyses. The AI is obviously not well-exposed enough to fat liberation speech in its training data to have picked up on this fat-affirming message as such; instead it drew on the imagery of growing fat as a metaphor for growing in power and influence, which is much more common in our culture—enough that the AI assumed that's what I was talking about.

So, this gets back to what I was just saying about the potential dangers of trying to force the AI to draw conclusions on ideas that it isn't well-exposed to: It'll do it wrong, both in assuming meaning where none is intended and in characterizing what that meaning is.

A second mistake that I'd like to mention is that, early in the podcast, the hosts characterize Silence as someone of "very few words," which isn't supported in the text (and if anything Silence tends to be verbose). What I think what the AI was picking up on is that Silence wasn't sharing her full thoughts with others, which is actually a really clever (and correct) reading, and somehow got from there—perhaps by way of Silence's deliberateness and her initial "passivity" in Sele during her convalescence—to the idea that she doesn't physically talk much. This is a fascinating conflation to make! I assume that what's going on here is that the AI has fitted Silence's personality type into a box, and this box potentially includes the character trait of not talking much, and the AI decided that there is enough supporting language in the scene to establish that Silence indeed doesn't talk much. No such language exists, but I see how an AI (and perhaps a human reader not familiar with Silence) might infer that it does. I'd be curious to round up some humans, make them read this scene, and ask them questions so as to implicitly invite them to say that Silence is a mate of few words without explicitly prompting them to do so. I bet you some would, even though in this very scene Silence is literally quite talky! Because it's easy to compartmentalize those things, I think: Silence's actual talkativeness, and her personality traits which might imply that she is not talkative.

A third mistake, which is, again, wrong in an interesting way is that there is a very clear up-front statement by the male host that Silence "is a woman," with emphatic tone on those words. This isn't the end of the sentence. He pauses, and there is some reaction from the female host, and then he finishes his thought: "...of few words." So, this is mistake not only because of the "of few words" part but also because of the "is a woman" part. I go out of my way to characterize Silence as near to genderless as possible, and while this doesn't apply to physical sex (she is female-bodied) there also isn't any discussion of that in the scene; so the AIs are drawing this purely from her pronouns, the discussion of conation (mindwashing / "mental merging" as the AI aptly puts it), and the clarification that Silence and Galavar are not bonking.

I wouldn't necessarily bother to call it out (I don't expect AI to use my paradigm of sex and gender), except that the way the host delivers his distance really emphasis the "is a woman" words, and I think that's deliberate; I think it's a very clever thing the AI is doing, piggybacking on a separate idea (about Silence not talking much) to insinuate without immediately mentioning it that there are sexual tensions in the scene. Now, it's wrong; there is no sexual tension—not between Silence and Galavar anyway. (And the AI elsewhere allows that the source material is vague on this point; I would say, from reading this scene in isolation, that that's fair.) But in fact this is the scene that woke me up to the fact that there can't be sexual tension between Silence and Galavar, at least not in the past, because she's still practically a goddamn kid when she kids to Sele, and Galavar is early middle-aged. I had never wrapped my head around that until I was laying out this scene in my thirties and realized that it'd be gross. So I wrote this scene with no sexual tension between them at all; the fact that Javelin originally mistakes Silence as a lover of Galavar before actually meeting her in person is part of the verismilitude of Javelin. But I can see how an AI without knowledge of any of these characters' histories outside this one scene might take the mistaken perception by Javelin as a positive indicator of a relationship between Silence and Galavar. So, again, I think we are seeing some of the limitations on this type of AI. In the grand scheme of things, many human storytellers would have sexual tension in a situation like this. And I think the AI failed to see that there wasn't any, or at least wasn't able to make up its mind.

I actually like this failsafe. The podcast hosts mentioned several times that the source material is unclear or vague, which makes it a lot safer for them to speculate. That, in my opinion, is a really solid implementation by Google of a way to lampshade the AI's risky assumptions to the end user. This way, if the podcast hosts go barking up the wrong tree at least they will appear to be doing it honestly, based on the text's own ambiguity. And I think all the instances where the hosts invoked this failure are justifiable instances, even if they aren't necessarily correct that the text is in fact ambiguous.

Another mistake, and one that's a lot more disappointing for technology which is otherwise getting so impressive, is that the hosts misgender Javelin later in the podcast after originally getting it right. (Editor's note: Actually I am not sure that they gender her correctly in the first place. I've listened to this thing so many times in the course of writing this that I'm not up for going through it again, so take the following with an asterisk.) I suppose this too is a reflection of how the AI actually weaves its stories: To them, the "Javelin" mentioned at any one moment in their podcast isn't necessarily the Javelin mentioned at any other moment. The AI''s persistent understanding of, for instance, the City of Sele as the setting for this scene does not imply a comprehension of what "the City of Sele" is but rather that it would be appropriate to mention a thing which is probably called the City of Sele in various instances where our human minds would in fact agree that it is appropriate.

A very understandable mistake is that the hosts infer that Silence's choice of the word "secretary" has some special meaning that goes beyond the actual meaning. (She chooses the word, having only recently become proficient in the Selish language, because of its relationship with "secrets.") The AI hosts are wrong in their speculation but they're right to be sniffing something there. (It is, after all, the title of the excerpt.) And you kind of need to have the other scene, with Galavar and Javelin that evening, to better understand why it's the title, and so I don't hold it against the AI for falling short here. I do like that they tried to mention it at all; I think they would have been remiss not to.

A final mistake that I'll mention is that, when talking about Silence's new immersion in Sele after having lived in Junction City, the hosts did indeed take an incredibly anodyne route of characterizing this discontinuity. I found that line of discourse to be very underwhelming. And it rises to the level of a mistake because it is not correct. Silence's adjustment to Sele is not complicated all that much by her past life in Junction City. That's just plausible-sounding tripe; it's not analytical at all.

There are plenty more mistakes, of course, but, moving on from mistakes to talk about specific praises, I really like how the AI hosts were able to pick up on the fact that Silence's behavior in this scene is all about her trying to prove herself. It is specifically stated in the text (albeit in a narrower sense, referring only to Silence's newly restored ability to walk partway across the city), so I'm not surprised that the AI picked up on it, but I'm glad that it did. That smacks of good design to me. It's a very important detail. And, assuming Google put its "top men" on the case for making these deep dives as compelling as the technology (and the company's contortions to avoid getting sued) presently allow, I would be inclined to consider the successful identification of this point as important to be a high-quality indicator that their efforts were constructive.

But, of course, this is the same tech that I found so underwhelming in the previous paragraph, so...you know. You win some; you lose some.

Another praise is that I appreciate the hosts for being gentle with me in describing "Silence's almost superhuman level of perception." It was equally within Google's power to invent an AI podcast host duo that will just absolutely trash you and roast you and make fun of you instead of acting like your work is the most interesting composition they've seen all year. Silence's capabilities are definitely possible, and I would even argue plausible, but I fully grant she is performing way above your average schlub. A character like her may not defy belief, but she strains likelihood. So I think it's fair and probably even authentic for the hosts to point out that she's "almost superhuman." I think not mentioning it at all would be a mistake. Silence is not just making power plays; she's an extremely competent person making power plays.

Anyway! Let me go back to something I said earlier that I said I would get back to:

I think this is first and foremost to my own credit for writing such a coherent scene in the first place

It occurred to me, even before generating this deep dive podcast, that I would like to feed the AI a scene that is much more difficult to parse, and see what that "deep dive" looks like.

So join me later when I do that. But for now, I think this is a good place to end it.

[lagraciaestaenelahora]

One hundred days in ig

https://www.instagram.com/100diascreandohabitos/

and podcast by notebooklm

[theothin]

taking a shot at generating vapid normie content farm takes on my stories and

oh no

you got a few things right and some other things VERY wrong

[thejaymo]

Enchanted Knowledge Objects in LLM UI

When we drop a document—a PDF, say—into an LLM’s context window, the document has a kind of gravity. It pulls and pushes the model toward certain ideas contained in the text, reshaping its context landscape and responses based on the content. Because I’m so sword and sorcery pilled, the word that comes to mind is that Knowledge Objects are a kind of Talisman – objects imbued with specific powers and properties that can be intentionally wielded to amplify the will of its user. 

[the-real-wholesome-bitch]

Mindless consumption and AI

Ok, so I am a computer science student and an artist, and quite frankly, I hate AI. I think it is just encouraging the mindless consumption of content rather than the creation of art and things that we enjoy. People are trying to replace human-created art with AI art, and quite frankly, that really is just a head-scratcher. The definition of art from Oxford Languages is as follows: “the expression or application of human creative skill and imagination, typically in a visual form such as painting or sculpture, producing works to be appreciated primarily for their beauty or emotional power.” The key phrase here is “human creative skill”; art is inherently a human trait. I think it is cool that we are trying to teach machines how to make art; however, can we really call it art based on the definition we see above? About two years ago, I wrote a piece for my school about AI and art (I might post it; who knows?), where I argued that AI art is not real art.

Now, what about code? As a computer science student, I kind of like AI in the sense that it can overlook my code and tell me what is wrong with it, as well as how to improve it. It can also suggest sources for a research paper and check my spelling (which is really bad; I used it for this). Now, AI can also MAKE code, and let me tell you, my classmates abuse this like crazy. Teachers and TAs are working overtime to look through all the code that students submit to find AI-generated code (I was one of them), and I’ll be honest, it’s really easy to find!

People think that coding is a very rigid discipline, and yes, you do have to be analytical and logical to come up with code that works; however, you also have to be creative. You have to be creative to solve the problems that you are given, and just like with art, AI can’t be creative. Sure, it can solve simple tasks like making an array that takes in characters and reverses the order to print the output. But it can’t solve far more complex problems, and when students try to use it to find solutions, it breaks. The programs that it generates just don’t work and/or it makes up some nonsense.

And as more AI content fills the landscape, it’s just getting shittier and shittier. Now, how does the mindless consumption of content relate to this? You see, I personally think it has a lot to do with it. We have been consuming information and content for a long time, but the amount of content that exists in this world is greater than ever before, and AI “content” is just adding to this junkyard. No longer are people trying to understand the many techniques of art and develop their own styles (this applies to all art forms, such as visual art, writing, filmmaking, etc.). People will simply just enter a prompt into Midjourney and BOOM, you have multiple “art pieces” in different styles (which were stolen from real artists), and you can keep regenerating until you get what you want. You don’t have to do the hard work of learning how to draw, developing an art style, and doing it until you get it right. You can “create” something quickly for instant gratification; you can post it, and someone will look at it. Maybe they will leave a like on it; they might even keep scrolling and see more and more AI art, therefore leading to mindless consumption.

[yeyshonan]

AI活用して生前葬のガイドを深掘りしてみた

「jFuneral もし生前葬をするなら」をAI活用して深堀してみました。 日本の葬儀ビジネスにおける「生前葬」のメリットとデメリット、そして生前葬がもたらす新しい葬儀文化の可能性について解説しています。 生前葬は、従来の葬儀とは異なり、本人が生きている間に自身の葬儀を執り行うもので、家族や友人との絆を深め、人生を再確認する機会を提供します。 生前葬は、宗教儀式に縛られない自由なスタイル、感謝の言葉を直接伝えられる場、家族の負担軽減など多くのメリットがあります。しかし、周囲の理解を得る難しさや二重の手間や費用がかかる可能性など、課題も存在します。生前葬が葬儀文化に革新をもたらし、選択肢の多様化、心理的なハードルの低下、そして葬儀業界のビジネスモデル変革につながる可能性についてです。 この音声ポッドキャストは英語でGoogle…

[bizarrepotpourri]

NotebookLM's Bleached Puritan Underpants

One more thing about NotebookLM and one that I am certainly not pleased with: while it's very, very euphemistic about mature content of whatever you're making it analyze and review, as you might have heard in the podcasts I've posted, there's one particular thing that struck me when I was running "Smuggler's End" through it.

I had to generate the podcast four times to even have it mention the protagonist's bisexuality. It's literally the big ominous swerve in the story, the homonymity between the protagonist's first love and the execution device, it's all spelled out in the story itself AND the behind-the-scenes post, and three times the algorithm just ignored it. The same algorithm that latched onto one behind-the-scenes sentence from "The Ultimate Disgrace" to praise the themes of female rebellion just refused to acknowledge that Madeleine Corbeau fell in love with a girl. Sure, I did focus on the carnal side a fair bit, but there's only so much you can ignore if you want to explain the context and how the Outsider does his usual thing.

Every time the fact that it's a "Dishonored" fanfic had the podcast talk a lot about the Outsider, in line with the short description he gets right when he shows up. But aside from that and the poster of Lizzy Stride that I mentioned in a specially edited version of the behind-the-scenes post that the algorithm also stupidly latched onto for no reason, it didn't caught the "Serkonos" name-drop despite going over the alt-text for the magazine cover I made for posting the story on Pixiv. Maybe if I run it for the fifth time, it'll figure it out.

It also motivated me to rewrite one small detail in the story itself and a whole paragraph in the behind-the-scenes post. Because I realized (no thanks to NotebookLM) that even if the description is fairly deliberate, the details may not click for everyone at first: why the silver face? Why the mention of rough wood?

[ultr8-v1olence]

guyss

actually i really love google's notebookLM

[topoillogical]

You could totally make an actually good ""AI podcast"" if you edited together a bunch of notebookLM clips. theyre so boring and vapid as full episodes but occasionally you get funny moments or things that sound real and human. It would be an AI podcast in the sense that the hosts are AI voices and no humans wrote any script, but obviously with human editing you cant call it truly AI. still. would be an interesting experiment

You'd wanna do it by like feeding one notebook some basic ideas about the topic you want, then feeding a second and third notebook different supplementary ideas, and then a fourth synthesis notebook that gets all the ideas and makes the conclusion. Or something

[printerpaintergamerguy]

Warhammer 40,000 Core Rules: A NotebookLM Podcast

In this episode of NotebookLM, we dive deep into the Warhammer 40,000 core rules. From the basics of movement and combat to the intricacies of psychic powers and warlord traits, we cover everything you need to know to start your own Warhammer 40,000 army. Whether you're a seasoned veteran or just starting out, this podcast is a must-listen.

Key points covered in this episode:

The basics of movement and combat

Psychic powers and warlord traits

Army composition and list building

Terrain and objectives

And much more!

Listen now on Tumblr.

(This week I got 2 machine spirits to teach you how to play with your tiny toy soldiers)

[thecurioustale]

I got more "deep dive" podcasts about my work from Google's NotebookLM

I've been using some of my downtime this week to play around more with Google's new NotebookLM "deep dive" podcast generator (currently free to use), which I wrote about in a recent post.

I had mentioned at the end of that post that, having sent the AI such a coherent and self-contained excerpt—about a young Silence asking Galavar for a secretary desk—and getting a very reasonable analysis out of the AI, all things considered, I'd be curious to send the AI excerpts which are more difficult to parse, and see what happens.

Well, I've done that! And more. And I've been really enjoying the fun. It is fun. It's a novelty, if nothing else, and may one day have the potential to be more than that. It's a very impressive new benchmark of AI capability. The AI hosts' feedback is usually pretty anodyne or outright dumb; it's cringeworthy like nostalgebraist mentioned; it's sometimes 100% wrong; and it's also spot-on-the-money sometimes; and in any case it's often accurate in picking out important things and talking about them, even if the talk isn't deep. And so it has given me a LOT of insight into how "mundanes" might engage with my work.

And I'd be lying if I didn't also say that "scientific" curiosity isn't my only motivation; this stuff also scratches that itch I mentioned of vanity and masochism to get feedback on my work. Usually I don't like feedback that goes out of its way to not say anything negative, which is how Google has programmed its AI here, but I find that I am finding it quite easy to weight the AI feedback differently from a human's, both in this regard and others.

So, here are my thoughts!

The first thing that I want to mention is that the first deep dive podcast (about the excerpt where Silence asks for a secretary desk) was the most satisfying one to date. I've fed NotebookLM another six excerpts since then (four Galaxy Federal and two After The Hero), and I'd have to say only two of the ensuing podcasts really "stepped up" to demonstrate the AI's capabilities when it works "well" like the original excerpt did. (And neither of them pleased me as much as the original.) The other four were seriously hobbled by mistakes and omissions.

I don't want to draw a stark "good" / "bad" binary here; the three I liked better and the four I liked less well are all relatively close together in the realm of my "approval space." But the three I liked better, and the secretary excerpt especially, left a noticeably better impression on me.

I'm not sure why, though it's something I'm going to explore as I go through my thoughts on the specific podcast episodes. But here is some speculation:

Perhaps Google is updating the tech so quickly, and/or the AI is evolving so quickly given that the deep dive podcast is enjoying a popularity spurt and lots of people are using it, that these updates to the tech are noticeable even within the span of the past few days, and are actually making its outputs worse for my purposes. Or maybe it's that this high level of use resulting from the popularity spurt is requiring Google to deliver a "cheaper" output product. One point in favor of that possibility is that all of the subsequent deep dive podcasts have also been shorter than the original one, despite the source material excerpts sometimes being shorter and sometimes longer (including significantly longer) than the secretary desk excerpt.

Another possibility is that my grading is biased. I'm really proud of the scenes where Silence asks for a secretary; this is some of my best writing for her. But I'm also really proud of the two excerpts out of the new batch of six that produced the best podcasts. Perhaps this makes it possible that any feedback is simply going to be rated more highly by me because I am in a better "mood" about it to begin with. The problem with this theory is that I'm also really proud of at least half of the four excerpts whose podcasts left me wanting. I've specifically been picking only scenes which I really like.

One strong possibility is that the nature of excerpts I picked has changed somehow in a way that I'm not picking up on. I have sent Google a variety of different scenes at this point, on purpose, in a manner that should produce varied results from the deep dives, but perhaps I have biases that I'm not aware of that are causing me to pick scenes which either relevantly or coincidentally lead to podcasts that I don't like as well.

Anyway! Moving on to other thoughts:

I had mentioned in my earlier post that, very much true to real life, the female podcast host was less relevant to and had less ownership over the podcast. That seems to be something Google is actively fixing, suggesting that the reason it was there in the first place is that it emerged organically (unsurprising) and the fix was not trivial (which makes sense). Nostalgebraist's podcasts, generated earlier than any of my own, were the worst offenders; my first podcast, where Silence asks for a secretary desk, was the next worst; and the next six seemed to improve on this over the course of this week, with last night's episodes being the best so far. Now I would say we're at the point where the female host is talking basically half the time (maybe even more sometimes, given that there is variance), and regularly gets to open, close, or even both open and close the episodes. Props to Google for fixing this; it irritated me.

I had seen someone else on Tumblr mention that Google has ordered its AI to avoid let's call them "hot-button" issues at virtually all costs, which has had the dismaying and predictable consequence of erasing queer people, fat people, love and relationships, and sex. I tested this for myself and it's true: The deep dive podcasts go out of their way to avoid this stuff. It never acknowledges that Cherry and Zoë are partners even when that's important to the text. It completely ignores my efforts to raise the visibility of fat characters and fat issues. (Nor is this stuff as preachy or explicit as I am making it sound here.) And one excerpt that I gave it, the longest of them all at about 28 pages, is a sequence of scenes from ATH, two-thirds of which is one big sex scene; yet the fact that sex happened at all is reduced to just one sentence in the entire podcast, with another sentence setting it up and a little bit of talk afterward about people looking for nondescript sources of pleasure in life.

This baffles me. I can't imagine there are many people right now who are using this new podcast tech to organize notes and reports the way Google intended. People are using this for fun, and I can't help but think Google is sitting on a goldmine. But Google needs to let people do with it what they are inevitably going to want to do with it. That means sex and filth and communism and space hentai and you name it. I understand Google's predicament: If they loosen the restrictions then the damn Nazis are going to use it to generate pages and pages of Nazi screeds that the forced-to-be-approving podcasters are going to talk about excitedly. And there'll be graphic violence, and sex abuse, and all sorts of awful things. And that stuff will get "exposed" in the news, and the public pressure will force Google to shut down the project.

But Google needs to do better on this. They need to give us as much functionality as possible in the areas that we're looking for it, especially when it comes to minority visibility, love and sex, and a certain degree of violence. The podcasters need to be allowed to talk about that and analyze it.

(Incidentally, the non-podcast parts of NotebookLM's functionality are comparably good in analytical prowess, and seem to have fewer and/or weaker restrictions on subject matter.)

One strong praise I have for these podcast products is that, because of how the AI works, the hosts tend to use the right names for characters and things. This came up, for instance, in an excerpt involving Silence Terlais (which they managed to pronounce correctly exactly one time!) and a mook you've never heard of named Brock Rudread. And Silence always goes by her first name, Silence, and Rudread always goes by his last name, Rudread, and the AI effortlessly picks up on this because that's how it is in the text.

Sometimes this backfires: For instance, in a Galaxy Federal scene, the hosts describe Cherry as Captain Seresa, because that's what her crew calls her and there are lots of "Captain Seresa" tags in the dialogue. But narratively I always call her Cherry in this part of the story, so when the podcasters then call her Captain Seresa it stands out.

Yet on the whole I like that the AI does it this way, because usually the result is satisfying. In another Galaxy Federal excerpt, the NotebookLM text synopsis of the excerpt ises the name "Basilisk" for the character Mereidi Basilisk, because that's how she usually has people refer to her, even though narratively I usually refer to her as Mereidi or fully Mereidi Basilisk. But in the associated deep dive, this "Basilisk" problem doesn't come up.

The next thought I have is that I am pretty darn confident by this point that I can "see" how the AI works. Like I said last time, it's just a really elaborate mimicry. It's pattern-matching. This is its principal strength but also its chief weakness. The AI cannot internally analyze subject matter to any deep, vibrantly-sapient extent. It can only recognize what the subject matter is in the first place (probably the most impressive feat in the whole process, for my money), recognize that a given subject matter corresponds to a given type of analysis that it has also trained on, and tailor the analysis to fit the subject matter's skin-deep specifics. A trivializing formulization of something deeply nontrivial.

Nowhere is this more obvious than when it causes mistakes. In reference to a Galaxy Federal scene, the podcast hosts mention a fee that comes up in the dialogue at one point. And in Galaxy Federal the space money is called batloos, because I specifically didn't want to use the word credits. The hosts reference by name the batloo (a "five thousand batloo fee"), but then analyze it by saying "That's a lot of credits." I know, by design, that "credits" are never, ever used to describe currency. Same thing when the hosts mention that the Starship Sevenge "comes out of warp" at one point. No it doesn't. There is no "warp drive" in Galaxy Federal. That word is deliberately not used in reference to interstellar travel.

Those are small-scale examples, but this same exact type of mistake occurs when attributing character motives and reasoning, and when sussing out the themes of a work. For instance, the hosts continually err—"get it wrong"—whenever their analysis looks at a situation involving character motivation and reasoning that is outside the mainstream or the norm, or where I as the author am trying to make some kind of particular point. The podcast hosts did that, for example, when they misinterpreted Silence's line about teaching Selish society to become fat as a metaphor for the society growing powerful; Silence really was just talking about normalizing the physical act of getting fat. In these six new podcasts, this same problem occurs over and over again.

The AI has the same problem the healthcare industry does: They identify the subject matter, go to the bell curve of associated analyses, and pick the most frequently occurring ones. What you end up with is a homogenization of worldbuilding, characterization, and plotting.

This is why the new generation of AI that has taken over AI space in recent years isn't really "smart" yet, at least in my book: It can't actually analyze. It can't actually look at situations in and of themselves, and read them and absorb them the way our minds do, to then carry out real analysis. And I'm not sure (because I don't know) if this is even within this kind of AI's sphere of potential. I'm not sure if it's something that's coming in the future or if this will never be solvable under this type of AI and will only be further and further mitigated for with additional refinement, training / data exposure, and processing power. And given that it takes several minutes apiece on deluxe Google hardware to generate these podcasts, I must reiterate what I said before about wondering whether or not this load can be downscaled / "miniaturized" or if there is going to end up being a serious floor to public access of this technology, like with air travel, because of the high resource-expenditure involved in utilizing it.

Many of the most significant mistakes in the deep dives comes from this fundamental weakness in the AI architecture. But there are also other types of mistakes, some of which are pretty interesting.

I'll get to others later, but here's one example: A moment ago I mentioned how the text synopsis of one Galaxy Federal excerpt uses the name "Basilisk," but this same problem doesn't happen in the podcast. Well, that's because that particular excerpt is written in the first-person from Mereidi's point of view, so the name "Mereidi" rarely comes up. But instead of the podcasters calling her "Basilisk" over and over because that's what other characters do, they kind of forget that Mereidi even exists and focus more on another significant character instead, even saying at the start of their deep dive that this story "is all about [this other character]." Which is just bonkers wrong, but in a very understandable way.

I also want to talk about the verisimilitude of the vocal styles. I think it was nostalgebraist who mentioned that humans are not going to listen to two robots talk to each other for ten minutes, so Google had to get the human speech patterns right—and succeeded wildly in doing so. I agree that Google succeeded at this very well. But there are also still plenty of artifacts: volume drop-offs, weird interruptions, phatic tics and cues and murmurs that don't land correctly and aren't spaced correctly with respect to the following dialogue, instances where the same host is talking but there's a clear cut at one point as if two different takes had been spliced together, and so on. These kinds of errors remind me of the errors in picture-generating AI, like hands with the wrong numbers of fingers, or lines that don't spatially map to objects correctly, etc. All AI outputs are empirical simulations of (standardized, homogenized) approximations of reality, and so sometimes the details are wrong in ways that the AI has no problem with but which stand out instantly to humans. Some of these errors are habitual; the female host twice (out of seven excerpts I sent Google) gets a lot quieter (not in her actual voice loudness but in the volume of the podcast itself) when she is the one closing a podcast episode. I wonder where that comes from.

Another thought I have is that, now that I'm seven podcast episodes deep with them, I am noticing that these AI podcasters run into the same limitation that many human content creators face: sameishness. Most content creators have a "formula" for how their content looks and feels, what it covers, what it's like to consume, etc., but what separates the appealing creators from the forgettable ones is the difference between whether their formula serves as a focal lens and/or a part of the human charm factor on one hand, or as a confining pen or even a straitjacket on the other. It takes personal charm and/or (but usually "and") the ability to create a wide variety of experiences within a formula in order for a content creator's formula to serve them well. Otherwise, the content starts to become boring and repetitive—sometimes very quickly.

I'm not at the point yet where I feel like the NotebookLM deep dive podcasts are "boring and repetitive," but I think most of this resilience is because I keep feeding new material into it—my own work no less, which I am obviously interested in. But the personalities of the AI hosts, and the ways in which they go about their "deep dives," are pretty superficial. Hell, the podcasters don't even have names yet; this is undoubtedly by design on Google's part—ever the Achilles' heel of social media companies and tech companies in general, trying to decide for us how we should experience their products and services rather than letting us control our own experiences like how it used to be.

For all the trouble that Google has gone to in order to make these virtual personalities sound human, they are entirely missing huge swaths of the human equation with regard to the hosts' personalities, preferences, opinions, defining traits, life histories and experiences, senses of humor, and pet issues. These podcast hosts are disarmingly friendly, which goes a long way toward masking the fact that they have no personality at all. Sort of like the Corporate World, in that respect. But this only succeeds in making the podcasts listenable; not in making them compelling. With an agadmator chess analysis video on YouTube, you just live for those moments when he has fun with a game, being delighted by something in the action or by one of his own thoughts or actions. That's as much a part of the appeal as his analysis or visual aesthetic is. But with these "Google NotebookLM AI-generated virtual deep dive podcast hosts" who don't even have names and thus must be referred to purely by functional labels, there's just no onramp for connecting with them on a personal level. And I don't know how much of this is the tech not being there yet and how much is the result of Google deliberately holding back on us, but over time it's going to increasingly become the latter.

Okay; that's all the stuff I was itching to talk about off the top of my head. (Though I'm sure I forgot some general points and will make them later.)

Now let's get into some specific episode-by-episode reviews.

I struggled mightily with whether or not to share these podcast episodes with you. Unlike the excerpt last time, where the actual story text was previously published, all six of these podcasts are about story excerpts which I haven't published yet. I'm not gonna just spoiler-post six giant blocks of story, many of which are in the 20-page range, for the sake of this discussion. Even discussing some of these scenes at all, without explicitly posting them, would be pretty spoilery in a couple of cases.

But it would also be pretty weird for me to post the podcasts here without publishing the story excerpts that they're doing their "deep dive" on. Yet it would be completely suboptimal for sure for me to not post the podcasts here and then go on to analyze them.

🤔🤔🤔

Here's what I have decided to do: I'm obviously not obligated to discuss any of these podcasts, let alone all of them. I just think they're interesting, and I want to write about them, so that's the why of that.

What I'm going to do is tackle these podcasts in subsequent posts. This post is already getting a bit long anyway. Maybe I won't cover every podcast. Maybe I'll generate new ones yet to come. Regardless, in each subsequent post on this topic I will either post or not post both the deep dive podcast and the underlying text scene. It's very unlikely that I'll post more than one actual scene, if any. And I'll probably share the podcasts in no more than half of the cases. And, whether or not I've posted the story excerpt and whether or not I've posted the podcast for you to listen to, I'll then get into my analysis.

And it'll all work out, in nice, neat little packages, because I am a smart and clever human who can do that sort of thing!

[angleformation]

Google NotebookLM : L'IA qui révolutionne votre organisation de documents !

[theothin]

out of curiosity I tried showing it the wikipedia article for gushing over magical girls

the pronunciations are. so bad. as are a number of other things, but I'm particularly amused by the understatement of describing the show as "suggestive", and the claim that the lord squad is less villainous than utena's group

meanwhile I'm increasingly starting to notice patterns in the comments that the podcast characters make - phrases and assessments that seem to show up a lot, regardless of how well they fit. which includes very shallow expectations for any given story genre, although I do like how both hosts agree on the appeal of being a magical girl

[jlabeatnik]

Some Videos from Google Re: NotebookLM

[venniekocsis]

They Assessed Me, and It’s Wild.

Google NotebookLM has a section where their AI will assess your web presence and produce a conversation based on your public work. You feed it links to assess. I searched myself in Google and grabbed the top links, along with my art store, my swag store, and a couple of my YouTube music videos. Listening to this, I thought, “Gosh. AI seemingly knows me better than I know myself!” It made me…

[yuusukewada]

第35回 M4配信：忙しい人のためのGoogle NotebookLM活用は「神」です

Google NotebookLMをご存知でしょうか？最近、私はこれを使って情報をまとめています。 Googleが提供するLLM（大型言語モデル）プラスRAG（検索拡張生成）で新しい情報を取り入れてできるだけウソ提案（ハルシネーション）を避ける仕組みのシステムです。これにて、まだAIが認知していない新しい情報などを一気にまとめたりすることができる機能です。 ブラウザでお読みになるにはこちらから： 35. 2024/11/02 忙しい人のためのGoogle NotebookLM活用は「神」ですhttps://clt1481914.bmeurl.co/11BDD713 今回のNotebookLMはAIのRAGという部分を提供してくれる機能です。これをPerplexityやGeminiなどと組み合わせて使うことで最強管理ツールとして活用できるようになりますのでぜひご活用ください。