NotebookLM – reverse engineering the system prompt for audio overviews
So it seems lots of people are having fun experimenting with the Audio Overview feature of NotebookLM — including me!
If you’re not familiar with this, Google’s NotebookLM offers a way to upload (or link to) up to 50 documents in a “notebook” and then get summaries of different types and ask questions of them.
Audio Overview: generate a (simulated) podcast
Their newest feature is that it can generate a podcast-style audio based on your documents. This is called “Audio Overview.”
When you first try it, it seems amazing that it can do this so quickly and sound so realistic! They got the style of a podcast down!
So of course, I tried this with about 30 different documents or groups of documents.
Some content I used: a report about AI and academic libraries, a group of articles I had saved about the climate impact of AI, meeting notes from a committee I’m on, a short story by Isaac Asimov, a blog post of my own about my time in Oaxaca back in 2013, the complete script of my husband’s full-length AI-generated film (still in production), the entire text of my book, Keeping Up with Emerging Technologies, and more.
Here’s a sample of what it generated for this article: Optimizing renewable energy systems through artificial intelligence: Review and future prospects. Energy & Environment, 2024.
What I learned
Some of those overviews came out better than others. Some limitations (for now) are:
- It’s always the same two voices, one male, one female. It seems random which voice is the interviewer and which one is the expert.
- You can’t direct it to pay attention more to certain documents or parts of documents in your Notebook.
- You can’t control the duration (it seems to usually end up between 6 and 15 minutes long)
- It works with English only, for now.
However, I’ve heard that Google may add more features that let you control it. I hope they will!
What worked well
• The podcast summary of a committee meeting was hilarious! One voice says, “When I think future, libraries aren’t the first thing that pop into my head.” Later they say, “Sounds kinda bureaucratic. Well, it’s librarians, what can I say?” (Made me laugh). The content was true to the meeting and what we discussed. Of course I shared it with our committee. It was based on an AI-generated transcript of a Zoom meeting we had.
• Combing articles on two different topics worked well. It did a good job of finding the relationships between those topics in the Audio Overview. (One article was about the problems with tech journalism, and the other article was about solutions journalism).
• Using only one document in a Notebook, instead of several. The Audio Overview did a good job of discussing this article: Net Zero Will Be Harder Than You Think – And Easier. Part II.
What didn’t work so well
• The overview based on my blog post about food in Oaxaca (from my time living there in 2013 as a digital nomad). It said that I tried eating some chapulines (fried grasshoppers that are popular there), but I wrote about not trying them, since I’m vegetarian. I suspect it might have to do with generative AI not doing so well with negative prompts — or not knowing the different between a negative and positive statement?
So sometimes it can get a fact completely wrong. But I didn’t find that in the other Audio Overviews I tried.
• The overview based on my entire book, Keeping Up with Emerging Technologies. It came up with some weird examples and didn’t really emphasize the most important parts of the book.
Using Claude to reverse engineer the system prompt
I kept wondering, what is the system prompt they use for this? So I tried an experiment.
1. I generated transcripts from four of the audio overviews, using this free tool.
2. I uploaded the transcripts to Claude and used this prompt:
I’m going to give you four different transcripts from some podcasts. I’d like you to compare them and tell me what they have in common for style, tone, common phrases used, and similar thoughts.
3. Claude listed 5 descriptions of the style and tone, 5 common phrases and language patterns, and 8 similar thoughts and themes.
I noticed that it didn’t really talk about the ending, so I asked this:
“What’s in common with the ending of each?”
It gave me 7 common elements of the endings.
4. Then, since it was including information about the subject matter of these transcripts, I prompted this:
“Now let’s ignore the subject matter for these. Instead focus on style and common wording. Please write instructions for how a similar podcast on a different topic could use the same style and format. Be very specific about how the dialog should be.”
Here’s what Claude gave me:
Certainly! Here are specific instructions for creating a podcast in a similar style and format, regardless of the topic:
1. Opening:
– Begin with a welcoming phrase: “Hey everyone, welcome back.”
– Introduce the topic as a “deep dive” into the subject matter.
2. Dialog Structure:
– Use two hosts who engage in a conversational back-and-forth.
– Alternate between short, punchy statements and longer explanations.
– Use frequent affirmations like “Right,” “Exactly,” and “Absolutely” to maintain flow and agreement.
3. Language and Tone:
– Keep the language informal and accessible. Use contractions and colloquialisms.
– Maintain an enthusiastic, energetic tone throughout.
– Use rhetorical questions to transition between points: “It’s fascinating, isn’t it?”
– Employ phrases like “You know” and “I mean” to maintain a casual feel.
4. Content Presentation:
– Introduce source material (e.g., articles, studies) early in the discussion.
– Use analogies to explain complex concepts: “It’s like…”
– Break down ideas into digestible chunks, often using numbered points or clear transitions.
5. Interaction Between Hosts:
– Have one host pose questions or express confusion, allowing the other to explain.
– Use phrases like “You’ve hit the nail on the head” to validate each other’s points.
– Build on each other’s ideas, creating a collaborative feel.
6. Engagement Techniques:
– Address the audience directly at times: “So to everyone listening…”
– Pose thought-provoking questions for the audience to consider.
7. Structure and Pacing:
– Start with a broad introduction of the topic and narrow down to specific points.
– Use phrases like “So we’ve established…” to summarize and move to new points.
– Maintain a brisk pace, but allow for moments of reflection on bigger ideas.
8. Concluding the Episode:
– Signal the wrap-up with “So as we wrap things up…”
– Pose a final thought-provoking question or takeaway.
– Use the phrase “And on that note…” to transition to the closing.
– Encourage continued engagement: “stay curious, keep those questions coming.”
– End with a consistent sign-off: “Until next time, keep [relevant verb].”
9. Overall Flow:
– Begin with the misconception or general understanding of the topic.
– Introduce expert sources that challenge or deepen this understanding.
– Discuss implications and broader context of the new information.
– Conclude with how this knowledge affects the listener or the field at large.
Remember to maintain a balance between informative content and engaging conversation, always keeping the tone friendly and accessible regardless of the complexity of the topic.
That looked pretty good. So I next prompted this:
I’ll give you the text of an article and I’d like you to use the instructions above to write a podcast script about the topic of the article. The title is “Will A.I. Ruin the Planet or Save the Planet?”
[I pasted in the text of this New York Times article].
Claude:
Certainly! I’ll create a podcast script based on the article “Will A.I. Ruin the Planet or Save the Planet?” using the style and format instructions provided earlier. Here’s the script:
Host 1: Hey everyone, welcome back. It’s time for another deep dive. And this one’s all about AI, but with a twist. We’re looking at its environmental impact. You know, that age-old question: Will AI save us or doom us?
Host 2: Yeah, it’s everywhere these days, isn’t it? But I feel like we’re hearing two completely different stories about AI and the environment.
Host 1: Exactly. On one hand, we’ve got headlines calling AI’s energy demand “obscene.” But on the other, some are hailing it as a potential savior for the planet. Talk about mixed messages!
Host 2: Definitely worth a closer look, then.
Host 1: Absolutely. We’re diving into a fascinating article from The New York Times that tackles this very issue. It’s called “Will A.I. Ruin the Planet or Save the Planet?” Talk about getting straight to the point!
Host 2: No beating around the bush there. So, what’s the scoop?
Host 1: Well, it turns out the global experiment in AI is just kicking off, but the spending frenzy? That’s in full swing. We’re talking about big tech pouring an estimated $105 billion into data centers last year alone.
Host 2: Wait, $105 billion? That’s… a lot of zeros.
Host 1: Right? And all those data centers need power. Lots of it. But here’s where it gets interesting. The article points out that while AI is an energy hog, it might also be our ticket to solving some major environmental challenges.
Host 2: So it’s a double-edged sword? Energy-hungry but potentially planet-saving?
Host 1: Exactly. Think about it like this: AI is like a super-powered tool. In the wrong hands, or used carelessly, it could cause a lot of damage. But wielded wisely? It could be the key to unlocking solutions we haven’t even dreamed of yet.
Host 2: That’s a really interesting way to look at it. But let’s break it down a bit. What makes these data centers such energy gluttons?
Host 1: It all comes down to something called generative AI. You know, the kind that can write poetry, code, or even have a conversation with you.
Host 2: Like ChatGPT?
Host 1: Exactly like ChatGPT. And get this – a single query to ChatGPT uses nearly 10 times as much electricity as a regular Google search.
Host 2: Wow, that’s… a lot more than I expected. So what does this mean for our power grids?
Host 1: Well, experts are predicting that energy consumption by data centers worldwide will at least double over the next few years. Goldman Sachs even estimates a 160% increase by 2030.
Host 2: That sounds pretty alarming. Are we talking about a significant chunk of global energy use here?
Host 1: It’s substantial, but let’s put it in perspective. Right now, data centers account for about 1% to 2% of total electricity demand. By 2030, that’s expected to increase to 3% to 4%.
Host 2: Okay, so not taking over the grid, but definitely a growing concern.
Host 1: Right. But here’s where it gets really interesting. Remember that double-edged sword we talked about?
Host 2: Yeah, the potential for AI to be both a problem and a solution?
Host 1: Exactly. Some experts argue that if we use AI wisely across the economy, it could actually reduce greenhouse gas emissions by 5% to 10% by 2030.
Host 2: That’s… a pretty significant reduction. How would that work?
Host 1: Well, it’s all about using AI as a tool for innovation and efficiency. For example, it could revolutionize how we design biological systems, leading to more efficient, less-polluting agriculture.
Host 2: So we’re talking about AI helping us grow more food with less environmental impact?
Host 1: Precisely. And it doesn’t stop there. AI is also being used to find new deposits of critical metals, improve geothermal energy exploration, and even optimize transportation and energy production.
Host 2: It’s fascinating how something that consumes so much energy could potentially help us save even more.
Host 1: You’ve hit the nail on the head. And there’s a historical precedent for this kind of technological evolution. Remember the concerns about cloud computing’s energy use in the early 2000s?
Host 2: Vaguely. Didn’t that turn out to be less of an issue than people thought?
Host 1: Exactly. From 2010 to 2018, the computing output of data centers jumped sixfold, but energy consumption only rose by 6%.
Host 2: So you’re saying we might see a similar trend with AI?
Host 1: It’s certainly possible. As Jonathan Koomey, a former scientist at Lawrence Berkeley National Laboratory, put it: “After the mania has calmed down, other incentives kick in. There is a huge incentive for the industry to become more efficient.”
Host 2: That makes sense. So what are tech companies doing to address these energy concerns?
Host 1: They’re working on several fronts – streamlining software and hardware, improving cooling systems, locating data centers in colder climates, and investing in alternative energy sources.
Host 2: So it sounds like the key is not just in how much energy AI uses, but how smartly we use AI itself.
Host 1: Absolutely. If we can harness AI’s potential while mitigating its energy impact, we might just get a lot of environmental bang for our buck.
Host 2: Well, that certainly gives us a lot to think about. As we wrap things up, I can’t help but wonder: If you could influence the future of AI development, what environmental considerations would be your top priority?
Host 1: That’s a big question. It really highlights the responsibility we all have in shaping this technology. It’s not just about the innovations themselves, but how we choose to implement and regulate them.
Host 2: Couldn’t agree more. The future of AI and its impact on our planet isn’t set in stone. It’s something we’re all figuring out together.
Host 1: Well said. So to everyone listening, stay curious, keep those questions coming, and keep the AI conversations going. We’re shaping the future of our planet, one innovation at a time.
Host 2: And on that note, we’ll wrap up this deep dive into the fascinating world of AI and its environmental impact. Be sure to check out the full article we discussed for even more insights. Until next time, keep learning and stay green!
That worked pretty well!
The transcript sounds just like one of those Audio Overviews. The next step would be to turn it into audio with a tool like Eleven Labs.
So I went to Eleven Labs and generated one script with a male voice and the other with a female voice. But of course I would need to edit those together into one audio file.
Since I don’t have time for that now, I’ll just leave this as a transcript. But it might be a fun project for the future to turn one of these into an audio file.
Feel free to use my prompts for Claude to generate a transcript for a simulated podcast of your own. Wait…. I have an idea!
I made a customGPT for this
It occurred to me that I could turn Claude’s prompt into a custom GPT. So here it is!
Simulated podcast transcript generator
Give it the text of an article and it generates a podcast-like script in the style of NotebookLM’s Audio Overview.
Why do all this? What is it good for?
This was an experiment to help me think about how this tool works. I don’t see using these right now as if they are “real” podcasts. At least not until Google improves the technology and gives you more control over the output.
But I do see promise to eventually use something like this to simplify information for people. It could also be helpful for improving accessibility.
For now, it’s helpful to think about how these are constructed, so it doesn’t just seem like magic.
I will be including an assignment about this technology in my upcoming course: AI Literacy for Library Workers. You can still sign up for this six-week online course. It begins on November 5.
Postscript
If you’re wondering whether Google trains on the data you upload to NotebookLM, the answer is no.
From a (real) podcast interview with Steven Johnson:
“We are not … training the model on this information so this information we’re just loading the information from these quotes into the the context window of the model the kind of short-term memory of the model and we’re using that to answer questions or be intelligent about it … there’s no chance that this information which is under copyright is going to be used to train the model or be shown to anybody else.”
And from Every (newsletter): “As an author, Johnson clarifies that no material uploaded to the model is used to train NotebookLM or Google Gemini; it’s only sent to the model’s context window, or “short-term memory.” Johnson explains that if you “have the right to use the material under copyright, you can use it inside of Notebook.”
Generated with DALL-E 3 (ChatGPT Plus). The cat represents my cat, Max.