🚀 Add to Chrome – It’s Free - YouTube Summarizer
Category: Technology and AI Tools
Tags: AI AutomationAI Content CreationAI Design ToolsGemini AppGoogle AI
Entities: FlowGeminiGoogle AIJosh WoodwardNotebook LMOpenAIProject MarinerSam AltmanStitch
00:00
Honestly, Google AI has been absolutely crushing it. Josh Woodward, who's leading up a lot of Google AI's efforts, came on the pod and showed some unreleased features, some experimental features, and some products that are going to make you a more productive
00:16
person. I brought him on, I'm not sponsored by Google at all, just so I can learn how to use these products.
I feel like I haven't spent enough time with Google AI products. So, I had one of the top guys, one of the top dogs come in and explain it to us.
This is a
00:32
primer on how to use uh Google AI product suite and what you need to know. So, I'm glad Josh came on and showed me what I need to pay attention to.
[Music]
00:51
All right. Uh Josh, um we're going to cover some of the most interesting uh of of Google's AI products today.
What are you going to take us through and by the end of the episode, what are people going to learn? Well, we're going to try to show a lot
01:07
of stuff today. So, we'll try to do these five demos in about the next 20 or 30 minutes.
Um and I'm really hoping you're going to learn kind of where a lot of Google's AI is at the frontier. So, we're going to talk about the Gemini app.
We're going to create some videos. will be able to go dive into research,
01:23
create UI and screens for code and then do some automation computer use stuff. And while while you're taking us through that, could you give your sort of inside tips and tricks on how to use these apps?
Because I know that some people, you know, people have played with
01:38
Gemini, people have played with Flow, people have played with Notepic LM, but they're looking for, you know, how do I get the most out of these products? Could you Yeah, definitely.
Yeah, we'll do that. Okay, cool.
All right, let's uh I mean let's start with from the top. Gemini.
All right, sounds good. So, we're just gonna we've opened a bunch of bunch of
01:54
tabs here, so you'll get to see how how we go. Um so, I think a lot of your viewers have used Gemini.
Hopefully, they use it every day. Um what you've got here is basically the home screen right where you land.
And I'm going to show you some stuff that's available in our kind of pro version of Gemini.
02:10
That's the one that's $20 a month. And one of the things we've been putting a lot of emphasis into is how do we start to create more interesting replies.
So I'm just going to paste in a prompt I had written earlier. This is teach me about photosynthesis.
Make a quiz to go
02:26
with it. Um and so you can see here in the upper left we're using the 2.5 pro model.
This is the great model uh that's great not only at kind of thinking which is what it's doing now. You can expand that and kind of see how it's starting to put all this together.
We've also got the 2.5 flash model, really fast
02:43
replies. Um, and so as this is coming together, what you're going to start to see is like not only a traditional reply as it breaks down and teaches you the concept, but also this quiz that is going to go along with it.
So, here comes the reply back. I'm just going to
02:59
close the thinking. Um, and you can see here it's given a nice little succinct answer.
And then you see it's linked to YouTube video, which is great. That's a new feature we've been adding in as well as this quiz.
And so what are the main ingredients for plants? What does it
03:14
need? And you've even built in a hint here it looks like.
So you can think about this. So let's see.
They probably need something like this. Do you want to take a guess here, Greg?
You know, this is bringing me back, Josh, to be honest. You know, like my elementary school ideas.
Yeah. Water and carbon dioxide seems about right.
03:30
There you go. There you go.
So that's um one of the things we're seeing with Gemini. Tons of students are using it, right?
This pro plan. We've just done a big student offer.
We'll start some more of that for back to school. So anyway, that's coming uh for a lot of new student teachers.
This is kind of fun. Um I'm going to create a new chat here.
03:47
And um one of the other things we're seeing people do a lot with Gemini, you asked about tips and tricks and what what the power users are doing is they're actually creating scheduled actions. So again, I'll just type in a prompt or paste in here.
Um send me a digest of the OKC Thunder info every
04:03
morning at 7 a.m. If you're an NBA fan, I love the Thunder.
Uh, so, um, this is going to go out and it's going to do a couple of things. It's actually going to construct kind of the first info digest.
Think if you're the president, for example, you'll get like a daily briefing. And so, this you can think of
04:18
as almost a daily briefing. And here it is.
Um, it's gone ahead and created this digest. And what's cool is every morning I'll just get an update about here's what the players are doing.
Here's any kind of offseason trades. And you can go in and edit this.
you know, in the past,
04:34
people would have created maybe Google News alerts or had to go through and dig and assemble all this info. So, this makes it really easy to do.
Um, so that's that's uh that one that on that piece, if you're say you're like an entrepreneur or solopreneur type
04:50
person, like Yeah. What are what are similar digests that you think would be helpful for someone who's, you know, a business owner?
Yeah. Yeah.
Yeah. So, we see a lot of business owners starting to use Gemini, too.
They do a couple of things. One,
05:05
they track how their own brand or their own store is doing. And so, it's good to kind of keep an eye on what people are saying about you, how you might be out out there.
The other thing they do is they do that for competitors. So, they may get their own digest, but they also study that.
The other thing we've seen
05:21
with some of these actions, Greg, as well as deep research. One of the the main features in Gemini, too, is they really will send certain products they're tracking and they may do research on those products.
I know someone who kicks off a new product every morning for a deep research report
05:38
u and you know, maybe they're thinking about a product they might want to build or put something in their inventory. So, those are those are ways people are using it.
Cool. Yeah.
Yeah. Um Sam Alman, the co-founder of OpenAI, just said that it is the era of the idea guy and he is not wrong.
I think that
05:56
right now is an incredible time to be building a startup. And if you listen to this podcast, chances are you think so, too.
Now, I think that you can look at trends uh to basically figure out uh what are the startup ideas you should be building. So, that's exactly why I built
06:11
ideaser.com. Every single day you're going to get a free startup idea in your inbox and it's all backed by high quality data trends.
How we do it? People always ask.
We use AI agents to go and search what are people looking
06:27
for and what are they screaming for in terms of products that you should be building and then we hand it on a you know silver platter for you to go check out. Um we do have a few paid plans that you know take it to the next level.
uh give you more ideas, give you more AI
06:44
agents and more almost like a chat GBT for ideas with it, but you can start for free ideabrows.com. And if you're listening to this, I highly recommend it.
Okay, the next one is um actually one that's been a little bit in the in the news the this morning, just about an
06:59
hour ago. So, um, we've been adding a lot of video into, uh, Gemini and, um, today's announcement is actually the ability to take this V3 model, which is unbelievable because it has incredible
07:15
realism and physics and everything on the visual side, but it also adds sound. Um, and so you can create these videos.
They're 8-second clips, 720p. Um, in this version it's now two times faster than we first announced it at Google IO.
07:30
Uh, and I'm curious, Greg, we can just take a take a request here. Is there a certain video or anything you're you're interested in or we can we can try one?
Yeah, I mean I I recently launched a new startup called ideabrowser.com which basically brings you new startup ideas
07:47
every single day. And I've been playing with ads uh AI sort of ads.
So, I wonder like could you do something like um like I don't know if this is possible but
08:03
you know you ever see those Tik Tok or Instagram ads where it's like a podcast studio but it's like a fake podcast studio. So it's like you know it's like it almost looks like a Joe Rogan studio.
Yeah. Um so it's almost like a fake podcast
08:19
studio with two two people maybe. Yeah.
Exactly. A host and a guest.
And what do you want them to be talking about or the sound effects or something maybe? Well, so so can I have them like say something like a script almost?
Yeah, in the new version you will be
08:35
able to. Yeah.
Yeah. So I mean like Yeah, I just found out about ideaber.com.
You know, I'm finding the most interesting startup ideas and trends there. Can't believe I didn't know it existed.
Okay, let's try it. Uh, I'm going to
08:51
make it a little shorter just because it's 8 second clip. Oh, true.
We can we can send this one off and then we can come back and look at it. You can see it's going to take about a minute or two to generate.
Okay, cool. So, let's keep that going in the background.
That'll be interesting to see. Um, the big new feature today is
09:08
um rolling out. It's at about 1% right now because we just started it is uh the ability to also bring an image as a reference in or a photo.
So you'd be able to maybe have a photo of that podcast studio you want and then or maybe have the certain people or whatnot
09:23
and kind of have that sort of animate and come to life. So that's pretty cool.
Which is so important, right? Like you need, you know, having that context ultimately is going to get better results.
Exactly. Yeah, you got it.
You got it. Um the last thing I want to show is um
09:38
is actually while this is generating I'll just create a new chat. Um there's some stuff we've been talking about and started rolling out around how we're going to make Gemini the most personal, proactive, and powerful assistant for you.
And that first P, the personal is something that um I just want to show
09:55
maybe teaser for, but uh keep it kind of short here. But if I go in, there's an area called personal context.
And you can see here, we've been talking about this. or testing it internally where with your permission you'll be able to link in things like your Google photos
10:11
data or your search data or your YouTube data and you can start to get some pretty interesting replies again with your permission your ability to kind of turn these on and off but I've been using it um when you're asking about tips and tricks it's really amazing if
10:27
you're planning like an upcoming trip um it knows that I have uh my wife and I have four kids it can suggest things based on places I've been in the past that I might like. So like, oh, you're going to go to Italy, you have four
10:42
kids, here's family friendly, familyfriendly restaurants near the coliseum. Um, or it might be like, hey, I checked your past trip photos and it looks like you have this interest in this hobby, like I don't know, pottery or sketching, and it will kind of suggest things that way.
So, I just
10:58
wanted to show this to you and and your viewers is like a little bit of a kind of a view of what's coming and so we can can share more of that when it's ready. Cool.
I appreciate that. Yeah.
I think um I actually tweeted this morning that, you know, I think it was I said Chrome's
11:13
3 billion users suddenly look like the most valuable data set on Earth. And I think it's really, you know, Google is such an interesting at such an interesting place because it it knows so much about you because we, you know, use the products every day.
We live in those products. So my hope is like that's why
11:31
I think a lot of people are getting a lot of good, you know, are starting to get really good outputs out of these products is because of that context. So just something I mean obviously I'm not like sponsored by Google or anything like that.
we have no relationship, but I'm just something that I've noticed and I'm hopeful that the more context that
11:48
Google starts, you know, feeding drip feeding into me because it, you know, because I use Chrome, um, because I use Gmail, because I use these other products, I'm hopeful that the outputs are going to continue to get better. Yeah.
Well, that's actually how we're thinking about it is, you know, for someone like you who's excited to link
12:03
this context together, we want to make that possible. And then you could see from that kind of look ahead, there'll be other Google products that you know, again, if you want to, you'll be able to to link in u and then use it as context.
So, uh, okay, cool. Well, let's see.
Let's check back in real quick and see
12:18
if our our video Oh, here it is. All right, here we go.
Uh, okay. So, here's our pro fake podcast studio.
Let sure got my audio on. Let's see what how come how it came out.
Just found out about ideabrowser.com. I'm finding the most interesting startup ideas there.
12:34
Yeah, cool. I mean, visually, by the way, this looks stunning.
Like, this is exactly what I had in mind. And I think like from my perspective, I'm looking at this is um people don't want to read anymore.
So, if you're designing a website, if you're
12:50
building apps and stuff like that, you know, they're used to Instagram stories, they're used to just consuming YouTube. They're used to consuming video.
Yeah. So, I'm looking at like, you know, my portfolio of businesses and I'm thinking like, okay, where can I create video to to make it more engaging to to
13:05
tell my story better? And to me, this is like an unfair advantage.
Well, I think one of the things we're seeing both with the VO video generation and imagine the image generation is Gemini makes it so easy to just quickly explore lots of ideas and then yeah, you
13:20
can download it, share it, drop it in wherever you want. So, yeah, I totally agree with you.
I guess the hard part becomes how do you come up with good ideas to to generate you know interesting content right people people you know on on on X that you know they
13:36
call that taste you know that's like the word of the month um do you have any insight as to like if you're trying to get the most out of some of these video models and you want to come up with good ideas what do you recommend to people? Yeah.
Well, I think
13:53
one thing is actually an interesting segue because we can start talking about flow, one of our other products, but is like I try to just look at lots of content for inspiration. Um, and so like if I if I cut over here just to show you like we this was a product we announced at Google IO about 50 days ago when
14:10
we're recording this now. And you know this is really an idea of like how do you make an AI film making tool where lots of people can create with it.
Um, and so we worked with some incredible creators. There's all kinds of features, but I think one of the things just to jump to it, Greg, is like one of the
14:26
parts of this product, which is fun, is called Flow TV. Um, and it's just a huge gallery of lots of different things people have created.
So, here's like always a fun one. This is like offseason s.
And what's fun is just like with your
14:42
keyboard, you can navigate through these different channels and kind of see stuff people are creating. And we've actually made it where you can show the prompts too uh for each of these.
And so one of the big themes we're seeing around all of this generative media creation is
14:57
around remixing and being able to kind of take inspiration and kind of put your own sort of touch on it. Um so this is this is kind of the gallery view.
You see people skydiving with glaciers and all kinds of crazy stuff. Um but how this tool works is actually kind of fun.
15:13
You you know you go out to labs.google/flow Google/flow and you have a landing page here where it's like different projects I've been working on. Giant penguin walking around or like mountain biking through like a lava field here.
But you can kind of go into these things and create different shots,
15:31
different scenes. And all of this, of course, is backed by both V2 and V3.
Um, and so you can go in and some of these that goes with you actually hear the
15:47
like lava magma crackling in the background as the bike is going through. Um, and what's interesting with this is we've tried to make it where say you generate something with a prompt and you just say add the scene and now you're suddenly in a really easy to use
16:03
lightweight kind of editor and you can do all the normal things like make clips shorter but you can also do things like extend them and this is really the power of AI starts to come through because you can say things like I don't know make a huge vulture come in from the sky right
16:20
and this is something where we can stitch this together. And when you send this off, it'll take a minute or so here and it'll generate.
And what's amazing about Flow and VO as a model is that we're starting to get to this point where you have consistent characters and
16:36
scenes. And that's been one of the biggest requests from creatives all around the world.
Um, and so there's a few other things that are cool about Flow. Maybe I can show you while this is generating.
So one is um we make it really easy for you as a creative to
16:51
generate lots of options so you can get two videos at a time or four at a time and this is really ex like propelling how fast people can explore a creative space. We also give you in this tool again because it's kind of for filmmakers creative types lots of choice
17:07
on the model. Do you want audio?
Do you not want audio? Do you want the fast version?
Do you want the high quality version? So a lot of control here.
Um, the other thing that's cool is we allow you to go text to video if you want to write, you know, big prompt or frame to video. And this is actually where you
17:24
can give it a reference image and then be able to kind of create things based on that. And so it's very similar to what I was showing you with Gemini.
Uh, oh, look, it just came back. So, I'm just going to play it from the top.
You can see we've now stitched together two shots. So, we're at 14 seconds total.
17:39
So, I'll just play it here. It's It's honestly like mind-numbingly good.
Like, you know, that's it's so good that it's just like I can't believe I just It's hard to believe how good it is.
17:55
It's crazy. And when you think about it, it's all generated here in seconds.
You saw that big vulture. So, I think what we're starting to see is and what's been really interesting as we've been building Flow, it's a Google Labs product.
We one of our big values
18:10
is we like to co-create with people. So, we've been working with filmmakers around the world to develop this tool.
And that's really how we see it as a tool. And what's interesting, Greg, is folks that are, you know, top tier Hollywood types, they see this thing and
18:27
they're like, "Oh my goodness, I can do so much with this." And it kind of raises the ceiling of what's possible. Then you've got people like me.
I'm not a filmmaker, but I might have an idea or I might have a project I want to work on. And now I've suddenly got this incredible tool that's so easy to use.
that makes it happen. Um, yeah,
18:43
there's a huge road map here of where we're going. Like if I switch over here to like the frames to video, um, I'm able to kind of bring in different frames here and sort of compare them.
This is super cool. We've got tons of camera shots like dolly in, dolly out.
19:00
You can jib, you know, all the things you might expect to kind of pan, tilt, zoom around this. And I think this is really where it's going is like making these things super easy and intuitive to use for people.
Yeah. And my guess is, you know, if you're not a creative director and you
19:17
don't know what dolly in means or dolly out, whatever, like I'm sure you could go to Gemini, say, I'm creating a script and right like, and I need help kind of thinking about how I can get the most out of this product. Yep.
And that's actually what a lot of
19:34
people do. They'll start in Gemini, they'll craft the prompts, they'll get help, they'll iterate really quickly and then you can bring that right into Flow or you can create it from Gemini itself now.
And so we're seeing a lot of that kind of back and forth. I think Flow, you can think of it, is really for folks
19:50
who might have a longer duration project. They may want to do something over several minutes or maybe we've seen people do up close to an hour of stitching things together.
U whereas Gemini is really quick, those little eight-second hits and that you can kind of put together really fast. Very cool.
20:05
Yeah. And then by the way, real quick on like the Hollywood versus individuals piece.
Like for me, I'll only speak for myself like I'm I'm most interested like of course I think this is going to change Hollywood, but um and make you
20:21
their lives more productive. But I'm more interested in like for me I'm looking at this and I'm like okay as a person who's building startups and who needs to tell stories.
Yeah. This this is going to help me compete at at a way higher level than I've ever
20:37
been able to before. I mean, of course, I could have done this, but it would have cost me hundreds of thousands of dollars to pay an agency to build it.
Yeah. What's interesting, what we're seeing is kind of this barbell effect where folks that maybe have the knowledge, have the expertise, they're taking tools like this and running with
20:53
it. And then we're seeing folks maybe solopreneurs or others who may have never had the skills or the access or the money or any number of things and now they suddenly have a tool that they can kind of use to customize for what they want.
Um it kind of it's interesting both of those effects are true um in this case. So
21:11
yeah. Yeah.
So that that's flow. Um and it's uh it's a lot of fun to to play with.
So you check it out. Um oh yeah I want to shift gears a little bit.
This is one I'm sure uh your audience has heard a bit more about Notebook LM and you've talked about it too. So thank you.
I think uh I want to show a couple of ways
21:26
again more just like power user things. So notebook you obviously a lot of people know if you don't you can bring in your sources you can get insights you can create podcast and sort of see kind of where the answers are coming from.
So that's kind of the highlevel idea behind notebook. It's been so fun to watch it
21:44
grow and it just keeps growing. So this is the home screen of notebook.
Uh just as a quick walkthrough if no one's ever seen it. You've got three panels here.
Here you've got your sources. You can bring in any kind of file that you want, docs, websites, YouTube videos, PDFs,
21:59
etc. You can chat with those sources and then you can also just start creating stuff in the studio panel.
And so kind of the most common use case a lot of people do is they'll load up some sources. And this is a notebook about the California gold rush.
I've been we were studying this, my son and I, right
22:16
before the end of the year. And um you can type in a question like what were the economic implications of the gold rush?
And the real thing that sets notebook LM apart from other tools out there is its incredible ability to sort of show its work and cite things. And so
22:32
you're seeing it come back right now as it streams in. And of course it'll give you these nice replies like here are the implications of it.
But as you scroll through this you might be like oh I really want to learn more about that. And when you click on it, you can see it just takes you right to the document and kind of where you found the information.
22:48
So that's that's kind of one of the core things. I think people have heard about the podcast, so I don't have to play the podcast here today, but I wanted to show you some of the new stuff coming um and some of the newer features.
So what a lot of people have started doing is actually you can manually add sources,
23:03
but you can also discover sources now. And you can just describe things like, "Give me more economic info about the gold rush." And what's going to happen there is when you send this out, we're going to take kind of a a shot at trying
23:19
to find other sources that are relevant to what you're looking for. And so here it's come back.
These are a set of websites. You can see some PDFs on the web.
What's really cool about this, Greg, one of my favorite features is it gives you the name of the source and then this little by line, it tells you
23:34
why we think this source is relevant to your notebook and what you might learn from it. So, it's super cool to be able to kind of see like, oh, this is talking more about banking or this is talking more about wages and labor market.
So, um with one click, you just tap it, they
23:50
all get added to your notebook. um it's made it really easy for people to kind of spider out and find other information.
And then once you've got the information in there, you can start to transform it. And one of the kind of sleeper hits uh that's gone really great
24:05
is this mindm feature. So this gives you the ability to take all that knowledge and you can just expand these and find kind of like all kinds of little trails down here like hard rock mining and what goes into it.
And as you click through these, it'll bring you back to your
24:22
sources. It'll help you understand it.
And so we think the big idea with notebook LM is it's a whole new way to understand knowledge. And what's been really cool is we've enabled a lot of sharing recently in the last couple of weeks.
So people are creating notebooks and sharing them either with their
24:37
friends or their colleagues at work or even in a class. And you're starting to get almost like this collaborative knowledge exploration which is a whole new level of Notebook LM and what's driving a lot of growth too right now.
Yeah, this is really cool. My quick use case uh for for this sort of stuff is um
24:55
I'm constantly looking at building or buying businesses and if I'm if I am going to do that, I need to get smart about a particular topic. So, for whatever reason right now, I'm like exploring like the book publishing industry
25:11
um and the music management space and the music publishing industry. So, kind of like media rights and stuff like that.
Of course, I can go and read posts and books about it. But, you know, when I'm in the car or I'm going for walks, I
25:28
just like to just listen. Yeah.
Just drop it. Just drop it in.
Y. So, you know, if you're like me, I think that using something like this to get like a very personalized uh level up
25:45
um is good. Of course, I still use listen to podcasts.
Of course, I still watch YouTube videos, but this is kind of like for me has been like a completely different experience. Yeah.
Well, and one of the other really interesting things we're seeing people do is notebook and um there's a ton of
26:01
students that use notebook and I think that's been another area it's great for kind of being able to go deep in the areas you want to and I don't know if you do this with the podcast feature. I'll get it going and then there's you know an ability where you can like have this interactive mode right where you
26:16
can call in and they'll be like oh hey Greg what's your question and you could ask your question and it's all grounded again in those sources. So it kind of makes this really personalized learning and knowledge exploration possible.
Totally. Yeah.
And yeah, that's really cool. I think
26:32
like a lot of people listening here like the these are the people that like to level up. Um and so yeah, I'm excited for people to try this out.
Yeah. Yeah.
Um All right, cool. We can switch.
We can keep going. Are you good?
We can go for another one. Okay, so the
26:47
next one I want to show you um this is a super new one. It's called Stitch.
Uh, so it's a Google Labs experiment. You can see it's in beta.
And what we're really interested in is how can you can see here design at the speed of AI. But how can you go from an idea to like a mobile or a web interface and we've got
27:05
a bunch of examples and we're kind of playing around with this where we make it really easy for you to kind of describe what you want and Stitch kind of builds it with Gemini. Um, so why don't I uh I'll jump into this one.
I'll just jump over tabs. This was one I was working on this morning just before just
27:21
to show you kind of how it works. But you basically describe something like make me an app for discovering California activities, show some destination, musical events, and trip planning.
And Stitch just like gets to work like it will start creating all of
27:37
these different screens for you. And what we're finding, what's super interesting, Greg, like if I click on one, I'll just go into the first one.
is California is like with one description and a tap, you can copy this code and you've got like the front-end code ready to go. Um, or this is super cool.
You
27:54
just tap Figma and it'll copy it. You can see in the bottom right and you've got a Figma a Figma file or like ready to paste in.
Um, and these are not screenshots. These are actual designs which is what's really amazing.
You can go in then in Figma if you want and move
28:10
stuff around, change it, etc. Um, and so this one is a I mean maybe a good example of a Google Labs experiment where it's hyperfocused on in this case, Greg, since you're talking a lot about how do you kind of level up or get started.
We originally thought like, okay, who might like this?
28:27
Maybe it's designers, maybe it's PMs. Turns out engineers really like this tool because they can describe maybe a screen if it's like someone trying to start a company and they get like good base level of designs and then they can edit it from there.
Um, and so front-end
28:43
code always maybe a little bit harder for some people. Maybe you don't want to do all the details or whatever.
This kind of helps accelerate that and get you to kind of a baseline that you can you can edit that draft. Basically, if you had to, you know, if you had to say, what is the one killer feature of
29:00
Stitch? Like what is, you know, what is the one thing that Stitch does better than any other vibe coding tool?
What do you think that is? Yeah, I mean I think right now we're really trying to make it where a lot of v coding today you might write a long prompt and then there's a
29:17
lot of back and forth. And so what we've tried to do with Stitch is minimize that back and forth and then give you the assets and the controls to kind of do common things.
So, like one of the things I love about Stitch is you can also just go up here and say edit theme, make it all dark mode, change all the
29:33
corner radius, change all the font, like common things that people get stuck on. We're really trying to focus on like the design quality here, but also making it really portable so you can move it to a tool like Figma if you want to go deeper.
And I think that's starting with
29:49
the design process is kind of how we're really thinking about Stitch. That's interesting.
Um because I think uh in general with AI like the chat has become the interface um understandably but I think that some things are just not meant for chat. Um and sometimes
30:07
chat is overwhelming. So I think having sort of a hybrid of chat with some you know productized buttons and stuff like that I think is interesting.
Well, it's one of the things it's interesting you're hitting on an interesting pattern we're seeing is how do you get beyond the chat box because you know and it
30:23
doesn't mean we're going to get rid of it. You can see Stitch obviously you can chat with it over here on the left and kind of edit if you want or bring in images as references but I think across Stitch across Flow definitely across the Gemini app you're going to see a lot more features like that photosynthesis
30:39
example where we're we're trying to get people into a more natural state. um chat is not always the most natural state for interacting with these models.
So yeah, totally. And yeah, it it's not and and I don't know for someone like me like I just find that sometimes chat is for me
30:57
and then sometimes I just I need a click around and that that it works better for me. So this is interesting.
Yeah. Do we have time do we have time for one more?
Is that Do you want to do one more? Can we do it?
Okay, let's do uh Oh, yeah. The last one I want to show you is um is project mariner.
So project mariner. This is our
31:13
project. It's really experimental again, prototype, fresh out of the lab.
We're really trying to understand how can these models navigate things like the web where you can give it tasks and it can go off and either do research or
31:28
data entry or other things and kind of put this together. So, what's really amazing about our Gemini models right now is they've gotten very good with the 2.5 series at planning multi-step actions and being able to also kind of raise their hand and be like, "Hey, uh,
31:45
I need help. I'm stuck here." U, and so that's one of the things we're exploring with this Mariner project.
Okay, Greg. So, here's the home screen of Project Mariner.
And you can see the giant experimental label at the top. And what we've been exploring this idea of computer use and agents actually back
32:02
since December. Um our first version of Mariner was a Chrome extension.
Uh it didn't always work all the time, but it was like eager and it tried hard. So what we've tried to do with this, this is kind of you can think of almost as like Mariner 2.0 know is pull it out of
32:17
the Chrome extension, put it in a website and in the background instead of operating on your local browser, we've created an entire infrastructure with virtual machines. And so what that means is you can spin off lots of tasks at the same time.
Uh and so what's really cool
32:35
here is with Mariner, you've got, you know, these tabs at the top, your active tasks, things maybe you've saved, tasks that have ended. And I'm just going to show you this is a test account I have here.
But we'll just pick one. I don't know, find me three drops.
Uh this is one of the sample ones to kick off. And
32:51
what's interesting is you can see here it's kind of preparing your session. This is actually firing up that virtual machine that I was talking about.
And it's going to go off and start creating kind of a plan for this action. And you can see there's the browser.
It's up. Uh
33:06
and you can actually see how the model thinks here on the right panel. And this has been fun actually as we've been testing this.
This is available for some of our Google AI subscribers right now. It's kind of in a limited test uh use case.
Um people sometimes will just
33:22
watch this thing uh because it's actually kind of interesting. It will navigate through pages.
It'll scroll. You can see its thought process.
And I mean I remember the first time Greg I saw this. It was like I was just shocked.
Like I was I was just kind of watching something that was like, "Oh, I
33:39
I know how to do that." And it know now knows how to start to do that. And so you can see we gave it a thing here like find three job possibilities for, you know, engineers in New York on this thing climat.
And it's going through it's now navigated to the site. It's finding a job posting for a senior data
33:57
engineer. Now it's searching through kind of a broader set of listings.
It's just going to keep going. Uh, and so what's kind of interesting is you go back to the home screen and you're now going to imagine like a whole dashboard of these tasks you can fire up.
So I'll throw another one in the mix. We were
34:12
talking before we jumped on about I'm in Oklahoma City right now as we're recording this. And this one is like kind of a fun one.
It's like go find five restaurants u in Oklahoma City, add them to my Google maps, places to visit. Um, and so it's going to again fire up
34:27
the browser. There's the browser goes to Google.
does some searches. And what's interesting about this is it'll start to do kind of the searches.
As it goes through, and we haven't sped this up. This is happening in real time right here.
It's going to start trying to navigate through and do the research.
34:44
And you'll eventually get to a point here in the video where it's going to pause and be like, "Okay, wait. Uh, I'm not signed in." You can see this is a signed out browser here, right?
Um, and it'll intervene and it'll say, "Can you help me?" And so these are some of the rails that we've built into the product
35:01
also to kind of let you know about what's going on. And what's amazing now is as people are using these, you can see that they're just running.
And so a lot of the early users may fire up like five or 10 of these. Uh and you know if something gets in in one of these stuck
35:17
states or it needs a human in the loop, it'll kind of change status and it'll be a little flag and be like, "Hey Greg, I need your attention here." Um, and this is a pattern we're seeing across a lot of our Google Labs projects right now. This one is for agents and kind of task
35:32
based stuff. Uh, we have another product for coding agents called jewels where you can deploy lots of coding agents to go fix bugs or build features.
And there's a really interesting you were talking about how do you get beyond the chat interface. These kind of dashboard interfaces or these bird's eye views of
35:49
what's happening. To me, this feels like a glimpse of what's coming in the future where you're able to kind of manage and oversee and give different assignments um in this case to kind of an AI agent that we call project mariner.
Yeah. I mean, you're essentially spinning up interns to do tasks for you.
36:07
It's interesting in a lot of these cases, it's sometimes they're tasks that no human could ever do. And I think to me those are the really interesting ones.
Uh what's an example of something like that? Well, one of the things we do um with our project Jules, the coding
36:22
agent, is um you know, we'll do a bug bash. We'll file a bunch of bugs or now we're kind of dictating bugs and we let AI transform them into bugs.
But it's like you can just put jewels on a GitHub backlog and it'll fix like 30 bugs in a minute like uh do you know what I mean?
36:38
So, it's like you can you can kind of imagine these leaps that are coming. Uh they're kind of superhuman in that way.
Yeah. It's one of those things where I think that we don't even know how people are going to be using this because it's so early,
36:55
you know, like there's going to be so many different tasks that like I'm trying to think in my mind, I'm like, okay, like how would I use this in my day-to-day, right? Like what are tasks that I have?
Yeah. Yeah.
Well, and I think the key to all this is how do you design these AI systems in a way that are both transparent and you can trust them and
37:11
intervene and like this is a good example. It just came back.
Look here. is like needs your attention.
Right. Right.
So it's like I tried to click this you can review it and if you read through the log here it's like we've paused because I'm trying to click the save button but since I can't log into your account would you like me to do
37:27
something else and kind of help here? And so this is actually a good example of how we're exploring and this is why it's all experimental right now but like when is the right time to come back to you and when is the time where over time maybe you build up trust with this and you kind of let it go.
Um, but all of
37:44
these are new kind of human computer interaction design problems as well as technical challenges. That's awesome.
Yeah. All right, Josh, you you you got me fired up.
Like I I first of all, I didn't even know that this existed. Um, you know, I've played with, you know,
38:00
chat GBT operator. I've done a tutorial on it.
Um, I want to do I want to play with this. I want to try it out.
I want to um I want to play with all this. So, I you know, thank you for for getting my creative juices flowing on this.
Yeah, definitely. Well, and if if you or
38:16
others like we put all this stuff on the Google Labs site, it's just labs.google. As far as I know, it's the only site that has like a hot pink navigation banner.
Um, but you can see as you scroll through there's Mariner, Stitch, Jewels. We talked about all these, but um yeah, this can be a place you can go
38:32
get some of the cutting edge latest experiments from Google and AI. Cool.
Well, I'll include that in the show notes for people to go and check out Take a First Spin. And and that's all I ask people do is like, you know, yes, it's great to Josh, I appreciate you coming on sharing all this sauce,
38:48
but it's, you know, stop listening to us and just start playing with the tools, right? Like the best way to learn is to is to play.
Definitely. Thanks for having me.
Thanks, Josh. See you.
See you.