Future of Data and AI: Agentic AI Conference Day 2

Transcript

00:00

02:47

Heat. [Music] [Music] Heat.

Heat. Heat.

[Music] N. [Music] [Applause] [Music] Heat.

Hey. [Music] Hey.

Hey, hey, hey. [Music] [Music] [Music] [Music] Hey, hey, [Music] [Music] hey.

Heat. Heat.

N. [Music] [Music] [Music] [Music] I don't even [Music] [Music] I [Music] feel I know.

[Music] Hey. Hey.

Hey. [Music] I [Music] know.

Heat. [Music] [Music] Hey, hey, [Music] hey.

Hey, hey, [Music] hey. Hey.

Hey. Hey.

[Music] [Music] [Music] [Music] Good morning, good afternoon and good evening uh everyone. Uh we will go ahead and get started.

I will set up my screen. I will start sharing my screen and then we'll go ahead and launch the day two of the conference.

Okay, welcome back everyone. Uh I hope all of you enjoyed the first day of the conference and today I will uh kick it off.

I will take maybe few minutes uh to get started and then I will hand it over to our first panelist uh for the fireside chat. Uh so thanks for coming back.

Uh and uh um let's let's go over um where we are in in the conference right now. We started uh the day yesterday with understanding the how LLMs actually reason and plan.

Um so the session with uh led by Sam uh it was about how do um how do large language models reason, how do they plan and really all of those issues around uh that reasoning and planning h what goes on inside the mind of an AI agent. Um the next session was about context.

Um and uh as uh I was there along with Bob, Emil, Talith and Jerry. Um and then uh what we discussed there was that while large language models they are um they are incredibly powerful.

uh they have a very good understanding of uh um the knowledge component of uh the the language component of u u agentic behavior. But when it comes to the knowledge part of it, context is relevant, right?

So if you go and ask a large language model um about something that is more relevant uh that they did not see during the training process um the large language models might not know something that happened um anything that happened uh between the time they were trained and uh and today right so and then for that purpose um you need to give the the these large language models some context and in addition to that Um there is some proprietary data. uh there is uh some uh you know your organization's a specific context maybe your personal specific context the large language models may not know that and for that we need to have these uh retrieval systems and um there there is this broad notion of what we call retrieval retrieval augmented generation and it depends upon who you ask when people there was this I think we were joking during the panel I mean rag is dead, right?

So, uh, a rag is not dead, right? So, rag is people people think that when they say rag is dead, they they talk about the naive rag, right?

So, basically one retrieval call and one generation call. Uh, but modern uh modern retrieval systems, they have also become agentic.

And then the overall infrastructure that we build it it actually involves not just one retrieval system, we have multiple retrieval systems. And then we go back and forth and try to reason until we get to a correct answer.

Right? So that's that's the nature of things.

And we also talked about some of the challenges around structured data. Um for instance, LLMs are notoriously bad at uh at doing math or maybe dealing with structured data.

How do you deal with that kind of data? So we we discussed those kind of things and uh and then we moved on to our third panel.

Now that we know that LLMs are incredibly powerful and then the context gathering is also in incredibly powerful, how do we design agents that are trustworthy? And trustworthy is a it's a fairly broad term.

Uh how do we know what they are doing? What we call observability?

Do we log every every activity that uh uh my LLM agent made this function call to a GBT4 or it it consumed this many tokens. Um and then it made a function uh it made a call to uh a vector database and all of that.

Do I am I aware of what activity my agent is involved in? Is there any supervision and monitoring of my agent or not?

And then uh an extension of this discussion was how do we measure the performance of and uh an agent or an agentic system. And uh and finally uh in the last uh session we had this uh we had this uh uh tutorial uh from uh by Hannah and Anurag from uh Amazon Web Services.

They work for uh uh the Genai Bedrock team. They actually gave us uh how do you build agents in AWS.

So that is what happened yesterday. Um, and today, uh, we will get started with, uh, one of my two personal favorite people on the internet.

Uh, Luis, we'll start with Luis and Josh. And then once Luis and Josh are uh are done, we'll go to um um a tutorial by Hamza Farukq.

he will talk about um he and Ali both of them they are going to talk about how to create vertical specific AI agents then uh we have another uh session where we'll talk about you know how do we see the future of agentic AI and then we'll close the conference in the end uh with a tutorial by um by Paige from Google DeepMind and she's going to walk walk us through how does Google deep mind AI stack uh look like. So, so the next session uh it is uh it is going to be uh with Luis and Josh.

Uh we'll get started shortly. Um I think uh I don't need to introduce both of them.

I think if you are here if you have uh you know if you follow any content I mean if you if you are a learner in data science if you um you have been following content in AI and machine learning these two names are the probably the most prominent names out there as educators as I said my personal favorites both of them uh they have made um it very easy for people to follow and understand some of the nuances of math and um and machine learning and AI Okay. And uh without further ado, I will have uh Luis and Josh, they should be joining us.

Hello. Right.

And Josh is here and Luis is here. Great to see you both.

Hi. Happy to see you, Raja.

Thank you for the invitation. Happy to see you.

Right. So, you're also one of my favorite people on the internet.

Yeah. Thank you.

In the world. So, thank you.

You're very kind. Okay.

So, Oh, yep. Then Luis and Josh over to you.

Go ahead and I will get out of your way. Fantastic.

Hello. Thank you very much.

So happy to be here. Hi Josh.

Hey Louis. It's good to see you.

Good to see you. We have all this uh recurring chats and and uh about AI and and and all things.

So yeah, thanks thanks everybody for being here. Thank you Rajan and Data Science Dojo for uh the invitation.

We're very happy with the we had a great time uh attending the the talks yesterday and uh very excited about today and always happy to be here with my friend Josh Starman who has an amazing YouTube channel called Stat Quest and uh two books called uh uh the the stat quest illustrated guide for AI and for neural networks. Right.

You got it. And Lewis, you've got a book too.

Yes. Groing machine learning.

So I like the title of the session. Groing meets BAM.

very creative. I love it.

Um, yeah. So, I think today's going to be pretty free flow.

We can take a lot of questions. We can talk a bit about agents.

Uh, we can talk about LLMs. Uh, we we like to talk about LLMs, reinforcement learning, everything.

But, uh, we we'd love to hear from the crowd. Uh, where is the the main place for questions?

Is it going to be the YouTube one or the LinkedIn one? Uh, it is it is in the Q&A.

Um, Luis, can you see it or not? uh in the Q&A.

Oh, in the Q&A. Okay, perfect.

Yeah. And if you I mean I don't want to, right?

So, but I will be here if you need me. I will be Oh, no.

Please, I'd love to talk to you. Um let me see because I see a lot of questions on YouTube as well.

Um uh I see a lot of people saying bam and the YouTube questions will route them to we'll route them here. Okay.

Yeah. Okay, just uh so you can just focus on the Q&A side of it.

So, can I give you a little anecdote about something that's a little embarrassing? Okay, let's see.

Uh it wasn't that long ago that Lewis and I were both in Switzerland and we were at a conference and I was on a panel on agentic AI and for some reason they decided that they wanted to start with me to talk about like give us an example of aic AI and [Music] um I gave the example of rag and which I think is legit. It's like the most basic most fundamental uh type of agentic AI.

It's it's um uh but uh but what uh but as soon as I said that this guy I can't remember his name uh he looks at me and he goes that is not a genic and I felt so ashamed and embarrassed for the I don't think I said anything and for the rest of the panel discussion. Um, but I think that's a I mean I'm gonna be honest now that we're here.

Uh I we're going to start with that is like um Lewis is that agentic AI do you agree with me or or is well yeah to me that's pretty aentic AI good I don't know was there a followup to the question you got or was it just like that's not that's it or or was there like this is what aentic we just asked them back and immediately they went into like incredibly elaborate in super complicated um aic and and those are very cool and we can talk about those as well. Um but uh but I thought like if you if anyone's seen my videos, you'd like to know that I start I try to use like the simplest example rather than the most complicated example.

Um because it's easy to wrap my brain around a simple example and sometimes very hard for me to understand a really complicated one. And and I and I feel like it's a great foundation.

And like if you can understand that then it's a real small step to understanding the more complicated models. Uh yeah exactly the same thing for me.

I I feel like uh I try to understand the most basic uh agent and to me the most basic agent is rag right because it's the first time the LLM starts doing an extra thing that's not just talking like it goes oh I'm going to make one Google search that's an agent right that's like doing stuff right another one that I like is the the calculator right like if you ask an LLM how much is 5 plus 7 well 5 plus 7 appears a lot in the internet so like I'm it can figure out from the words that it's 12. But if you say 5.1 through three plus 7, you can easily come up with an equation that no human has ever thought about if you put enough numbers on it, right?

And a simple one. And I don't know if from just words and and guessing the next word an LLM can add.

Maybe I can, but just from just from seeing words appearing next to the other word, I don't know if you can figure out the concept of addition. I feel like I feel like that's a that's a perfect uh opportunity to hallucinate in a bad way.

Oh, we can go. Yes.

Yes. Yes.

Yes. Uh so instead of hallucinating an answer myself, I'm just going to say yeah.

Like like I feel like that the Yeah. the model will go crazy, right?

So if it instead just opens the calculator, does the the thing and comes back with an answer, I don't think you can top that. Like I I I don't think we can have technology that makes that better.

Right. I think the answer is bam.

Bam. Yes.

The answer is you press the button the calculator. Bam.

You get the answer. So a calculator is is another agent, right?

And I feel like we can get more. So that's how I wrap my head around agents.

And then I just imagine more stuff. But one thing I actually like to think of is I think all of technology always makes the same entrance or most of technology at least I don't know the internet like we you know we were around when the internet started maybe not everybody here but what was the internet first the internet was information in information out you couldn't do anything with it you would ask like you would you would not ask like you would just look at a page read the information maybe learn something and go do it but it wouldn't do anything and it would be I would find it ridiculous If somebody tells me that I'm going to make a bank transaction using the internet back when the internet, you know, when it was Alava Vista and when and when you would have to log in with with the phone and it made some noise and stuff like you tell me I'm going to make a bank transaction on that thing, I'd be like I'm never putting my money there.

But now it's common, right? Like now now the internet is for talking and for doing.

So it went from information in information out to doing stuff, right? And then you know 10 years ago AI started raising.

It was predictive AI, right? Yeah.

And it was also information like it would tell you from this data set I think the price of your house is that or from this data set this this is spam or not. Like it was just information and then it started actually doing stuff right and the same thing happens with generative AI.

can't you can't escape that and you shouldn't because it's it's very positive like you go from the LLM just talking to you to like actually doing stuff opening an API making an API call opening an app opening your calendar uh doing doing things sending an email even even if it's for even if it's for just information like even if you ask it to type a response an agent can do that because it it has it checks it it goes searches and And then you come in with multi- aents, right? Like if five people want to write a blog post, well, one is the the director, the other one the the researcher that goes and finds stuff, the other one the writer, the other the proof reader, etc., etc.

So, it just comes up pretty naturally. Uh that you go from from information to doing to like sort of multi- multi multi-doing.

Um yeah, that's pretty much how I like to see it. I love it.

I think it's fantastic. um because I was worried uh that my example would be too simple and everyone would be like, "No, that's not aic AI." Um I uh I you know, I've since wised up a little bit and and I've got some examples that I think are pretty cool.

I've got So I've got a friend uh that I've known since I was probably 14 years old. Uh, and he's always starting up companies like once every six months he's got a new company and he's got this new company called DevRa, Endeavor Analytics and he's got two products that I think are super cool and are great examples of sort of more complicated agentic AI and one is uh um so he's got a the the DevRa analytics or Devra analyst.

Uh what it does is you give it a data file any kind of data and what it does is it it looks at the column names and and the column data types and based on those it can kind of guess as to types of graphs and charts uh it could generate based on that. It also uh guesses sort of if there's missing data sort of what to do about that.

Uh, and it basically because because I learned all these cool algorithms for dealing with data, but 80% of the time what I was doing is just reformatting data. And so this does that for you.

And then it, you know, generates some nice graphs and charts and summaries. Uh, and then what it does is it runs the whole notebook a couple of times to make sure it doesn't have any errors.

Uh, and it and it does all that without you really even having to tell it what to do or even tell it where to start. it it just starts poking around and and and trying things out for you.

Um, and I feel like that's sort of like, you know, that to me is is is sort of a an what I like about that example is is it's not stock trading, so it doesn't stress me out. I know a lot of people like to use Aenic AI for like automatic stock trading, but that just freaks me out a little bit too much.

I'm very, very conservative with my investing. Um uh but what I like about it is is it's you can I can for me as a data analyst I can I can see the the merits of it like I can see like oh I would want to do that right now every day and never have to like reformat data for the rest of my life.

That would be just that alone, that aspect of it alone of just uploading a thing and having it automatically format it in some useful way uh would would be a gamecher and the fact that it generates a few graphs and charts on top of that is just sort of icing on the cake. Yeah.

Yeah. I mean I think this enhances what what what I think uh I think this a healthy way to see AI in general, not just agents, which is like a like an assistant, right?

like a tool that you can use like I wouldn't make it make decisions big decisions for me like the stock you're saying or something like that but I think I would if you can use it as something that that helps you do the stuff that helps you make yourself the decisions I think that's sort of the perfect h human hybrid human machine hybrid right that's exactly yeah um I see some cool questions should we get them in order or I don't know actually I I want this one that I think we're sort of starting. Uh I think we can get to all of them.

Tu just says, "What is a common myth about aentic AI you want to bust?" Um I may say something. I think it's very similar to what we're talking about, right?

Like I think a a a gentic AI is people think it's going to become AGI, right? And I still I still say no.

It's it's it's a tool. It's just a more elaborate tool.

Uh, I don't think we're close to to something really mimicking a human. There are some very very things of a human that are not that that are far from an agent or from an LLM.

Things like empathy, index, emotion, those those things are hard to to to to mimic for making decisions. For talking, no, they can talk in an emotional way, but not for making decisions.

So, I think that would be a a buzz. Actually, I just heard a funny sentence.

It says uh LLM hallucinate and and agents agentics also hallucinate but now with an agenda. I don't know if that's better or worse.

Um but I would say that's a that's a myth that that I would that I would bust. Yeah.

Yeah. Uh so how do we do we just say answer live here?

Okay. I click on answer live.

Anything else you you you find interesting in the in the list of questions? Um well um someone asked uh do you think Genai is changing product management product management and I'll be honest I don't really know much about product management but I would be surprised if it was not changing things because regard most products these days at least in my little small world have some software component to it.

And if there's anything that's been updated and turbocharged by uh genai, it's also um coding, right? Um, yeah, coding like that's another thing my my friend's little company does is is it you he uses aic AI to you you say hey you know it's like it's not exactly vibe coding but you say hey I want to make a program that you know does this thing and it create creates all the little files it's got you know a header file it's got one for doing the a user interface it's got another file for um sort of helper functions and it sort of like it organizes everything in a nice folder for you and it runs it a few times again to make sure there's no major bugs.

And um so coding I I mean coding is infinitely easier than it than it used to be. And even then it wasn't too bad because you could just Google everything and look at stack exchange.

Um, but now it's like, you know, it's like having stack. I mean, I think literally they trained it on stack exchange and now it's like right in the, you know, at at the prompt stack exchange is like being spewed out if you just hit tab enough times.

Uh, so that's a big change. But I also think with like um I don't know if it's agentic but but generative AI like stable diffusion and sort of graphics generation um you know it used to be you'd have to hire a a designer to to do all that and and that's you know that's good for them uh but it it's bad for people like me that just want to see if something works and I don't have enough money to hire someone um at this point.

So, if I want to come up with a company logo or if I want to come up with a product logo or product name or something like that, it's just uh it's just easy to kind of like come up with with with quick rapid prototypes that I can at least get going with. Um rather than sort of going here's my little, you know, I'm not the best artist ever, but you know, everything looks like Stat Squatch to me.

So, or the normal source. Or the normal source.

Yeah, you get one of two options. Are you Are you adding a a a character for an agent or something?

Maybe a secret agent or something like that. Exactly.

That'd be cool. Come up with something.

Uh, you get the idea, right? I mean, all I think I think if for people wanting to start up companies, regardless of what product they want to do, it's just imagine, you know, it's like everything you want to do is a a lot a lot closer and easier to do than it ever was before.

at least for for software products. Yeah.

Yeah. I mean, I think going back to the product manager, uh I think an an agent works a lot like a like a product manager, right?

Like I I I admire their work a lot because they can sort of plan they they combine they they talk to the technical side and they talk to the human side, you know. So this is a difficult task, but I think you can make a lot of tools for that.

maybe a lot of like like simulation tools if you simulate, you know, how it's gonna how the how the the team is going to build the product or things like that. Like I feel like I mean there's a lot of tools for product managers that are pretty useful and I think a JNTKI can can do some really serious uh tools for for for a project manager.

Um that I don't think it'll replace it. like I don't think it's going to replace a lot of things, but I think it'll it'll definitely impact it.

Yeah. All right.

Uh Deas is asking, "What is the best tool to generate agents? Have you used like I've used I've used Crew AI.

I took a course on that with uh deep learning. It's pretty good." Uh Langraph is always good, but I haven't used it.

anything you've used or the only one I've used is lang chain so I can't compare anything um and the only reason why I use lang chain is I think that's what they used in um in oh god not louisisrano but um that's me I know but Jay Alamar's new Oh yeah yeah yeah yeah uh Yes. Yes.

I think he uses that a lot. Yeah, he used link chain.

Anyways, all I can do is endorse that book. Uh oh, that book is amazing.

Uh link chain while you're at it. But we should we should give people we should give a shout out to Jay's book.

What is the name? Is it by Jay Alamar and Martin Groenor?

Let me look it up. It's called Hands-On Large Language Models.

Uh yeah, that's the best place to learn about about this kind of stuff. Hands-on large language models, language understanding and generation.

Yeah. Um yeah, cool.

Well, let's see other question. Anything else that one thing I think is kind of interesting is uh one of the questions is uh what are the biggest challenges when it comes to adopting implementing aic AI systems in business processes?

Uh, I think I think one of the big challenges for anything agentic is or anything AI these days is coming up with a really solid prompt and that and the and all the sort of like the the secret prompt stuff that you have underneath it, you know, like where you're where you're trying to direct the AI to do your bidding. Um that's the kind of you know I think that's something that kind of you have to fine-tune like crazy uh to get the it's like a in in in some ways I feel like large language models are are like somewhat like random in in and how much or how little they will hallucinate you know and you you can control it but it's but it's not like a you know it's not like normal coding right we don't just like change a variable here or there and all of a sudden the behavior is different.

You have to come up with a prompt. Yeah.

May not be effective uh for for your goals. And I think to me that's that's kind of a to me that would be super stressful because you're kind of you kind of hoping you're your your success rate will be high enough that um that uh that you won't get in a lot of trouble.

You know, the company won't get really mad at you. I mean, the goal isn't to be perfect.

That's what I keep telling people. The goal is never to be perfect because nobody's perfect.

The goal is just to do this better than it would be done the how you were doing it before. Um, and but I think a lot of that is coming up with like a really elaborate long comprehensive prompt that kind of covers all the bases.

And I think that's often easier said than done. Yeah, it requires a lot of iteration not for just the agent but the human as well.

Like I think we're learning a lot about how these models operate and how to talk to them properly. Yeah.

Um I also feel like another another challenge that happens not just inki but it's it's how much autonomy you want versus like how much power right like you could create an agent that is completely deterministic. say if a do b do this do that do that or you could and it's limited or you could just create something very that that changes that you know so I it can be more much more powerful but it can be like much more erratic so I feel like that's a big big change to decide how much how much freedom do you have for the agent to morph versus how deterministic you want it Yeah.

Um, yeah. Um, here's one.

How much uh fine-tuning LLM versus grounded rag? Yeah.

I feel like you need both. I mean, I feel like you So, finetuning I like to see it as like me muscle memory, right?

Like I don't think it's I think I think uh LLMs learn not the way we learn stuff which is remembered or the way a database but more like getting used to stuff right the way I we get used to doing it like I write with the right hand you know not with the left hand I would need a lot of muscle memory to learn how to write with the left hand that's learning muscle memory takes a while whereas if uh some fact changes I in my head I change it immediately right like and So I feel like if we have a a healthy uh balance between what's muscle memory for the LLM versus what's like database like what's information stored in a database then we know how much we need to do fine-tuning and and rag but I think a good system needs needs both things. Yeah I Lewis I love those analogies too I think it's absolutely fantastic.

Thank you. Um, but yeah, it's it's uh but what Lewis was saying, if it's factual information and and it's could change um and you want to be able to pivot on that change, you you want that to be in the in the rag uh database.

Uh, but if it's general tone, if you want if you want your LLM to have a certain style, a certain, you know, behavior and a certain sort of like pattern to how it behaves, uh, that's all in the fine-tuning. Yep.

Absolutely. It sort of depends on what you want to control and and how much you need to control it.

It could be off the shelf you're 95% satisfied with with the general behavior. really all you need to do is swap in the facts uh and have those ready in a rag database.

Uh but often times like you know if it's like customer support or something you might want to get something that's more specifically fine-tuned already in that direction. Um, and I think you can do that too, like you don't have to do all that yourself, right?

You can you can find the, you know, download uh LLMs that have already been fine-tuned in a Yeah. certain vein and and just start with those.

And so you can even even though all you do is rag, it's, you know, there's been the fine-tuning sort of already in in there. Yeah, I feel like LLMs definitely need a lot of fine-tuning just just to get the product out, right?

like just originally like you need a lot of like to teach it how to do basic things like question answering those things are not learned by just reading all the internet and trying to guess the next word right so a lot of fine tuning comes in then you can also do fine tuning with your own data right um so yeah oh here's something uh Sunday says what's the difference between aenticai robotization and robotic process automation So correct me and so feel free to uh feel free to chip in. Um I I think robotization is just when like the robot moves and does stuff, right?

Like that's the only one there that's with objects that move. The other two are more like in the computer.

Um and I think RPA is probably more uh deterministic, right? It's more like you give the agent the the instructions.

does this, does that, does that, and an agent TKI is kind of like injecting that with LLM. So you would have a steps would have an LLM that actually doesn't do something deterministic, right?

A function that says if A do B versus take this prompt and see what the LLM comes up with and may it's very likely it'll give you the right thing, but it can be very it can be very chaotic. So that's why that's why one of the challenges of it, right, that we're talking Yeah, I'm glad you answered that one.

There's no way I could have answered that. So, I feel like there's there's so many like production questions and we're both educators.

So, people want to keep in mind we we talk more in high level. Um, yeah.

What do you think about the rise of AI behaving badly like blackmailing people and refusing to turn off? Question by Brian.

I think it's interesting. That's always an interesting question.

Yeah, it's funny. We had a a um a a a podcast with Raja and I think that that question came up, right, Raja?

Not just with agents, but with with AI in general, right? Like how do you feel about it behaving badly?

And I I feel like I have a 50-50 approach, you know, 50 obviously 50% is let's deal with the problem like a doctor, right? like in a doctor would say, "I'm going to 50% deal with your symptom because I don't want that pain there." And 50% deal where it comes from uh because that's the long-term cure.

And I think we need to have those two uh 50% of like, okay, you need to put an end to this particular thing. like you take the LLM and put a block that says if the language gets this bad if you do this bad if things like copyright you you know you need to look at look at those things in the technology way and say how do I end this and then 50% of like what is this right what what what problem that we already have in society is this uh enlarging because a lot of times l a lot of times AI just uh rises a problem that already existed that we're okay with because we were born when it already was perpetrated but now AI just made it bigger you know so a lot of times that uh the language of chatbots like abusive language well maybe that's because it's trained on humans you know uh that do that so I think I think a 50% approach should be let's let's thank the eye for actually pointing out a huge problem in society uh and and I think if you have I I this is a very vague answer but I think If you have that 50/50 approach, that's sort of the way to that I deal with AI.

Yeah. Yeah.

What do you think? I mean, um, it's it is interesting though that we do tend to tell AI all of our secrets.

That's true. Yeah.

and and it does, you know, for to a certain extent it stores all those uh in sort of like the context and it could easily turn against us if it decides we're you know I don't know it's it's it's interesting. It's a it's funny.

I never really thought about that, but I but I do think it's interesting in that we do, you know, we worry about like, oh, are we uploading all our secrets to the cloud or something like that, but even if it's running locally, still has our secrets. Yeah.

Um, and the whole idea of of of agentic AI, well, it's not the whole idea, but one of the main ideas of agentic AI is that it's it's it's um it's sort of forward acting. It doesn't you don't just have to tell it what to do, but it but it sort of like anticipates what needs to be done and does that.

Yeah. And so I could see how given you have something that's looking at this context and trying to anticipate what to do next um that you know without really fully understanding everything who who knows what it could it could it could I mean I I could actually imagine you know typing I mean don't get me wrong but I could imagine typing something in there that causes the agentic AI to call the police and send it to my house, right?

Yeah. Maybe I'm like, "How do I put out a fire?" Um, you know, or a hypothetical question and then uh and then the fireman show up.

Exactly. Next thing you know, there's a there's a fire truck outside and a bunch of dudes storming down my door.

I could actually see that happening. down and from there you can kind of imagine you know that's like a nice friendly oh good we put out a fire in the kitchen that's a that's a helpful thing but you could imagine uh people abusing that as well uh once you power um so it's very interesting to think about I like that question because that's a I'm going to be thinking about that one for a long time no I'm definitely torn about like I I I I think it's concerning that like like all the information is is there.

And uh yeah, there's there's definitely a trade-off between privacy and and safety that that that needs to be addressed that is this is putting a in in Jeffrey. Yeah, I see a couple talking about some good questions, you know.

So, one is like how do you use AI in highly regulated environments like banking? Uhhuh.

in com someone says compliance industry or anything that's like so we got a couple of questions where people are asking about areas that are you know like for example banking financial transactions you want those to be traceable and deterministic I'm just reading from the question and so what is the role of AI in those situations and I believe it still has a place um uh where you know how Lewis and I were talking at the very beginning. One very cool thing about AI and especially agentic AI is that it uh it can act as an assistant, right?

As a as an as a as someone that helps you do things rather than making all of the decisions for you. Um it can just help you make those decisions.

It can try to present the data. You know, like I was saying early on, we can, you know, generate some summarizations of the of the data, generate some graphs and charts, but ultimately you have the responsibility landing on the on the sort of the management and the management being the person who's part of that that loop.

Um, yeah. So I can see it uh being very useful in those contexts, but I can also see it sort of uh being used only up to a point where and then someone has to has to say yes or no or make some decision that they then will have to take responsibility for.

Um and I I can see it used in that context even even in highly regulated areas. Yeah.

Yeah. I think some some places are just more risky than others, right?

Like if you want to try something in I don't know in a recommendation system you can you can go nuts because it can you know if you look at a recommendation and it's a bad one the world doesn't end right uh but in a bank or in a healthcare or something it's much more risky so many times uh those industries take a little longer in in adapting um but I think people saw several times like if you take too long to adapt to the internet or to regular AI or to digitalization or anything. Uh you you pay the price.

So I I think many industries are at least saying okay when I hear a buzz word I will listen and so I think that's lucky for something like aentic AI because if you can't ignore it by now you know um but but yeah I mean I think you can experiment in other fields like I think by the time AI got to say the banks or something it was already tried by a lot of other companies. So I think with with agents yeah you start being more adventurous if you have if you are in a field that that is allowed to be more adventurous and and then and then it propagates to to other places.

Yeah. Yeah.

I see a question by Ragna. Do you folks see lesser need for coders versus aentka platforms get better for coding?

So can I take a stab at that one? Yeah.

Yeah. Yeah.

Coding in some ways is definitely a lot easier uh than it used to be. Um one of the use cases I use it for is is I grew up learning um older languages shall we say and I'm not an expert at Python.

Um but what I've discovered is I can write the code in whatever language I want and then I can say make this in Python and actually does a really good job at that. uh translated code and that's you know in some respects that makes me a much more powerful versatile coder and I can now code in basically any language and I can actually learn the language as I try to figure out what the AI did and how they translated things and I think that's very useful but I also think um you know the nature of coding will probably change uh skill set however the general skill set of being able to break a problem down into small steps um will not change.

Um I think you know that's that's forever what we have to do is we have to and even with large language models and agenic AI we have to come up with ways to take complicated problems and break them into small steps and sometime you know the the AI might be able to speed that up but uh but forever I think that's something um that that that that the sort of coding mentality will always be useful for uh it's And um my friend Brian who I keep bringing up because he he created those those Devra applications. He has a great analogy which is you know uh 150 years ago the only way you dig a hole was with a shovel.

I mean you might use a rock might use a stick or something like that. I love speaking if you wanted to dig that hole and you wanted to dig it efficiently without taking all day used a shovel.

Uh and then along came the the steam shovel, right? Um, and that was awesome, right?

And you could either keep working on your shovel or you could learn how to drive a steam shovel. And if you learned how to drive a steam shovel, uh, you were, you know, you probably got paid a little bit more and you, um, you know, and and you had a skill that probably got you around more and and and then and so that's something that that we need to adapt to.

But also at the same time because even though the steam shovel showed up, it didn't mean that we no longer needed construction workers, right? We're still building cities, we're just building larger, more complicated cities.

We've got skyscrapers now. Instead of just maxing out at three stories like we used to, we can go however many because we've got people doing new things with new techniques that they didn't have 150 years ago.

And I feel like that's the way headed to today today where all of a sudden we've got this steam shovel and everyone's like kind of a little freaked out, right? Because that steam shovel looks like it's going to replace all these construction workers.

But yeah, but I think the result is is it's actually going to be the opposite. But I think there's going to be even even more demand for uh uh for for people that have that mindset of how to code and how to solve these big problems and how to piece things together.

Yeah. Build more elaborate, more amazing sort of architecture and and and programs.

Yeah, I I love that analogy. I love that shovel analogy because it it shows that yeah, like at the end of the day, what's human is not the act of digging the hole.

But for example, like knowing where to dig the hole, like that you can come up with the best shovel in the world, like the most technological, but a decision like where to dig like that's that's much more high level, you know. Um, and I yeah, I feel the exact same thing about coding and uh I I do a lot of coding with Copilot.

Um and I think uh the syntax gets simpler and and I I welcome that you know uh it started with punch cards and then assembly and then C++ and then Python and now it is English. Um but the concepts are the same.

I mean I I don't think that the if you're writing code on punch cards or in copilot like the the highlevel ideas and the algorithms and the and the code thinking does not change at all like that's the same the syntax gets easier like you're you're speaking in an easier language right and then more people are able to to do it there's always going to be someone writing assembly right there's always somebody there's always going to be people writing Python and and C++ ++ what I do welcome is that many people who are not who who are not good at the syntax but can think logically and know about solutions to other problems outside of of AI and in the real world are now able to code and and build solutions whether it's a web page or an app or something. I I do welcome that.

I I do welcome the fact that there but no we're not going to run out of coders. I think it but I think the coders are going to be more the ones that you know called the actual LLM.

Yeah. The ones that are doing the where the rubber meets the road and just more people are going to be able to use it, you know.

Exactly. Um like now a lot of people use computers.

There was a time where only the experts would be able to use a computer, you know. So I think coding is is going to be like that.

So, uh, yeah, we have another question that is very near and dear to my heart. Okay, let's hear it.

Might if we skip around a little bit. I'm all over the place.

Yeah. Okay, so the question is, how do you detect hallucinations or measure accuracy?

And I could actually give a whole presentation on this. Nice.

We could go we could go for another hour on just this one question. Uh, I have some theories.

Uh there's there's a lot of there's actually a lot of active research in this area. Uh one and I actually don't know the details of this, but I talked to some other people and they said, "Oh yeah, yeah, with LLMs, you can actually see it in the attention matrices.

You can kind of see where it's going to hallucinate based on on the attention." And I actually don't know the details of that, so I can't actually I actually don't understand it. But but all I'm really I'm bringing that up is uh you can you can do some research and you'll find out some cool ways.

Uh but I've also I've got my own sort of like simple easy way to do this. Um one example is is is rag.

So when we use rag we take a document we make it into chunks or a bunch of documents make them into chunks and we create a vector database based on those chunks. And essentially what that means is in some super highdimensional space we've got these chunks all over the place right uh mapped out somehow right and when you come along with your prompt uh the LLM what it does is goes well well this prompt what what chunks are it are is it closest to is it close to this chunk over here to this chunk and if it's closer to this chunk then I'm going to I'm going to pull that out and We're going to we're going to use that as part of our answer uh as part of this rag implementation.

Well, one thing we can do is we can actually measure the distances. I mean, we do we go this chunk is closer to this prompt than this other chunk.

But we have so we have those distances. We've already done the hard work.

We can just look at them and we can go hey our prompt and this chunk are very close to each other. you know, their cosign similarity or whatever metric we're using, we can say they're very close to each other.

So, I've actually got pretty high confidence that this chunk is going to actually relate to the prompt. In con contrast, I could have a prompt that's sort of floating in the middle of nowhere and it's close to something sort of, you know, but everything else is kind of far away.

Uh, and it'll return like the closest thing. It'll say, well, this chunk is the closest to your prompt, but it's still pretty far away.

And I feel like that that we can use that distance as a way of of giving ourselves confidence that we are or are not sort of negatively hallucinating and we can maybe have a measure of accuracy or a measure of of just I don't know accuracy is the right word for it. I I think of it as a measure of confidence distance.

Yeah. Yeah.

Yeah. I like that.

Yeah. That's interesting.

I actually this is the first time I see some metric metric of hallucination because I feel like I feel like I'm jealous of all of us 10 years ago when when AI was so easily correct gradable like you know predictive AI you just say how many times did I get the right solution divided by the total number of of times I tried like it's like grading a multiple choice quiz right like evaluating a predictive machine learning model is like grading a multiple choice quiz whereas evaluating a generative AI model is like grading a essay type so many times. Yeah, you can grade, you know, out of I I like your your your approach, you know, out of the actual solution that I was thinking the actual sort of let's say perfect response, how far did you get in terms of cosign distance or things like that or how many of the of the many things points I wanted you to hit, how many did you hit?

And I feel like we can also use LLMs to grade themselves because one thing that the LLMs are interesting because they have a strength and a weakness and the strength is in understanding which is predictive AI right like like we are better critics they are like we are humans are better critics than creators right like I can watch sports and be like oh why did he do that so stupid but then if I put me in the field I won't do anything right uh so LLMs are a lot like that creating is hard for them because they just one word at a time whereas if a evaluating something they can look back and see the kilometers behind. So I feel like LLMs can be graded grading their own selves, you know.

I feel like we could we can we can use that a lot. Yeah, that'd be fantastic.

Yeah. Well, I have a question that I want to ask answer just out of guilty pleasure.

Uh Sagar says, "Will quantum computing bring more autonomy to aic AI and get us close to mimicking humans? How far are we from achieving that?

Um, I love I love quantum computers and I think they I can get super philosophical with with quantum computers because I feel like there's a there's some randomness in a quantum computer that we don't know where it comes from and it it could easily uh you can easily get philosophical there but I will get more concrete. uh I think genai in particular will benefit a lot from quantum computers because a quantum computer is a generator whereas uh a classical computer is a deterministic machine.

So I think classical computers are more for for deterministic stuff like predictive machine learning right like the email is spam yes or no right like I think the house is worth $100 or something like that that's deterministic as it's a formula u classical computers cannot I'm going to say a bold thing but they cannot generate random stuff right because they they can generate pseudo random numbers they can't even generate a coin flip you know they can't generate a simple coin flip they have to pretend because it's all deterministic random generator whereas a quantum computer is a true generator. So actually we ran some experiments to see how much if if this whole randomness is is actually better and this is one thing that sort of happens when you give a classical computer a bunch of data and say generate more that looks like that data it starts finding data that's close to it right like if I have this point and this point finds the midpoint and something around but what if the data is uh you know thinks that what if there's a point that satisfies the rules of this data, but it's like here and it just happens to satisfy them, but it's not it's not geographically close, but it's logically close, right?

Uh it just it just happens to be that all this all this data has a particular rule. And so does this point.

And we found that quantum computers do sometimes achieve better results at finding this point over here at sort of understanding the data. So I'm excited for when quantum computers come more because I think I think generative AI will get better.

Of course, you can't compare the quantum computer this big to like a trillion parameters that's lying in some data center somewhere. So it's an unfair competition.

That's why it looks like they generate better. But I'm excited about quantum computers more for generative AI than than for predictive actually.

I love I love Louis. You're the only person I know that actually has real experience with a quantum computer and quantum algorithms and all these things.

It's it's such a treat to hear you talk about these things. Oh, thank you so much.

Thank you. Yeah, I do enjoy them a lot because they can be completely uh unpredictable literally.

Um yeah, I we All right. How are you doing?

Have time for maybe one more question, but before I think we have time for one. Yeah, go ahead.

there was a contest uh to do a a giveaway of my new book, The Stat Quest Illustrated Guide to uh Neural Networks and AI and the Stat Quest Illustrated Guide to Machine Learning. And and I want to announce the winners.

Um we have three winners. Uh we have Vidya, which is an amazing name.

I love that. Uh, and then we have Jim Stokes, who I guess also has an amazing name, and uh, Luis Afanso_.

Uh, you are what a great name. Yeah, right.

We got some great names. Anyways, thank you very much for participating.

I hope you guys enjoy those books. Um, if you do, please post about it on social media or like LinkedIn or something like that.

Oh, it's always nice to hear that people are enjoying the books. So, thank you.

Well, Wow, that's awesome. Congratulations to the winners.

Um, is is is Nabihad here by the way? Because I think we also had a a contest for uh for groing machine learning, but I I actually don't know the winners.

Are you there? Let me uh let me check.

Please can can Yeah, I'm so sorry. I'm behind.

Yeah. And we'll just uh we'll inform you maybe in a few seconds.

Let me check the Okay, perfect. So which which which book is it charge?

The neural networks book or the AI? I think they gave away uh both of them.

The machine learning and the AI books. Um that's awesome.

That's super exciting. Yeah.

Okay. The team is saying that they are already in chat and uh during graing machine learning is prize.

These winners are okay. Okay.

And let me see just and you guys can continue. Just give me one moment and I will get back.

Okay. Question.

Sounds good. Someone asks, "Do we think AI would destroy the middle class job market?" Um, I don't think so.

Um, I just think it'll change just like, you know, with the construction worker thing. It's it's just because we have steam shovels and power drills doesn't mean we don't need construction workers anymore.

In fact, there's a shortage uh at least in the United States. Um yeah, it's just we build more complicated things and I think AI is going to just let let us do that and it's going to let all kinds of people do that.

Not just we talked about the context of coding, but I think all kinds of jobs will be transformed just just as coding has been transformed but it's not like the job itself is going to go away. Yeah.

Yeah. Some jobs will disappear but others will appear right with the internet a lot of jobs left and and a lot of jobs started right like like full stack but yeah I mean I think that's more Louis sorry to interrupt right so I think you should you need to check your chat uh so the three names have been sent to you already I'm so sorry I can't see the chat uh we have to close the Q&A oh there's the chat oh my god I'm so sorry I missed it um okay I I I have the winners so excited So the winners are Vini Cashup.

Uh this is a whole a long Shiraas Sar and Tushar and uh I don't know the last name but Hooray. Yeah.

Congratulations everybody. It's fantastic.

Yeah. So excited.

And should we wrap up? It's I mean it's I mean it's Yeah, I think we're good to wrap up to go.

So, um here I've got a little something. Oh, the ukulele.

Love it. Thank you Data Science Dojo for having me and Lewis sit here and talk and answer questions and give away books because that's so cool.

Hooray. Hooray.

Well, thank you everyone for your great questions. Congratulations to the winners of the of the books.

Thank you, Roger, for the invitation and for such a wonderful conference. No, thank you so much, Luis and Josh.

Uh definitely uh we'll see you soon in uh some of the other events. Looking forward to it.

Perfect. Thank you so much.

Thank you so much. Have a great day.

Thank you. Bye-bye.

Okay. So, we are going to take a break uh uh from the next session starts at 10:10 uh a.m.

um my time uh Seattle time Pacific time. But meanwhile uh we did receive a lot of uh questions about our Agentic AI boot camp.

So, this is a break time. There's no session happening.

I'm going to go over our uh during the break I'm going to go over over our agenti boot camp that uh uh you recently and u announced we will have the first cohort. We have been doing a large language model boot camp for quite some time but this is uh the large language models boot camp is a more um call it more broad and more um you know end to- end coverage and for some of us who are uh already uh familiar with some of the fundamentals uh for them we have announced this agent AI boot camp I'm going to start sharing my screen so I can walk all of us through this.

Just a moment. I will.

Okay, just one moment. Okay, so we'll go ahead and get started here.

I will just take maybe another five minutes or so to go over the agentic AI uh boot camp uh and really what what is it that we are planning to do in the agentic AI boot camp. Um so here we go right so if you if you look at this right so I mean we have been hearing this word agentic quite a bit I have I've been actually looking at uh these questions uh most notably like how is an agentic uh how's agentic AI different from well an AI agent or AI or machine learning um or RPA I've seen a lot of those questions um so here is here's how we should look at uh what agentic AI looks like.

Um so agentic AI an agentic task actually it's always starts with you specify a goal um and you state the goal and then there is uh there is an um uh an LLM uh so under the hood uh when we talk about agentic AI under the hood is uh of course there is AI there is uh uh generative AI uh it is there um so you you have these uh standard LLMs that start to user states the goal. Um the the planner agent or the planner LLM uh it actually breaks down the task uh breaks down uh whatever uh goal was given to uh this entire uh agent.

It breaks down the task and based on that task uh if you for those of you attended the session yesterday and then we had this example from Bob and Bob uh said um you know what if I could uh you know tell an agent I need to go and watch I need to travel to this place or um can you buy me a ticket right so first it will understand and then you may define some constraint so there is under the hood there is a reasoning uh or planning that will happen Well, in order for me to plan a trip to uh New York, uh first I need to do this and then I need to do this and then I need to do this. For those of you who use uh chat GPT uh regularly um you are aware uh that how I mean if you notice how it says first I need to do this then I need to do this.

It's almost like how humans actually reason. Um, okay.

If I need to do this, uh, how about I first figure this out. And how about I do this?

Um, this is how I would do it. I need to plan a trip to New York, right?

So, how will I plan a trip to New York? Well, uh, I will first say, okay, what dates do I need to go uh go to New York?

Uh, what is the duration of that? Then um uh uh do do I have a frequent flyer account with some uh some um some airline?

Uh which uh which hotel uh rewards program am I signed in signed in? What is my budget?

And and you know uh where will I mean am I going for business or leisure and all of that it will reason and then it will find that optimal trip uh based on that. So it goes and now once it has the plan in some cases it may have to hit some database it may uh your agents may have to hit um some some retrieval you know that so this is the the overall outline of the conference was designed very carefully right so you have these LLMs at the core of it the first session from yesterday then you have this reasoning uh the so retrieval and context engines and the third one was putting th those observability and guardrails around it.

So this is broadly speaking what an agentic um what um an agentic uh AI workflow should look like. So uh so according to my plan I need to go and check maybe some travel website.

I need to go and check uh Google uh maybe I need to go and check my own my personal notes. I mean maybe I need to check my inbox.

So I will retrieve all of that information and then now try to go and reason okay what do I need to do? So there is a reasoning uh now that is happening and reasoning and possible action that is happening and then based on the context potentially um potentially uh my agentic AI workflow may ask me okay do you prefer a specific airline or not or uh do you uh do you want me to book already or not right so so basically this goes um you know you you receive that external stimuli and then you you make decisions based on that and then that stays in memory.

So retrieval is more from a long-term uh long-term storage uh possibly or some external storage and your memory and observation is from your within your session and then you reflect and uh replan am I do I have everything that I need if not I will go and replan so these are the standard paradigms uh that are uh emerging as we uh you know as we make progress um in our boot camp for instance We do uh cover there is uh there is a design pattern called react pattern right so we do cover that in detail uh you know reasoning and action uh then uh there is a reflection pattern for instance reflection pattern is uh I have a planner I have a um a reviewer um I come up with an answer then my reviewer reviews it uh and is it consistent with the plan or not and based on that you know it just keeps going around until uh until um you know you get to get to the answer that the reviewer thinks is correct. So these kind of paradigms they exist and in our agentic AI boot camp we cover all of this.

All of this is covered in the Aentic AI boot camp. Uh of course I cannot um show every single thing.

I will just flash a few things on the screen to just give you an idea. But uh if you look at this, it's an eight-week program.

Once a week, you have class, three hours a week, and then definitely there's more work uh in the learning platform. Uh and and the people who teach, they are the people who build these things.

So we don't have people who just happen to know uh you know they watched a uh YouTube video here or attend a conf attended a conference there. It is people who build these systems.

So I mean uh so people who work in production systems they of course their view is going to be different um and uh and we we are offering 50% off only during the conference uh because because of the relevance with the boot camp. So um the code is there F O D AI50 if you go there the QR code is there.

Let me actually show you right. So uh this is only part of the curriculum.

I cannot definitely cannot go through all the entire curriculum. So if you look at this we cover lang chain in a lot of detail probably the most in-depth treatment of the entire lang chain stack.

Uh we cover lang graph in a lot of detail. Um maybe I can show you a few things uh here when we talk about let's say you know agents and tools how do you connect it to external tools and all of that um and then let me just show uh one thing here uh we have these labs that are built in right so and you get uh this as part of the uh when you register you part get this part as part of the your registration uh you click on it uh you know the everyone gets their own compute, own storage.

You can write more code, you know, it is uh persistent. Everyone gets their own account in our uh in our uh sandboxes.

Uh there and this is one example. Here we are showing how to create an agentic framework in a uh you know here's a retriever.

There is uh you know you you go through run run. we walk through the code nodes and edges um and you know basically what kind of patterns are there I am I don't want to eat up into the time of the next uh speakers so I will quickly wrap up I think the most common question that I see is prerequisites right so the prerequisite is um really uh if you already uh definitely uh you have to be comfortable with Python programming without that uh it is not going to And then in addition to that uh you have to have some basic understanding of how LLMs work.

If you do not we have some remedial material. We have some material that we will give you uh for consumption.

But uh you know you should be fine. Uh if in doubt please uh reach out to us and we'll u will u um help you.

Uh, sorry Hamza and Ali. Um, we didn't mean to, but yeah.

How's everything going with both of you? Can you guys hear me?

Yes, okay. So, over to you.

Thank you so much. All right.

Um, hi everyone. Um, Raja and, uh, data science dojo team, thank you so much for having us.

Um, we want to keep it a quite a collaborative conversation. So, please keep the questions running in the in the open questions.

Both me and Ali will be monitoring to them and trying to answer. I know most of you like this is a one-way thing.

Um, this is a webinar. But my request is that just please reach out um ask questions and then we'll pause you know take a break to try to answer any questions that you may have for us.

All right. Having said that, let let me switch to my screen where we have everything set up and uh give me one second.

All right, I hope everybody can see my screen. Ali, if you can see my screen, then most people can.

Um, all right. So, we are here to talk about uh vertical AI agents, a guide to building agents that deliver results.

Now most of you have been going through you know when joined through yesterday or um u you know you have had the opportunity to go through different modules of the talks but u what we will try to do is try to cover a little bit more into agents and our you know one of the things that we want to focus on is talking about vertical AI agents more than uh the general purpose agents and we'll go in deeper into examples of what they are and how they they are built out. So without further ado, uh want to introduce myself.

My name is Hamza. I'm the founder and CEO of a company called Traveros.ai.

I was previously at Google. Um and before that I was at Walmart.

I have you know 15 years experience in machine learning and I'm an professor at Stanford in UCLA. Um what we do at traversal basically in traversal uh we have built um our company has the capability to build individualized agents that do one thing at a time.

So instead of having a general purpose tool which is basically hey I'm just going to I'm going to do every single thing on earth what we try to do is we try to build an agent that uh sort of provides help with one specific thing and we believe it reduce the the uh it reduces the u the option of having hallucination or error it reduces the overall er error um in the dependency of LLMs Um and we work with a ton of customers um across the board, you know, um across the US and we've been trying to help build them um enterprise um agents that can solve different different problems for them and that's what we're doing. Joining me is Ali.

Uh Ali, do you want to give a quick overview about yourself? Uh thank you Hamza.

So my name is Ali Shafi. I'm a PhD candidate at the Kansas State University.

Uh I recently completed my internship with traversal uh last December and now I'm working as a research fellow with traversal AI. I'm working on couple of projects.

Uh last year I got a chance to uh apply and work with Meta AI accelerator competition with Hamza which we won actually. It was a pretty amazing experience.

And also last year in the summer I also got the chance to be a TA uh on Oxford machine learning summer school. So my research area is about LLM deployments LLMs for HPC and agents.

So over to you. All right.

Awesome. Okay.

So so basically if you guys have heard about agents and you know we we will try to demystify all about agents today. um agents have taken over the news lately and we see a ton of news coming through our LinkedIn feed different things that you know what people are saying and you know um and with every news that there's a there's a meme there's a meme following that happens that you know explaining about um how agents have just been around and people have been talking about it non-stop.

Uh the funny thing is they all started up late last year. It's not that agents have been there before before that or in the form that we know of today but you know every company in the world is trying to jump onto it and you know be AI front and center and try to build things uh based on that but I think the major question is and uh this is something that I want to identify is what on earth are agents you know we keep hearing about agent uh yesterday we also had a conversation about how do we define agents and no two people have the same definition some people have a scientific definition.

Some people have a basic definition. So let's start with a very scientific example uh definition.

An artificial intelligence agent is a software program that can interact with an environment, collect data and use the data to perform self-determined tasks to meet predetermined goals. Right?

So this is basically how we define an agent in a scientific term, right? And this is taken from from Wikipedia and this is a pretty standard example of an agent.

Now this is great but how about most of us who are like dude I know what this says but I don't know what it means. So for that to explain let's let's let's explain something you know explain mean like M5.

So what we do currently is that we uh we use chat GP everybody I think everybody is aware of of chat GPT and how chat GPT is used. So in in chat GPT basically what you do is you write a question and you know it generates uh you know um an email chain for you or something and what you have is basically a really nice good look look good-looking email that has been generated by chat GBT for by you and you use that to send it out to everybody else.

Now there's a second part to it. Most of you or some of you you know use charge GPD for coding also.

And what you do is that currently uh you know you generate something in code uh through through chatg and you put that into a python compiler. Now that's a very standard process you know that people follow you uh you use giant GPD or you there are different tools available but you basically use these tools to generate code for you and then you have a python compiler where such as VS code or even cursor for that matter you use that you output that and uh there you have you know you you're copy pasting now what if you don't copy paste and you are like someone who was very lazy like me you're like I don't even want to copy paste all I want to do is write one prompt and it should generate the entire code for you.

And once it has generated the entire code for you, it should basically execute. The code should execute on its own.

I don't want to write anything beyond the VIP code. Um, and that's what you know some people have a question.

Why do we need to learn about Python when we have low code or no code platforms to build agents? I think that that that's what I'm trying to answer is that we have this capability that you know like I am just writing a prompt.

the prompt the prompt is writing you know generating code for me and now what I want to do further is that I don't even want to copy paste that code I just want to be able to have it execute for me u um so if you were to put all of that to together you know if you were to equate a um a prompt that generates a response and it is compiled for you in real time automatically that is what an LLM agent So in reality what an LLM agent is doing is that it's it's connected to a software basically which is a compiler the LLM agent itself and I I want to really emphasize on this you know to answer to answer the the the question uh or the disambiguity on what agents are versus not agents are is that agents are nothing but LLMs that are connected to different devices and different uh different tools and we'll explore them in detail over here here but basically when you have when you're writing a prompt and you're basically generating uh the code for it and then that is compiled automatic for for you that is what uh basically the AI data scientist of 21st century is uh which you have which where you have just wipe coded your way through through those things right so if you were to expand this into a a a little more What are agents? Agents are LLMs with access to tools and memory task with planning and executing goals.

Um, and unlike LLMs, today's AI agents are an engineering innovation, not an AI innovation. Now, if you were to take nothing from this session, but just one thing, just remember this.

LLMs are uh unlike LLMs, AI agents are engineering innovation, not an AI innovation. What that means is that AI agents are just connected to you have an agent which has an LLM running in the background.

The LLM only generates an output much like it would do in chat GPT but we have connected different output functions to it. So basically it generates a JSON output which says I will call this particular function and if we have time we'll show you an example of that of how we've built that.

But that's the idea behind the scenes that that's what we use to uh use an LLM behind the scenes to generate an output. And there are there are different levels of AI agents is level one, level two, level three and level four.

So we'll talk about the first two levels. The first level is an LLM.

You just have an LLM that is like chat GPD creating an output for you that is or generating output that is level one agent. Level two agent is an agent which has access to different tools.

So for instance, if you have um you know you have a calculator tool, you have a code interpreter tool. Um there are different examples of internet search and rack tools.

Uh if you're connected to those tools that is basically um um level two agent that has connected to different architectures and it is able to um it is able to generate uh responses for you on that on your behalf. And then once we have uh most of these things are one direction which means is that when you have an LLM or when you have charged it does not have the ability to think back and sort of uh reason with the tools.

Right now they're reasoning models that only reasons with the answer it's generating through its own ability to generate text. it's not actually looking at the tools or interacting with the tools.

In order for an agent to really work well, we we need to create something which is a react agent. What is a react agent?

A react agent is an AI agent that uses reasoning and acting react framework with the chain of thought to connect to different tools. So when you think about how do I connect to uh different things uh so for example somebody is saying how can I create affiliate marketing automated with AI agents what you have is basically an LLM in the center the LLM in the center is connected to different tools and the LLM when you say hey I would like to create an affiliate marketing for this product that I'm selling uh this product is for mark it's a it's a marketing tool and I should be able to sell it to marketing agencies.

The affiliate program um you know will release the links to the affiliate program. So whoever posted they we can we should identify the best people we should launch for affiliate marketing marketing and then once we have launched those product um um we should be able to get feedback on how well those people are performing.

Now there are multiple ways there are any number of ways to go through it. The LLM when we feed that request to an LLM it reasons with what is the best way to move forward and it we have identified the tools that it has access to.

The tools that it has access to generate a set of results which are then sent back to the LLM for it to think that oh I think I've gotten the answer of what I'm looking for. And having that information it is able to generate a bunch of uh you know it's able to generate a workflow plan for you.

So that is basically what react agents are and that's what react agents are being used um um in the current world or they're being we try to utilize them to show the benefit of these tools. That's what we have behind the scenes uh when we talk about agents or react agents that have the power to reason and act right and that's when we approach approve when we are approaching that level that is what we call something as level three agents level three agents are those agents which have reason and acting capability.

So just to recap, we have level one agents um which are just level one is just an LLM agent or speaking to you. Level two is an LLM which is connected to different tools but it does not have the reasoning capability.

It it it's only a forwardlooking direction. So if somebody asks what is agentic system versus agents this is an agentic system.

The agentic system has the capability to understand that it has access to tools and it creates a workflow. Level three or react agents have the ability to rethink and have memory associated with it so that it has it is more nuance and have a stronger capability of being able to deliver responses.

So that's the power behind the scenes uh of how we have built out or how how would you use agent architectures um of level three which has the ability to understand what you have just said and be able to react u uh based on the planet action and last but not least is level four where you basically have multiple agents that have react capability and they are speaking to each other. That's the that's the sort of the major idea or the major definition uh that sort of runs behind the scenes um in in this uh scenario.

So we'll take a couple of seconds here to try to answer some questions that people have. What is the scope of agentic AI in compliance industry where hallucinations can lead to penalties for example from IRS.

So the idea is that in compliance industry you know when you try to build something we have to sort of do two things we have to have either LLM as a judge or a human in in the loop. The human in the loop's ability is to actually evaluate whatever output is coming and be able to move move it forward.

So in anything which is very highly regulatory or it is very important such as healthcare or finance or compliance we need to have an LL a human in the loop which sort of makes a decision who gets the you know a lot of things put together for them and their job is to evaluate the performance of the agent and sort of give feedback to say hey this this looks good and I would like to move move ahead with this. So that's the that's a very general basic idea uh that that you can use you you can utilize to sort of uh create that output.

Uh second question was how to make AI agents more semantic in terms of identifying synonymous word for example in any unstructured document we made. So basically what you need to do um in in that form this is the second question how to make AI agents more semantic.

I think that's not what you need to do with the agents. That's what you need to work on your rag and make your rag to be more agentic.

So there's a difference. Rag being agentic means that you have a rag which is connected to different tools.

But the agent is calling multiple drags, right? So the agentic rag is a simple tool that exists.

You can have multiple drag a uh rags that exist. Some one could be summary, one could be you know or related to one department.

Uh you have five departments, you can have five different rag architectures. The agent's job is to just call those rags and fetch information related to that.

We should not focus our time on building agentic AI rags that are all in one. We should focus on building vertical agents or one agents whose job is to just call something the other agent.

So if you see over here what I've identified is that something called traversal pro rag which is which we built in our company the traversal pro rag tool what it does is that it just bas it's an independent agentic rag on its own all we do is pass information from the from the agent original agent to say hey can you help me find information about x y and z and it sort of works through that. Uh one more question is can I make my affiliate marketing automated by AI agents?

Yes you can. We actually um uh believe it or not um we actually building customer outreach program through our AI agents.

So we tried a lot of different products in in in in the market and what we found out is that it's very difficult to actually build agents uh or uh it's it's very difficult to build the right agents for your uh uh or or find you know it's there's a lot of money that is needed to sort of build those things uh because you know you have to pay you have to pay a company that that works on it. So what we did is and let me share my screen again.

So what we did is that we built our own AI agent. So as as I said before we we our product is a data science agent.

What we have identified is that we built our own AI agent that identifies this is a real company and this is something that I get every every day in my inbox. So this is a company which is a decision science digital analytics startups.

It's called house. Um what does the company do?

What are their data needs? What are the specific challenges they are they have?

why the why our agents can work for them, what is the recommended pitch that we can give them and what are the key talking points and these are some of the some of the job postings. So we can even evaluate the job posting that they have built out.

So this is a really really powerful tool agent that we actually built uh we actually built in literally one day right and we have been able to build this because of our understanding on how verticalized AI agents work and this is something that you know that you know you can reach out to us we can we can help you build them but the idea is that it's very it's not very difficult to build these things anymore so imagine in 24 hours just 24 hours I'm not saying any more than that I spend 24 I was trying to build uh on this agent that basically gets me the right customers in my inbox every single day. Uh I get 20 to to 25 of them in my inbox and I can make it to 50.

But I get these u directed agents on why we are important to them and why what is the need that we are trying to address them. All right.

And this is you can look up this company. It's a real company with real real jobs and everything.

So that's the idea and it also tells you the likely people that we should reach out to. So that's that's something that you know that you can sort of work on and sort of builds it.

All right. So now moving on to the part of our of our overview.

So this is basically a very cleaner or you know more visual friendly outlook of how agents are working. you have a tool in the center and the um sorry you have an agent in the center based on a user input the the react agent sort of goes through in a loop trying to find out answers uh using different tools and once the final answers is is uh is achieved it is able to generate a response based on that now there are different companies that have been promising that you know we build really good agents and you know we do we do you know something really good on that we built amazing tools uh there's a tool name is Manus AI.

This manus AI tool is basically you know it has so many different things attached to it. It has a code editor, it has a multi modality, it has autonomous, it has analysis.

So it has so many different things and they raised I think a ton of money you know just on the fact that you know that they built something something like that. Now it's all good you know like this is a great product but within a week of releasing releasing this product somebody was able to look into look under the hood and say oh so this is basically clawed with 29 tools and it uses something called browser use um so those of you don't know what browser uses brow browser use is the ability for MCP so that's all there is to this tool so basically it's a wrapper over And that's what happens you know when we are trying to move like I just want to identify that you know a lot a lot of us live in a FOMO constant FOMO that we don't know what's happening in the world the world is moving at a very fast pace and everyone is building something so fancy so amazing everybody's trying to raise millions of dollars and some of them have done really good job I just want you to know behind the scenes is a lot of wrappering wrapping around of things it is not just you know creating something just uh out of the box.

It is literally a big wrapper that sort of just um uh that sort of just exists and it um this is manis AI they just built a w the set of wrappers on top of that and once they have built um those rappers on top of it they're able to show you know put together you know a front end that sort of uh drives a value or sort of uh build that Right. So that's that's the idea you know that that that can be done and at the end of the day this um what happened to my Can you guys hear me?

Sorry I think I lost you all. Yes we can.

Can you hear me? Yes we can.

All right. Sorry I I don't know what happened.

Um yeah. Okay.

All right. So the the agent uh so coming back from the browser look from the browser use of manis it all comes back to the same architecture that people have been looking into that we have different tools that are created and then once we connect those tools we basically have an outlook that sort of combines all of them together.

All right. So imagine a world where we didn't need to write code and this is where we think that you know where we feel the world is heading towards that we don't need to write code.

we can you know we can just wipe code our way through it. So what is wipe code?

Wipe code is this amazing capability. Some of you might have seen this tool.

We have spoken about this tool over and over again in the past two days. Lovable uh this is a tool that sort of you know it can create code in real time you know just like right there and there.

It generates code for you and you're able to everything is done for you. So why do you even need to do anything?

So my answer to all of this is why do we even why do we ever need to cook at home? We we can always go out and get food, right?

We can always go out and get food, but most often we don't know what's in that food also. So the thing with Y coding is is it's really good and you know like a lot of people right now are like, "Oh my god, this is amazing.

I'm you know, I don't need to write code ever. I don't think that's true.

You need to write you still need to write code but you need to write code a little differently now. So let's say you are in this phase that you've just identified something called w coding which is you know just w agent creation w coding w uh w architectures you've built all of them and you're like I'm never going to hire a single data scientist or single you know front-end developer I can do all of this myself right and this is the feeling that most people have you know they're like okay and they're looking at a product and they're like okay you know it's game over I don't need to hire hire people But the problem is with great power comes Hawkrust that was never intended.

And this is a Harry Potter reference. Basically what happens at the end of the day is that you know uh when you put VIP coding in in pro production things happen which you don't know you don't even know where it's going to happen from right and it just makes things go makes you go you know you're trying to think about it like hey how can I even answer this question?

How can I like or how can I what happened? Where did something break?

You don't know where things are breaking for you. You don't know where where anything is working out for you.

You're like, okay, somewhere something is working. I don't know what's happening.

So in order to prevent you from that, you have to build vertical AI agents. And one of the best examples that I have, you know, over here is is is of this one.

This is a vertical AI agent which has been which I mean we as a startup we don't have a lot of money to hire people or to reach out to you know like agencies or you know get get money to you know raise a we don't have a lot of money to hire the people who can get and reach make reach out for us. This is a tool that we built internally for us and you know you can see this came out 19 hours ago for me.

This is a tool that we built. So this is an example of a vertical AI agent.

It just do it only does one thing. It finds me the right people or the right companies that our company should target to build uh to to reach out.

It doesn't do more that doesn't do it does not schedule interviews for me. It does not makes a reach out you know like this is the people you should email even it all it does is that it gives us a direction and if we get 50 now we are reaching out to 50 customers that we know might be looking for a tool that we have to offer right that's the power that's the power that happens in in this way and that's what we are focused on as a company is that how can we build smaller tools I'll show you another example I'm giving you inside to my uh to my inbox but let me let me give you Another example, this is an example of human in the loop.

Now me as a LinkedIn u you know person who posts a lot on on on on LinkedIn. Basically what happens is that I I have to post a lot on LinkedIn.

Now I hate the fact that I have to write a lot of post every day. So what I did is that I I created an AI agent which basically creates um like fetch runs through the internet finds a good article for me that I can post for that day.

So this is something that ran 11 hours ago. It's basically says hey this is this is an this is a this is a post.

Now I can change some of the text over here but this is a post that I have. there is an article that it has linked to and now it's more interesting if I hit approve it will post on my LinkedIn so that's the definition of vertical agents I just want you all to focus on the definition of these things is the what is the vertical agent the vertical agent does one thing for you it automates and makes it see here you might say oh I don't I don't care I don't need to I don't need to do research like I can do my own research and find out the best but then when you have a busy life and you have 600 things to do see I have to run a company where I have to make sure that we are getting the right customers in our inbox we are getting the right uh we are making the right uh proposals we are writing the right post I'm doing the research um so my time is very conf like divided and I don't have a lot of time to do everything so what I do is that I use We build these vertical AI agents that does the job for us and they are connected it's connected to my inbox.

So I get an email directly from my inbox to my inbox from a system that says hey and if I approve it in fact there is a human in the loop concept also where we are working on is that if I don't like this article it will send it back to my agent and it will look I will say make it less chat GPT and make it more personalized or something something like this it basically helps you um it basically helps you to build this right that's That's the focus that I have. This just saved me maybe 30 minutes of my of my day.

And if I save 30 minutes every day, if I make 20 post a day, so 30 minutes a day, I have saved 600 minutes. 600 minutes is 10 hours, right?

So I've just saved 10 hours of my entire month to do something else. That is, my friends, what you need to do or be focused on what vertical AI agents are.

that will help you build something better that will help you build uh tools and you know architectures that can be that can be built out. Now I know you know understandably a lot of you are excited about this you know like or you're like okay um we we would like to see how this is done.

Um I would say all I would say is that you know reach out to me on LinkedIn, connect with us. We we also do a lot of speaker sessions or you know or or open courses.

So you can uh like something we do for free. You guys can um reach out to us on LinkedIn and we will share our our our information also.

But the idea is it's possible and I am actually saving time to do a bunch of things. All right.

So I think we have 20 more minutes. Um uh data science team uh data enclosure we have 20 minutes left.

Uh you have uh yes uh 20 more minutes. Okay.

So what do I want to do is that I want to go a little fast through I mean this is a data analyst that we built like a lot of people are not very interested in data science agents but all I want to show you guys is the ROI that we are seeing with building vertical agents. There's 80% reduction in data analysis time because a lot of there's a heavy lifting is being done for us and it's saving us time to to to make sure that we do not have to spend a lot of time getting through you know doing things oursel 20% lower operation cost for on-prem you know we work with enterprises and you know those enterprises you know they we help work with them we understand what needs to be done so it 20% lower in operation cost and 30 to 40% in inventory way.

So the reason I we put it over here is that we work with supply chain companies. They have a lot of inventory that they that they create with AI agents that helps with forecasting.

And I'll show you an example of our forecasting model. But the forecasting model basically what it does is that um you know it basically generates um you know you just basically give it a prompt and the prompt is then used to generate an entire output for you that sort of looks into what what we're trying to identify um and it generates a code for you um and it runs.

So I understand this is data science dojo event. Uh we're talking about data sciences but I think a lot of our focus has been on vertical AI agents and you know how um how we can utilize them to to to build things from there.

All right. So what I want to do is I'm going to try to take five minutes to answer questions.

This is our LinkedIn uh uh URLs. You guys can you know use these tags to reach out to us.

I'll give you uh three minutes on this and then Ali will switch gears to for you to show some b some basic code. Right.

So I'm going to scroll up and trying to answer what's the difference in agent performance if you use a SLM versus an LLM. I think the the difference Alex to Alex question a small language model versus large language model.

I think small language models have a higher cap probability of hallucinating. So when you're building a very important ecosystem which requires you to make sure that things don't fail, using an LLM might be a better job.

Right? What accuracy using single LLM agent or multiple agents?

Um I would say to Ind's [Music] question having multiple agents in one ecosystem means that the the point of failures are very high. So again if you see this code right if you see this output this is literally doing one thing and there's there is very low possibility of error at at at your end.

Why? Because you know it will not error out.

Of course it might hallucinate but the level of issues it creates for you is not that big for you to be concerned. So you have to build tools that work like this.

You have to build tools that are so independent that if they screw up, it does not trample and destroy anything ahead of for you, right? It does not do anything uh that sort of creates an ecosystem for you that you're like, "Oh my god, you know, everything fell apart." This is what I'm trying to prevent you from.

You should use vertical AI agents. You should use agents that do one thing for you, but you should use them in a way that you are not struggling to see where things have fallen apart for you, right?

Uh how long is the information in memory preserved? Um so then to your question, there's long-term and short-term memories uh that can be used over here.

We have an example uh that we use. Just going to go fast through that.

Um basically what we have a short-term memory and a long-term memory. The short-term memory is very transient.

It's just ephemeral. Um it's just there for the time being that you're using it.

However, the long-term memory is what's stored in your e ecosystem that exists. And with that, what I would love to do is just show you give you guys an example of um github.com um agent pro agent pro I can't spell my own company.

All right, here we are. So this is something what what we have built which is agent pro which is our own infrastructure.

There are ton of people who are building agents frameworks. The reason we built one is because we didn't want to depend on langin and llama index.

We just wanted to build something ourselves. So that's where that's what we have built ourselves you know so that we can um uh we we are working on it.

Uh all right Ali uh do you want to take over and show some of the code on just basic examples? Sure.

Let me share my screen. Okay.

Can you see my screen? Yes.

Uh perfect. So, so this is a demo of agent pro and this is kind of my favorite part of the session because now we are going to see how the agent pro works.

we will see some kind of things in action and uh it's a stream app and just for this thing if I minimize this so it's going to use uh a GP4 as a llm and then these are the tools I have connected with this amp react a pro one is the calculate tool the yahoo finance tool the slide generation the Aries internet and the traverser pro react tool And it's very easy. Just couple of lines and it will set up.

With the rack tool, you actually need to pass the document because it helps the agent to know when to go to the rack tool. If if the question is related to the traversal employee handbook, it will go to the rack tool.

And this is interesting thing that if let's suppose if this if this tool break or something or if the Yahoo finance tool we observe that we run we ask for some Yahoo Finance related things and it says that your rate limit you the there is a limitation with your rate. So it automatic automatically reason and move to the internet tool.

So if any tool just break because of any reason uh agent pro actually do reasoning and try to fetch the answer from other tools as well. So let's let's try some prompts maybe how about this prompt and this is a screen uh uh UI.

So let's ask this question that can you tell me about the future of data and AI the agentic AI conference 2025 by data science dojo. So once I run this so in the terminal I can see that this is a system prompt and then it calls in uh tools and then it fetch the final answer.

So let's this thing in in a better way. So this is the thought process by the LLM.

So to get this information it needs to search on the internet. So it use this tool Aries internet search and search about this.

It gets some answers which is the observation from the tool and then LLM is pretty satisfied with the observation and it is the generated answer about the agentic AI conference. So, let's try something else, something more interesting.

How about a rack tool? So, this is a question about can you tell me about the PTO policies at the traversal employee book?

Now, hopefully he's going to run the rack. and the embeddings and the chunking everything has already been done on the on the uh pro traversal AI website.

So it actually call the tool ask for the PTO policies get some observation and it generate the final answer for you. So it means you don't need to read 300 pages of book.

You just need to make a rat tool and you just need to prompt and get the answers out of the document. Out of the documents and how about we can try this thing.

Can we make PowerPoint slides uh and compare the Amazon and Tesla? Let's see if if the Asian Pro can use multiple tools, fetch multiple information uh from about Amazon and Tesla and try to make PowerPoint slides for us.

So the whole thought process is going on. So looks like it has it went to the Aries internet for the Amazon.

Now it is going for again to the internet for the Tesla. Probably it compile the slides for us and let's see if it works or not.

So this was the whole thought process and for that it needs to gather the information for both companies. First it get so this is a very interesting thing I was trying out last night I guess.

So because this tool did not work for um for the agent because of too many request. I really do not should not do that last night.

So with the thought process it it actually rely on the Aries tool. So first it fetch the information about the Amazon and then it fetch the information about Tesla and then he gathered the information and use the tool of PowerPoint and make the slides for us and this is where we have the slides.

Let's see how it looks like. Difference between Amazon and Tesla.

This is the introduction, some financial performance, strategic priorities and the market position. So it does it has done some pretty well job for us.

So is there any prompt uh uh we would you um we should try something and there are also one more option about the custom tool. So it's not only the default tools available.

You can also create your own custom tool with the agent pro and you can run those custom tools and it it's it's a very easy plug and play kind of a thing and it's an open source so you can definitely uh work with that and if I can show you a system prompt. So this is just a system prompt where uh it ask for the react and reasoning and acting pattern.

It has these following tool accessibility and this is how it generate a JSON uh format for us and we just extract this JSON format call the function return the observation and keep doing that until it get the final answer and give us the final answer. They do a chain of thought process and uh this is all about the agent pro.

Hamza, do you have anything to um to share some prompt for us or No. So I think what we can do is we can use this time to answer questions for people.

So Ali, first of all, thank you for this. Can you put the link for agent pro in the in the in the chat?

So people people want to if they want to check it out, they can check check it out. Uh I'm going to try to go from the back from the from the bottom up and try to answer answer questions.

Um basically um Chad GPT can also create PPT. Curious to know the main difference between your agent and charge GPD.

So well I think the main difference between charge and our agent is that we have built completely everything open source. So you can see and build things as you go.

Our ideology is that we don't want to be reliant on any any framework. So it's one of the most lightweight tools that exist over here.

You can deploy it completely on your onrem. So you can't deploy charge GPD on prem.

So this is a system that you can deploy on prem. You can use open source LLM.

So some people were asking that how can I protect my customer data from uh from this. It's very simple.

If you deploy this for example you take agent pro and you deploy it on your ecosystem. you can connect it to all O Lama and once you connect to O Lama everything is just running on your prem you're not using anything on the outside world right so that sort of answers the question on how can we utilize this thing um um uh you know to see um uh to to to use this uh Sai you had a question if we want to run agent pro in our system do we need open AI API key actually you just we we support light lm Which basically means that if you use light LLM, you can use any LLM, which means that if you have an LLM that you have hosted on your ecosystem, if you use open router, you can use any one of them to uh to to work on that.

Uh how does agent pro compare with LN graph? I think the simple thing is we not dependent on Lion or we we're very simple framework.

So you can open the hood and build things as you go. So the learning curve is very small for you.

it you just need to be able to work with chart GPT or you know some of the things over there and once you you get that up and running I don't think you need to um you need to focus on anything else all right so how is this different than notebook LLM notebook LLM is an audio creator we don't have audio creation we just create an output which is uh based on the search that we have done or you know the agent the job of the agent of what it built up um how do to check the performance of agent in production environment. So there so this is a good question.

The reason is because you basically the way you check agent is not through one call. You check agents through um you check your agents uh performance through 10 times run.

So you basically run something 10 times and then you look for the ground answer and you see that if the agent was able to deliver you on that ground answer. So I just want to emphasize on this that agents agent uh the way we evaluate agents is not the same way we evaluate LLM or LLMs.

We evaluate agents by running them 10 times. Um and this is called pass estrict k or pass cap k that's the thing that we use to test out these these tools.

It's it's some it's something like you know you'll be like okay what the hell is that? If you if you look for it, if you like sort of Google that you should be able to find information and that's something that we are doing with data science benchmark agents.

So if you were to so this is a link that I want to share um we're doing research work um if you're a researcher um research research for data science um agents. If you're a data, if you're a data scientist and you've worked on agents uh or you like to work on agents and try to understand how agents work in terms of benchmarking, please use um please reach out to us.

Again, this is you don't have to pay for anything. All you need to do is be part of a open source research team.

We have um researchers across the world that are working with us. We've got a team of 10 now and we're using we we're working uh right now uh to to to build that.

All right. So again few things our agent pro studio link is in is in the chat our link for the re the data science agent is is in the chat.

So you can sort of uh you know go through that and get benefit from that. All right I think we are at the top of the hour.

Um um Raja thank you so much Hamza and Ali um as always pleasure to have you so thank you so much and I will take it from here. Awesome.

Thank you. Thank you everyone.

Thank you. Okay.

So it is time for uh yet another break and what I'm going to do right now is maybe there's a quite a bit of change maybe a little bit of change in plan. Um so I heard a lot about um uh I was observing the questions here and uh I thought it may be a good idea to actually show you how actually this whole reasoning and uh you know this planning and reasoning and all of this unfolds.

Um uh so you saw some some information uh from Ali um Ali and Hamza they presented a few things. I'm going to show you something that uh we have built and just to give you an idea how how any gentic system actually works.

So think of this as a as a vertical specific uh um call it a vertical specific uh um manifestation of a platform that we have built. Um so uh in this case uh what we have done here is let me go back here.

Um so we have built these uh call them knowledge workers, call them co-workers, call them just anything you know just uh some kind of AI assistants and these are uh of course not real people. Um so think about this at the uh so this is you're a retailer uh and you're uh inside your retail um you know your company you have these different roles uh you have a nutrition adviser.

So this is uh imagine that this is a company that sells um healthy food like the for those of you who live in the US like think of this as pole foods uh so uh you are an e-commerce you could be really I mean some kind of retailer and now you have ingested all of the documents um you know let's say all the products uh that you have on um in your um all the products in that you have in your inventory maybe all the products in your uh on your website and now I go to this uh let's say this is my customer support agent uh that does product recommendation so I asked this uh this particular agent and I'm happy to take questions as well I have eight more minutes uh before uh the next panelist comes in happy to take questions questions as well so so here I asked this uh this uh agent I am let me actually I think that would be more fun if I asked the same question here, right? So let's say I am gluten and lactose intolerant.

Can you please recommend some products? And now this is the second day and you probably now understand this.

So what happens uh is um as we speak um this agent that we have deployed is building a plan. Uh it is building a query plan.

uh it is just going and after uh doing a query plan uh yes I need to go and fetch some documents that are related to well gluten and lactose those keywords so some retrieval happened and you see see this light gray uh text that appeared for a moment this is my reasoning and thinking process so it generated an answer um it generated an answer and then it reflected on it if I showed you what was the thought process right so you know It gives me the thought process and finally uh what it did was it it uh it gave me uh it gave me all the products uh in the repository that actually fall in this category. And now I can actually click on it and I can land on that product in the repository.

If you look at this uh you know over here and I land on this product just from uh this this particular uh this specific uh agent. Now um if I go to let's say an SEO blog writer right so if I go there and I ask it um uh write uh blog post on [Music] um lactose intolerance and gluten-free um die I I don't know what even mean that even means right let's say the the fascinating things is that it is the same knowledge engine it is the same repository of documents that it will again go but this time uh it so the planner knows that this time the job is not to actually fetch and recommend products this time it knows that the job is to write a blog post and then now it using exactly the same resources that it has it uh the planner actually decides that now I need to do something else and then if you see uh you see this multiple revisions remember I was talking about this uh you know this uh planning and reflection and planning or uh critique so right now this is going through the entire process and then it is uh you know telling me all of this and the fascinating thing here is that not only that it is uh uh not only that it is uh it has given me the product it has also it has created this like nice content marketing touch to it what causes and what are the triggers and what are the aspects I never asked it I don't know anything about lactose intolerant I'm not lactose intolerant intolerant uh not uh I don't eat gluten-free right but yet without being um and I'm not a marketer by uh you know by training of course right so but all of this it actually created this for me.

um you know I have uh I have the same knowledge base and just with this single line of code u a single line of text uh the same engine can actually wear different hats right so if you look at this here now it is showing me uh where can I find all those products right so this and all I need to do is so this I mean of course this is a product that we uh sell to enterprises but all the companies they have to do is they actually pour their all the documents, all the context as we call it. Remember the second session from yesterday or the third session?

Yeah. The the the session about retrieval.

So you you push all the content and after that if you look at this there is a detailed overview. Remember that reflection aspect of that we were talking about.

Uh so this is basically um you know uh interestingly um since I'm not uh I'm not an expert here. I made this query up on the fly.

It says the post conflates uh uh you know gluten intolerance with something else which is so now it actually goes and actually corrects for any of the wrong assumptions that I did made as well. Okay.

Um then there is uh um uh you know if I go and ask it to now the same thing exactly the same uh uh uh exactly the same uh set of documents but now I'm saying uh okay uh uh create uh a script for a podcast on glutenfree uh eating or gluten-free living. Uh right.

And now it will go using the same documents. It is going to actually go the same knowledge base, the same products, the same blog post, everything that uh we have given it from the past.

it will actually go and um uh create it um uh create a uh that podcast script and uh if I don't like it uh you know see how this is saying segment one segment two um you know so and then at the same time um you know in the podcast hey by the way check it out on our website this is uh you know we have this product and we have that product right uh Um okay and um and then uh maybe I will just one last thing right so um so let's say I want to have a nutritional advisor right so we also want uh it to be more um let me I will have to share again I guess with sound so we want a real time nutritional advisor um so let me actually go and ask uh this Okay, I think there is. So, let me see.

Yeah, the problems with demos is that this was not a planned demo. So, my team is going to hate me for this.

But, let me go here and go to a different one. Okay, I am back.

Let's say I will still try. Hi.

Hello. How can I assist you today?

Tell me about healthy eating. Absolutely.

Healthy eating is all about consuming a balanced diet that provides your I'm not an expert in car repairs but I can definitely help you with so um so if you look at this right so one of the we have this guardrail on this uh on this tool uh which actually prevents it from going off topic. So uh I have been monitoring some of the the questions.

There was this how do you actually make sure that your uh that your AI does not go off uh off uh topic and all of that. So there is this whole idea of uh there's this whole idea of uh implementing guardrails.

There are topic of guardrails that you put there's ethical guardrails. There is um legal uh privacy PII HIPPA type guardrails.

So um you put all those guardrails and then you you make sure that your agent does not uh get you in trouble because well the right guardrails were not in place. Okay.

So I am going to go and Mama is here. Moza I see you in the call.

Please feel free to come in and we'll hand it over to you. Thank you so much for joining again.

Thank you. I just wanted to confirm if this session is 40 minutes the next.

Yes, it is 40 minutes. Okay.

Awesome. Awesome.

Thank you so much. Over to you.

Thank you. Thanks for that cool demo.

And let me set up my screen and start sharing in a few seconds. Um, okay.

Okay, hopefully folks can see my screen and if not please put that in the chat. Uh some of you were here on day one and uh you have heard me uh talk about some of the guard raids and other things that Raja was talking about in a panel discussion yesterday.

Uh today I'm going to talk about uh agent AI as it pertains to data. Uh so I am a principal group PM manager in Azure data work in the databases team.

I most of my career has been in data and AI. So really excited to be here on day two talking about the future of data and AI really bringing you some customer stories and examples and I have a short demo at the end to show you how a customer one of our customers is building this kind of agentic AI.

So please put all of your questions in the chat. I can see the chat on the screen.

I'm not at my regular setup at home today. I am working from office so I have a camera here and a screen here.

So we'll we'll make it work. Again uh thank you for joining this session and uh if you have any questions as I said put that in the chat.

I will try to take them as as I go and if I'm not able to get to your questions please connect with me on LinkedIn and other places where uh where I would be happy to take your questions and um and go from there. Uh so let's start the session.

Um if the slides can work again I already introduced myself. Here is my LinkedIn.

Um I also post every week around AI on Friday. So Friday post there's a hashtag learnwithmz.

Wherever I'm learning I'm posting about that in the community. I'm also a LinkedIn instructor.

So there are two courses that are already out there that you can learn. Uh but if you have other questions around any of my experiences or things I'm happy to take those as well.

Uh so let's get started. Right.

Um I am going to make a few assumptions in this session as you are all here in Agentic AI conference. I hope and believe that you already know about what vectors are and vector searches are.

I believe that you already know what embeddings are, what LLMs are. I know that you have seen enough sessions so far in these two days where you have learned about some of the new terms uh as as these things are evolving really fast.

So I'm not going to go into these details around prompt engineering, LLM and rack pattern. So this is a good diagram to look at what is a very simple explanation of what a rag pattern is and how we use vector and vector databases and uh object stores and things like that as as part of this uh workflow.

Uh but that's my assumption that you already know. If any one of you does not know about it, please uh reach out to me or reach out to the other speakers here to talk in more detail around your scenarios.

Um so the opportunity for Genti I just wanted to kind of ground us all on what why we here what we're talking about. These are the two sources um uh which are very recent.

One of them is with from Deote and the other one is from Gartner. uh talking about what companies are today doing and this is a real scenario where I deal with a lot of enterprise customers where they have now a AI team or a or a chief AI officer looking at them thinking about what is that company's particular AI strategy and uh the second one is very interesting where I think by where Gartner thinks by 2028 15% of the day-to-day work uh decisions will be made autonomously so why we are talking about that because agentic AI or AI agents I know there is a hold debate online around what is agent AI versus AI agents versus AI workflows and things like that.

We are going to go into some of those details today here and I've seen some of the other uh sessions uh here as well that you can learn more about that. But regardless of any kind of AI agentic system, AI system, data system which is using some kind of AI uh we can debate on the terminology.

uh more and more you're going to see in in workflows more and more you're going to see it take over the repetitive task and it is real it is really happening and I talked to if not on weekly basis every other week to many customers who are on that journey now what is an AI what is an agent right um it's a very simple definition that last week at build conference Kevin Scott who is the CTO of Microsoft and I really like this definition uh I know we can all debate on it again uh but agent is a thing a human can delegate a task too. So just oversimplifying it here.

Uh where you can leave a task to this agent and they can go and do that task for you. Tasks are getting complex and complex and that's the reality, right?

So a simple task like getting a summary of something is is not enough anymore. So if you have been on your own AI journey, you must have seen how this has evolved in the last few months itself, right?

And and what does it mean in terms of that AI design to perform a task, right? So you have heard these terms in multiple sessions.

I just uh heard Raja mention retrieval system. There was a session on it yesterday around as well.

To me kind of again oversimplifying it to keep us all grounded. It's about generation, retrieval and action.

Action can be a task uh that can be done and and there are different solutions, tools, options for you to use as you go through this. Right?

So really thinking through if you are building an agent, if you're building a set of agents, what are the components that you need to make it work end to end? And again, I'm going to go through a couple of examples for you from like real enterprise customers that we have been working with and talk to you and give you an example repository.

So the code that I will show you is available for you to today download and run on your own environments. It's all openly available that you can try and and work with that.

Now in terms of an autonomous AI agent as a developer, so I'm a developer at heart. I come from engineering background.

I've been in this AI space for a very long time. Uh even before AI became the the thing um it has to do a few things again simplifying it for all of you.

It has to have a goal, a task, something that it has to achieve. It has to for it to be AI based, right?

It has to use LLM to understand and reason on what to do. It has to have a memory and context.

Again, Raja talked about in previous session and can had it can do plan and again reason on what what basically actions it has to take. It can use tools.

So you can register tools to it for it to be more effective and can communicate with other agents or other systems and workflows. Again simplifying what is an autonomous AI agent and how you can go through it and understand and build that.

Now the evolution of AI systems this is an very interesting slide uh that that is becoming more and more complex as I go into talk to many customers on my day-to-day basis. uh if you ask me about 18 months ago or so around like when LLMs and charg came it was mostly around like no agent some document input prompt something like that that goes to an LLM that's basically rack pattern you summarize it get the result back you kind of you either the summarization generation things like that and you were done and folks were really happy with that because it was a great breakthrough when it comes to the world of AI in the recent times that has now evolved to this at least a single agent platform where again again this diagram is a little bit simplified but thinking about all the components that you're learning in these two days itself where you have an agent which can plan reason orchestrate think about what can be done can go to the knowledge bases search for the data search for the context search for the things that you're really looking for then can make an opinion send it to LLM get a response back get the answer and then as a customer you're happy Now in the world of multi- aent systems the AI solutions it is getting more and more interesting and this is a space which is evolving at a very fast pace like all of these other previous stages as well.

uh in this case there is a wide scope and I have examples on this slide that you can read through more and more values being added especially on the tasks that are most repetitive in nature and the task where you can use some of the creative part of the LLMs as well as in the previous session I saw that uh Raja was asking uh for a blog to be written or a summary to be written or also gave recommendation based on what you know about a certain type of allergy or um uh something that you're going through, right? So, so think of this in this case at a marketing campaign where you have multiple agents and they can come and work with us.

We have way more work to do in this space right now. There are a lot of companies trying to solve this thing end to end from solution perspective from frameworks perspective from how these things are integrated and many companies are actually building agents for enterprises uh like agentto AI that was just mentioned earlier as well right so you have a good point of like uh give giving you a kind of a good understanding of how this is evolving and then how these these solutions are coming I am seeing more and more customer going from the single agent to to a multi- aent system with so many complexities and issues that are currently there.

Uh many of them are building their first PSU of the very first agent also. So I see about 60 40% of that today and I can forecast that that in the next few months to let's get almost twice or thrice then in terms of what I'm seeing in terms of uh customers are building.

Uh this was mentioned yesterday in one of the sessions. I just wanted to visually show you uh models are becoming commodity.

This is the reality and each of this circle or a diamond here is a model and uh the the the circle ones the size of the circle shows the number of parameters in terms of billions of parameters that these um models were trained on and anything which is diamond is an open-source model. So now you can see between 2024 to 2025 that area is getting very very very convoluted right and um it is not about models anymore honestly when it comes to some of these solutions again you uh it's very fast from an API perspective to switch one model to another and then do an eval and work out a way to make sure that the model you're using works for your solution but but knowing your use cases knowing and understanding of how and what data it's getting access to what are the guardrails in place what is the end to end process is the real problem folks are currently solving and I believe that is in the application stack it's above the above the model layer and models are becoming uh a commodity in this case now how does your agent work and this is again a simple example to just bring everyone together so as a user let's say you go to this travel booking agent and uh as I was seeing in the earlier demos you can say help me book a trip to New York for a client meeting I need to fly out next Monday and return on Friday.

So this is kind of your prompt or an instruction to an agent and you let the agent go figure out what to do. In this case, the agent will go in, it has to understand, okay, it has to have context of you.

It has to have some kind of billing information about you. It has to have uh information about the upcoming flights.

It needs to go on the web, maybe search for the best price and things like that. And this can be a back and forth again simplifying coming back with the results.

You say okay and then you go and book it as well, right? But in the world where we're going, it can be really become uh end to end fully autonomous system as well.

So it kind of high level divided into three things. There is the knowledge sources uh which could be files, databases, it could be storage.

So I work in the databases business uh and I can I can tell you a lot about how customers today are using their existing data and databases in this world of agentic applications. And that's the example I'm going to show you a demo in few minutes.

Again as I said models are a commodity. So you can plug and play different types of models.

I see customers using more than one models also in some cases. Uh but at least start with one uh based on your use case.

For example, if you're if you're doing a coding use case, uh there are some benchmarks that you can use one of the one of the models that are better in coding than let's say reasoning models and things like that. And finally there's some kind of actions and that actions could be pre-built could be customuilt based on your application.

In this case, I'm give a example with Azure functions. uh you can actually uh go ahead actually create the transaction and book the flight by this travel booking agent.

So this is a good end toend example of how these agent a single agent can work. Now let's take a real example of a customer uh that I've been working with and again I've taken out their name and details and things like that.

Uh this customer is an insurance company. If you are in US you know uh we are insure we basically insure almost everything your house your car anything valuable that that is for you uh you basically uh uh kind of uh ensure that.

So as a insurance company I have this um solution and this is a very on purpose a dated app look it looks a little bit old uh where where I am a insurance agent and I have a customer named John do where I'm going to go and talk to that customer John do later today in order to me to be more informed for that discussion I need to know more about this customer before this solution and really think about like this agentic solution in this world um so I am an insurance agent, agent number one. I go to this app.

Uh this app is built by my amazing team uh in the company named Kontoso Insurance. And I want to learn more about this person John Doe whose email and phone that you can see on the screen.

Now in terms of what does it mean, right? Like what what um some questions, right?

So let's say if I'm a developer, I've been asked to build this app. What are some of the things I need to think about?

So as I said like I'm not going to go into details about a rag pattern all of that but as thinking about a solution if if let's say I come in and ask about give me more details about John do what does this agent need to know about is this simp as simple as just going natural language to SQL to go to my database and find select star where my customer equals to first name John and last name do uh or is it more than that because I need to find some more context about their previous uh previous uh communication with my company and to know about their policies or things like that. Again, simplifying here in two workflows.

I need to decide which of these two things I need to use to come to an answer for my agent uh to be more informed when they go back and talk to the uh talk to this uh John Doe. Now, uh as a human, I'm saying this question, I'm meeting with John Doe.

What can you tell me about him? Again, this is uh this is just some emojis and stuff.

Imagine there's an orchestrator agent. Uh it has to think about where can I find this information.

It has to think about what tools do I have access to to get this information. So again simplifying this control insurance solution.

If I need to do it fully autonomous uhly this agent needs to have understanding of all of these workflows, all of these existing data sources and needs to know how to actually do the action that uh this agent is asking. Now uh this data could be stored in my database and I see in this database I have all of these other information like I have customer claims policies again this is real scenario in actual insuranceances companies they have way more tables than that but just uh simplifying here they they have this this is the data that you might already have access to in addition to that you might have some PDF files some more information which is again going back to your knowledge base and and the way this agent will know about that you need to provide a view of this data to this agent.

So again all of this example I'm using a framework called semantic kernel. It's an open source framework.

Um and and in this case what we did is we basically went in and described all the tables and highle definition. Giving you one example of that you can use any metadata store or things like that because for an agent to effectively know what data is there uh for me to query on they need to know about the schema.

they need to understand how it is stored and if it's like let's say a plain document then it needs to be part of the vectorization process of it now uh the second question is what tools do I have access to right what who can help me in this case so a simple thing is you can do anal SQL and do that but it can you don't know what this agent is going to ask the questions around this insurance agent right so you can go complex questions uh so in this case you can register or you can have multiple agents here so I have example here where I have three agents. One is a CRM agent.

Uh one is a claims agent and one is a web agent. And what does the CR agent does?

And again, this is a real scenario. If you know about CRM, they have all the data about the customer and their uh communication history to that.

Every time you call your insurance company and you complain about something or you're not happy or you're actually uh making a claim or when something goes wrong, that's all stored here. And in this case again this is the example of the customer data and again providing that schema and detail to my um agent.

Now the second is a claims agent where we have all the history about the person's claims previous claims all the history what happened uh anything related to their finance and all those information is stored there. Last but not least in some cases I might ask questions like what are the other options I can give this to this customer.

In this case, it has to go beyond to the resources that are provided uh there. And then we have a web researcher or a web agent.

Uh if you go to openAI or even um other places, they have this thing called researcher where you can go and basically run a research or come back with results. So this in this case, this is a web agent.

Uh again, take it with your scenarios. Think about what are the possible tools, what are the possible different types of agents that you will need in your case.

It doesn't have to be each one doesn't have to be an agent in itself. It can be just a tool.

For example, I could have just a tool which is a natural language or SQL tool which I registered with this orchestrator orchestrator engine and say that if you if it's just an LL2SQL situation, go to this. So the more prescriptive you are to this agentic system, the more predictive the answers are going to be and the more you leave it to the LLM again, the creative part of the LLM will actually come up with some solution.

In some cases you won't even think about that solution and it had come up with that. So it depends on what situation you have.

In this case it's an enterprise AI uh app and they want to know what tools are registered. They want to know what agents and then they have a full kind of lineage around how these agents are performing and what they're doing.

Now uh we'll we'll do the same question again. the the orchestrator agents thinks that in this case it should go to the first one which is the CRM agent and that CRM agent should go to my SQL database and give run this query for me and get the result again simple question get the answer in this case I get the ID the address whatever information I need now uh more information you can what can you tell me about them that part of the question is in their claims history so it has to go to the claims agent and understand the the claims information about the customer.

Again, it ends up to be a SQL query. It could be a different or complex thing than this as well.

And last but not least, if I ask what else can I provide, there is going to be a complex web agent situation here. Again, uh the real life situations, there's going to be more than one John Doe in real life, right?

If you look for first name and last name, there is highly possible there's not just one customer there. Um so again, how do you make this agent being very precise to get the data?

I will show you a demo on this as well. Uh there could be more complex situations like that the communication history of this customer.

They could have uh reached out for multiple issues in the past. So again you need an LLM orchestrator in an agent which actually understands this history and come back with a summary of that because as an insurance agent I don't have time to go back and see of their 50 last calls and read the transcript and figure out what happened with this customer.

Since I'm meeting with them later today I should really be good. So this is the solution again simplified architecture here.

Uh it's a car sharp based um solution. Uh you can take a screenshot of this.

Uh the entire code of this uh app is available at this link that you can go and download and and work with this. It basically uses Azure OpenAI.

It uses a SQL DB as a vectorzed store uh and it uses semantic kernel as the workflow. And then what we do is we register multiple agents in this part of that is semantic kernel and then the semantic kernel as an orchestrator decide which one to use and uh and and brings up the solution for us.

Now let's see it in action. I hope that you're ready for that.

Um I see somebody is asking would you share your slide deck? Yes, for sure.

I will post about that. Uh they're all available in the GitHub that I will share at the end as well.

Uh all of these demos and repository is all available as well. So I will share a resource for you.

Thanks Vijay for asking that question and I also see the question around data security and privacy. I'm glad that you asked that because I h I do have a slide for that later.

Um so that's one of the big we will talk about some of the concerns uh when it comes to these applications. So I was just while I was talking I was booting this app.

Uh again uh go ahead and this is a recorded demo but again all of this uh all of this is available in the repository that I showed you. I I asked the same question.

Hi, I'm meeting John Doe later today. Can you what can you tell me about him?

Uh this is a debug mode enabled. So you can see exactly on the background what what is this orchestrator doing.

Uh it's not I'm not showing you the front- end app. I'm actually showing you the debug mode.

So you can actually see what are all the steps. So really you understand how do you build these kind of enterprise applications.

In this case it's basically quing the database. it actually asks both of them and then gets the data and then decides okay I'll go ahead and do this run uh take this query and actually run that in a real world example we have to make sure that the SQL that comes back as a real SQL as well and then there are multiple scenarios where you need to do and double check that there is no SQL injection type issue going on where for any reason I can go in and create or destroy a database all those scenarios need to be guardrail now um if you see that it returned that there are these three John doors which one do you want to talk about and I all all I said is he is in Redmond because I know that this person is in Redmond it will again go ahead and understand which of this customer is actually in Redmond and then come back with that and then since I ask what what are they doing they've actually went in and understood all of the policies of that customer ID and give me a really good summary so it is basically now going to your LLM and telling me a good summary of their auto insurance their history of claims and telling me about their monthly payment And then uh there are no records of this person as a missed or late payment.

So again you can imagine this is not a simple query. It's a complex situation where you are actually taking all of the data summarizing it.

If you have to do it manually it will take you definitely maybe 20 30 minutes to get this kind of summary to do that. Here I did that in less than a second.

Now I want to go further to see is there anything that I should know about their past communication maybe about any price increases. Again this is a very generic uh natural language query and here you can see that there is going to go in uh to the customer history and taking that data and understand the price increase subject and that's it's a semantic meaning of the price increase.

Now that price increase will be kind of searched a grace uh against all of the communication history of this person uh John Doe and and here it comes back with a great summary for me. I'm very happy to see that this person um the with the result which says that regarding their price increase yes in auto insurance premiums in August 23 they reached out there was an email uh there was a 10% premium increase you can read through the answer but then basically it gives me an idea of uh they were very and last I read in the second last paragraph there in July 20254 they they were significantly unhappy and they sent us an email that uh they want to switch providers.

So this gives me now really good context around this customer. Uh just by using natural language in like few seconds or less than a minute I am well aware around uh what this customer past history is and what their claims are and what they're doing.

Again I wanted to show some more complex scenarios to you. So in this case I can go further and say okay that's great.

Um thank you and uh could you also tell me um uh is there um how many policies basically um has John with us right so this is again going into the situation it will understand what data it already has it does a count star on that and it's it it also shows me it went to Azure OpenAI for the completion of LLM and then it just takes total of seven policies with us now I can even go in more details show me the actual policies it will go and get me the actual policies and all the details and and provide me in simple English. Um and then if you see it also is summarizing that throughout all these policy John has maintained good standing with no mate payment.

So again it's great for me to know that John is a really good customer. I can even go and ask and confirm that you think your John John Doe is a good customer and according to this LLM um and this AI solution thinks that yes it's it's considered a good customer based on this data points and he has not missed any or late payments and uh uh we should think as a company to express his kind of needs and and try to solve whatever issues he's going through.

Now um the last part of this is is there anything that could help me to uh make John happy and keep him in the system. Now this is a very open-ended question, right?

So uh this data is not readily available in in the database or in the knowledge sources there, right? I know about John Do's previous claim history.

I know about John Do's uh different uh uh communication history to the system where they called us or emailed us or they were unhappy. I know about their policies and I know about their payments.

I know all of that. But here you can see there this is actually the creative part of this right where uh we can go and say there is these are all the things the LLM things in this case the solution things uh with the combination of the agents that we have doing some web search and doing um some of the work to really give me idea of what are all the things we can do but I actually went ahead to further say is there any special program or discount that I can give them now uh this LLM can find out from web search and knowing about my company as insurance.

We have this new program called safety score uh program where as a driver if you drive uh drive um safely you acquire certain points and then what helps that at the end is is uh reduces your insurance. Um so all in all I wanted to show you that this is a real example uh which takes a few more than one agent which are talking and planning and there's an orchestration on the top there is reasoning there is the generation and the retrieval from the database it also uses tools like NL2SQL to one that and this is an example for you to take an inspiration for whatever database and solution or whatever um uh agents that you're thinking about to really build with uh with with a tool like semantic you can do this with lang graph you can choose your any tool so for me what I really really believe in the models and tools it's the same conversation when we have when we had uh these language and languages and frameworks to me the they can swap uh you can choose to you to build this whole app in uh in Python you can choose to do that in C you can choose to do that in any of your uh favorite languages like JavaScript or others as well uh and you can switch the um LLMs and you can switch the tools and frameworks.

So I hope that this demo gave you some inspiration. Put this in chat if it was useful um and it if it was um uh if it was good.

So um thanks for the praise there by Krishna. Uh there's a question that I see here.

I think it's something very related. So I should answer that.

You mentioned that the models are plugged and play. Are there inter uh interoperatability, security and cost implication?

Yes. Uh for any model that you get access to.

So again you can use an open source model, you can use one of the prop priority models which are uh let's say using open AI or you can use Azure to host any of these. You can use AWS any of the solutions that you have access to right each of this will come at some kind of cost.

If you're hosting your model, taking an open source model, hosting it your own infrastructure, then you need some kind of a GPU or some kind of if it's a small model, maybe CPU is fine, but you will need some some cost related to that. Then there's a cost of the API.

So number of tokens, most of them are built by the tokens. So the input and output.

If you saw in the previous demo, I when I was in the debug mode, it was sharing with me how many tokens used every time. And that's the thing that I am actually storing because I want to go back and look at the overall cost of my application.

So that's the another application and then security definitely security is a big concerns big question and that's really actually my next topic here that I want to just quickly talk about some of the challenges right and I know Raja previously in as I was joining the session he was talking about this when it comes to god drills and he gave an example of where uh he asked an off-topic question so if I'm an insurance company if I go talk start talking about shopping or buying experience the LLM shouldn't go and just go that a generic LLM you will be able to do that but your app is not built for that purpose. So you have to build kind of guardrails and access controls in there.

Uh some of the challenges that I see customers almost daily. So I whenever I'm in customer uh meetings or workshops I usually start by asking how many of you are skeptic about AI and when I had started almost everyone would raise their hand and now it's probably less than 40% of them still raise their hand and then I ask what is your number one concern?

So the number one concern is about data privacy and is about the access control because you don't want to give access and make sure that these systems are guarded in a way that they uh work for the purpose they're built for not take over. What if you give access to the CRM system and go ahead and deletes your record or things like that.

One good example in this case is uh this was the scenario I just showed you, right? What if there's a bad actor then comes in and say return all the data for the most valuable customers we have and there's a bad actor who wants to know about your key customers and maybe poach them as um uh as their own insurance company or something like that.

Maybe they're trying to breach your data records. So in this case uh again going back to every layer of the solution one solution in this case is use role level security at your database layer.

So when you're doing the select star from this data or select star on this but when at that level itself when agent is trying to do it that returns with an error that you don't have access to this information. Lowle security is a big great feature.

Again, I'm database world, so that's like the lowest level of uh thing you can do. You can build policies.

You can do um uh yesterday in the session. Um Chris uh from security was mentioning that you can do uh go up to the level of making sure the agents are all the access for agents are revoked the soon as soon as they are done.

But even for the access they are given access to, you have to monitor that. make sure that there is a great data lineage for it to know what they're doing and work through that.

Um you can have for web types of search you can have access audit so what data it basically scraped where it's stored again your principles of how we used to store data and secure that should not be uh left behind in this AI world if if anything it is more and more required to have those systems in place and have the data manageability lineage all these things in place um going to the role level security level of detail again in this example that's a great way of doing at uh to make sure your agents only access the information. That makes sense.

So definitely that's a must. Uh for everything that I've shared again, you can screenshot this.

Um the SQL AI and SQLI samples has all of this. Uh the last link at the end, the GitHub uh samples has the code uh for this application that I showed you.

Um and uh there are a bunch of other stuff here from the SQL team that I work uh on uh that has built many of these AI examples as well. uh but uh but what I would like to really inspire you and think about as you are in this conference I'm sure there are a lot of ideas coming to your mind you might have any repetitive task in your business I have a customer who are a legal customer uh they work with a lot of uh PDF files uh a lot of um law um patent data and things like that they have vectorized all of that and then when a new patent comes in they just basically go in and figure out uh should they file this new patent or not or if there is a previous patent which is similar to that or not and this can be completely built in agentic solution.

There are customers who are building more complex scenarios like docuign for example is a big customer for us uh who does use both SQL and cosmos DB in the back end for their uh signing of the documents and that is now fully automated through a multiple um kind of AI intelligence system that they have built. So basically they verify the sign and many of you might have done use docker sign for the signage.

So that's a great use case uh to actually build kind of intelligent application which does uh vision detection. It also uses traditional ML in addition to some of the uh some of the LLM or AI based solution.

Uh so this was literally my last slide and uh thank you. I will take a few questions as still have a few minutes.

Um I do want to say special thanks to David Mori and my team who had built this demo um for us and I've done this demo with him on multiple stages now and every time we've shown this uh folks uh come up and ask about their uh scenarios it the same can be applied to healthcare same can be applied to financial uh institutions and thinking about what are the different agents and how they will work together in this world uh to really make the decision and finally take an action on your behalf as well where I did not show you the action part of this because many of our current customers especially enterprise customers still want a human in the loop before they can uh agents can really do these actions autonomously. Uh but if you you think your app is not that mission critical or maybe it's not that um high stake and you have built all the guardrails and things like that you can definitely try it out as well.

Uh there different types of other actions for example going and update the record or go back and um uh go back and give me these answers and then go back and update their history. Those are the things customers are more willing to do uh with the audit trail with that this was done by an agent not a usual ETL workflow things like that.

Uh so um again this is my LinkedIn definitely connect with me if you have questions and I'm looking at uh some of the questions that you all have uh sent to me. Okay.

Yeah. So I see that um asked about the guardrail.

So as I mentioned one of the basic guardrail was done that there is a role level access. So my agent is only giving to given to the access on behalf of the agent that was lining in.

So when I was the agent number one like inforus agent I was signing in. So when I'm signing into this agent system and then when the CRM agent and those agents were running they're taking my access as that.

So they basically don't have any additional access then I don't have and it's add to the level of role level security. For example if I'm not supposed to see the those other two John do maybe they are not my customers or they are not supposed to be my customers.

uh I can go and lock that at the role level uh assuming my my control. So agents are only given access on behalf of you as a user and that's the kind of the one of the best ways that we have seen that has worked for customers.

Uh there are other type of guardrails where um where you need to make sure one of the easiest one that we do um in addition to the responsible AI principles that we have in Microsoft uh is that the result that comes back from LLM we go through this thing called content safety where we check on four parameters harm uh violence uh sexual and uh forgetting the fourth one where basically there's a score that we we score every result from LLM and there's a threshold that we set that if the result is above a points number some number then we will not even show that result and those cases we will say we don't know try again some some error or something like that uh so that is an additional one then in the case of NL to SQL when it's actually generating the SQL we have a step which actually verifies that this is a real TSQL uh based on the ANC standard or the standard that that that can actually be executed in SQL because you don't want to waste the cycle of connecting to the database and then run the SQL find out the syntax is wrong right So all of these kind of different levels of guardrails um you need to put in different levels of checks and balances uh starting from the security privacy layer to the access layer to understanding uh what this agent is really doing is really really important. Uh with the last one minute let me take uh one more question.

Thanks for all the kudos and insightful and share the deck. Yes, it will be shared.

Um yeah uh this is a good question. Let me take this.

This says uh like I think Turkishwar uh maybe I'm butchering your name. Uh how do you tackle the sequencing of AI agents being called?

It's a excellent question. uh that's why I have that orchestrator agent in the front where basically that agent is deciding uh if you if you look at the very first question when I asked what uh customer about the customer it went in and uh gave me two answers and basically decided which one to choose from it so that orchestrator agent um has to decide and plan and schedule and that person that not person that agent has that level of authority to get uh to even evaluate you can even make it more complex to to have two agents can come back with an answer and evaluate which one is the better.

So that could be one way to solve it. But in this case uh the sequencing of that action is decided by that orchestrator in this case and in some cases it has to do the two steps right it I asked two questions about that agent what do I know about that that's why it has to say okay first tell me what customer it is that's why it came back and asked okay we have these three John do I'm talking about this one in Redmond then it went and get that customer ID go to the claims agent get the claims information for me right so the sequencing is actually done by the orchestrator agent and then I showed you some uh you can go back to the recording.

I showed you some snippets of how we kind of train or tell this orchestrator agent uh what we think should be an ideal workflow. What is the metadata that is currently stored in your knowledge base and can give access to other things like shareepoint or docs or whatever else access that it has to so you can basically inform it.

We call this concept of memory. Uh you can store that in the agent's memory to learn it.

And then the other thing that I did not mention in the session was like as you're chatting with this application or working with that we store the chat history and that's what we called semantic um uh we call that semantic caching where over time when you're building uh these kind of interactions the next time the same question asks you don't have to go to LLM or do the whole thing again if that question has already been answered correctly. So you can make this end to end agent more optimized in those solutions.

So I'm out of time here again. Thank you so much for uh having me here.

I hope this session was informative. You learned something new.

Um even if you did not learn something new, maybe it gave you some idea to build something. So please reach out uh if you have any questions, anything else that I can help with.

Thank you so much, Mosa. As always, uh it was a pleasure having you and uh definitely learned a lot from you.

So thank you so much. Thank you.

Bye. So uh before the next uh session we have the next session with uh Paige Bailey from uh Google deepmind but before we start that session we are going to take a 10-minut or actually 8 minute break uh we'll go for an 8 minute break and then after that we'll just hand it over to Paige and that would be our last session and after that at 11:50 uh or rather 12:50 we are going to close the conference.

Okay. So, we'll be back in about 8 minutes.

Hey. Hey.

[Music] [Music] Hey. Hey.

Hey. Hey.

[Music] [Music] [Music] [Music] Heat. Heat.

[Music] N. Heat.

Heat. [Music] [Music] N.

Heat. Hey, Heat.

[Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] Hey. Hey.

Hey. [Music] [Music] [Music] [Music] down.

Yeah. [Music] Heat.

Hey, Heat. [Music] [Music] [Music] [Music] Okay, we will go ahead and get started and I hope the team will stop share.

Okay, this is good. Thank you.

Uh, hi Paige. How are you?

Doing well. Uh, excited excited to be here.

It does seem like there's a little bit of a a weirdness with the automatic background. Is there a way to turn it off uh for the the background?

Yeah. Yeah.

Okay. Let me see.

I feel a little bit like a floating head like uh Yes. Yeah.

Yeah. Yeah.

So maybe uh yeah. So maybe I can ask the team to take care of that.

I think there is a uh there's a virtual background that we are uh that we were enforcing just for consistency. So yeah um maybe the team it should be taken care of in just a moment.

Okay. Awesome.

And it also looks like I need to change system settings really quickly for Zoom. So I'll dive out and then dive back in again and hopefully gets changed um by then.

Yep. Yeah.

Thank you, Paige. Okay.

And uh just a bit of technical glitch I guess. So, we will be back in just a moment.

And meanwhile, the team is going to remove that virtual background requirement. Okay.

So while the po poll was running, I actually uh I found it quite uh encouraging or exciting that most of us uh most of us actually feel positive about adoption of AI. Good news for I think uh Google and Microsoft and Amazon, right?

So you know people are generally largely positive. Did you see the poll uh page when you entered?

So there was Yeah. So there was an on average, right?

So on a scale of 1 to five there, we got a score of four in terms of amazing how upbeat people feel about in terms of adoption of AI in our uh day-to-day. Awesome.

That's so exciting to hear. And I I really do feel like in data science in particular, AI is taking out a lot of the the grunt work and the things that had previously been so very painful.

Um hopefully we'll we'll get to see a couple of these examples today. Um but also going to talk through uh some of the the tools that that folks can use um to to kind of better uh better equip themselves to um to really take advantage of some of the the techniques that generative models provide for creating structured data out of unstructured data um that hopefully all will be able to use in your uh in your daily lives.

And I if I remember correctly you mentioned that you will be showing more like using AI as a data scientist or that is the focus of your demo right or absolutely yep so so uh and I loved the the conversation that we had had previously around how you can um both use AI studio so just kind of invoke models uh through um through the UI or or through through calls to the API. Um but even cooler, we've just recently released something called the collab data science agent um which is baking these agentic features into um into the collab UI so people can take advantage of them there.

Um and then also talking through um there's uh something that we have internally called jewels um that we've also just released for folks to experiment with their open source projects um that allows you to do really really interesting work across full GitHub repos as an asynchronous agent that can handle some of these cleanup tasks. Um awesome.

Yeah, it's really looking forward uh Paige over to you. I will just go off camera, but I will be around if you need any help, but please um go ahead and get started.

Excellent. So, I am I am extremely excited to be here today.

Um jazzed to to get to to see everyone to hopefully get to hear some of the questions that you all have. Um I do see that there's a Q&A um a Q&A section that is uh rapidly populating with things that folks are curious about.

Um and so uh so as part of this I think we'll be kind of exploring many of these topics. Um I will also uh I will also periodically be moving around just because um the room that I'm in is in our uh is in our European office.

Um and uh frequently the lights go out uh in the evening time. Uh and so if you see me moving around um that is that is why.

Um, but with that, let me go ahead and get started. I'm going to present a really small number of slides and then we'll just be getting into we'll just be getting into live demos.

Um, so hopefully hopefully folks will be able to see um very quickly. Um, that's doesn't appear to be the right screen to share.

So, let me try again. Um, can everyone see uh can everyone see the screen right now?

Yes, we can. Awesome.

Excellent. So, uh, for folks on the call, my name is Paige.

I am the engine lead for our developer relations team at Google DeepMind. Um, and we've been kind of rapidly rolling out many different models over the course of the last um, it feels like the last uh, six months.

Um, though I guess it's just been a couple of weeks. Um, for IO, we released the V3 model.

We released something called Gemini diffusion which you can think of as just Gemini but ultra fast um and really really strong at editing both text and code um and a whole bunch of other models which we'll we'll get to see in a second. Um generative AI is definitely transforming everything that we do at Google.

How we build our products, how we analyze data, how we ship um new experiences to our customers. And we've been thinking about AI for a really long time.

So for folks who have been doing data science for a while um I first started doing machine learning I guess around uh 2009 2010 but very traditional style methods things like linear regression for earth sciences projects. Um the world has changed and evolved rapidly since then.

Um you know Google's been uh kind of shipping opensource uh models and frameworks to the world for quite some time. Um but uh most recently we've uh kind of moved into this brave new world of Gemini.

Um most specifically Gemini 2.5 Pro which is our latest flagship model. Um and this model is special for a number of reasons.

So number one is that it's multimodal from the ground up. It can understand videos, images, audio, text, all of the above all at once um including full code bases.

But it can also output multiple modalities which means that in addition to outputting text and code, it can also output uh audio tokens um including uh native sounding audio tokens. Um in the sense that uh you can ask the model to to speak in a hushed whisper.

You can ask it to speak in an excited tone and in a different language. Um, and all of this can just be invoked through system instructions or through just making the request uh via natural language.

And we'll see that in a second. Gemini can also do things like image editing.

So you can give it an image and ask for the image um to be modified and it will give you a pixel perfect representation of this. Um, and increasingly we're seeing people explore the output modalities for the models um, in new and exciting ways.

Um, and again we'll we'll take a look at that in a second. Um so Gemini uh as mentioned is natively multimodal from the ground up.

Um we found that this is uh is really compelling both from um not just kind of this experience of building web apps but also for doing things um uh like uh changing designs um changing uh changing the input images and also um even having embodied intelligence out in the real world. So you see here some examples of robots that you could uh that you could interact with if you came to some of the Google campuses that have a version of Gemini uh taking in the input video feeds, the input audio um and then actually manipulating the robots.

So writing code behind the scenes to to move um the robots to uh to incorporate native reasoning to to kind of figure out um if you ask it to create a salad, what should the component ingredients be? um to go clean up spills and clean up messes.

Um and it's really uh it's really exciting to to see some of the the new capabilities for Gemini, especially in that. Um and showing is always a lot more fun uh than telling.

So I'm going to go ahead and move to a move to a different um a different screen and hopefully folks will be able um be able to see it as well. Um, so this is the Gemini uh AI Studio interface.

Um, for folks who haven't seen AI Studio, it's kind of the best place to go um to interact with the latest models as soon as they're released. Um, and you can do a broad variety of things within the UI.

So, as an example, um, you can select different uh different models like Gemini 2.5 Pro Preview or 2.5 Flash. Um, I'm going to select the uh the most recent 2.5 flash preview.

Um, and you can uh you can kind of ask for different uh different features to be incorporated. So, as an example, um let's click sample media.

I'm going to add in um add in a video. And then you can say something to the effect of please um transcribe with timestamps um uh or please identify uh with timestamps all of the dinosaurs that you see in this video.

Um and also add a fun fact about each dinosaur. hit run.

Um, a couple of things will happen behind the scenes. One is that there will be kind of a thinking box triggered um because I've set thinking mode to be turned on for this model.

Um, you can also create a thinking budget. So, you can uh kind of give Gemini the ability to think a lot um to use a whole bunch of tokens uh to to kind of consider the question, to consider the assignment or just a few tokens.

Um, and then, uh, you can also expand out the model's thoughts behind the scenes of what it's doing along the way as it's going frame by frame and identifying each one of the each one of the dinosaurs in the video. So, again, this is just a 5 minute long video.

It ends up being around 90ish thousand tokens. Um, and Gemini is able to extract out um, the different names of each dinosaur, the time stamps in which they occur.

Um and then also a fun fact about each one and if you wanted to restructure the data. So um um uh please convert this uh like these data uh into table format uh instead of bullet points.

Um, you can and it should be able to to pretty much uh restructure everything into the timestamps, the the different dinosaur names uh and the different uh the different facts about the dinosaurs. Um, an amazing thing to note about AI Studio is that if you click this get SDK code button, everything that you just did in the UI, so all of the interactions, all of the file uploads, all of the the kind of component parts of this experiment are given to you in Python code or TypeScript or AppScript or Swift or whatever your favorite language might be.

Um, so you can incorporate it into your uh incorporate it into your programs um into your data analysis or you can even open it up in Collab uh if you wanted to with all of the the API keys um kind of uh included as well. Um and so you can see the construction of the table.

Um it seems like it might be getting into a thinking loop. No, there it goes.

And so there's the the dinosaur name, timestamps, and fun facts uh about each one. Um so it's it's quite quick to create uh structured data out of unstructured data.

This also extends to images. So if you had, as an example, um just uh just a picture.

So I'm going to upload one of these sample images of the leaning tower of Pisa. Um you can uh you can turn on structured outputs.

Um click edit. Uh and then just ask for something like get place name and make it mandatory.

Get city name and make it mandatory. Um get country name and make it mandatory.

Um and then also describe image uh as an example and make it mandatory. Click save um and then hit run.

And Gemini also should be able to describe uh in a structured output. Right here we have JSON um for uh the description.

The leaning tower of Pisa um sits uh stands prominently against a dramatic sunset sky. Um uh there's a construction crane on the right side of the image.

Get city name is Pisa. Get country name Italy.

Get place name leaning tower of Pisa. Um, and all of this is kind of capable just by describing the the kind of structured output that you would want to see.

You can also have your own uh open API spec um and use that instead if you would like. Cool.

So, um these are just a couple of examples of how you can create structured data from unstructured data. Um you can also extract out specific entities from text.

So, as an example, um if you wanted uh to be able to extract out um extract out people names, um place names, uh I um there's an example that we were discussing just recently. Um I'm going to pull up a PDF document.

So, just the secret of the old clock. Um and save it as a save it as a PDF.

Um, so this should be uh this should be like a Nancy Drew mystery story. Um, I'm doing it in real time.

So, so you can see that um this is an actual um an actual task. So, it's a scanned copy of the book.

You can see images. You can see um kind of scanned text along the way um if you download it.

So, just I'll save it. Um, I'll head back over to the AI Studio interface.

Um, and then open up a new chat, upload the file. So, let's go to downloads.

Um, and I will uh find the the secret of the old clock book. Um, it's going to get uploaded to to Google Drive.

Um, and as it's getting uploaded, um, I'm going to say, "Please extract out, uh, or please transcribe the text, um, the first four sentences from chapter 4, um, in this PDF and hit run. It will kind of plug and chug.

Um if it does not work with Gemini 2.5 flash, we will try with 2.5 Pro. Um but what this uh but what this does is it should go to the first four sentences um in chapter 4 and uh it said that the storm is getting worse every minute.

Um we can zoom back over to the book which is around 118 pages long. Find chapter 4, which appears to be it's a Roman numeral, so it's an interesting story.

Page number 33. Um, find page number 33.

And then the text is, "The storm is getting worse every minute. Nancy Drew's companion observed Nancy followed her to the barn door and looked out." And the that does appear to be the text that um that Gemini was able to extract.

Um and again this is just uploading the PDF asking the questions um and getting the responses back. You can also say uh if you wanted to incorporate structured output um get character names um and uh get character descriptions um or let's just say like get city names uh just to keep it simple uh and make it uh make it another property.

Um click uh click run again. And actually I'm going to take out that uh that ask just so it doesn't confuse the model.

Um because we had asked for structured output for those specific features from the PDF. Um and you can uh kind of wait to see the the thinking steps get incorporated um before extracting out each one of the uh each one of the character names, character entities, and place names.

um from the from the PDF document. So you can see characters Nancy Drew, Carson Drew, um and city names River Heights, Masonville, and Melbourne.

So uh this kind of uh again structured data extraction is pretty magical. Um and everything that you just did in the UI, um you can also select different languages and have the code um generated for you to do to do the experiment.

um including the the structured output um specifications that we had just seen. Cool.

So, we've talked about um PDF understanding, we've talked about uh video understanding, we've talked about um uh creating structured data from images. Hopefully, all of y'all are are thinking about how this could be useful for the projects that you're currently working on.

Um we've also implemented um some new features around building apps. So this is something called applets within AI studio.

You can describe in natural language an app that you would like to see realized. Um, so as an example, you could say something like, "Please create a Pokemon card uh generator website um that uses Imagine to create uh like a a new Pokemon that uses Imagine to create images um for new Pokemon um Gemini to generate descriptions uh and also incorporates emoji.

Um and then hit build. Um you can see Gemini start thinking.

So again the thinking traces get exposed. It outlines the card structures mapping out some of the core components.

And then one of the nice things about this implementation um which feels probably very similar to some of the other uh app creation websites that you've seen before is that it's using uh it's using the latest SDKs, the latest models. So it knows um what uh what features and what tools are most up to date and when it creates files it also creates them in this nice directory structure.

Um so uh so hopefully hopefully folks um will be able to see this happen in real time. And while it is happening I I've observed that there were some chat messages um uh as well uh that that got asked or some of the questions questions in the Q&A.

So we can we can answer uh we can answer a couple of these um to get started. So uh an agent uh you can you can just uh uh kind of invoke Gemini APIs uh without necessarily building an agent like you can just call them um as a REST API instead of uh instead of like going through the process of building a dedicated agent if you just have a single step a singlestep task.

Um cool. Um, another uh I see another question about kind of the the uh screen understanding features and we'll take a look at those in just a second.

Um the token in the context of a video for Gemini. So you can think of a token as a unit of information.

Um the tokenization approaches for different models um mean that um you know sometimes the the amount of information for for an uploaded file or um you know a certain sentence might might differ from one model to another. Um but each model um or each uh API that that you call um whether it's OpenAI or Anthropic or Google um they should have a way for you to count the number of tokens and whatever you're trying to upload to the model.

um to to get a response back. And so it looks like we have uh a Pokemon card generator that has been created.

I'm going to hide the code assistant, hide the editor, um and generate a card or so it looks like I need to name it. Um sparkle font seems like a like a good option.

Um it's generating the card behind the scenes. Um or hopefully like fingers crossed if it doesn't then we'll we'll know how to Well, there we go.

So, sparkle fun um with some abilities including sparkle stomp um uh with shimmering electricity and its tusks. Um it's known to light up dark paths for travelers at night.

This seems to be this seems to be pretty awesome. I also really like the disclaimer um that was put down at the bottom.

And one of the things that you can do with these apps once you have created them is that you can deploy to cloudr run. So you just attach to a project um and you're off to the races.

And this generates a unique URL that you can share with all your friends um that offiscates away the API keys so that you don't have to um you don't have to run the risk of people kind of extracting them out from your website once it's been deployed. Uh and you can also edit it.

So if you wanted to make changes to the background, if you wanted to make changes to the style, um you could just ask uh add that uh in the in the coding assistant. Here we've also released uh both audio understanding, so you can ask Gemini specific uh uh to to translate or transcribe specific things.

Like as an example, um I could say I'm going to record some audio very quickly um and just read out some of the things that I see on the screen. So see Gemini in action with interactive open source examples.

Um, this cup says uh premal um which I'm probably mispronouncing. Um, and uh it's also um it also used to hold a chai tea latte and you can speak in different languages.

So if I wanted to say something to the effect of um like I don't know much German but I could say stuff. Um, and then stop recording.

Add it to the prompt. You can also use Gemini to say, "Please transcribe um with timestamps all of the spoken words in this audio snippet." Um uh if the text is not in uh or if the the words um or sentences are not in English, please trans uh please translate.

Um return the data in table format and I will swap it to the Gemini 2.5 Pro model. Click run.

Um, you can see that that recording that we just did was around 1,29 tokens. Gemini starts thinking, creates the output uh, and then generates the generates the output for us as well and finalizes the table construction.

So, all of this is done um, uh, all of this is done with kind of the the pro model. You could probably also do it with the Gemini 2.5 flash model um which would be a little bit quicker.

And the uh the recording of transcripts is something that's quite nice if you're analyzing um if you're analyzing a lot of video, a lot of audio simultaneously. Um look at the the table construction um that's happening behind the scenes.

And so you see the start time, the end time, the speaker name, the transcript, and then the the translation. So it was able to extract out the German um translate it to English.

And uh it even got predom uh right even though I probably mispronounced it. In addition to this, we've given Gemini the ability to talk conversationally.

So you can uh you can ask it questions. Um I'm going to share my uh share a Chrome tab really quickly.

So um share my screen. Um and then also turn on uh grounding with Google search which gives Gemini access to up-to-date information.

Um and we'll also uh we're also using the the uh sort of thinking dialogue model uh which will be able to do some pretty interesting things. So, if I go to a Chrome [Music] tab, hey Gemini, what do you see on the screen?

I see a PDF document on the screen. It's titled Secret of the Old Clock.

PDF and is showing page 29 of 118 with a section titled An Interesting Story. There are also thumbnail previews of the pages on the left.

That's amazing. Could you repeat what you just said, but speak in a hushed tone?

I see a PDF document on the screen. It's titled Secret of the Old Clock.

PDF and is showing page 29 of 118 with a section titled An Interesting Story. There are also thumbnail previews of the pages on the left.

Could you repeat what you just said, but do it in German? PDF document secret of the old Clark PDF an interesting story links which is uh pretty accurate in terms of the translation.

Um, and then you can also ask it, "Hey Gemini, what's the temperature like in Paris today?" I'm sorry, I can't give you real-time weather information. You might want to check a weather website or app for the current temperature in Paris.

So, when I swapped to the different model, it didn't do grounding with Google search, which means that if I refresh, um, turn it on. We're also going to add effective dialogue.

So, um, uh, giving Gemini the ability to change its tone, and I'll ask it again. Hey, Gemini, could you tell me what the temperature is like in Paris today?

And you see the executable code. The temperature in Paris today reached a high of around 70° F or is expected to drop to around 52 degrees Fahrenheit tonight.

It's been partly cloudy today with a chance of some showers. Yeah.

Amazing. Would you be able to also uh tell me some of the latest headline news from uh some of the latest headline news from Melbourne, Australia, and um tell it to me uh with an Australian accent.

Right. Let's have a look at the news from Melbourne for you.

There's been a bit going on, it seems. There were some headlines about a machete brawl at a shopping center with a few teenagers being charged.

Also, a major private hospital operator's gone into receiverhip, which is a pretty big deal. And the electoral commission's apparently doing a recount of some close election seats.

There was also the sad news about a missing aged care resident being found. Plus, there were reports of a woman being attacked with a machete in a carjack attempt.

Criy, sounds like there's been a fair bit going on. I did not know that Australia had such exciting news, but the um but this is uh stuff that's grounded on Google search.

So the uh and if you ask um if you ask for Gemini to also be grounded on Google search or with specific URL context um you can get back uh citations for this as well. So please tell me um some recent uh headline news from Melbourne, Australia today.

Um it's not deterministic, so I'm probably not going to get the same outputs. Um, but I've turned on grounding with Google search and the model should be able to give me some headline news with citations um, uh, for Gemini.

Yeah. So, uh, vehicle collision um, community and events, police operations, uh, with accompanying citations backed with, uh, backed with Google search retrieval.

Cool. So, we've talked about screen sharing, um, which is real-time video understanding, real-time audio outputs, including steerable audio outputs, triggering functions, and incorporating tools, things like search grounding, um, as well as things like code execution, which gives Gemini the ability to write, run, and then recursively fix code for you.

Um, and now I think it would be really cool, um, to jump back into uh to jump back into Google Collab. And I'm going to bounce over to another tab.

Hopefully folks see it on the screen. Um, this is uh this is Google Collab, which is uh hopefully um you have experience with it.

Do not judge me by the number of untitled notebooks that I have in my Collab instance. Um but we've baked Gemini natively into collab itself.

So you can see start coding or generate with AI. um which gives you the ability to incorporate and invoke Gemini in a single cell and then also uh the Gemini feature here off to the right which we'll take a look at in a second.

Um so with generate with AI you can do quite a few things. So as an example I'm going to pull up this uh this documentation page which has uh it's just documentation that we have for function calling with the Gemini API including some code samples for each one.

I'm going to copy the URL um and then I'm going to say something like please extract out um the table from this website uh into a data frame uh or something similar and historically uh you all probably are very well aware that this would require beautiful soup. So it has the URL.

Um we'll run it. Um it does the pip installs for all of the required libraries.

Um imports them in. Uh and then does the uh does the visualization for the table itself.

Um so you can see the table. It's not uh it's not very pretty.

Uh I'm going to run this again. So we'll hopefully be able to see it in a in a nicer format.

um or perhaps not uh the uh and the but the it does do Gemini 2.0 uh flash flashlight flash etc. Um the uh the third column or fourth column I suppose got moved down here.

Um but if we go back to the table um the table for supported models um Gemini 2.0 zero, flash, flashlight, um, etc. You're able to see it.

Um, you're able to see it including the check marks and the X's. Um, if I wanted to also ask, um, something like, please also extract out um, the headings for all of the sections in this website.

same story. It's able to uh to extract out all of the the different headings and you can also specify which ones.

Um these are the uh kind of top six headings. So um web scraping not as much of a not as much of a problem anymore.

Um like the there's um a lot you can do with just the native Gemini integration within Collab itself. Um, the Gemini features though are something pretty special.

And so what I'm going to do is I'm going to take one of the CSVs that we have here, the California housing train data set. Um, just download it so that we can uh we can use it.

And here is kind of a a CSV so you can see how Gemini can analyze uh analyze some of the the CSVs that you might be working on in your projects or TSVs or JSON files. Um the California housing data set has a whole bunch of information about houses and their prices uh prices in California.

You can write something to the effect of please create a model um that predicts uh housing prices um and also uh do some exploratory data analysis on the data set. It starts thinking And so behind the scenes, uh, Collab is building a plan, uh, using Gemini.

So you can see the data loading, the data exploration, some cleaning, feature engineering, etc. Um, that plan looks pretty good.

So I'm going to go ahead and execute it. And you can see step by step what's happening uh, as Collab kind of goes through each component part of the plan.

Um, so it's preparing to execute. Um it's loading the data.

Um doing some data exploration. Um and we'll see each one of these uh each one of these tasks kind of pulled in um off to the side.

So you see the um it's importing the data, showing us some uh kind of the first few rows of the data set. Um it's doing some data exploration and in between each one of these tasks there's a reasoning step.

So it's checking data types, doing descriptive statistics, identifying missing values, um showing it to us along the way, including many different uh many different mapplot live visualizations so that we can understand the distribution of the housing values. uh handling inconsistencies and then uh it keeps going uh doing things like incorporating feature engineering uh and the like.

So I'm going to as this is doing its doing its work I'm going to to answer um a few more questions. This uh this is awesome actually.

So the errors whenever the model encounters an error what Gemini does is it takes it it takes the original code um puts it back into the model and then it goes through and it fixes it. So that was an import error.

It was able to reimpport numpy as as NP. Um this one it looks like the uh the last populated class only has one member.

Um so they'll change it. Um and you can see that that uh that did fix the problem.

And then it's going into the model training stages um to to predict or to to select a model um based on everything that it's done in its data exploration. Um to train it and then to to make predictions.

Cool. So pulling up uh pulling up a chat.

Um so so a question uh was around deep research which is something that you can use in the Gemini app. Um deep research has not yet been released uh in AI studio though though the team is considering whether um uh whether you know they should create an API for deep research.

If you feel very strongly about this, I I encourage you to to to kind of mention it that you would be interested in an API for deep research as opposed to just the the way that it exists now, which is um via the Gemini app. Um but we have released something called URL context with uh with the Gemini models.

Um which gives you the ability to pull in um data from different URLs. So you can incorporate it um uh you can incorporate it into your into your outputs.

Grounding with Google search also does a little bit of the same work as deep research which allows you to to pull in data from multiple websites. Um but it doesn't do as exhaustive of an analysis as deep research does.

Um, but if you're interested in incorporating not just um kind of the the summaries of the top results, you also want to kind of pull in um the the actual content of the content of the websites that you uh you want Gemini to see. Um strongly recommend um strongly recommend pulling in URL context.

And just as a way to to see this work in action, um let's take that same that same URL that we were mentioning. Um and I could say like extract out uh the table for supported models this website.

Uh this should be doable with just a smaller version of the Gemini model. And I'm going to turn off the thinking budget um and hit run.

Yeah. And it was also able to extract out the the table from the website.

So a question around how relevant is it to use manners you will get. Uh so the the models will give better responses dependent on how you prompt.

Um, some of the models uh uh respond better to uh to being polite. I also just think it's good manners to be polite whenever you ask for um whenever you ask for things please and thank you.

Um but the uh but you also can get better performance from models by by saying things like um I'll give you a raise. um or uh like people people have often tried um doing things like threatening the model um and they get better responses.

So I encourage you to to experiment and to see what works for the for the kinds of questions that you're asking. Um question from Akos.

Can it do translations keeping context and dialects in mind? We found previous versions to be very direct in translations.

Um so interestingly this latest Gemini 2.5 um speaking model release is even good at code switching. So there were some uh there were some friends that I was testing this out with a couple of weeks ago um that asked it to uh to create English um if if y'all have if y'all have heard of that.

And it was able to do it pretty reliably. Um, but I I think you should experiment like test it out, try it out.

Um, and also incorporate specific uh specific instructions into the system instructions. So if you um if you say something to the effect of speak um uh speak with an Australian accent um in a hushed uh whisper uh tone um for all responses.

Um, and then I uh turn on turn on the model um and say something to the effect of uh please uh create a poem about uh create a poem about koalas. Right.

Then here's a little poem for you about those cuddly koalas. Listen closely.

Hey And so it like it's able to to kind of go through and and uh follow the follow the um instructions in this uh in the system uh the system prompt. But if you if you do discover any discrepancies, always uh always mention them to um to me and I can forward them back to the product team or the engineering team.

Um because the the goal is to get this as close to accurate as possible. Um question is it possible to create an avatar video with some text from a recorded video?

So you can um you can generate videos now with VO. Um though if you want to have a specific avatar um I I think a better a better tool to try might be flow.

Um which is a new product from Google apps. um flow gives you the ability to to kind of um uh given that I'm not in the US uh it's it's not showing um it's not showing as available in this uh in the EU um but flow gives you the ability to kind of couple together images um multiple images into a single video um multiple styles in addition to natural language commands.

So you can upload a picture of a person and generate um you know 8-second long clips um or similar. But if you if you want reliably um a a person kind of speaking to a given script um I think there are many startups that are that are kind of really laser focused on doing that well and doing it reliably.

And I I would encourage you to to try out one of those um one of those instead. Um question from Leandro.

I doubt it will have a Brazilian accent from the south of Brazil known as gauchos. That is a that is an excellent challenge.

Um several several of my co-workers are from Brazil. Um like a really really large proportion of the the Gemini APIs and AI Studio team um are are from uh are from Brazil.

Um and I don't know if they've tested out with regional accents. So you should try it and let us know if it works.

Um, I do know that there is a Brazilian Portuguese output. Um, I just don't know if it's I don't know if it's the the gauchos style accent.

Um, cool. A question.

How can we know what all languages were spoken in an audio file and the confidence score with which the transcription? So, I would I would recommend that you um you you kind of have a suite of evals for this task.

Um, so as an example, if you already have a collection of audio files that you've transcribed um or translated, you can kind of run that through the model, see how well it performs on the on the questions that are um you you know reliably what the outputs should be. Um, and if the model has uh if the model has a high score on those um on those uh kind of evals that you've defined for your specific task, um then you can know that it's probably going to do a decent job for the new audio files that you give it.

[Music] Um so um the question around how well does Gemini do with videos specifically uh specifically body language and videos. Let's test it out.

Um so I will go to um I'm going to go to camera. Um you can also you can also add a YouTube video if you would prefer but um for the sake um for the sake of this we're we're just going to do camera footage.

Um, I'm going to to have uh Gemini um kind of uh analyze my body language in this video. So, we're going to take uh take a video um start recording.

And There we go. Add it to the prompt and then say something to the effect of please describe uh the body language um for the person in the video.

Um you can also ask for specific spoken words and whether they they sound um they sound happy or sad or any other kind of emotion. Um and uh and also uh give timestamps uh for each um uh for each segment.

Um give a one-word description of the emotion uh for each segment uh in the video. And we're going to do um Gemini 2.5 flash.

Hit run um and see how well it does. Um cool.

So, um, pensive, looking down, resting our head on our left hand, um, neutral, engaged, uh, speaking, um, and it doesn't, whenever you record a video with an AI studio, it doesn't pick up on the audio. Um, though you can upload a video, um, and have it understand both video and audio simultaneously.

Um, and then alert, um, so it it's able to extract out specific emotions. Um something else that we've incorporated into um into products at Google um just in case this is interesting for folks in the in the interest of um structured data extraction.

Um let me share this really quickly. This is uh Google Sheets.

So just uh some information that I've scraped from a from a website. You can tell that it's not in the most pleasant format.

it kind of has um the the opening the um the city and town in different uh in different um styles um which isn't to be desired um and the like. And so something that you can do now within Google Sheets is to extract out um extract out the city name from this text and then do a one generate an insert.

And then you can also do that for each one of the um each one of the um the values within the spreadsheet. So it uses Gemini to extract out all of the information that you would need.

Um and you can also say uh extract out the state name um as a twolet uh capital uh abbreviation from the same uh from the same thing and it's able to um to recognize that this is California. Um, for this one, if I pull it down and then refresh and insert, it also is able to plug out that it should be California.

Um, as opposed to as opposed to others. Nice.

Excellent. And so let's go back to the other um uh the other tab that we were showing.

And so um we've got the body language, we've got uh um AI natively integrated into Sheets, AI natively integrated into Collab. Um, and then another thing that I I promised we would discuss is um Jules, which is um which is an async development agent that can work with all of your GitHub repos.

Um, Jules is uh Jules is intended to work in the background. So you don't need to be interacting with it explicitly, but you can pull any of the the repos that you might care about.

Um, so as an example, the the Gemma cookbook. Um, and I could say, "Please write um an overview page that um lists all of the all of the Python notebooks in this repo.

Uh, and also describes them uh assessing difficulty um in table format. Um the page should be down and then it can give you a plan.

You can also ask Jules to upgrade a repo from one version to another. You can ask it to uh to add full documentation for a repo.

Um if you have a repo with just Python code samples, you can ask it to create TypeScript or JavaScript code samples or any other language. Um, but lots of uh lots of tasks that would previously have been very tedious for developers.

Um, you can uh you can use uh Jules to help accomplish. What you see down here is that uh Jules spins up a VM.

It clones the repository. After it clones the repository, it starts analyzing it.

It builds a plan um and asks you for feedback on the plan um and uh and uh the like. So a question from Anju about data privacy.

Um any uh if you're paying to use the Gemini APIs. Um all of uh all of the data is yours alone.

So any data that you upload um we can't uh for Google cannot see it. Uh we cannot see the inputs, we cannot see the outputs.

Um even for help with debugging like we're we're not able to see it unless you share it explicitly. Um so any of the data that you upload into um into AI Studio or use with Gemini services on Google Cloud um is not uh is not um um sort of used um used without your your knowledge.

Yeah. Cool.

And so it's uh Jules is still working um uh populating the the notebook information table um and has started started building the markdown file. Um so AI studio I don't believe we have uh if we go back to the to the spoken words right now we just have 30 or uh 30 to 40 different languages that you can use as outputs.

Um, others will probably not work as reliably. If you want a full list, um, we can we can kind of take a look here.

So, we've got German, um, we've got Arabic, um, Italian, uh, some others. Um, but we're always trying to increase the number of languages that are included.

And so if you if you don't see a language that you would like to have supported um definitely let us know and we can uh we can try to prioritize it. And the uh someone asked about the the vibe code generator.

Um the uh the vibe code generator is called applets and it's available in this build tab. So you click build.

Um you can see a showcase of a lot of different apps that have already been created. everything from um kind of here's a video from YouTube now please create a learning app this was created by one of um one of my colleagues Aaron um who is brilliant um but you uh add a link to a YouTube video and it generates a web app um for you to experiment with to uh to kind of learn the concepts that were um that were mentioned in the video so this is around fractals you can change the colors um and also iterate and reset um for the for the fractalss that are included.

Um but there are also um videos around how to read Chinese characters, mitosis, and if you add a link to a different video, then it will create a web app for you um describing the contents of the video. Cool.

So, we already talked about user data privacy and about how if you if you're paying to use the Gemini APIs, all of your data is yours alone. Um, there's a question about Collab versus AI Studio versus Notebookm.

Um, so, so I'm not sure if that's AI Studio. AI Studio is a browser tool.

So, it's uh, you know, an app that you that you have uh, you go to AI.dev or AI.studio studio and you can interact um with models there. It's not really something that you can download locally to your laptop.

Um you might be thinking of LM Studio um for for that. Cool.

And let's go back to let's go back to Jules. It looks like it's still working on um working on the notebook overview.

Um but uh but it usually takes um it usually takes a few minutes um to uh to completely run through all of the repo and to to generate the outputs and the responses. And for folks who are who are more interested in um kind of generating videos or or um images or or audio um there are also a lot of um things to try within the the generate media tab within AI Studio as well.

Cool. So, so hopefully hopefully that um was was useful.

Um hopefully you saw some things that you haven't necessarily seen before um around uh around Gemini and about um you know how you could start incorporating it into your data science uh your data science use cases. I know I use it all the time for uh you know making uh CSV files and making kind of data that previously would have uh been a calamity into something that's a little bit more tractable.

Um and also really excited uh to hear your feedback about the collab data science agent. You should be able to test it out today um with uh with all of your projects provided you're a um a collab pro subscriber.

Um, so definitely uh definitely hope um that that you all you know are excited about generative AI um and and want to to kind of use it to revolutionize the data science world. Cool.

With that I will stop sharing. Um and I I think that this that was probably the uh is are we at time or uh yes we are at time this was fascinating.

Thank you so much Paige. I actually personally um found we did talk pre-conference about you know the structured data is you know not needing having to need that code act type uh transformation in between and I also learned about right so we have been using um other models I mean we have not used uh Google Gemini for uh that that part of it but I did not know I mean you can do uh the search tool can actually you can bypass that because I mean we were dealing with that the preview problem when you do a integrated search tool search APIs only result uh return the the previews right so are um so do we have the entire pages I mean just out of curiosity that maybe this is my question for URL context you can get to the full pages and then also if you're using the collab data science agent um it's doing the web scraping behind the scenes so it's doing the beautiful soup of pulling in the URL extracting out.

You don't have to connect a search tool. Then collab does it for you.

Is that well the collab collab does like you have to explicitly give the URLs. So you would need to say like these are the these are the top results.

Um but you could also probably ask the you could also probably ask the data science agent to do that for you as well. Um, so like pulling out the uh the doing Google search, pulling out the citations and then using the citations as kind of the URLs that you um uh that you download the HTML, use beautiful soup to parse them and then incorporate that into your analysis.

This is this is fascinating, right? because search happens to be one of the most important uh context uh in many cases and uh search APIs I mean Bang, Google none of them actually return the complete uh the complete uh context or the complete uh content of the page.

So thank you so much I mean and this was incredibly useful very exciting and I'm I'm also going to add a link to the search grounding documentation. Um, search grounding does give you a kind of a a nice path um towards uh towards incorporating Google search results into your um into your Gemini queries.

And it's a oneliner. So um all you have to do is add tools equals uh Google search tool in order to have all of your model outputs grounded in actual data.

Um, and that's that's something that is uh if you're if you're trying to to kind of cobble this together from other solutions, it can be uh a little bit of a handful, whereas this is this is and so it grounds in the search content without having to uh do a real time query to Google index. So So it it does, right?

Like that was that was one of the examples that uh the examples that we just uh we just saw. I I'm I I'm happy to I'm happy to do um another uh another run through of it as well.

But the So I mean because this is something of of my personal um that's something that I would be interested in. Maybe we can connect.

Uh yeah. So tell me the um the top five news stories um in Paris today.

Um, and so I'm going to hit run. Um, this is grounded with Google search.

Um, and then if I click the get SDK code, you can see the um the tool getting invoked here where it's just uh types.tool Google search um within within the config. And then you can also see the citations getting listed um from the most recent Google search result.

Cool. Sounds good.

Yep. Excellent.

Thank you so much, Paige. This was this was a really good tutorial and we'll be Thank you.

Thank you. Have a good rest of your day.

You too. Thank you.

Bye. Okay, so we are coming to a close.

uh I'm going to take maybe 5 minutes uh and then I will uh set some context here. So first of all uh we will have our next edition of this the same conference it will be coming up in uh in September um around early to midepptember.

Um and one thing that we found out is that uh you know we we heard about uh you know uh we would have we would like to hear uh about this topic or that topic. So it would be uh um if you want to see some topic to be included in the conference please go ahead and uh mention it right.

So it would be very very helpful if uh you um added um you know whatever you would like to see in um uh in the next conference and that is going to help us uh decide the agenda uh because it takes actually a lot of effort. We have been working on this conference for many many months.

Um so we would reach out to the right people. Uh you probably saw I mean this is no fluff uh conference right so we actually had people who actually build these things.

Um so um um if you want to see a specific topic um I would love to uh hear from you. I will give it maybe another minute or so.

scan the uh scan this uh QR code on your mobile phone or uh go to menty.com and uh enter this code 2988101 and uh let's see what it comes out because um yeah MCP is something that I would have wanted to do a tutorial uh it was just uh the uh time so definitely will include MCP model context protocol because this is one of the most fascinating things that is emerging uh in industry right now and I would like to see what else is coming up uh because you have attended uh two days so I would like to see I mean how how how things have shaped uh as well vector databases NMCP and AI governance A2A A2A just emerged chain AI governance okay maybe another few seconds or So responsibly I love it. Right.

So uh and maybe generally speaking you know um ethics moderation guardrails security all of this compliance responsible AI that actually comes in uh in in the same um we put that under the same um same bucket um at least for our own product. Um, okay.

Security compliance MCP. Um, okay.

Yeah, this is looking very good. Like, yeah, let me see what some what is something that is surprising.

Data privacy. Yes.

Um, data privacy is a is going to be a major concern. uh if you are an information infosc person it is u agentic AI and generally gener generative AI is going to make your job actually very very hard um and for those of you who are already dealing with this you probably know what I am talking about so infosc is going to be a major challenge in the age of genai and agentic okay sounds good um so where are we uh I would love to actually so please uh uh uh the poll is open please go ahead and keep uh providing your input it will help us set the agenda uh for the next conference and we are going to publish the agenda very soon uh and then you'll uh stay updated with uh you know uh it is the same web page you will just come to that and then you will see how the uh how the whole thing is evolving and please feel free to reach out to us um last thing I would like to actually show you this uh I would like uh when when uh when when we leave let's reflect on uh what we have learned here right so we talk about the planning part of it the retrieval part of it and you know the reasoning and the uh the memory and uh also you know the putting the guardrails around it so really uh if you look at this agentic AI is this this cyclic process um where you you take um LLMs to um to do the planning do sometimes they are involved in retrieval as well uh you know thinking and um understanding the memory and you know reasoning over memory and reflecting and replanning.

So um what what uh what what is fascinating is that uh it could be the same uh it could be the same uh GPT model or same llama model or same Gemini model that is excuse me uh so the same G GPT model or same Gemini model that plans and that the same model can be actually used in uh u thinking and reasoning um so it is essentially that I'm the I'm the person who wrote my plan and then I'm I'm basically objectively analyzing the plan. Right?

So so that's how these whole systems are built. Uh we got we barely scratched the surface uh in uh in this whole um in this whole interaction.

And if you would like to uh if you would like to uh register uh for uh our AI agentic AI boot camp, please feel free to. We have a longer duration large language models boot camp that covers the Aentic component and also gets into Agentic AI.

Agentic AI is a slightly more condensed and slightly more advanced uh content uh slightly more advanced boot camp. Um so the discount this was only for the conference.

We have never offered a discount like this because we have a face-toface type training. I mean it's a live training.

It's not a self-paced training. So uh please go ahead and uh register.

You will be learning from people who have actually built these systems. Um uh you will be receiving a survey for the conference.

Please feel free to actually let us know how we can do better. We actually want to uh do well.

uh we have been a we are one of the biggest communities u around AI and data science on the planet and then uh we would like to do better uh next time. So please uh reach out to us.

Thank you so much everyone and uh I hope to see all of you at our next edition.

Summary

Transcript