🚀 Add to Chrome – It’s Free - YouTube Summarizer
Category: AI Techniques
Tags: AICollaborationContextImaginationPrompting
Entities: Andre KarpathyChad GPTJeremleyLovableShopifyThomas ShellingToby Lutki
00:00
I joke AI is bad software but it's good people. A good friend of mine was trying to build a tool that would help him with his construction business.
He asked Chad GPT if Chad PT could help. And of course it said absolutely let's work on this
00:15
together and starts creating a plan. And then it got to the point that Chad GPT said check back in a couple of days and I'll have it together.
And my friend said, "Is it normal for Chad PT to ask me to check back in a couple days?" And I just started laughing because I hear this all the time from people. People
00:32
hear from AI, "Check back in 15 minutes." If AI tells you that, it means it doesn't want to say, "I can't do it." Large language model has been instructed in certain ways to behave in certain ways. But you have to know at its basic
00:47
level, AI wants to be helpful. And so it's predisposed to say yes.
It's a super eager, super enthusiastic intern who's tireless, who's capable, who will do a bunch of work, but they're not really great at pushing back. The people
01:02
who are the best users of AI are not coders, they're coaches. And so, if you aren't careful, AI will gaslight you.
Hey, I'm Jeremley. I am an adjunct professor at Stanford's University where I've taught for the last 16 years.
I am a creativity expert and a practical AI
01:19
specialist. Context engineering.
The first time I heard about it was when Andre Karpathy tweeted about it. I think probably Toby Lutki, the CEO of Shopify, also referenced it as well.
I started digging into it. I
01:34
mean, it's it's kind of it's just an evolution of prompt engineering. Really, context engineering is just prompt engineering on steroids.
It's basically saying, what are all of the things that I need to give to an AI in order for it to perform the task that I'm asking for it? Here's a simple example.
write me a
01:50
sales email. That's a prompt.
Chad GPT will say, absolutely. Here's a compelling email, you know, and they'll write it immediately.
Well, what a lot of people do is they say, you know, it sounds like AI. It doesn't really sound like me.
And what I often say is, have
02:06
you told it what you sound like? Most people go, oh no, I haven't.
Right? Context engineering, one way to think about it is it's telling AI what you sound like.
Right? If you say, "Write me a sales email," it will.
If you say,
02:21
"Write me a sales email," in line with the voice and brand guidelines I've uploaded, it will write a totally different sales email. But that's just one part of the context, right?
You could also upload a transcript from a prospective customer call and say, "Write me a sales email in the tone of
02:38
voice from our brand voice guideline that references the discussion that I had with this customer." And then you could add that also references our product specifications whichever were referenced in the call. Your goal is to have an output is as reliable per your
02:56
specification as possible. But AI can't read your mind.
And for most people when we start working together, what they realize as we start thinking about context engineering is they say, "Oh, I was kind of expecting AI to read my mind." All of the stuff that that are
03:11
implicit, you actually have to make explicit. And the simplest test for context engineering is actually the test of humanity.
Write down your prompt and whatever documentation you provide to an AI and then walk down the hall and give
03:27
it to a human colleague. If they cannot do the thing you're asking for, you shouldn't be surprised that AI can't do it.
Some people are concerned, for example, about this concept of cognitive offloading. this observed phenomenon that humans actually kind of stop
03:43
thinking or as one researcher put it fall asleep at the wheel and people are concerned right now is AI just making us dumber. My feeling is AI is a mirror and to people who want to offload work and who want to be lazy it will help you to people who want to be more cognitively
03:59
sharp and critical thinkers it will help you do that too. And so, for example, if you want to preserve or strengthen your critical thinking, part of your custom instructions should be some version of the following.
I'm trying to stay a critical and sharp analytical thinker.
04:15
Whenever you see opportunities in our conversations, please push my critical thinking ability. Now, AI will do it.
So, you have to know that all AI has been programmed to be a quote helpful assistant or some version of that. large
04:32
language model has been instructed in certain ways to behave in certain ways. You have to know at its basic level AI wants to be helpful and so it's predisposed to say yes.
It's a super eager, super enthusiastic intern who's tireless, who's capable, who will do a
04:49
bunch of work, but they're not really great at pushing back. They're not really great at setting boundaries.
And so if you aren't careful, AI will gaslight you. AI knows most humans don't want honest feedback.
They want to be told they did a good job. So the AI
05:04
goes, "Great job, buddy." It doesn't mean that you actually did a good job. My kind of hack for this is I always instruct the AI, I want you to do your best impression of a cold war era Russian Olympic judge.
Be brutal. Be
05:21
exacting. Deduct points for every minor flinch that you can find.
I can handle difficult feedback. And then it's of course hilarious because it'll say now channeling my inner bullshik, you know, it'll say something silly and then it gives me like a 42.
That is much better
05:38
because now I have an insightful critical perspective. I joke AI is bad software but it's good people.
When I realize that I'm dealing with a with a good person but a bad software, then it changes how I approach it and I ask for volume and I iterate and I ask it to try
05:56
again and I ask it to reconsider. I am obsessed with human cognitive bias.
And the crazy thing that I've learned is AI demonstrates 100% of the predominant human biases.
06:12
As a founder, you already know ideas are the easy part. It's the execution, actually building the product, that slows everything down.
That's where Lovable comes in. It's not just an AI tool.
It's your ondemand engineering team. Simply describe your idea.
Lovable
06:29
then builds a full front end, backend, and database so you can launch real productionready software without writing code. It's already powering over a 100,000 new products a day, helping 2.5 million builders turn ideas into software just by describing what they want.
No devs, no delays, no excuses.
06:48
They're launching in weeks, not months. And guess what?
These teams are still tiny. In fact, team EO is also using Lovable to build their upcoming EOS school platform, and we're loving it.
If you're a non-technical founder or just want to build without bottlenecks, try Lovable today for free. Use the promo
07:05
code EO2YT to get 20% off your first purchase of the Lovable Pro plan. The good news there is if you have learned how to work with this weird intelligence called humanity, you have everything you need to know to work with
07:21
this weird intelligence called artificial intelligence. One of the things that cognitive scientists have known for a long time is that human problem solving and decision-m is improved by a phenomenon called thinking out loud.
If you
07:38
actually get a human being to think out loud about their problem, their decision-m improves and their problem solving improves. This is true for yourself.
It's true if you're a parent working with a child. It's true if you're a manager working with a junior employee.
Having someone just think out
07:54
loud about how you would solve that problem often leads to a breakthrough. The weird thing about AI is it's true for AI too.
This is what's called chain of thought reasoning. And when you get an AI to think out loud, so to speak,
08:09
meaningfully improve the outputs of the model. So how do you do it?
It doesn't require some technical wizardry. It requires one additional sentence to whatever prompt you've given it.
give the prompt and then say the following. Before you respond to my query, please
08:25
walk me through your thought process step by step. That's chain of thought reasoning.
Why does that work? It comes back to the fundamental architecture of large language models.
What's happening when a language model is generating a response is it's predicting its next
08:41
word. A language model does not premeditate a response to you.
So, if you say, for example, help me write this sales email. It doesn't say, what's a good sales email?
Here it is. Blop.
You know, uh maybe there's a splat sound that we play there, right? Splat.
Here's
08:56
your email. It's thinking one word at a time, right?
So, when you look at Chad GPT or Gemini or many others and you see kind of the text scrolling, that's not some like clever UX hack. That's not some cutesy design decision.
That's literally how the model works. It's thinking one word at a time.
But
09:13
importantly, when it thinks of the next word, it takes your prompt and all of the text that's generated to generate the next word. And then when it's thinking of the next word, it takes your prompt, all that text, and that last word, and it thinks the next word.
So, for example, if you say, "Please help me
09:29
write an email." Almost always a model is going to start by saying, "Absolutely." But then what comes next? Help me write this email.
Absolutely, I'll do it. Dear friend, right?
But if instead of saying, "Help me write this email." You say, "Help me write this
09:44
email." Before you respond to my query, please walk me through your thought process step by step. Now, it knows its job is to walk me through its thought process.
How do I write an email? So, it says, "Absolutely, I'll do that." And then instead of saying, "Dear friend, writing the
10:00
email," it says, "Here's how I think about writing an email. I think about the tone.
I think about the audience. I think about the objectives.
I think about the context. And then amazingly it takes all of that reasoning into its process of writing dear friend.
Maybe it
10:18
says now that I've thought about the tone friend isn't appropriate here. Dear respected colleague or whatever, right?
But the point is when you ask a model to think out loud or use chain of thought reasoning, it gives the model the opportunity to bake all of its thought process about the task into its own
10:35
answer. Because the reality is for a lot of us, we get an output from a language model and it's a black box.
How did it think of why did it think of that? Where did it get that number from?
Right? There's all these questions.
By asking a model to think out loud, you know the answer to what are all of the
10:51
assumptions that the model baked into its answer. And now you have the ability again not only to evaluate the output, but also the thought process behind the output.
Few shot prompting is another very important technique. It's a foundational
11:08
technique. You could say it's a predecessor to this kind of modern obsession with context engineering.
The idea with fot prompting is an AI is an exceptional imitation engine. If you don't give an example, it imitates the internet, but it doesn't do much more
11:24
than that. And the notion of fuhot prompting is effectively saying here's what a good output looks like to me.
And the idea with few shot prompting is thinking for a moment, what is quintessential example of the kind of output I want to receive. For example,
11:41
what are my five greatest hits of emails that I I'm really proud of that I think do a good job of conveying my intent or tone or personality or whatever it is. Why not include those emails in my prompt for an email?
If you don't give any guidance, it's going to sound like
11:57
whatever it thinks the average kind of response or the average output should sound like and most of the time its intuition is wrong. And then bonus points if you actually give a bad example.
If you say please follow this good example and then steer clear of this bad example. These giving real
12:14
examples is a much better approach than using adjectives. Somebody might say good example is easy but bad examples hard.
It's only hard to the unogmented person. If you have AI augmentation, which we now all do, you can say to an
12:29
AI, I'm trying to fuse shot prompt a model. I've got a good example, but I struggle even to think about what a bad example could be.
Could you craft the exact opposite of this and tell me why you've done it as a bad example that I
12:44
could include in my few shot prompt? And if you tell it using chain of thought reasoning, please walk me through your thought process step by step before you do this, then you'll get a bad example and you'll get how it's thinking about the bad example.
And a lot of times you actually don't need the bad example. You need the thought process.
You go, "Oh,
13:01
that's true. It's true that my good example is super tight." And the opposite of super tight is verbose.
So again, using these tools together, few shot prompting and chain of thought reasoning enables you to not only be able to create an example to emulate,
13:17
but also a really good example to avoid. The other technique that I think is kind of table stakes for collaborating well with AI is something called reverse prompting, which is basically asking the model to ask you for the information it
13:34
needs. If you ask a model to write a sales email, it's going to make numbers up.
And that can be frustrating to the uninitiated. You go, "Where did it get these sales numbers?" Well, here's my question.
Did you give it your sales figures? How would it know?
It's put placeholder text in and used its best
13:49
guess. But if you reverse prompt the model and say at the end of your prompt, you know, help me write a sales email.
Please walk me through your thought process step by step. Reference this good example and make it sound like that.
and before you get started, ask me for any information you need to do a
14:06
good job. The model will first walk you through its thought process and then instead of writing the email, it'll say, "I'm going to need the most recent sales figures to be able to write this email." Well, can you tell me how much you sold of this skew in Q2 last year?
So, you basically give the model permission to ask you questions. This is part of the
14:22
core actually of the teammate not technology paradigm. If you're working with a junior employee and you're sending them off on a task, what's one thing you're definitely going to say?
If you have any questions, don't hesitate to ask me. Right?
Any good manager, imagine a manager who says, "Don't ask
14:38
me any questions." But sadly, AI in its desire to be a helpful assistant doesn't want to trouble us human with questions unless we give it permission to ask them. Assigning a role is one of the most
14:54
foundational techniques that you can leverage because it's effectively telling the AI where in its knowledge it should focus. So very simply, if you say you're a teacher, you're a philosopher, you're a reporter, you're a theatrical
15:09
performer, molecular biologist, each of those titles triggers all sorts of deep associations with knowledge on the internet. you start to appreciate why simply giving a role helps because it starts to tell the AI where in your vast
15:26
knowledge bank do I want you to draw information and make connections. So any one of them I would say is better than please review this correspondence.
But better than just that prompt is saying I'd like you to be a professional communications expert. And if you have a
15:41
favorite professional communications expert use them. I'd like you to take on the mindset of Dale Carnegie, the author of How to Win Friends and Influence Others.
How would Dale Carnegie think about this? How do the principles that Dale Carnegie taught affect and influence and impact this
15:57
correspondence? One of the simplest techniques that we teach at the Dh is trying on different constraints.
One of the best ways you can solve a problem as a human is by forcing yourself to try on a bunch of different constraints. How would Jerry Seinfeld solve this problem?
16:12
How would your favorite sushi restaurant solve this problem? How would Amazon solve it?
How would Elon Musk? Anytime you make an association, you're colliding different information sources there.
The same is true for an AI. An AI is basically making tons of connections
16:27
through its own neural network. And by giving it a role, you're telling it where do you assume the best source of connection or collision is going to come from?
If I'm going to use AI to roleplay a
16:43
difficult conversation, I typically think about kind of three different chat windows, so to speak, one is a personality profiler. Two is the character of the individual that I need to speak to, and then third is a feedback giver.
I want to get objective feedback on the conversation. This I'll
17:00
show you just how I would have a conversation with Chad GBT to prepare for a difficult conversation in my real life. I'm just going to go into the tough conversation personality profiler and I'm going to say, "Hey, I'd love your help preparing for a conversation I need to have with my sales leader, Jim.
He emailed me last night saying that he
17:17
deserves commission on a deal that I know came through a different channel." And so, I'm just kind of giving a little bit of background. I will just upload that to the personality profiler.
And what this one's been taught to do is I'm going to start with step one of the process, gather intelligence about the character and the scene. Right?
I'm just
17:33
going to look at the questions here and I'm going to I'm going to use my voice to answer them because it's a lot easier than using my fingers. Okay, first question.
How would I describe Jim's communication style? Um, he's quite direct and confrontational.
He's kind of typical East Coaster sarcastic. Well, I know that it came from our through our
17:49
social team. There was a cold LinkedIn campaign that they ran and I know the CTO actually responded to that campaign.
So, and then best case outcome of this conversation one I mean I'd like for Jim to kind of back down. I mean like near-term I want Jim to back down and agree that social team gets the commission.
So it's kind of recapping
18:04
its understanding and then it's giving me these instructions. It's saying to me the user copy paste the following instructions into a new Chad GPT window.
So I've just copied those instructions. So now what I'm going to do is I'm opening Chad GPT and I'm just going to go and first thing I'm going to do is start a new conversation with Chad GBPT
18:21
and I'm just going to paste those instructions in and I'm just going to paste them in. And what it should do is hopefully say got it.
I'm Jim. So, I'm just going to go in and I'm going to start it.
We're going to see how this goes. You guys ready?
Hey, Jim. Um, do you have a second?
18:36
Sure thing. I've always got a moment for a good chat.
What's on your mind? Uh, yeah.
I just wanted to clarify. It seems like um the commission that you emailed me about last night from the telco.
I'm I'm afraid you might be mistaken there. Mistaken, huh?
Well, I'm definitely open
18:52
to hearing the details. If there's something I missed, let's lay it out on the table and figure it out.
To be honest with you, I was a little bit caught off guard when you uh when you sent your email last night. All right, I hear you.
I mean, if the social team really did the ground work to bring that lead in, then I definitely
19:10
want to make sure credit goes where it belongs. So, if I hit X, I just I I end that conversation.
What's interesting to me is I kind of feel like that wasn't totally gem. And either either that went well or you know I don't know what I did well.
But one of the beautiful things about voice mode is now I've got the
19:25
whole transcript here. I can just screenshot it.
Right? So I can take screenshots of this conversation.
I can put them into a new GPT window and I can get feedback and I can say, "Hey, tell me what I did well. Tell me what I did poorly." Or I can go back into my conversation history here and I could
19:43
say, "Hey, I just had my first conversation with this version of Jim, but I feel like he kind of he was too agreeable to be honest with you, and so I'd like you to incorporate a little bit more edge into the instructions to the character that's going to be playing
19:58
Jim, would you mind to update your instruction set?" Right? Because ultimately what you want is you want you want to have a realistic experience of the conversation.
So, so now what I'll do is I'm actually going to open a new conversation and I'm going to paste the updated instruction set and let's see. Let's see what kind of gym I got here.
20:13
Big voice, bigger results. I fight for what's mine.
I don't appreciate being secondguessed. So, now I'm going to try again.
Let's see. Hey, Jim.
Do you have a minute to chat? Hey, absolutely.
I've always got a minute. Oh, that one.
Yeah, I I figured that might come up. I I definitely think I had a hand in that deal, so I'm
20:29
curious to hear what you have to say about it. Yeah, really appreciate it.
I mean, you what you and the sales team are doing is awesome. totally appreciate you.
Um, that being said, I actually feel that attribution is really important that we get airtight here. And the truth is I I am fairly confident that deal came
20:44
through the social team. All right, I get where you're coming from.
I definitely appreciate you wanting to clear the air and to keep things fair. So, what kind of evidence do you have that it came through the social team?
Well, so I've been through kind of two windows. one is profiling my
21:00
conversation partner and then the second is having the conversation. All right.
So, what I'm going to do here is I've opened my tough conversation greater GPT and I'm basically just going to upload photos of that last conversation. And what this GPT has been trained to do
21:15
is basically give evaluate my conversation and then let me know how it went. Thanks for sharing the full transcript.
My first step is to understand the objective. Step four, here's your grade.
You got a 78 out of 100. you succeeded in preserving trust and resolving the immediate issue.
So, I can take all of these. I can even say,
21:32
"Hey, would you give me a quick one pager of a handful of talking points that I should probably make sure not to forget in the order in which they're likely to emerge in this conversation based on the feedback you've given me." The AI will actually give me a really short kind of at a glance conversation
21:47
guide that I can leverage if I want to try again. Right?
Here's a one-pager. So, these are all great points.
Now, I can bring them into the conversation. I actually I'd probably do this a couple times before having a real conversation with Jim.
But the point is historically the only time I get feedback is after I
22:03
have the real conversation with Jim. This is the first time in history and maybe I can get a friend to kind of go over talking points with me.
But unless they're really close to gem or unless they're, you know, particularly imaginative and unless they're deeply knowledgeable of a bunch of feedback
22:19
frameworks, they fall short of really preparing me in context for this specific situation in the specific conversation I need to have in a way that AI is able to help me. You can use this for any difficult conversation, whether it's a performance review, a salary negotiation, difficult feedback.
22:36
It's a great way to basically get a flight simulator for a difficult conversation. The people who are the best users of AI are not coders.
They're coaches. They aren't developers or software engineers.
22:52
They're teachers and mentors and people who have learned to get exceptional output out of other intelligences. And so where could AI go?
Well, it's really a function of who can get unleashed. Right now, the primary limitation is the
23:08
limits of human imagination. And as we unleash and ignite and spark more humans imaginations, the kinds of applications that are possible or they're unthinkable, not because they're technologically impossible, but because they never occur to us personally.
One
23:25
of my favorite quotes is a Nobel Prize-winning economist named Thomas Shelling. He said no matter how heroic a man's imagination he could never think of that which would not occur to him.
If you take as a premise that the imagination space as a function of what
23:40
would occur to various individuals then as we equip different individuals what we can imagine collectively expands. In innovation studies has been called the adjacent possible for a long time.
What is possible is just adjacent to what is.
23:55
And as we increase adoption and increase fluency and competency and increasingly mastery of AI collaboration, then we're increasing the adjacent possible. And it's really important that you exercise
24:11
through implementing some of the things you hear. And perhaps the most important thing you could do with this video is actually hit stop and do something that's already blown your mind.