ChatGPT – The Super Assistant Era | BG2 Guest Interview

00:00:00Chat GPT originally was entirely freeand the reason for that was that it wasintended to be a demo and we were goingto wind it down after a month.>> We then realized that the demo wentviral and people loved the demo and itwas actually a product and but werealized to be a product you can't takethe product down every time you're atcapacity. So we you ship subscriptionssimply because it could shape thedemand. It was a way of gracefullyturning users away when we had to turnaway someone. You know, you guys are at900 million VT active users now, andthat growth has been incredible. Thenext billion users, where they where arethey going to come from?>> We've got about 10% of the world comingto us now. 90% left to go, right?There's so much more opportunity.

00:00:48>> Well, Nick, so excited to have you here.>> Thank you for having me, Peru. You'vehad uh quite the journey from Germany tothe US for Brown.>> That's true.>> Most recently at Instacart delivering uhgroceries in 30 minutes to nowdelivering AGI to billions. I'm surethat was the plan all along.>> Yeah, clearly total master plan.>> Well, well, tell us about your journey.What how did you get to OpenAI? I knowit's a fun fun story. And your yourthree and a half years or so at OpenI,how have they gone? The only throughlight in how any sort of employmentdecision has been entirely people based.So I don't claim any credit for for uhjoining OpenAI um or predicting chat GPTor anything like it. But um I uh um hitup someone who I admire a lot who I gotgot to know at Dropbox. um Joanne who uhworked worked here at the time and sheuh I asked her to get off the Dolly 2wait list and she told me I had aninterview if I wanted to get off thewait list. So um I took the bait and gottotally nerd sniped in the process andand here I am.>> There you go. The Dolly 2 weight listwill get you.>> It's a great recruiting tool.>> Nice, nice, nice.>> We should do more weight lists probably.>> Yeah. Yeah. Yeah. Well, you know, thebig uh the big

00:02:04super cycle we're in is is is Chad GPT.Now, I assume over a billion users onthe monthly side, 900 million weeklyactive users that recently reported upfrom zero three and a half years ago. Uhyou could have, if I imagine what thedashboard of Nick Turley looks like, itcould have users, it could have payingsubscribers, it could have daily activeusers, it could have retention,engagement. I mean, there's like 15things, maybe all of them. What what isyour northstar? How do you how do you opwhat are you optimizing for? Uh what isNick looking at in his daily dashboard?>> It's it's funny, right? Because it'ssuch a young product. It's been to yourpoint three and a half years. And umthis kind of question, it kind ofchanges as as as you evolve and you andyou grow up and you ask yourself, youknow, what are we really building here?And uh to this day, right, then want tobuild a super assistant that canactually help people achieve theirgoals.>> And ultimately the thing we care aboutis like is our product doing that? Is itactually helping you do the thing thatyou're, you know, um coming to theproduct to do? And it's so different fordifferent people, right? Some people umare are trying to get healthy, otherpeople are trying to start a company,>> um um learn a new topic, um>> do their taxes. There's all thesedifferent things that you

00:03:19might bedoing. Um, and the true measure ofsuccess is whether or not that we'rehelping you do that. And um, obviouslywe look at WAU in particular because,you know, we want to know if you'recoming back to the product. We look atretention. Um, but we, you know, we welook at, um, all kinds of stuff inaggregate because really there isn'tlike this one single thing that you canoptimize for.>> If you were to allocate a 100 units ofpoints>> to these metrics, which metric can youlike distribute the 100 units acrossthese metrics in order of importance foryou right this second? It's a goodquestion. I I care a lot about long-termretention and I would put all my pointsthere. Um because um I'm really proud ofthe retention stats we have.>> Huge,>> but ultimately the sign of durablevalues, whether or not people are comingback in three months because that meansyou're really solving their problems.And>> yeah,>> I think things like revenue, they followfrom that. Um>> yeah.>> Um versus like, you know, trying to goon those things directly. And we'vewe've had a lot of success making veryprincipled decisions u on this stuff.Like one good example is you GPD4 usedto be behind a pay pay wall because wecouldn't serve it to everyone and thenwe had GPD4 which was a totalbreakthrough>> in um in in our ability to inference itand um>> so we just gave it away for free andthat ended up being totally revenuepositive and retention

00:04:34positive becauseit just provided access to the tech andI think when you make your decisionsthat way and you focus on the customer>> you end up with a great product andrevenue obviously follows too.>> Phenomenal.>> Yeah. Well, it uh it shows up in thenumbers. You know, I I posted this chartuh yesterday on the data that we have,you know, from a third party. Theretention curves for chat GPD aresmiling. Look at that. Just like that.And that is a rare that is a very rareoccurrence, you know, as as we know. Andum why why do you think like if you wereto give us a narrative on that smilecurve? What is the why do these smilecurves exist? What are you seeing inChad GPD that has people who have whohave maybe turned off for a couple ofweeks or months coming back and why arethey coming back?>> Look, there isn't one single thing. Youknow, the way you build a retentiveproduct is lots and lots of littlethings and really trying to make itbetter systematically. I will say thatyou know with AI and in particular chatGBT I found that it takes people sometime to really understand all the partsof their life they can delegate rightand I think many users for that it's amulti-month process for them tounderstand how can this thing help meand what are all the different ways>> that um I could plug chatbt into my life>> um and

00:05:49um but you know when I thinkabout some of the breakthroughs andlevers we've had um things like searchand personalization they they havehelped solve those user problems becauseum you know search provides way moredaily value to you.>> Um it used to be that chat was a prettyworky product. You know we'd see usagego down on the weekend. We usage youknow go down during the summer monthswhen a lot of people were off from work>> and um>> today you know we're we're mobile first.The vast majority is mobile and we seeall these personal use cases and I thinksearch was a big investment that got usthere>> and personalization makes chat so muchmore relevant for you right because itgets to know you over time. you get toknow it. Um and um those are two thingsthat have materially moved um the waythat you know people come back to theproduct. But um there's lots more to do.>> Yeah.>> Um and you know, as mentioned, I don'tI'm not resting on our um retentionstats even though, um we're obviouslyvery proud.>> Nice, nice, nice. Um and you know, theother thing that I I got wrong aboutChad GPT was this is two two and a halfyears ago. I was like, well, you know,let's look at who's going to win thisconsumer AI race. You typically theseconsumer markets are winner or take mostwinner or take all. Look at search.Google has near 90% plus market share.Three and a half, four trillion marketcap.

00:07:04>> Mobile, same thing with Apple. Social,same thing with Meta. I was like, well,AI, Meta has all the distribution.Google's got all the distribution.They've got 34 billion users. Well, itwould be a flick of a switch for them toroll out their AI. But I was wrong.That's not what happened. Chad GPT turnsout uh you know you guys are at 900million VT active users now and um thatgrowth has been incredible clearlydistribution was not enough right>> so the same question for distributionwhat are the levers for us that havegotten us to the scale is it modelquality is it you know product qualityis it you know features is it theexperience or product improvements likememory and personalization or searchlike>> same question like what would you saydrove historical>> growth and and successWe've got about 10% of the world comingto us now. 90% left to go, right? Sothere's so much more opportunity to toto re reach more people and introducethem to the way that AI can can can>> benefit them, right? But when I when Ilook backwards and I only say thatbecause like the next billion usersmight be very different in terms of likehow you um engage and reach and providevalue,>> but when I look backward, it's beenroughly sort of>> oneird one third one third betweensort of

00:08:19classic friction removal type ofwork. Like one of our biggest umum moments when you look at pure impactum was you know removing theauthentication wall>> and Sam will say I told you so because Ithink that was his feedback from likeday one>> u like can't you shouldn't have to loginto the chatbt but it's like stuff likethat that you do for any product and itdoes matter some things never changeright>> um and but you know and then you knowanother third or so isort>> you know are what I would sort of sortcore product investments and they'rereally typically things that we've donetogether between research and product.So>> search and personalization are reallygood examples of that where we cametogether and we figured out not justUIUX evolution but also how to postrainum these changes into the model and itwas really the moments when we cametogether. Um um another recent exampleis like we have these writing blocksthat render when you you know ask aboutqueries um where you're um you knowtrying to write with the model>> and like putting really good craft intothose experiences really matters and umum you our users love it and then>> another third of of the growth has beenjust model improvements like step bothstep changes like going from um GBD 3.5

00:09:35back then to GPD4 then going from GPD4behind a payw wall to O suddenlyavailable to everyone, right?>> But a lot of it is also the iterationthat isn't splashy, that doesn't warrantlike, you know, a named release. You weum I'm really excited about the updateswe just made um with 5.3, 5.4, etc.>> Because um that is when we like take alot of user feedback and we you know,methodically address it and obviouslythat shows up um>> in our retention as well. So sort of onethird one third one third betweenclassic um friction removal um andaccess>> um core product investments and thenpure model improvements.>> And so the question that I've reallybeen waiting to ask you is how do we getthe next billion>> and and you know talk about that alittle bit. There's a lot of it it seemslike at least from the outside fog ofwar. If I was a consumer today in themarket to pick my super assistant, umyou would have a couple of greatoptions, you know, uh plot out there,they're having some great traction lastcouple of weeks.>> Um Gemini mega distribution, Uberdistribution, um and us by the by the bythe leading product today, at least inuser numbers, the next billion users,where they where they going to comefrom?

00:10:50First of all, just tocontextualize that goal. You we careaboutum two things at the end of the day.Obviously, reaching more people isreally important. It's the directmanifestation of our mission um to theworld where you the more people we canintroduce the benefits of AI uh thebetter that is. Um but we're also reallyexcited to go deeper. Um and that meanstaking the same billion users that findvalue in chat GPD today. Um and actuallyproviding more meaningful value in theirworld like actually helping them achievetheir goals. not just answeringquestions, right?>> Um, so I'll talk about how we get tomore scale, but I think it's importantto remember that, you know, the way thistechnology is evolving is, you know,we're going to go beyond pure chat botsum pretty fast.>> Exciting. Um I think on on scale youknow it's shocked me how many peoplehave found value in chatbt as it workstoday because I don't think delegationis a natural skill for most>> and chatbt is a prettyit's a it's it's a power tool right youcome to it doesn't tell you what it'sfor you kind of have to discover it onyour own and you have to use it and thenyou'll learn about this prompt that wasreally cool and then maybe you you're onTwitter and you learn about another oneor you're on Instagram and you learnanother But the product it's it's like araw appliance. And I think

00:12:05to you knowon one thing we really need to nail aswe you know reach the next set of users>> is a product that has a bit more of anaffordance.>> U because I think for most peoplethey're very very busy.>> Um and they everyone I think in theworld has intelligence constrainedproblems like problems that moreintelligence could help with>> but you need to frame that to people.>> Yeah. Yeah.>> And I still feel like we're a little bittoo much like a computer terminal and itneeds to feel more like software like orum you know an operating system ofsoftware,>> right?>> Um so that's one thing. Um another thingum that gets at the same constraint isis beginning to be proactive.>> Um in a world where a lot of folks are,you know, too busy to delegate theirproblems to AI or don't quite know whereto start, I think being able to help youproactively is really really importantas well.Um, but you know, I think all these areare product evolutions that we couldmake on on top of the current tech. Andthe thing that gets me particularlyexcited is productizing our nextgeneration tech or reasoning modelsbecause the truth is when you look atreasoning and chatb today.>> It's relevant to a very small group ofpeople. It's relevant for the people whoare trying to get the most out of chatGPT.>> But I fundamentally believe thatreasoning it's transformative. And ifyou can figure out how to

00:13:20productizereasoning in a way that works onpeople's behalf without them evenknowing>> um and that looks very much like youknow>> the model doing long horizon tasks onyour behalf. It doesn't mean youencounter the concept. It just meansit's benefiting you right.>> Um so there's so much work to do and andyou know>> the the the product certainly has toevolve to to be relevant for for forthis kind of scale.>> Yeah. One of the one of the things thatI've been hoping for a while and youknow Brad made a bet two years ago umwhen can Chad GBD help me take actions.>> Can Chad Guby help me be more proactive>> and um I think his bet expired end oflast year. So he's he's we're verycurious.>> When is that coming? And I'll frame thatfor you because you know with with withwith search engines in Google you knowtwo decades ago you'd have gotten the 10blue links. You could have spent an hourgetting the answer. you can now get theanswer instantly with chat GPT.[clears throat]>> Uh and it feels like the next step is isactions>> 100%.>> And it feels like the next step is youknow with you know pulse is a is a greatproactive product.>> Um that that that feels you know it's Ihave a pulse that runs weekly>> but what I would really like is like heyNick spoke about something and just justfind me make sure I know that Nick spokeabout this

00:14:35or or like hey this XYZ thinghappened that I cared about a lot.>> Um when is when is one of that going toget proactive? um what is the modalitygoing to look like?>> Yeah. Yeah. So there's there's twoconcepts I think. Um there there's chatshould be doing stuff rather than justanswering>> and then there's chat should be beingproactive and I think when you put themtogether you start feeling like it feelslike a super assistant. Um because Ithink these things compound>> um>> on the action taking piece you strictlyspeaking in chatcan do stuff today. Um the action spaceis just very limited, right? It cansearch the web, which means it can useyou search tool or browser in the sameway that a human would.>> Um it can make images. It can do all theall these things, right? But it doesn'thave clearly doesn't have the sameaction space that a human with acomputer would have. And that is what weaim to build. Um>> and if you timing is everything on thesebets, right? And I don't pretend to begreat at timing either. U you look atpast attempts that we've made like thechatbt agent for example,>> which kind of has capabilities likethis. um it was just slightly too early.The models weren't quite good enough tohit real escape velocity. And theproblem is if you don't have escapevelocity is that users don't learn totrust it. They don't even try.>> So when you look at a lot of thingspeople were doing in the originalversion of chatb

00:15:51agent, it was thethings that happened to work likemigrating your uh uh file server intothe cloud or something like that. Usefulstuff but very niche.>> Yeah. Um and um as this stuff getsbetter um we just have to get it to apoint where people try to use it forreal meaningful problems in their lifebecause then we can start hill climbingand this has been the magic of chatbtwhere chatbt upon launch was good enoughto get real attempts at use cases evenif they didn't initially work like chatwas a pretty bad writer originally itwas a bad software engineer but peopletried and got enough value out of itthat we could take those use cases andmake them great>> and I do think we're about to get tothat point with general purpose agentswhere>> it works well enough that you get atleast partial credit>> and because you're getting partialcredit you get really good tasks backand then the magic begins because um youknow once you have>> a a set of use cases that you can umclimb the hill on we can make themawesome. So on task I think we're closebut I think even people inside of openwould have had a hard time predictingexactly when this gets good. Um we'vebeen excited about it for a while>> on um proactivity.>> Um Pulse was a really great first stepbecause

00:17:06>> what we wanted to build was a formfactor where you're not prompting themodel like the model's prompting you.>> Um for the reasons that I describedearlier, which is, you know, it's sohard for people to delegate and tofigure out what their problems are. Whatif the eye understood your your goalsand the things you're interested in andjust could start being proactive on yourbehalf?um pulse is limited in the value it canprovide for you because it's notconnected to your life and it can't takeaction. So, it's producing informationfor you>> and people love that. You know, I lovethat. I've got mine running, too.>> But I think the magic begins when youhave actions and proactivity becausethen it can begin speculativelyactually, you know, um detecting, hey,you just landed where you were supposedto go. Um you know, I'm I'm I'm going tocall a cab for you. or>> um you know if you're at work it's likehey I proactively ran this analysisbecause I saw your metrics dropped>> um so I think these things reallycompound and we need to nail multiple ofthe building blocks to really achievethe transformation um and the formfactor that we hope for>> as you were answering those questions Inow have uh 15 more questions for you[laughter] so so so I hope you have 15more minutes but um>> okay one by one we'll start with whatyou said on actions and tasks>> got it on timing tough to Okay. But isthere

00:18:21a shape or ordinality of tasks andor or agents that that you think, hey,this is the kind of thing that's likelyto come first whenever it does?>> I mean, the thing that's already comefirst is the uh domain specific agents,right? If you look at what's happeningin in code,>> we're we're we're fully there.>> You know, it's it's mindbending, butwe've got so many engineers um who whodon't open their IDE like ever. Andright>> for me as someone who you know used tocode and then unfortunately got veryvery busy it's brought me back in thegame.>> So um codeex and you know products likeit is clearly a product that has escapevelocity where people are absolutelyusing it for all kinds of agentic workand if you just take what people aredoing>> and make it work even better>> you kind of get all the way there. Um,you know, I won't be surprised if yousee this happen for other forms of sortof quantitative knowledge work justbecause it happens to have theproperties that code has. It's testable.You know, if it worked or not,>> um, it's, you know, very RL friendly.>> Um,>> but um, uh, the the domain specific onesalready work. I think the thingeveryone's working for is, you know,general purpose agents that just>> kind of work for anything.>> Yeah. And I

00:19:36you know that's why I thinkyou need to to win a consumer becauseit's very hard to train people into likeokay>> um it can work like deep research was aconsumer product and it's really was ourfirst agentic>> uh thing out there>> um but I think what consumers want is Ican just ask it anything and we'll dowhat>> um what needs to be done without anysort of retraining and um>> we'll get there just a matter of time>> at least a psychological goal is uhflight bookings bookings restaurantbookings, shopping,>> all this stuff. Um there are so manyconsumer problems and those are just thetype of things that you would kick off,right?>> Yeah.>> The minute you have productivity,there's there's things you don't eventhink of as agentic tasks.>> Um like you're trying to get in shape.You don't think of that as a task youwould delegate.>> Um unless you have a trainer, in whichcase you do, but most people don't,right? But if the EI knew that, it couldtotally start working in the backgroundfor you over very long periods of timeand and getting you, you know, um,here's your fitness plan. Okay, Iactually signed you up for this thing.You could imagine it being quite helpfulif it's aligned with your with yourlong-term interest.>> You're going to give Ozanic a run forthe money. [laughter]>> We got to be careful what businesses weget into, but uh, hopefully we can help.>> That'll be great. Cannot wait. Cannotwait. Uh the second thing you said

00:20:51wasum you know pro proactive users and thatmight require us to go beyond chat bots.>> What's an example of a modality thatmight take chat GPD beyond a chatbot?>> So chat will always be close to myheart. It's the way we grew up. Um, andit's an important modality to stay likeI I I think>> it's less about chat and more aboutnatural language to me where you knowthe fact that you can express yourselfto the machine in ways that are verynatural to you>> whether or not that's text whether ornot that's voice whether or not that isyou know um structured UI that isrendered by the model>> that is just very very powerful andthat's here to stay but>> u I think the thing>> SA server>> that's right uh uh for those that don'tknow that's the name of our codebase umshort for super assistant server Umbecause you know it's proof that thiswas always the vision and is always thevision. Um but uh you know the the thething that that'll change I think isthat chat is a great way of expressingyour intent. It's a good way ofcommunicating with the machine but it'snot a great output. Um where in manycases what you want back is an artifactlike here's your your your you know planfor your trip. Here is the analysis.>> Here is um you know an outcome that Idelivered for you. you know, I just madeyou five bucks.>> Like,

00:22:07this is what I want my AI doingfor me, right? Yeah, totally. I mean,this is what people care about, right?And and and and I think chat will alwaysbe there as the way that you sort ofdisambiguate your intent and you kickoff the task,>> but I don't think it's necessarily thethe the final deliverable. And I thinkthat's that's that's the way in which wecan evolve. So hopefully that's a verygraceful transition because I'm verylucky and it's hard earned,>> you know, to to have a billion peoplecoming to you weekly for a thing thatthey love.>> Yeah. But I think it's a great jumpingoff point because we have so muchunsatisfied intent from people wherethey're trying clearly trying to dosomething and chat is helpful enough>> but it could be so much more helpful. Umand I think that's where we evolve.>> Yeah. And you must be sitting on so muchof this data where people are showing upto chat and attempting as you said uhthree years ago they were at leastmaking the attempt.>> Yeah.>> So you might have at least the frequencyhistogram of like hey here are all thethings that people want to achieve withus. We do uh we have like really awesomeyou know um classifiers that runautomatically. It's fully privacypreserving but gives us a sense of youknow what use cases people have>> and it's important right because whenyou make a new model you make a modelupdate you want to know what use casesjust got better what use cases gotworse.>> Yeah.>> Um and that's not always always trivialto figure out unless you have reallygood

00:23:22analytics on the system. But somuch of my learning is actuallyqualitative where I will just you know Ihave a habit of reaching out to a fairlyrandom set of users to just figure outwhat they're doing>> and I've never worked on a product wherethree and a half years later you'restill learning every time becauseusually by that time you know what theuse cases are that your product can youknow deliver on>> but our tech is so unusual in the factthat I keep learning about somethingcrazy I didn't know was possible.>> Wow that's awesome. But basically abillion users, I suspect a smallfraction of them are power users>> who are uh getting maybe thousands maybemaybe maybe tens of thousands of of ofvalue on on their $200 subscription.>> Yeah.>> Um the vast majority is is you knowmiddle of the pack and then and and afew call it casual users who are youknow start using Chad GPD as searchmaybe or or or teach me about AI or orhelp me with my homework. Um what isyour focus like maybe in thoseconstituents power users, casual usersand and early users or however you frameit, what is our focus on for each ofthose three factions?>> Yeah. Yeah. Well, first of all, I feelaccountable to our entire user base. Infact, our non-users too because youproducts like like Chachet can have realexternalities on on

00:24:37all humans.>> Yeah. But when I think about sort of theway we build,it's really useful to imagine theextremes. Um, one extreme being a userwho doesn't care about AI at all.>> Um, who has a busy life. Um, and um, um,needs to be convinced of, you know, thevalue that we can provide because thatforces you to really nail the interfaceand to expose the capabilities uh, thatare hidden in the model in a way thatpeople can actually gro. And then theother useful extreme is you know um ourour power user base because power usersare the users who teach us what'spossible. Um>> um it's actually impossible for us to doall the product discoveryum on our own>> simply by because of how empirical>> this technology is um and how much youactually learn post launch.>> So building you know um for each ofthose extremes can be valuable. Uh butour user base is incredibly diverse andum people have so many different usecases and this is why you know I like tolook look at all kinds of differentsegmentations not just frequency butalso you know what use cases are youcoming to us for. Um>> but um definitely huge variety in the HPuser base.

00:25:52>> Yeah>> I look up to Mac OS for example as anexample where it really works for peoplewho don't understand technology at all.it's entirely magical but if you are apower user you've got terminal you gotsettings you configure almost anythingin Mac OS>> and it's really beautifully done wherethe complexity is progressivelydisclosed>> so um you can interact with it and lovelove the simplicity of it all but youcan also got all the knobs anddevelopers love it right>> and so I think this is kind of theinspiration for how we want to be inchatbt that doesn't mean we always liveup to it but um it means that buildingfor powers is extremely important>> and you know that's not just a propertythat I'm think is sort of aestheticallyexciting. It's also really important inAI because it's the power users who showyou what's possible.>> They are actually doing the productdiscovery>> because it would be impossible for uswith such an empirical tech to do allthe product discovery on our own.>> So the type of user who subscribes toChad GPD Pro who used codeex before itquite worked>> who is now the strongest advocate of ofcool tools tokens and teaching us what'swhat's possible. um that is anincredible valuable incredibly valuablemember of the community>> and it might not show up in your youknow um weekly active users as just onenumber right um but

00:27:07this is exactly whythere isn't like a single north star>> and you really need to need need to takethese different segments very seriouslyso um I love building for power users>> um>> and uh you know you you asked on youknow token consumption etc it's sofascinating to see there's people whoget incredible value um out of theseproducts and u watching what they do isvery informative.>> Okay. So, so we're very focused on theentire user base. Um learn a lot fromthe power users. Um>> you know the other thing I might say isthe power users right now are getting um>> a lot of value almost too much value>> and a lot of>> no such thing.>> No such thing. the the the analog thatis most common is the Uber and Lyft ofthe 2015 era,>> right?>> And you know, it took it took a while,but I know you were thinking about it alot. I know you guys are thinking aboutpricing quite a bit.>> Yeah.>> Um maybe tell us a little bit aboutpricing. Um>> you know, right now pricing is prettysimple. uh is there a is there a pathfor for for folks who are getting a lotof great value to price that productdifferently and and and you know meetthem where they are and and and and andthe other way on the other side>> I mean pricing is there's no world inwhich pricing doesn't significantlyevolve when

00:28:22>> the technology is changing this quicklyright chat GPT originally was entirelyfree and the reason for that was that itwas intended to be a demo>> and we were going to wind it down aftera month>> we then realized that the demo wentviral And people loved the demo and itwas actually a product and but werealized to be a product you can't takethe product down every time you're atcapacity. So we you know shipsubscriptions simply because it couldshape the demand. It>> was a way of gracefully turning usersaway when we had to turn away someoneand it felt like the fairest and mostequitable way of doing so is saying heyyou know if you really need this productpay a subscription fee and you got it.Then we figured out how to make theproduct stable and we had the choice ofdo we keep the subscription thing or dowe go back to free and we realized wehad consistently more tech that wecouldn't scale. GPD4 being the firstexample because we had way too many freeusers to serve GPD4 and we put it behindthe plus plan. And so, you know, the waywe stumbled into subscriptions was wassort of accidental by trying to justsolve for the user. Um, and it felt likethe right way at the time um to toprovide maximal access to our to to toour tech. Um,since then we've had so many otherbreakthroughs including test timecompute where you can scale up>> um um

00:29:37intelligence>> um um kind of um as much as you want.Yeah,>> more or less. And um you know, it tookus, you know, in the entire industry alittle bit of time to turn that intoproduct value, but we're here now where,you know, our our our ourpower users want to use more and moreand more intelligence. And you it'spossible that, you know, in in thecurrent era having unlimited plan islike having unlimited electricity plan.You know, it just doesn't make sensebecause like, you know, people may needa lot a lot of electricity and they'regetting a lot of value out of that.There's a reason you can't buy that,right? So, um, obviously I want to bereally thoughtful about the way that weevolve our our plans and SKs andsubscriptions, but you would beincredibly sub, you know, surprised ifit didn't change given the magnitude andprofoundness of of the technicalbreakthroughs that we've had and theproduct breakthroughs that follow.>> Yeah. Um, and you know, relatedly, so soI I imagine you're going to havesomething for the power users.>> Mhm.>> Um, what about the other side? How do weget um the casual users into the intothe wheel and and still uh monetizethem?>> As mentioned, you know, our our businessmodel is will evolve and the northstaris access, right? We we we would like topick

00:30:53a um way of of of of you providingan off we want to pro provide anoffering that maximizes um the number ofpeople can who can access our mostpowerful tools. I think for the longesttime that has been subscriptions.>> Um, subscriptions have the downside ofthe fact that, you know, in many marketspeople don't>> have credit cards or they don't usecredit cards to subscribe to software.Um, and we're interested uh in otherways that can maximize access of of thetech. Um,>> our ads pilots are in that spirit. Youknow, we we really view it as a tool ofbringing chatt and our intelligence mostbroadly>> to anyone around the world. M>> um and it is an example of how weconstantly need to evolve and figure outthe best way to to you know bring thedemand in line with what we are able tooffer.>> Makes sense. Makes sense. You know theads the ads piece is um is is has been atricky one because you know Sam hashistorically expressed reluctance aboutads and um>> you know we've got to maintain a lot oftrust um with while delivering that. So,um, I guess what changed?>> I think we've talked about this severaltimes in my my history at OpenAI, andevery time it came up, we said if if

00:32:08wewere to do ads, we'd have to be reallythoughtful about the way we do it. Um,so the first thing we did, you know,starting, you know, end of last year wasto really engage the company on if weput ads in chat GPT, how should weapproach it? What should the principlesweigh? what be how do you preserve thethings that are magical about chat GBTwhile getting the benefits of ads whichyou know is is is our ability to bring>> um our most advanced tech to to toanyone um regardless of their ability topay>> and um I really love where we ended upon the principal side>> on the experience side>> um we're very very early but on theprincipal side I feel really proudbecause you know it's it's veryimportant that the answer of chatbt beindependent as an example>> respecting user privacy is veryimportant and there's a lot to learn umfrom you know the way that tech hasevolved over the last few years>> um or really last decade um>> um and I I like that the principles areout there before we've even reallygotten started um like we're very earlywith our pilots>> um you know it it's kind of interestingI was obviously very anxiously andeagerly looking at um our support uminbounds and data and um the most commoninquiry about ads is not you know how do

00:33:23disable ads or turn off ads, but it'slike how do I run an ad?>> Um because the entire ecosystem isreally excited to be part of the storyand to figure out a way to talk to chatbusers. So,>> um there's a lot more to come. Um but II I'm I'm very eager to get this right.>> Yeah. Yeah. I'm sure you guys will. Umswitching gears, Nick, um something youand I have spoken about a little bit is,you know, distribution and partnerships.Uh there's a couple of big partnershipslast year. Um Apple Reliance with withGemini, those are two big user bases,right? A lot of India, a lot of the iOSusers. Um talk tell us a little bit howyou think about partnerships for ChadGPT to to to meet the meet the userbase. Um and maybe specifically on thosetwo as well.Look, um I think partnerships are agreat way to to bring two two two twoproducts together and to you know>> um exposeum something like chatt to um people whomight not other otherwise haveencountered it.>> Um the thing that I care about most whenconsidering something um like apartnership is what is the what is theuser

00:34:39experience and can we make itamazing? Um because at the end of theday um when you look at um what's goingon in the market um you can get users toclick on things, you can get them to tapany sort of product um especially if itlooks like a product they recognize etc.>> Um but if the experience isn't trulyawesome um you people will churn or theywill you know at least not retain in theway that you know we've been lucky toretain them on chatbt. So for thatreason you know I'm super interested inin paths like that. Um but it needs tobe great. Uh it needs to acrue to theuser. Um we are very lucky to have um agreat brand and a recognizable productum for many many folks and I want tomake sure that anything we do isaccretive to um um to all that.>> Nick, you are uh you're a master oftrade-offs. You must be making a lot oftrade-offs right now.>> Uh um tell us about tell us about someof the trade-offs you're making. tell usabout, you know, an a trade-off that youmight be making that people don'tappreciate uh from the outside.>> There are a lot of trade-offs um indeedand um for different reasons, right? Umum the what I encounter a lot is tradingoff

00:35:55delivering on the people on on the usecases that exist in the product todayand making them better versusproductizing you know step changetechnology that's going to generate awhole another set of use cases>> because when you think about how chatbcame to be it was a totally open-endedproduct. It was basically a userexperience around a technicalbreakthrough.>> And we couldn't have told you all theways that people find it valuable,>> but putting it out there was reallyimportant because it allowed us us todiscover and the world to discover whatwe can do.>> And then postgum we can obviously very systematicallygo and improve on the things that peopleactually want to use it for. And whenyou're at a company in this moment whereyou both have such amazing traction withwhat exists today and the mostmind-bending breakthroughs on theresearch side, the balance you have tostrike is making the core product youhave better today with all the thingsthat matter, latency, um reliability, ummaking the use cases really great thatpeople come to>> with um you know providing access to tothe step change um And um uh you we tryto get the balance right, but we're asmall team and we don't always get itright.

00:37:10And for that reason, it's one ofthe most difficult um trade-offs that Ihave to deal with.>> Nick, I imagine one of the hardesttrade-offs you guys make here is thoseuh GPUs that are melting between chatGPT, between codecs research. Uh how doyou guys uh allocate the GPUs?>> That is a very good question. Um andI'll let you know when I figure it out.Just kidding. you know, we we've gottenwe've gotten a lot better at this. Um, Ireally hope, by the way, to be at apoint one day, and I've yet to reachthat point, where>> we don't have to face this trade-offbecause it's really painful to have realuser demand>> uh for products that you can't serve.>> Um, like you know, if if you only everworked in software, that's entirelyunusual dynamic, right? Where you just,you know, or you were limited by thiszero sum resource out there.>> Um,>> the marketplaces have it. Um um but youknow I think pure software doesn'tdoesn't really have the dynamic right.>> Yeah.>> So um you one thing we try to doobviously we prioritize our existingusers um first we want to provide a fastreliable product um and that is criticaland table stakes.>> Then when you look at new capabilitiesthe sort of na naive>> u business school thing to do would beto probably you know look at revenue

00:38:25incremental revenue per GPU or somethinglike that. Yeah,>> but this is where it's more an art thana science because we often have newbreakthrough capabilities that areentirely zero to one. Deep research wasone of those you know we couldn't havetold you is there going to be demand foryou know consumer demand for a researchproduct but if you don't productize ituh um to find out>> um you you will never know. So you knowthis is where we have to be a little bitthoughtful on how we balance you know umthings that are no-brainers that peopleare really going to love with thingsthat are um brand new ideas. Thenobviously on the research side, there'sa reason that Mark has the job he hasbecause a big part of um his his job isis figuring out what research to fundand um um you know, obviously GPU is abig part of that. So um very nuancedtopic that we're continuously gettingbetter at, but um for me the priority isalways on our users.>> Yeah. The other takeaway that I had isyou can you don't have line of sight toa time when you won't have that problem.>> It's it's been so fascinating. Umbecauseyou know we obviously have beenincredibly lucky to uh encounter moreand more users who want to use ourtechnology but then the value that we'reable to provide for each user is goingup as well>> and um you know GP

00:39:40or GPU consumptionyou know it correlates pretty well withthat value>> um and uh when you just look at tokenconsumption per user um especially inthe enterprise too which is you know uma a massive opportunity>> um you see um a lot of very GPU GPUhungry hungry workflows and um um yesdemand keeps going up even as prices godown.>> This is a fascinating insight. Peopleused to think that humans were you knowyou can't kind of make more humans wellon a takes nine months and then 19 years[laughter]but uh but you're saying well that'sactually more more less finite resourcethan GPUs. Yeah, I mean on the humanside um you can hire uh more humans andobviously we've been doing busy doingthat and bring the best talent um acrossfunctions to OpenAI. Um in the worldwith agents you can also get moreleverage per human um um you can makeyour humans very effective um at theirjob to do more>> but GPUs are zero sum um and if youdon't have more GPUs you really have tofigure out how do you make very veryhard trades and hate making hard tradesfor users.>> Yeah.um hence the desire to um um have moreGPUs. But uh it's it's useful to startwith the most zero sum trade-off whenyou do your planning. So I thinkstarting working backwards from GPUs

00:40:55umis usually pretty pretty good idea.>> Yeah. You know, one of the we have allthese external data sources for chartsof users and usage and>> act activity and retention all thosethings. What we don't have is tokens peruser over time.>> And I bet that chart is like a sweetline going this way. I think internal ispretty good. Our internal uh employeesis a pretty good indicator for what'sabout to happen. And um yes um thecharts are are are mindboggling.>> Yeah. Yeah. Yeah. Fascinating. Uh okay,a couple of quick ones on the presentbefore we go into the the landscape,which is, you know, shopping. Uh youknow, we we just moved into a new house.We took some photos and we were hopingthat all our furniture would magicallyappear that Chad Gibbly helped us paint.But>> you know a lot of a lot of uh recentupdates on chat GPD shopping. Tell ustell us about it. What are you thinking?Yeah. On on shopping as Chad GP as ashopping assistant. Shopping is one ofthose use cases that exist organicallyin chat GBT today and they work. Um youcan ask chat GPT about any purchase youmight be planning and get prettyexcellent advice. Uh but it's also oneof those cases where the experience thatexists in chat um today is it's not it's

00:42:10not the perfect experience that youwould want um because shopping is veryvisual for example. So you're going towant to actually see products and imagesand be able to compare and contrast notjust read you know walls of text.>> Um people care about the sources of youknow um um um you know where can I learnmore about you know a given product etc.Mhm.>> Um and um so there's a lot of work to doto make this discovery really reallygood and allowing people to use chat asa as a um as an assistant to find theright product to buy.>> Um and that's where our focus lies ismaking that really great and making thatreally great in a way that works for ourretail partners as well. Um because as Imentioned earlier, there's huge appetitefrom the ecosystem to be part of the theChachiBT journey. Um and um nailing thediscovery piece is has has been um themost promising um focus here to date.>> Nick on chat GPT you must see a breathof information. You must see a breath ofuh use cases that people are doing withChad GPT and and tell us something aboutyou know what what does the worldunderestimate about Chad GPT that youhave maybe been surprised by or or orlistener might be sub surprised by?there's been a real change in the waythat people think of chat

00:43:26GPT>> um over the last year or so where umit's increasingly like a true thoughtpartner to people>> um it's not just a thing that you knowanswers your question um but it's athing that you can it's a sparringpartner that you can actually thinkthings through with>> and u that shows up in all kinds ofdomains ranging from life advice whereyou know if you got a>> relationship problem you can actuallyget a lot of value chat is helping youthink through how to handle it and howto talk to you your partner about it. umall the way to a work setting where youknow you're you're working on um um ananalysis or you're like trying to figureout how to frame something or you're youknow trying to build something and ChadGPT really shows up as um as a um secondbrain um of sorts and>> uh I think that's qualitativelydifferent in terms of how the mentalmodel it occupies with people>> and you see in the usage patterns andthe use cases that exist and I think themore we nail things like you knowproactivity which we talked aboutearlier in task etc. I think the moreit's going to feel like a teammate inthe workplace and like a super assistantat home>> and uh I think that's going to ummeaningfully change the the use casesthat people come for.>> Yeah. You know, I've been the the mosthighstakes thing

00:44:41I do with Chad GPT isuh we have a we have a new baby>> and when the baby's uh crying at 3 inthe morning,>> Chad GPT,>> what you know what's what's going on?>> First of all, congrats. Second of all,I've heard this from all parents in mylife. the chat has become indispensableindispensable>> as a thought partner and it makes senseright if you have a really specificscenario or you think it's a specificscenario to you chat really comesthrough and can um you know help you youknow um um build confidence and I thinkthat's such an empowering thing>> right like and I imagine new parentsaren't always the most confident aboutwhat is the right thing to do and ifchaty can make you um you know feel likeyou you you are you have agency andcontrol and you know you know uh I thinkI think it's really valuable.>> Yeah, it's huge. Well, thank you formaking Chad GPT. It's literally gettingme an extra hour of sleep every day.>> It took a village. But that is a greatmetric. You know, that should be thenorthstar metric is like incrementalhours of sleep.>> That's a great one.>> Incremental hours of sleep, incrementalhours of joy.>> There you go.>> I mean, you joke, but like we talk aboutthis a lot and um because spirituallythat that that is pretty close to whatwe>> hope we can do, right? is is is help youre reach whatever you considerself-actualization.>> Yeah.>> Whether

00:45:56that that's sleep or joy or anyother goal you might have.>> Yeah. Yeah. Yeah. Well, thank you forthank you to the village.>> We're going to switch gears and talkabout the landscape.>> Sure.>> There's a lot going on on the field. Umyou know, how would you frame Chad GPT'sdifferentiationuh to to to people out there? Um there'sa lot of different products out there.Look, it's the best time in history tobe a consumer of technology.>> It is indeed.>> Um because uh you got options and thecompetition is intense. And I thinkthat's beautiful and it's actually goodfor us too because>> if you were to premortem why a companylike OpenAI does not achieve itsmission, it's probably focus because ofthe sheer number of opportunities thatbecome possible when you approach AGI,right? and having competition and andoptions out there, I think um it itforces us to focus on our customers tooum and on the things that really matterwhich aren't always the most flashythings, right? Sometimes it's latency,reliability, um the quality of the userexperience.>> So, uh I think it's a really good thing.>> Yeah.>> I think the biggest differentiation ofJet Beauty is the team behind itbecause, you know, we're not static,right? anything we build will get copiedand you know sometimes in

00:47:11in in in waysthat are high craft sometimes in waysthat um you know are sort of um you knowum check boxes and um it's reallyimportant to us that we evolve thecategory um and um build the superassistant that we've always imagined andI think the way the reason that I haveconfidence that that's possible at aspeed that outpaces you know um umthe the dynamic of being copiedis that we have an amazing team. Um, andthat we have an amazing team acrossresearch and engineering and design andall the different functions that ittakes to make something amazing. And Ithink our unique ability has been tobring those functions together to buildsomething that is sort of at theintersection of useful and possible>> right in that moment.>> Um, so you know, my best answer for youis we keep pushing forward and we hopeto be um um expanding what people thinkof this product as. you know, last uhwinter we had obviously uh you know,what was called Code Red. Google had a agreat model. Uh there was a lot of, youknow, talk about it. Mark Beni offswitching very vocally to to to toGemini and us delaying ads and healthagents and shopping. Basically hit pauseon everything, making tragedy better.>> Talk talk to us about that moment. Um

00:48:26both both about what led to that um andwhat was happening in that moment.>> Yeah. Um so first off code reds are atool we use um to create focus and asyou can imagine when you're in a placelike open and this is what makes usspecial to um special to work here is isis>> there are so many different things>> going on it's a research lab we arepursuing many different ideas right>> and there's been these moments where uwe've wanted the company to cometogether um um to solve a problem>> across boundaries you know no matterwhat your project might have And>> end of last year, we had one of thosemoments where we felt like we we need toshow up for our users. We need to focusthe things focus on the basics. Um likereliability, performance, um the waythat talking to the model feels ummaking personalization really great. umall these elements that you know our ourusers care about and um I loved it umbecause it was really an opportunity towork with a bunch of folks who I don'tnormally get to work with on making theproduct great>> and we just exited the code red um>> which we knew we would um with thelaunch of 5.3 which you know is is agreat model for the everyday user umit's great to talk to and um 5.4 fourwhich is um workhorse if you're tryingto do real knowledge work

00:49:42>> and um you know undoubtedly we're goingto continue to use the tool um um of acode red whenever we want to createfocus um but u I'm excited because Ithink um chat GBT um is is is in a greatspot>> yeah so code red is over now>> that's correct>> it's not the new normal>> um it's not the new normal uh we want itto be a special thing but it is a tool II suspect we will continue to use>> that's great that's great and Um maybetangibly if you were to point at how didcode red change chat GPT uh or maybe theops or or how the team operates.>> The thing I try to get like you knowfoster with the team is focus. Um so wecertainly more focused than we were sixmonths ago>> on um you know the>> uh the things we really want to nail andsome of those things are very behind thescenes like>> latency reliability>> those kind of things.>> Okay. And some of those things are likevery considered efforts like youinvolving chatbt into the superassistant and um>> so focus is the main lasting artifact.Um and as you imagine um>> um it's hard to stay focused sometimeswhen there's so much going on in thespace but um that's the hard job and youasked me about trade-offs earlier.>> Um um getting getting the team to focuson the things that really matter tousers is certainly one of them. Umthat's

00:50:57always worth it.>> Yeah. you know, in the back of my mindas I ask you that question>> is all the other founders that are thatare in the arena right now. Uh and andjust a reminder that hey, Code Red is atool for for for you. Wartime at balanceas we used to call it is is a tool.>> Yeah, I think you know every companydoes it differently um in terms of howyou how you get stuff done. Um, but Ithink it it's really valuable to haveterminology that um, you know, meanssomething>> um, that, you know, signals to peopleit's okay to drop your other stuff andit's okay to, you know, focus on thisthing together even if that wasn't youroriginal job.>> Yeah.>> Um, so I think it worked really well um,at a place like OpenAI. Um, but Iimagine startups would have anequivalent.>> Yeah. You know, one of the things thatcaught everybody's imagination uh, onour team was uh, what Peter was doing atOpenClaw.>> Mhm.>> Incredibly potent to put all the toolstogether. Obviously, Peter's a greatbuilder. Congrats on on on bringing onPeter to the team. Um, tell us a littlebit about what what what Peter's workingon and when when might the uh uh thebillions on Chad GPT have something toto to see there.>> Well, first of all, I'm very excited forfor Peter to be here. Um, I uh um um wasexcited to have another German speakerin the house. He's he's Austrian. I'mGerman. So, we were we were

00:52:12umexchanging Guten Morgans. Um but the uhum yeah the the open claw is soinspiring because um it brought to lifein in many ways um a vision that thatthat you we'd had um um in differentforms know admittedly>> um around this kind of AI that is fullyembodied that you know exists acrossdifferent UIs that can do stuff for youthat has state that has an interactionpattern that feels a little bit morelike talking to a human, you know,because you um you open cloud allows youto you interact in a very very naturalway where you can you send many textsback and forth and it's very curt and sothere's a lot of elements of of of ofOpenCloud that I think were veryclarifying to to folks across theindustry and um but the best you know II'm I'm super excited to just like learnlearn from Peter and bring in uh intothe company and figure out what we cando together. So there there's there's alot more to come. All right. So, now onto the most fun section. Rapid fire.>> All right.>> You ready?>> Sure.>> Well, we we'll start with my most my mymy favorite game, which is long short.Uh, pick an idea, a startup, a business,a product that you love, you think youyou're very bullish on. Yeah,

00:53:27I'm I'm ifI were starting a company today, I I I'mreally excited about these companiesthat are going into companies andgetting extremely hands-on and doingeffect effectively professional services>> um with AI because we've saturated allthe emails>> um and you need to get proximate to theproblems. So um that's it's it's thosecompanies that that um I'm I'm payingattention to.>> Fascinating. So this is, you know, thisis an example. This would be like, hey,you're going and either acquiring or oror going inside an operating firm thathas scale and and and and and a hummingengine.>> Exactly.>> And making that a more efficient engine.>> Yeah. Or or or just like, you know,you're doing contracts for for for forcustomers that have really hard problemsand you're actually going in andcommitting to solving the problem.>> Um outcomes.>> Yeah. because like you know therethere's a reason I think that we made somuch progress on math and coding um butnot on many other domains because thoseare domains we are approximate to we aspeople who work in labs and>> there's all kinds of other domains thatwe are not as proximate to and if youget proximate I think you can you knowbuild something transformative and Ithink this is more important now>> um precisely because um you know theeasy problems have been solved um theobvious problems have been solved by themodels>> credit where credit is due I think notnotebook spoke

00:54:43LM is awesome anddifferentiated and helps me learn newstuff. I think it's great.>> It's so good.>> Yeah,>> it's so good.>> I think this is the example of you caninnovate and you can build something umtotally different. It's awesome.>> Yeah. Yeah. Yeah. It's so good.Particularly for uh I found it for somemore technical uh uh learning to be avery approachable way to to>> totally learn. And it's really cool. Ifeel like AI an underrated capability ofAI is to just transform things into adifferent medium.>> Mhm. And I think that's so important forlearning like we just la um um launchedthese like dynamic um math blocks whichallow you to visually understand mathinside chatb learning is obviously a biguse case for us too>> and I think just being able to transform>> things from text to visual you know soonfrom you know visual to video and likeall these different media is amazingbecause people have such different waysof processing information um and somepeople are like auditory learners somepeople are visual some people likereading uh so I think that's reallymagical Um and um a great great angle totake.>> Yeah. Amazing. Amazing. Amazing. Uh youknow, one of the things I think about alot is education and and and educationfor for for kids now in school. Um theworld's changing so fast.>> I'm not

00:55:58sure our education system ischanging that fast.>> Yeah.>> What advice would you have for forstudents who are in school now? Um youknow, who might have to adapt fasterthan the system around them might adapt?It's a really good question andsomething that I've thought a lot aboutum myselfand you know the I think the mostimportant perma skill in this era is uhcuriosity I think because

00:56:32>> if the machine can answer all yourquestions you better have goodquestions.Um, and the only way to have goodquestions, I think, is to>> pursue the things you were actuallyexcited about from an early age andthroughout your entire life.>> Yeah.>> And, um, I reflect on this because theonly reason I'm here and working on thisstuff is because I thought it was neatwhen I got, you know, nerd sniped in>> um, in the interview process, right? Andit was like,>> that's right.>> This is so cool. And and so no matterwhat you're doing, I think that's animportant skill is to be curious andlearn to stay curious. And I think I'mconfident that um if you foster thatskill um you will know how to adapt toyou know an evolving>> landscape of tools and AIs and and jobs.Um so that would be my advice.>> Yeah, curiosity is has always been thepoorest scale. Our friend uh Bill Gurleywrote about it in his book Running Downa Dream. Um but curiosity have to checkthat out.>> Yeah. Um, what what is a job that uhgets more valuable, not less, as uh AIgets better as as AGI arrives?>> Well, I think maybe the easy answer isbeing an entrepreneur.>> Um um because it's the best time tobuild ever>> in terms of like being able toself-actualize your

00:57:48>> Yeah.>> your idea. Yeah,>> but maybe maybe one that is maybenonobvious is I think writing actually>> um is very important and it's notbecause the AI can't write you know AIwill become amazing at writing just likeany other domains but because I thinkthe skill of writing>> forces you to be very clear>> of what you have to say>> and even though prompt engineering isobviously going to go away and has goneaway to much extent>> the idea of expressing what you want>> to a machine>> requires you to be a pretty good writerand a very precise um um writer. So I Iwould say that that is a you know anyprofession that involves very clearwriting and therefore thinking I thinkis is is um um well set up. Yeah, 100%.Honestly, I mean there this is the wholething about slot, right? There's so much>> that's the other thing. I think there'sgoing to be a permanent need for highquality, trusted, authoritative contentand tools like CHAP can help youdiscover that content.>> Yeah.>> But I think the need for amazing um umcontent is is is also here to stay.>> Uh and final question, what has beenyour AGI feel the AGI moment? When didyou feel it?

00:59:03I've had so many honestly.Um, and it'sit's definitely not stopped. Um,a few weeks or so after I joined OpenAI, GP24 had finished training>> and I remember trying it out and itactually it didn't impress me at all.Um, nor anyone else that week because itkind of didn't work. And it's because wehadn't figured out how to post train it.And uh I think seeing it go fromkind of wait is this really a thing orwas GPT3 kind of it to wow actually thisis an entire step change with what feltto me at the time who didn't understandmuch about AI at all as like just sometweaks or some a little bit of finalstretch work>> was profoundly humbling because you canrealize that you might it might not looklike we are close>> to really powerful useful AI But weprobably are. Um,>> and then the moment that, you know,really, you know, there was two thingsthat GBD4 did thatfelt like AGI to me. One is it could dopoetry and I didn't think it waspossible for AI model to do poetry. Itjust kind of>> fundamental philosophically, it justdidn't feel like

01:00:19>> in scope>> and then the other one was it couldproduce code that actually worked andcompiled. And then my next moment whereI stared at the ceiling just in awe waswhen I realized GPD4 could just simulatean entire computer terminal um um like afull computer with commands etc. And I'mlike, wait, how would this be imbued ina in a in a language model? And there'sbeen so many moments since then,honestly. Like reasoning was a moment.One of the moments was when I think Markand I were uh uh giving a demo ofreasoning uh in front of the um wholecompany. Um and um this was a momentwhere we were still trying to kind offind use cases that were hard enough forthe AI for the reasoning to, you know,make it make a difference. We're waypast that point, we know. But at thetime, you know, I think we were havingto do a puzzle in front of everyone andum I think one of the moments that mademe totally feel the AGI is like um wewere in the middle of the demo andeveryone started laughing and I waslike, "Wait, what what is funny?" Andthen I stared at the screen because wewere showing this chain of thought as itwas streaming out of the model. Um andthe model swore and said like, "Oh, damnit. May I have to adjust because Irealized I had made a mistake in thepuzzle." And uh the fact that they didthat, but in particular the fact that itdid that in a way that was entirelyemergent

01:01:34from the you know RL processcompletely blew my mind um and you knowmade me you know um uh feel quite humbleabout what else these models might beable to do. So um that was one of thosemoments.>> Yeah.>> Uh and then most recently watchingpeople use codecs like like watchingpeople have>> walk around with their computer open uhbecause they don't want the task to end.Um yeah uh watching people who havenever coded in their life make stuff andbring ideas uh to life that feel feelslike an AGM. So honestly it just it'sjust accelerating for me and it doesn'twear off at all. Um and uh everyone hasa different thing obviously but thoseare were some of mine.>> Yeah. You know it's uh 10 years agothere was a product called kite. I don'tknow if you remember it was for softwareengineers. It was like an AI codingproduct. Mhm.>> That's when I that's when I felt thehunger for for for for for personal AIand you know nothing happened for 10years and then everything happened inthe last 10 months.>> The timing thing is really hard becauseit's it's actually quite>> possible to predict where things willend up I think um in terms of the kindof product form factors you're going tohave but to know when it happens it'sreally hard for me to make statements onanything between>> sort of eventually and in three months.

01:02:49>> Yeah. because of all the ambiguityaround, you know, um um>> well, that's a tight enough window, youknow, now and three months is a tightenough window.>> Three months is pretty pretty okay. Tryto stick to the three-month plan more orless. Um um though, you know, my teamwould probably tell me we don't, but I Itry. But yeah, anything in between threemonths and eventually is is isdifficult. Um>> yeah. Yeah. Yeah. Well, thanks for doingit. You've got a lot going on. This wasa total treat. We so excited to see allthe great products you release for us.Um,we can do anything to be of help, let usknow.>> Awesome. Thanks very much. Thanks forhaving me.>> Of course, man. This was fun.

01:03:36As a reminder to everybody, just ouropinions, not investment advice.