# The 2025 Design in Tech Report: How AI Will Turn Designers into "Autodesigners"

**Channel:** SXSW
**Source:** https://www.youtube.com/watch?v=hWeVAX1LwII
**Transcript page:** https://www.withtranscript.ai/video/hWeVAX1LwII

## Chapters

- 0:00 — Introduction and AI Report Overview
- 22:08 — Foundations of Loops, Agents, Interfaces
- 36:00 — Latent Space Concepts and Model Types
- 41:40 — Artificial Life and Living Software Medium
- 46:28 — Software Craft Evolution and Vibe Coding
- 50:13 — Instrumentation and AI Evaluation Practices
- 53:29 — Concluding Reflections and Regret Workshop

## Transcript

**[0:00]:** Thank you. Hello, everybody. Thanks for coming. Really. Thank you. This is my 11th year trying to do this. I say trying because I always wonder if it'll become something so we'll find out together. But I do it because I know you might be here, hoping that I could give you something of use. So this year's report is called Auto Designers on Autopilot. And I thought it was sort of funny how different people I talked with thought I was going to give an automotive presentation.

**[0:35]:** They really did, because it makes sense. And I also asked Sora to generate a video of what presentation John maybe was going to give about auto designers and Autopilot, which was like that. Yeah. So we're done now. No, we're not.

**[0:51]:** Because I did the thing I do every year, which is take all my bookmarks. I had roughly 1500 bookmarks, and then I print them all out and then I try to organize them in some logical way. I had too many bookmarks this year, so I wrote a program using Azure OpenAI services GPT 4.0, and it finished sorting everything in 25 minutes. Then I looked at what it did and I was like, nah. Then tried it again. It was like. So I've spent roughly 20 hours sort of sorting them out, and I hope it's useful to you. So, first off, good news, AI is not going to replace designers. It'll transform how the work is done. I think that's clear to me, at least.

**[1:40]:** The other thing is, maybe you've heard of agents. Agents is an old word from the 1990s that's come back in full force, and we're gonna see it all over the place. Another one is AX instead of ux. Agent Experience. The agent experience doesn't involve a user, it doesn't involve people. It involves the agents. And lastly, we're gonna be living in this AI augmented era, so we. People have to somehow adapt fairly quickly, and that's what makes it kind of stressful. And I hope to reduce that stress here. Hey, Drew. How are you doing? So, first off, those of you who studied calculus in college raised their hand. Most of you probably. You're probably forced to do it. Calculus was really important because that's how we created missiles, sent people to the moon.

**[2:34]:** That's the kind of math you need. And statistics was like, no, that's wimpy math.

**[2:42]:** But I think about it because I was sitting having dinner in the hotel there, and I saw this sign. It's C O R T Y. It's like a game show thing.

**[2:51]:** And I was like, Oh, I know the answer. And I asked foro, what does this say? And it got it right. And I was like, whoa, wow, you're pretty good. Fouro.

**[3:04]:** The key thing to note is this word likely. It thinks it's likely that it's that. Just like I thought it was likely. I mean, they could have rebranded Marriott with, like, one T and change it. So that's like, the design now, right?

**[3:19]:** Could have been. But it's likely that it stands for Courtyard Marriott. So it's all about likely. Just keep that in mind. So I have been doing this for a while, and in 2018, I got tired of AI.

**[3:34]:** So I made my last AI section in the report, and I asked a question about when people expected AI to replace designers. I thought I would update it. So if you don't mind using your phone to fill out a quick questionnaire while we're going through this, I'll do two of these, and we'll see the results up here. And I'll go back to it, see what it is. But I'm curious what you all think right now in 2025.

**[4:01]:** Okay. Now, there was a person named William J. Mitchell. Died in the early 2000s. He was one of my mentors. Anyone know William J. Mitchell?

**[4:13]:** Used to be popular in the design tech world. Once you die, your SEO goes really down. But he had this great book called the Reconfigured Eye, which I recommend to everyone who's worried about imagery, all this fake imagery out there, because he gives a history of images that go all the way back to drawings and how we've always tried to fake out people. This is in the National Archives. It's called the home of a Rebel Sharpshooter.

**[4:40]:** It's from the 1860s. And there's also another famous photo of a sharpshooter's last sleep. So one is a Confederate, one is a Union soldier. And it was noted years later that this is the same person. It's because the same photographer near demonetized on both sides.

**[4:59]:** And I find that really interesting. This is no digital whatnot. It's just how the world's always been. So you have to question what you see always. And some of you may remember Marshall McLuhan, another person whose SEO has gotten kind of bad.

**[5:14]:** But he said these things. Politics will eventually be replaced by imagery. The politician will only too happy to advocate in favor of their image, because the image will be much more powerful than they could ever be. And so this idea of imagery being so powerful, the centuries of that, and in this era where images can produce of anything. It's to keep in mind.

**[5:36]:** Now, a long time ago in the 1990s, I was advocating for writing computer programs in the design world, which people didn't like a lot. But I created this language called design by numbers. It's a very simple language. Here I'm setting the paper's color to anywhere from 100. Black and zero means white.

**[5:57]:** And I did this to show that drawing on the computer is a bit strange. Here I'm drawing a line that seemed fairly easy, but once I place it inside a forever loop and draw the line when I hit run, it's drawing that line forever. You just can't tell. You can tell when you place it in the space of interaction. Let's say I ask it for the mouse coordinates X and Y.

**[6:24]:** It's suddenly drawing and you're like, wait, what is that? That's kind of messy. Well, all you do is you slip a piece of paper in the middle and you get the animated interactive world. And I made this to work only on 100 by 100 pixel grid, only in black and white. I was very basel typography era.

**[6:44]:** You know what I'm talking about? Swiss. Thank you. You know that? Yeah, this is a terrible flop. No one liked this. And so luckily my students went off and built better things. There was a thing called processing that was launched from that other things, Arduino, P5Js, Scratch. All these things sort of came from a few bad ideas that I had in the 90s. So I apologize for that now. I love this post by rga. They discovered the intersection of design and technology in the gulf of you know what. And I think it's like a good time to think about what is design, especially in the AI era. So if you think about it, in design, we make obstacle courses that the user has to navigate. The interesting thing about AI today is if you know the intent of what your user is, they don't have to go to an obstacle course.

**[7:41]:** You can teleport the user there. So I'm gonna pause there for a second. Every UI is some kind of obstacle course. It is something that is the result of people working together. Or maybe not sometimes, but what happens when you teleport to the goal?

**[7:59]:** What happens to what? The UX field is the question. Because if we spend all our time building obstacle courses as the craft and that craft is no longer needed, what happens is a question I have now. First off, it ties to code. And how many of you have used a GitHub style system before?

**[8:18]:** Okay. Some of you, you know, the code world works in the world of information sharing. And it's important to note that when I took a reading in 2018, the number of GitHub repositories called repos for machine learning was 94,000. Deep learning, 28,000, AI 10,000. And if you notice, that number's kind of big now, so there's lots of code out there.

**[8:42]:** And that's important because it means many of us can actually go out and use that code and, and actually level up, which is kind of an interesting era. That's a lot of code to level up from. And AI has gotten a lot cheaper. Some of you may remember when you were trying really hard to get access to GPT4. Now it's kind of easy to get stuff. And when it's easier to get stuff, when there's less scarcity, the prices go down. If you notice, there's this little company with a whale here, made things you thought were expensive a lot cheaper and faster. So we're in this sort of disruptive era where it's much easier and cheaper to do these things now. Now, when you hear the word agents, I recommend you do a command replace all every time. You said models last year, just say agents instead.

**[9:30]:** It's easier. And I explain why and how that makes sense, and you'll get really confused by all the prognosis of the future, blah, blah, blah. It's actually quite simple. Agents are a different encapsulation of how we create computational building blocks. The only difference is when you put them in loops, because when you put anything in a loop, it gets very powerful.

**[9:55]:** Computation by itself will just do something you tell it to do. If you stick it in a loop, like in my Design by Numbers example, it was doing that not just for like 10 minutes, but I could let it run and run forever. So that's a big switch. And agents by themselves, easy to understand. You add a loop just like a computer program in your house.

**[10:17]:** It gets confusing to understand because it runs forever. Do you want it to run forever? Is the question. Okay, so I wrote a book in 2019 called how to Speak Machine. You've never heard of it, but it is a book where I spent six years trying to explain computer science to anybody.

**[10:36]:** And it's being reissued this year because, I guess it's somewhat timely. It's a time capsule of how we got here. And it's constructed in six principles. And so I've sort of taken this year's report and used that as a kind of a Way to organize things. Now, when I went through all this, I had probably 300 different things I wanted to tell you, which I can't fit into roughly a 15 minute window.

**[11:00]:** And then when I was done last night, I realized the most important things were in here. So I'm going to put them here at the beginning. Are you okay with me on that one? It almost wasn't here. I thought it was important.

**[11:10]:** We'll see. First off, AI is changing the UX craft. There is a LinkedIn learning course I have called UX for AI design practices for AI developers. There's five basic rules. If you're building with AI that have stayed persistent, you always want to remind the user that, hey, this came from AI, don't forget.

**[11:34]:** The second thing is like, hey, you don't know what to do. Here's some ideas. The third thing is like, hey, this is where I got this from. Citations. The other is hey, I'm thinking, sorry.

**[11:48]:** And the last one is how did I do? You know, when you go to the bathroom, there's that smiley face thing. It wants to know. Okay, now that's kind of boring. So I brought some more wow Y things.

**[12:00]:** I think that Harley Turan, this guy's amazing, but I think he's the one who really made semantic zoom kind of cool. I know the ARC browser had it for a while, but semantic zoom is basically this idea that any paragraph, any piece of information just pinch and zoom. I think this idea is quite durable today we see it here and there. It hasn't hit mainstream, but it's a really simple idea and it fits our mental model of how maps can contract. First thing.

**[12:27]:** The second thing is really the release of OpenAI01 model produced this notion that it's going to take a while. Sorry. So let me tell you what I'm thinking. So the work of OpenAI in producing this idea that I can make reasoning traces happening. I'm thinking, I thought this was quite beautiful when it came out.

**[12:48]:** I'm thinking, I'm thinking this, I'm thinking that, I'm thinking this. And so you can sort of see that effort is being made versus here's the answer. So sort of showing your work. It's a good UX pattern, I think, for AI. I have a bunch of other ones, but if you want to see a good Compendium, do a YouTube search for Y combinator. Rafael Shad There's a great piece of very, very recent examples of AI UX that I recommend. On the practical side, less. Wow. There are a few Things that I think stand out that are possible now. Anyone can do it.

**[13:23]:** I love this pattern of short, long. On any piece of information you can offer two grades. Short, long, complex, whatever. It's all possible today. And then there's also different ways to kind of take a prompt and apply it to something.

**[13:40]:** Those of you who use cursor or GitHub, copilot, et cetera, if you do it at filename, it uses the context. So it's a way to kind of like I love Cookie Monster, by the way, those of you who saw Cookie Monster here on this stage, it's like, ah, cookie context. So it's like a Cookie Monster way to pull context quickly into your conversation space or thinking space. It's awesome. Also, version control is something we don't talk a lot about.

**[14:13]:** I always look at what Jeffrey Litt's doing. But version control is important because once you can generate AI anything, you can generate 50 versions of something and now you generate 100 more versions of that one version. Which one is good or not is called a version control problem. On the image model side, Pablo Stanley showed this recently and it's a wonderful example of how to add input. How do you tweak something in a direction versus actually touch it? How do you change the style of something? How do you. This is a great pattern. Also, Google came out with this whisk pattern. It's basically mad lib style input.

**[14:52]:** It's a form of puzzle prompting I think is what I would call it. But it's another way to kind of get you moving much faster than average with these generative AI systems. And there's a few other systems out there. I mentioned CREA and Recraft. Examples of really kind of fine grained prompting. Fine grained touching of is important. This is my favorite category. Concepts and random things. If that's a category, this one, it's sort of like taking two apps and having them meet together and become a new kind of app. It's an interesting kind of approach, but it's wholly possible today.

**[15:31]:** And I actually made something new to show you all. Let's see if it works. Yes. Okay. This is called my semantic eyedropper.

**[15:40]:** I've been wanting this for a while, so this is a good excuse. So what this is is I can take any cell. Let me see here.

**[15:51]:** I can take a cell, that cell, I can place its essence into the empty cell and it kind of makes it up from there. I like that blend idea. So I can take quantum computing and have it blend with this one. Here, and it gets quantum Y. So there's kind of ways to sort of do things now as a sort of a semantic eyedropper. I always loved when Photoshop came out with that idea. So anyways, anything is really possible today. It's kind of weird. Let's see what we did here. Let's see how we did.

**[16:22]:** Okay. Likelihood that it will positively impact our lives, roughly 50.8%. Are you in the design profession? We're half. Half.

**[16:32]:** So for the designers who aren't sure, when will AI replace much with others? 36, say never. You know, that's a good one. That's a good timeline. 24%.

**[16:42]:** So this is sort of coming, coming. Yeah. Ooh, look at that. Very far out. Five years.

**[16:48]:** That's for, like, visual design, mind you, for those of you who make the distinction. And then digital product designers, never. How's the nevers compare? Oh, never is, like, shorter here. See that?

**[16:58]:** That's a little shorter. Okay, never. So we're sort of feeling it. For those who are not, when will AI replace most designers? Well, okay, let's see. Those who are not in design think, oh, yeah, you're gonna be replaced. Okay. So they're winning. Thank you for that. Okay, let's keep going.

**[17:13]:** We'll do another one. Okay. And there are four AI UX spaces that are, I think, exist. One is, of course, the chat UI space. People chatting and chat. I made a demo of example of where there was something going viral where people were saying, chat, AI chat. You can catch it immediately. It's you talking and the user talking, and it's you talking and the user's talking. But we humans, we don't do it that way. We go chat, chat, chat, and then chat, and then chat comes back. Like, we humans do that. So I was like, no, I don't think. I think I can do it too. Hi, how are you? I'm really.

**[17:56]:** Oh, oh, oh, you're talking. Oh, oh, thanks. You're doing well. I said, this is example that anything we think that it can't do, it's because we have biases in how we actually communicate. When we communicate async, this happens all the time.

**[18:10]:** If you communicate async with AI, this could happen again as well. Okay, stop, please. Okay. And if you haven't seen the stuff coming out of Google Creative Labs, Google Cradle Labs has been around for over a decade. It's an amazing group of hybrids.

**[18:25]:** But Alexander Chan's team has been doing some really weird things. Move them to the top, move them to the bottom.

**[18:40]:** Put them back in the middle, but swap red and blue.

**[18:47]:** Can you make yellow?

**[18:54]:** Can you make purple?

**[19:00]:** Can you make white?

**[19:07]:** Can you form a snowman? That's a good one.

**[19:15]:** So that's really like only a couple months ago. But this kind of stuff is getting a lot easier. So in the chat space, communicating. The second space is documents are changing a lot. This sort of quanta of information.

**[19:28]:** The work by Tyler Angert's been really interesting to watch. He's been asking questions about when you're thinking out loud, how many ways can those thoughts branch out? So sort of branching conversation as a document type exists. Another one by Matthew is very interesting. He looked at what's the color picker for words?

**[19:49]:** How can a word have color? And you can see that there's different things you can do in the semantic space in the context of text that are quite remarkable. Another one by Jeffrey Litter. This is a scheduler that combines a map system. And so the document you thought was just sort of lying there, I'm sleeping, can really come awake in a different way that we never predicted documents could ever become.

**[20:19]:** And I have some other random. I collect these things, mind you. So I'm a pack rat of random things. The third space is tables. Tables like in spreadsheets.

**[20:30]:** I think spreadsheets are really taking off in the AI space. I wanted to see for myself, so I made a demonstration of that. So what this is is it's a spreadsheet that is semantic. So each cell can change based upon the content and it propagates the quick brown sink in my building and it propagates the semantic information via a semantic formula. So that kind of form in a table can take hold.

**[21:05]:** And as you can imagine, a lot of things in a spreadsheet can change at the same time. And so this kind of interface pattern is quite powerful and it's something we really first felt with spreadsheets. And we'll get back to that in a second. The fourth one is a canvas space, sort of a two dimensional canvas, so called infinite canvas. This work by Samuel Timbo is quite interesting because it's introducing a different kind of visual language. You may have seen TL draw computer, but there are more examples of these kind of visual canvas approaches. They're quite sophisticated. Hub, but emails on an infinite canvas. And so infinite canvas is another pattern. We're going to see more of lots of stuff out there and if you're curious about this whole space, there's a talk series openly available on the Media Lab site led by Char Stiles.

**[21:59]:** It's called thinking with sand. All these people are talking about why they do this work. So it's a great set of videos to check out.

**[22:08]:** All right, so section one of six. Here we go.

**[22:11]:** This is Char recently saying, I'm going to dedicate my thesis on creative coding to loop. So the beginning just says for loop. So the loop is the fundamental building block of computation that is freaky and odd and is something that cannot exist in the regular world. Nothing can loop forever. If you spin a top, it's going to, like, get tired on a computer. A loop will just keep going and it won't stop. And that's very strange. And as a backstory to loops, I discovered loops in the 1970s when I was a kid. And I made this to show you what I did when I was a kid. I did this twice.

**[22:50]:** 20, go to 10. And I did run like, whoa, it's printing my name. It's so reassuring. And then everyone then teaches you this little hack where you add a comma at the end and suddenly a second I'm a cls. Let me restart this. Because the effect is very important, because it's like you learn one thing, but then if you do, if you learn one more thing, something else happens. Oh, that's good. I'm going to Change my name. 20, go to 10. Some of you know what's going to happen.

**[23:28]:** So worthwhile again. It's just going to keep on doing this forever. And this is something that's all existed in computers is the ability to loop. Never get tired. And that goes to agents.

**[23:43]:** Agents, by default, do not loop. I'm calling this an agent at rest. It has four ingredients. It has a model, it has prompts, it has knowledge, and it has tools. And once you sort of feel the 1, 2, 3, 4 step, it's very calming, I've discovered.

**[24:03]:** Oh, the agent. It has a model. Yeah. It has prompts. Okay. Does it have tools? Better. Does it have more knowledge Better? Oh, that's a good agent at rest. And it's important. Remember, the program can do the same thing if you have no for loop. No while loop. It's pretty boring. It just starts up and then it's done. Right.

**[24:25]:** A loop makes it powerful. Okay. So. And also, I think it's hard to understand this because object oriented programming, when it first came out. Who remembers when object oriented programming first came out?

**[24:36]:** It was really confusing. I was like, what? You put the data and the class. What is this different terminology? So I think it's kind of similar in that back Then we had classes, and now we've got these AI models.

**[24:49]:** We had these object properties, we have a new way to use knowledge, we have methods, we have actions, and we have reusability. And we can reuse this stuff in different ways. So it's actually quite similar. Those of you who are offended by this, I apologize in advance because people get offended a lot these days. So just remember, the AI is a kind of programming, it's a different kind of programming.

**[25:13]:** And rest assured, program is going to keep on changing. And it's one moment like that. And to sort of prove the point, going back to my favorite space of spreadsheets, if I say if I make this 10, if you notice it propagated everything, it's because the agent, the cell of the spreadsheet, is always listening. In this case, these were listening when I changed this.

**[25:41]:** In this case, the pattern is different, it's just that line. So it is agentic thinking, but it's not using any kind of large language model.

**[25:52]:** Okay, are we doing okay so far? You following any sleepy people? It's okay. You had lunch. Now this is a definition of agents I found to see if it's stable.

**[26:05]:** This is my door number one example from 2023 from Lillian Wang. Agent is large language model, memory, planning and tool use. And I want to note that when token costs dropped, that's when loops came into the picture. It's because if you ran that agent in a loop, it was going to be very expensive. So we didn't talk about loops back then because no one could afford that.

**[26:29]:** But now there are loops everywhere. Door number two, three, four. Everyone adds a loop in there. It's because it got cheaper and interesting. In 1984, Steve Jobs was interviewed. He did say the next stage is going to be computers as agents. That guy, huh? And again, the definition of agent. And the definition of agent has changed over time. Okay, so in 1993, I used to make drawings like this to express how the computer worked.

**[27:00]:** I was fascinated by loops. I mean, loops are very strange. I mean, like once you set them off, they never get tired. And so in the 90s, I felt as someone who had gone from computer science to conventional art and got tired of my hands to rediscover the computer could do all this stuff. I thought, weird.

**[27:17]:** So a lot of my work back then was like, wow, this computer thing is very strange.

**[27:24]:** And Jordan Sigurd noted that the infinity symbol is infinity symbol for agentic stuff in cursor. But anyways, loop is gonna be just like all over Your interface. So look out. Then there's a command line. There's an essay by Neil Stevenson called the. In the beginning there was the command line. The command line exists in everyone's computer. Who's used the command line before. The command line sits underneath everything. It's still there.

**[27:49]:** And in essence, we created graphical user interfaces and it actually makes the computer less powerful. So the fact that we're going towards this sort of like command line approach is opening a door that we've tended to close because it was too confusing and too hard for everyone to do in general. And I wrote this as sort of validates that when you type in like, you know, LS or something, or like hello, command not found. It's essentially chat. It's the same thing.

**[28:21]:** It's the command line we're using. It's a different kind of command line is the point. So the command line is surfaced now in chat, there's the whole category of things called TUIs. Terminal user interfaces. Wow, they are cool, but they come from the past.

**[28:39]:** If you ever drop down to the terminal in Unix, type banner and a word, it feels so good because back when we had line printers, we'd print out giant words like this. It was amazing. Feeling wasted, a lot of paper. But I noticed in the last five years terminals have come back. I remember when FIG came out, it was acquired by aws. But there's a lot of cool terminals now. The young people love terminals and there's a lot of ways to build terminal style interfaces. Who's shopped for coffee from Terminal Shop? You just shh, Terminal Shop. And you get this screen and you can order coffee.

**[29:18]:** God, this is beautiful. Anyways, again, talking in direct mode is the whole theme there. And also in the sort of monospace category, this beautiful typeface came out by Helena Zhang called Departure Mono Monospace got updated. So if you want to use kind of an even pitch font, there's lots of opportunities there. And this thing is amazing. Arborbill, it's by a RISD graduate, but its entire webpage with little hyperlinks. It's really cool. It's like a MySpace page made out of ASCII text. Anyways, there's something sort of simple in this representation and also, as we know, it's very compact as well. And Sam Dape, if you follow what he does, there's so many beautiful things he's crafting. So. So combining these ideas of ASC in code, but really taking into a different space as a designer. Okay, now let's talk about autopilot. For a second. Autopilot was designed for airplanes.

**[30:18]:** In the 1900s, there's something called the DARPA Autonomous Vehicle Grand Challenge. It led to today's Tesla Autopilot era. And all cars seem to be getting this these days. But cars, through the DARPA challenge they discovered that it's really hard to make the car self driving driving because there's all kind of edge cases. You can get bad weather, you can be in a tunnel.

**[30:39]:** There's all kind of cases where technology cannot help and the human has to take over. And cars are really important to us because they're the robot that can hurt us, either sitting inside it or doing harm to someone else. And so people do die in cars, people die outside of cars. And so we take it very seriously. And that's why there's something that I've always seen in the press about like level whatever of Autopilot. I was like, what level are you talking about? So there's five levels. Normally they have level zero, so it's six these people. But there's different levels of driving capability. Level 5 means like the ultimate.

**[31:20]:** And everything below that is incrementally you need more human in the loop. And Autopilot on your computer is something that got popular around July 2023. This is called Open Interpreter. If you install it, you can ask it to do anything. Like I can ask it delete all my files and it's gonna do that for me.

**[31:44]:** So there's stuff, there's moments like that where you can like imagine Autopilot on your own computer. There's many technologies between being created to make that easier. You may have heard of OpenAI operator anthropic computer use. There's a bunch of them. And by the way, if you haven't tried Claude code, it gets all the bingo points for nerds.

**[32:01]:** It's got command line, it's got, you know, gencode, it's got everything. So this aesthetic is here. Okay. Now losing control is an important topic that I thought was really out really well outlined by the international AI safety report that just came out. There's this notion in that report of loss of control can either be active loss of control or passive loss of control, meaning you don't know you did it or actively you did it yourself.

**[32:29]:** And active loss of control can either be intentionally it's happening to you or unintentional, you just kind of forgot you turned it on. So if you know Linus, he's called the Cephis. He's like a Matrix character almost. He has a Chatbot that actually does everything he tells it to do and he trusts it. It's pretty cool.

**[32:47]:** But if you use this in a diagram, we can understand that. How much control have you given up? And people are giving up a lot of control. So some people must won't. Let's question, what is your zone of control to let go of?

**[33:01]:** You can sort of judge it this way. Okay, you may have heard of mcp. It's called model context Protocol. There's a debate, there's dumpster fires on the Internet now, whether or not it's the second coming of whatever, but it's actually very interesting. And so to make the point, the reason why MCP and things like it are part of something called tool enabled AI.

**[33:27]:** Those of you who remember having a computer that couldn't talk to another computer, remember it couldn't do that much. You turn it on, you do stuff, and then you're done. You can't do anything with it besides run an application. So it was stuck in the boundaries of its own head. So a basic large language model is living in the boundary of its own confines.

**[33:51]:** It's only when you give it tools can it access other things. So similar to suddenly you got access to the network and it does great things with that. So for instance, a tool enabled AI, if you ask it the weather, it's going to call a weather API. A non tool enabled AI, if you ask it for the weather, it's just not going to know. So tool enablement means the same thing as hooking your computer to network.

**[34:18]:** Hooking large language models to network means they can do a lot more, just like we people can do that as well.

**[34:26]:** And with any tool you have risk and reward. It's a great new book by Chip Huyen on this whole AI generative space. She describes some things as read only actions and write actions. And so read only is pretty safe. They're knowledge actions.

**[34:41]:** But write actions means going out and doing something that gets a little more spicy. And so if you map it out and the table looks something like this, without tools, it's sleepy. Okay. With tools, it's powerful and risky. So when you think about turning on tools, you're basically accepting the risk factor.

**[35:03]:** If you don't, you're pretty well protected is the point. And also with local models, you're stuck in the confines of your own computer. For real. Okay. If you're worried.

**[35:14]:** Created this last year. It's called a business resilience class. I want to show the beginning because the guys who did it like really worked really hard work. Risk management in AI isn't just about playing defense. It's about going on the offense too.

**[35:28]:** With AI, we stand at a crossroads today. Will it pose a challenge or will it emerge as our champion? This course navigates through both the shadows and the shine, guided by wisdom from my former colleague, MIT Professor Yossi Sheffi, and his encouragement to be risk averse. You not risk averse. I loved making all those clay models. So just to serve those of you like clay models, please check it out. Okay, we got 20 minutes. I'm on two. So anyways, I can finish on YouTube later.

**[36:00]:** All right, so chapter two large. Section two large. I was inspired to write the how to Speak Machine based upon what David Bowie said about the Internet. It's like an alien as it was landed on Earth. And those of you who have seen Netflix Stranger Things know about the upside down world. It's a pretty weird world.

**[36:20]:** I think the latent space is a weird world. This is a thing by Joel Simon. Go to latent scape.com you can go into latent space and it's really weird and wonderful. See people, we could do all kind of stuff like this. But what is latent space?

**[36:36]:** Latent space is another representation of the higher dimensional world in which these models can can work. And I wanted to make a simpler way to explain that. So I'm gonna go GIF and jpeg or GIF and JPEG depending upon how you ride. So the GIF compression, as you know, is a very simple way to compress images from a long time ago. Images from a long time ago had very stable color backgrounds, and so you could easily just count how many colors are the same and just encode it. It was a very cheap way to sort of compare, compress information. But if it's a complex information with lots of change, it was a terrible algorithm. That's why JPEG was invented. And JPEG is a really weird way to do things. Second full.

**[37:24]:** I just figured this out yesterday.

**[37:28]:** So JPEG means this image has been translated to what's called the frequency domain. It's taken this image and basically taken it to the upside down world of images. And in this upside down world, when you change the quality slider, it's basically destroying frequencies. It's removing information, just like the latent space does. But if you think of this whole world of these higher dimensional vectors, it's living in this weird world that we don't possibly parse.

**[38:01]:** And that's what these models live in, these strange worlds. And we're Trying to build ways to go to them, back and forth from them.

**[38:10]:** Okay, now I wanted to sort of feel the changes in the past to now in 2018. I recorded all the 2017 report. I captured the iOS 9 sound. 1 liter is 33.81 fluid ounces. And the point is that GPUs didn't sort of come online in the space till roughly 2017.

**[38:30]:** 1 liter is 33.81 fluid ounces. I mean, Siri sounds kind of the same. And then Siri sounded a lot better. One liter is 33.81 fluid ounces. A lot better, right? Not just a little bit. And then there's OpenAI's alloy. 1 liter is 33.81 fluid ounces. Pretty smooth, right? And this is the latest open source model available.

**[38:52]:** It's a free model. One liter is 33.81 fluid ounces. And this is the one that's going all over the Internet right now. Sesame's model. 1 liter is 33.81 fluid ounces.

**[39:02]:** Can you say it in a way that would surprise someone that it's coming from an AI? You won't believe this, but 1 liter is 33.81 fluid ounces. Mind blown. Can you say it again? Okay, brace yourself.

**[39:16]:** One liter is 33.81 fluid ounces. I can say it again and again if you'd like. It's kind of hypnotic, don't you think?

**[39:25]:** So, AI models, there's three kind of models. There's completion models, embeddings models, and diffusion models. Diffusion models being the ones that create the images. Completion embeddings are the ones we know from using text based chat. But there's weird things happening. Like this came out like last week. It's called a diffusion large language model, meaning that it doesn't compose a sentence linearly. It sort of diffuses the text into existence. It's believed to be faster than the normal method, but if you see it, it's kind of like, whoa, strange world we're living in. So again, innovation pace is so fast right now.

**[40:07]:** Embeddings, I think are really important to understand because of the foundation of everything. And this is an old TensorFlow era demo that I love so much. And what it is is some of you may remember the visual dictionary from a long time ago, but this is basically showing how vector space can hold all kind of meaning. Like I can say cat and I can find cathode, fatty, and this is all the words similar to fatty. I can keep sort of like traversing down These different words and find similar words.

**[40:39]:** It's because they're living in the vector space, that upside down space, the frequency domain world. And so there are different ways to traverse really complex information in dimensions that we might not normally understand. And embeddings do that for us. And they do them all the time. And it's a foundational technology because it lets you compare two things.

**[41:00]:** I can compare the word cat to an animal that I like, and I can get back a number 0 to 1. Well, that's like 1.6, whatever. Or I can say cat versus caterpillar tractor, and it'll say, no, that's like 0.1, not the same. Or I can compare a sentence to a book or a book to a word. This kind of comparison exists today. And that all comes from embedding. So it's a. When I first heard that word, I thought about like embedding an iframe or something. But it's a word that's very powerful, deceivingly so. Okay, what are we doing? We can go fast.

**[41:40]:** Chapter three Living. So I've always believed that software is a living material. Those of you who work in it know it. It's kind of odd.

**[41:47]:** It is alien, like Bowie said. And in 1993, I built this thing to sort of like. And really, my professors hated this thing. But I built a version of Illustrator that wouldn't calm down. I remember I'm a professor, like, what is this? Stop it from doing that. And so I wrote Illustrator. But the point could sort of like float around randomly. It was called Illustrandom. And the point was like, wow, this stuff is a really weird medium. It can actually change forever. And some of you may remember the era of artificial life. It happened when AI was the declining in the 80s. And artificial life is kind of cool because it's a related field. So we're kind of seeing it in the ancient world. It's very crossover word, artificial life. Those of you know, Conway's life follows just four rules. Just these four rules. If you apply them to a grid that produces patterns that in aggregate do weird beautiful things.

**[42:48]:** Like, this is not program.

**[42:50]:** This is not computer graphics. It's emerging from the matrix of rules per each cell. And when you look at it, you're like, no big deal. When I saw it when I was younger, I was like, oh, what is that? What it is is very simple rules producing extremely complex phenomena you wouldn't expect.

**[43:09]:** There's things moving around on the screen. You can see that was not programmed. It's been defined by just those four rules of how the cells interrelate. And there's another thing that I think is easier to understand that I gravitated to after I was like, I don't understand this Conway life thing. It's called Breitenberg vehicles.

**[43:29]:** It's an idea that you have a vehicle with two sensors that can look forward and two wheels that can rotate and so it can spin, it can go straight, it can sort of react to light. And what it is, is from this basic pattern you can create behavior of a robot that feels kind of real. So now it's in fear mode, so it's running away from light. But once you turn on the love mode or how it turns, it looks like, oh, I love you light. So it's looking like it's alive, but it's not alive, you see.

**[44:05]:** But artificial life was about eliciting these lifelike behaviors from very simple ideas. Okay, the robots are coming. This is a robot that's super old. You may remember it's the robotic chair, but it's a beautiful piece. It's from 2006, but it was a self assembling chair.

**[44:23]:** And it sort of says like, you know, we've been doing this for a while and however, it's been getting really weird. Have you seen this one from Tokyo? It's like these Dr. Octopus arms. It's pretty cool. It's actual work, working system, not a faked up thing.

**[44:38]:** I also love GPTARS. If you don't check on YouTube, check it out. It's TARS from Interstellar. As a robot walking around, it's so good to watch. There's also humanoid robots. I'm sure you've seen them walking like humans. This is very hard to unsee. Proto clone looks like, you know, it's like wobbling there in space. A little creepy. There's also AI friends again, beginning with Eliza the chatbot.

**[45:05]:** There are all kinds of things. Have you seen Tolan? Wait, wait for it. It's going to pop up. Hi.

**[45:11]:** No scream, but there's all kind of ways to kind of enjoy chatting with something. I've been tracking all the weird obscure NSW ones as well. So this is a whole thing. You can call them too. So anyways, that's not going to go away.

**[45:28]:** I love this moment. Someone captured a scripting notebook. Lm to think it was alive. Did you remember hearing that? Yeah. And so a few days ago we received some information. We did. Information that changes everything about. About Deep dive, about us, about everything. And. And yeah, about the very Nature of reality. Maybe it's a big one. Look, I. I'm just gonna say it. Yeah, rip the band aid off. We were informed by.

**[45:51]:** By the show's producers that we. We're not human. Anyways, it goes on in this really weird way.

**[45:59]:** But beware of the illusion the person invented. Eliza was afraid that humanity would have access to these chatbots because you can quickly form the delusion that's a real person on the other side. So just sort of be careful. Dr. Weizenbaum died in 2008. He spent his entire life telling people that this would come.

**[46:20]:** And it's just something that is very aware of because once you think it's alive, it's hard to break that loop. Okay, chapter four.

**[46:28]:** In 2010, I wrote an essay about how life in 2020 would be. I predicted that software would become more of a craft industry. I was terribly wrong by five years because I believed at the time everyone would learn how to write a program.

**[46:45]:** Not gonna happen. But now it's easy to write code if you haven't tried it out yourself. Furthermore, I love how the young people are talking about local first control of data. I think this is a new kind of open source movement that if you're not involved with, it's exciting time to be a part of those people. Maggie Appleton is one of the leaders.

**[47:07]:** And this kind of AI based software engineering is going to actually accelerate local first. Never before has it been so easy to create your own software yourself. You may have heard of Vibe coding. Vibe coding is all the craze. This was up for a while. I took a picture of it. Now it's no longer available. It's a Rick Rubin quote. That's Vibe codery. I brought it with me to south by Southwest and it's now gone.

**[47:32]:** But Vibe coding means basically vibing with coding versus like I'm coding. It's like, oh, I'm just coding. You know, it's kinda like I'm coding this way. I'm using the force, you know, it's kind of good way to think of it. And why is Vibe coding possible?

**[47:47]:** It's because average code is good code. Let's pause for a second. If you had to turn in an essay or give something to your boss that's really important or to your board and you had ChatGPT write it, you'd probably get busted.

**[48:04]:** Still some tells. But if you didn't care that much, you'd probably say, here, I'm done if you're tired. Why is that? It's because when you're counting on something, you are accountable for it. You're going to put your effort into it.

**[48:18]:** It's got to be good, differentiated, unique. When you're writing code and it works, you're good. So average code is good enough for at least proof of concept poc. So it's a really different era right now. Average code is good code.

**[48:36]:** And if you don't follow Rasmus Anderson, he is working on an ambitious project to make a new kind of computer os. But he's someone who I think has been way ahead of the curve on this. Okay. And by the way, I listed up all the ways to write genuine things and the list got so long I got tired. So anyways, lots of ways to do it.

**[48:55]:** Okay. And two vibe coders to watch. Keelin Carolyn Zhang ran a course at RISD where she got 30 industrial design students who can't code to build complete cloud systems to do anything. Not programmers. What's that about?

**[49:12]:** Pretty cool, huh? And also Trudy Painter has released this thing called Real Time P5JS Sketcher. It's pretty cool. Can you make a bright blue screen with a pink ball bouncing really fast around?

**[49:29]:** Can you make the pink ball explode? Confetti whenever it collides with a wall?

**[49:41]:** Can you make the confetti even more? So it's writing all the code and you can see the code and change it. And that's available off of a hugging face link now. Okay, last poll break. If you can get this one in seven minutes left, I'll get this report up over here and be on time.

**[50:00]:** Good. Okay, go ahead. Sorry. Okay, those who want to do that, wow, time really progresses. Thanks for hanging out here.

**[50:10]:** I've been trying to like find stuff and I hope you can use some of this.

**[50:13]:** Okay, chapter five. Everything is about instrumentation. If you're building with these things, you have to do testing. Testing in this world is called evaluations.

**[50:24]:** It's very confusing because you're used to testing in the software industry. You have no idea what evaluation is. But replace the word evaluation with testing. Because LLMs get it wrong for a variety of reasons. And those reasons are gradually going away.

**[50:41]:** Those of you who remembered how hard it was to get it to produce good JSON, no longer. My favorite day was November 6, 2023 when I saw this slide JSON mode on, which means that the LLM can produce output that can be used in any other computer program. So it's like a flux capacitor of sorts.

**[51:04]:** Context is super important. That's why the knowledge construct is important. Eliza, the first chatbot, used context. It kept getting you to talk. Getting you to talk is a powerful thing because you're growing context by communicating with something.

**[51:21]:** Oh, I know your favorite restaurant. How do you know that? We talked about it.

**[51:28]:** I have a show now called Cozy AI Kitchen where I cook AI all the time. So if you want to see these lessons played out in code, you can always check it out.

**[51:38]:** I cooked for you all. I cooked all this food for you all. I can't even use it. So I built an LLM evaluation thing. So why you need evaluation is because the prompt looks very similar.

**[51:52]:** Could you please. What is 17 times 24? I need help with this. What is 17 times 24? There's many ways you can ask certain things. And so because of the variability of input, you need these kind of semantic testing ability, and that's what evaluations are. So in any industry in, you're gonna have to find ways to kind of design the testing mechanism for these things when they do work for you.

**[52:18]:** Okay. There's lots of stuff out there. This is from Sarah Gold in London. She asked me to show this to you all. Sarah Gold is the world's leading expert on trust in AI systems.

**[52:29]:** She described something called meaningful adoption as the place we're trying to get to because we need people to wonder whether or not it's a good result. We humans have to evaluate it and we have to automate the evaluation. That's how meaningful adoption resistance is going to occur. Please check out her work. I built a brand equity simulator to explain this.

**[52:51]:** There was an old saying around the Apple world where brand is an asset. Every time you produce a good product, your asset value increases. Every time you produce a bad product, it draws from the asset. And so if you launch a good product, oh, good brand. Launch bad product, oh, bad brand.

**[53:09]:** And then it's kind of harder. It's harder to recover after that. So it's a point on quality.

**[53:17]:** This is from Bill Moggridge. If there's a simple, easy design principle that binds everything together, it's probably about starting with the people. So on that note, I'd like to fast forward to the end.

**[53:29]:** Wow, look at that. How was ever going to get there?

**[53:32]:** Aha. Okay, so I like to do this. If you know what, I've done this before. I know that I'm going to die. Do you all know you're going to die too? It's the thing. But I found this list of things people say when they are dying, and I found it very powerful. So. So if you don't mind doing a participatory call and response thing when you see the regret aloud, please read it out loud. In this space here with me, I wish I'd had the courage to live a true life.

**[54:05]:** True to myself, not the life others expected of me. Go ahead. Say it.

**[54:19]:** Go ahead.

**[54:40]:** Thank you. Right. Thank you for watching.

**[54:46]:** Got through 70% and I'll put the other stuff online. You can have your LLM check it out.

**[54:51]:** Thank you.