Conversations on Applied AI - Stories from Experts in Artificial Intelligence

Nick Walton - Using Artificial Intelligence to Augment Human Creativity

March 09, 2021 Justin Grammens Season 1 Episode 15
Conversations on Applied AI - Stories from Experts in Artificial Intelligence
Nick Walton - Using Artificial Intelligence to Augment Human Creativity
Show Notes Transcript

I’m so happy that Nick Walton took time out of his busy schedule to join us and talk about human and AI collaboration. This was one of my favorite conversations on the Applied AI Podcast! Nick is the CEO and founder of Latitude, a company working to develop unique, immersive virtual experiences with cutting-edge machine learning technology. Nick attended Brigham Young University, where he received a Bachelor of Science Degree in Computer Science and since graduating has been working in the areas of autonomous driving and as an AI research assistant before starting Latitude in 2019.

Latitude is currently hiring!

If you are interested in learning about how AI is being applied across multiple industries, be sure to join us at a future AppliedAI Monthly meetup and help support us so we can make future Emerging Technologies North non-profit events! 

Resources and Topics Mentioned in this Episode

Your host,
Justin Grammens

Nick Walton  0:00  
I think we're gonna continue to see especially human AI collaboration is going to increase because I mean, what a great tool to get rid of writer's block, for example, right? Like you, you don't really know what to do next. So you feed into an AI how to generate five possible things. And that just gets the wheels in your head spinning. That's my view on AI in general, is I think we're gonna see much less AI completely replacing humans and more AI augmenting humans, because I think that collaboration is even more powerful than either one alone.

AI Announcer  0:26  
Welcome to the conversations on applied AI podcast where Justin Grammens and the team at emerging technologies North talk with experts in the field of artificial intelligence and deep learning. In each episode, we cut through the hype and dive into how these technologies are being applied to real world problems today. We hope that you find this episode educational and applicable to your industry and connect with us to learn more about our organization at applied Enjoy.

Justin Grammens  0:57  
Welcome to the conversations on applied AI podcast and this episode, we're speaking with Nick Walton. Nick is the CEO and co founder of latitude, latitude develops AI powered games designed for players freedom and self expression. And they are the creator of the ever popular game named AI dungeon. Nick attended Brigham Young University where he received a Bachelors of Science degree in computer science. And since graduating has been working in the areas of autonomous driving, and as an AI research assistant before starting latitude in 2019. Thanks for joining the call, Nick. Yeah, happy to be here. Awesome. Well, I'm sure we'll talk a lot about latitude and AI dungeon and stuff like that during this this episode. But maybe one of the things that I wanted to start with is, you know, how did you get interested in this field in the first place? Well, so AI in general, I've been interested in robotics for a long time since kind of the start of my bachelor's, I actually started out as a mechanical engineering major got interested in self driving cars, and eventually decided to switch to CS because I was more interested on kind of the deep learning side of things. So then interned at a couple different self driving car companies, including the latest was avora innovation, which was started by Chris Urmson. He kind of led Google's self driving car group for several years, right in the Utah area. No, that was in Pittsburgh. Okay. Well, they split between Pittsburgh and SF. I was in Pittsburgh, okay. Okay. Yeah. So then I kind of stumbled into a dungeon. I was at a hackathon and playing around with some different deep learning stuff. And I saw that you would come out, and they just have the smallest model. So I downloaded it and started playing around with it and was like, Hey, you can do some interesting things with this and had recently started playing d&d. And so I was like, Hey, can we make like an AI Dungeon Master. So built basically a prototype by the end of the hackathon, that was pretty fun. And then I just kind of worked on over the next several months. And then when the largest version of GPT two came out, I used that I found some data to fine tune it on, and then released it as just Python code that you could run in a co lab notebook. And then that got probably like 100,000 plays in the first week. And so that's when I was like, Whoa, like, it just exploded. And so that's when I realized there's something here worth kind of exploring more into. And I turned down a job offer I had for going to work at Aurora so that I could work on this full time as a starting latitude. Very cool. So what I mean, was it started because of AI dungeon, or had you guys already been kind of experimenting around with other games? No, it was started because of a dungeon. So my co founders, my brother, we'd always talked about, like starting a company someday. And we had talked about like, oh, it'd be cool to build, you know, these kind of infinite world games. But we both kind of expected that to happen in like five to 10 years, not so soon. And so it was kind of like a trigger of like, Okay, this is the opportunity if we ever wanted to do something like this. So we should jump on this. That's awesome. Well, I commend you guys for taking the leap. And well, honestly, taking a risk, right. There's a lot of people that don't start companies just out of honestly, just fear. Yeah. I mean, it was a little easier for me where like, I didn't switch to deciding to start a company until we already had this cool thing that had 100,000 users, right, that makes it a little easier. But I mean, yeah, there is still risk are the 100,000 paying users. No, they were Yeah, no, they weren't paying. So one of the things that gave me enough confidence to jump ship was, we did have quite a few Patreon supporters, like our Patreon got into probably the top 100 of Patreon, within like, a few weeks, and so that kind of gave us confidence that like, okay, people are willing to support us, and this is something we can go forward on. Nice. Well, so for people maybe that don't know about AI dungeon, could you give just a quick sort of synopsis of what it is? Yeah, so it's basically like an old text adventure like zorich or Colossal Cave, but the difference is, rather than pre programming every possible action and the outcome, we instead use a natural language AI that's been trained on text adventures to say, okay, given this history and your latest action, what's what's going to be the next thing that happens? And so it's

generated dynamically every time you do an action. And there's two cool things about it. One is that allows you to do any possible action. Whereas before, like, I don't know if you've played zorich. But it's like, if you don't do the right action, it's like, sorry, I don't understand what you said. Right? Right. Whereas in a dungeon, it almost always, like virtually always can think of something that happens. And so that's one of the cool pieces. The other piece that's cool, is the experience is completely different every time. And so you get this cool phenomenon where you have a different experience every time. And then because your experience is something no one else ever had. You share it. And so that's kind of how we got huge user growth is people sharing their cool adventures? Because they're so different each time.

Yeah, very cool. Now, you mentioned GPT. Two, is that what you guys are using for this?

So we use GPT. Two, up until a couple months ago, when we switch to mostly using cheap d3. So now we have we have kind of two model tiers. So free users use a model that we call Griffin, which is basically in between 52 and 53. And then dragon is like the full GPT three model. It's a massive model, but the quality is massive.

It's been all over in the news that it's almost the GBT three is almost like scary how good it is. And that even were you. I mean, is it? Is it publicly released to everybody? Or were you guys were able to access it first? I guess I was curious. Yeah.

I mean, so so we were the first kind of, as far as I'm aware, the first production application built off of GPT. Two. And so because of that, even before open AI had even launched their API service, we were chatting with them about, you know, what would we need to make it work? So we were one of the earliest testers even before it was announced. And so yeah, we've been we've been using GPT. Three for for a while, but it was only after they announced it that we like officially made it, you know, the premium feature of AI dungeon? Well, and what

is the status today? Can Can anybody go out and get three? Or 63?

Yeah. So no, you have to apply to kind of a beta program for open AI to get access to it. Okay, I think the waitlist is pretty long for them, because they've had so much interest that they're just trying to, like, digest that.

Yeah, for sure. I've heard it's a pretty costly model. And maybe you guys got and I don't need to interrupt, you know, you understand your guys's terms with them. But I mean, I've heard that. I mean, obviously, they've done a lot of r&d, and they've they've done this thing, so they need to make it up. But I've just I have been reading that it's gonna cost companies a fair amount if they want to use their API's.

Yeah. And and with good reason, I think to a degree, it is the most expensive AI model to run than anything you can kind of commercially access, right? Like, if you look at anything that Google or Amazon offers some machine learning API, like GPT, three costs significantly more to run just in terms of compute. And so I'm not I'm not super surprised, like I'm, we deploy GPT, two at scale. And so we have, at least for GPT, to have a bit of an idea of what those costs look like. And I think the other thing that's valuable for us from open AI, is, they're gonna continue to do tons of research and development and improving these models. And then we can just kind of plug those into a dungeon and get the benefit there.

Yeah, no, that makes a lot of sense. That's I know, I've talked with other companies just really around the idea of Yeah, who's gonna make your model better? Do you really have the time and effort and expertise? Or is it better just to make more of a SAS play? Right, so these guys offer new services, and they obviously have PhDs and in speech and audio and all that sort of stuff? And in this case, text, I guess, right,

we're kind of hybrid. So like, we do a lot of ml development on our own, especially with like, kind of like auxiliary systems. So we have the main we have gt three do kind of the main story generation. But we also have things like quest detection, where we have a model that can detect whether or not you completed a quest that's separate. So I expect us to continue to do our own ml development, but we'll be able to focus more on what are the things that are AI dungeon specific, and let open AI improve the things that are generic to anyone using GP three.

And so to be clear, you they're running the model on their servers and stuff like that. You guys are just using their API's?

Yeah, okay. Yeah.

Which is what I thought that that's all that they allow right now. Right? Is this only API?


Cool. Well, I mean, so this whole idea of text and being able to write stuff, obviously, you guys are using it, you know, to sort of simulate what a dungeon master would do. What are some other I guess, just thinking bigger picture? Yeah. Have you seen some other interesting applications of this, since you're sort of in the field? And when working in this space?

Yeah, it's interesting. I think there's a lot of cool promising demos that it may take time to actually be effective. It's like there's interesting demonstrations of like creating a web app, create React Native components. Yeah, I would have to play around with those to know what the quality is because obviously, it's a little bit hard if you can only see the cherry picked examples, right to fully tell and no, my expectation is that that will still take a bit more work that I think we're kind of the Perfect application because one of the issues with GPT. Three is it's great at generating text. That seems like it could be right. But it's also good at generating text. That seems like it's right but isn't right. And so I think entertainment is perfect for this because like for a dungeon, doesn't really matter if the AI says that Germany is in South America, or give some some weird answer. But if you're trying to do applications that need to be cracked, I think there's still some some potential issues there with with usage that people will have to figure out before it's able to extend fully to those use cases.

Good point. Yeah, that makes a ton of sense. I mean, I think there's always been this thinking that still humans are more creative. I guess that that the that the creative aspect, whether it be music, or art, poetry, or whatever, still needs to be done by a human and couldn't be done by an AI. But it feels like in some ways, those are starting to be disproven. I don't know what what you've seen or what your thoughts are on that.

Yeah. I mean, it depends. There's like different types of creativity, right. So I think I think for things that are truly new, those will be things that it will be humans that do because largely these creative, AI's are trained off other things, right. It's trained off books, it's trained off texts. And so I think to do things truly new, it will likely be humans. But I think AI generating things allows completely new creations. So like, the way we think about it is we're kind of reinventing how games are made. And we don't see it as AI replacing game designers, but rather augmenting them such that rather than needing a team of 100 people to build this massive open world game, you might only need like five, who are kind of the directors of the game, they can kind of scope out the world and the lore and some of the larger pieces. And then the AI can fill in all the detail. Right? And so I think I think we're gonna continue to see, especially human AI collaboration is going to increase because I mean, what a great tool to get rid of writer's block, for example, right? Like you, you don't really know what to do next. So you feed into an AI have it generate five possible things. And that just gets the wheels in your head spinning. I love my view on AI in general, is I think we're gonna see much less AI completely replacing humans and more AI augmenting humans, because I think that collaboration is even more powerful than either one alone.

Yeah, good points. good points, for sure. And then of course, that's going to percolate. I mean, I always sort of asked people on on the program, like how do you define AI? I mean, if you actually had I don't know, if you have a standard textbook definition, everyone has has their own. But if you have an elevator pitch your definition for it?

Yeah, that's a good question. And it's relevant, especially in gaming. When I say AI, and what people in traditional game companies say ai ai are usually very different things. So they usually think of like, you know, pre programmed rules for NPCs that make them seem like they're acting human ish, but it's not machine learned. And I guess my definition is, generally, there's usually some learned component where it can do informational tasks that we typically associate with humans. But I'm not saying my definition is the right definition. But that usually when I think of AI, that's kind of what I think

I was kind of thinking about, you know, I don't know, if you've seen sort of those pictures of like, you know, procedural programming that we program has always been done is is, you know, there's basically a series of tasks and some logic, and then there's an output that happens and sort of this idea of machine learning, deep learning sort of flips that model around where you're just feeding in data now, you know, and it's basically learning it. And the output is basically the rules and algorithms because of it. They don't need to be programmed, you know, it seems like the machine is going to evolve over time. I mean, I'm assuming what you're seeing with AI dungeon, you're saving the responses and figuring out how people responded. I mean, are you looping that back on itself at all inside? Yeah,

we're still building that piece. We don't currently do that. But we are gathering the data and doing that with the expectation that we will do that open. I've done some really cool research lately around using reinforcement learning to improve text models. And so I expect us to be using that within the next couple months, strategies like that. And I think that really does open up a new door. Because what happens when you have a game that can optimize itself for player experience, right, they can continue to learn from what the players are enjoying and improve the experience. I think there's a lot of really powerful things that can happen there.

What tools do you guys use internally? You know, are you guys like a tensor flow shop? Are you guys using pytorch? Like, what are some of the tools that you've used even? And whether or not even at your existing company or the stuff that you've been working on in the past that

people should maybe explore? Yeah, so I mean, the ML people on the team, we've used both TensorFlow and pytorch. Generally, where we like we don't have one we necessarily always use because one thing that's different I think, with using large language models, is often what you're doing. And this is quite different from a lot of traditional MMO development is rather than defining your own model and train your model from scratch on data, you're using these massive pre trained models and then fine tune off of those, sometimes we'll just use whatever those are written in because we want to try out this cool new model and experiment with it sooner rather than later. One of the models that we've we've found really successful for general purpose language tasks is t five. So t five is what we use for request detection. It's different from GPT, two and three in that it's sequence to sequence. And so it's better at doing things like fill in the blank, or, you know, answering questions, at least from our experience. And so, yeah, those are some of the things we use cool.

Have you guys played around with or, you know, thought about service transfer learning? Have you taken a model trained on something else, and then try to apply it to where you're at, I guess, that seems to be like the holy grail of generally,

using pre trained language models, and then fine tuning them is essentially like transfer learning. I think the difference with like safe using transfer learning with vision, often you do stuff like freeze weights at earlier levels, and then only train the top level weights. Generally, with fine tuning, you don't do any weight freezing, you just have this model that's trained on its own text, and you just train it a little bit more on your specific data set. So we do that a lot. Yeah, that's what we do with quests. And with GPT, two and three, I think there's a lot of power in pre training language models on general amounts of text, and then fine tune them for a bunch of different types of tasks.

All right, cool. Yeah, that makes that makes a ton of sense. I mean, what you guys are using is really more around fantasy, right? So fantasy adventures, is kind of how you guys would be some of these tuning things. I mean,

yeah, we do any type of story. So it's just it's more of like second person adventures, because it could be sci

fi, it could be modern. Makes sense. I yeah, whenever I think of dungeon, I always think of a course of Dungeons and Dragons. And that's, that's where my mind goes to. But you're right, I guess these stories can evolve. However, they're gonna evolve.

Yeah, yeah. And and fantasy is probably the model performs the best because that's where we have the most data. So

when I use the site, and was experimenting around with it, you know, I've been kind of following you guys for a little while here. You know, it feels like it's me and one Dungeon Master, you know, is there a group component to it? I guess, with regards to more of a massive, you know, online game? Yes, you guys, you guys are either building out or, you know, we're gonna be multiple people.

We do have a multiplayer right now. And so that's more of like, you know, you've your d&d party, we're still working on improving how that experience is, in the long term, we'd love to do something where there's like, a shared world that a huge number of players can play in. But we're still figuring out what that looks like on a technical level. Sure,

we touched a little bit like on tools and stuff like that, and you're deep in this field. I mean, if someone is five years younger, or 10 years younger, and they're looking to get into this field, you came up through computer science, I mean, is that is that a good place? Like what classes should people maybe take along the way? are college and universities in your mind actually even doing some of this good stuff? Or have you really learned a lot of it more post school? Just kind of curious what your thoughts?

Yeah, I mean, like, so universities in general can give you a good CS background, which, which is valuable. For me to kind of get into the deep learning, it depends, like you have to find the right pockets within universities. And some universities may not have a pocket that can help you get in that. So for me at Brigham Young University, there's, there's one professor David Wingate, whose lab is super strong on deep learning, and just getting into that lab and connecting with all these really bright students is really what made me be able to come up with a dungeon and have kind of the experience and know what to do. There's multiple people from this lab that work at Google brain or work at self driving car companies or work at you know, Nvidia doing deep learning. But in general, if I didn't find that lab, I don't know if the university would have like helped me that much. I think it's a matter of finding the right pockets of people who you can collaborate with and learn from and just be encouraged in in going in that direction.

Sounds like this was born out of a hackathon. Is that is that right? Yep. So it feels like there's a good aspect of getting challenged, I guess, of going through hackathons. I've run quite a few in the past. Yeah. And so I've always found those to be like, really interesting. And actually, a couple startups have come out of our hackathons here. In the Twin Cities.

Yeah, no, I'm a huge fan of hackathons. But it's kind of like, Are you familiar with reinforcement learning? There's kind of like greedy verse exploration balance. Uh huh. Yeah. So it's kind of like that. We're like, sometimes in our careers and life, we do greedy optimization where like, okay, where do we want to go? What's the clearest route, we know how to get there. But sometimes to be successful, you really have to do that explorative route, because if you don't do exploration, you may not discover these cool new paths that no one has even told you about or that hasn't existed, right? So if I hadn't been playing around at this hackathon, I didn't expect to make a company out of it. Even after I built a dungeon I didn't see it as anything more than a side project until it kind of exploded but I think there's a lot of value in just doing things for fun because you enjoy it and maybe something awesome will come out of it and maybe not but like there's still so much value and just building for fun.

Good point. We're kind of talking about life and and having it all be about experiences and the more experiences you can have. Obviously the richer life you're going to have a but also be More people you're gonna touch right? So I like those sort of greedy versus non greedy.

Yeah. And I think there's something to be said for if you approach entrepreneurship from, I want to be an entrepreneur, and what's a good idea I can have, I feel like you almost always come up with an idea that may be valuable. But there's probably five to 10 other companies already working on that are doing that. Whereas I feel like a dungeon when I built it, literally no one had ever done anything like it like it was so so different. And I don't think that's something I could have come upon. If I'd set out from the outset with I want to build a successful company. I think it's the it's the kind of thing that only could have been discovered by just playing around for fun in areas that like people hadn't really played around in before.

Yeah, there's a good book, I know if you've ever read it by Linus Torvalds, the guy that started Linux, I think the title book is just for fun. Oh, interesting. And it was written a long time ago, I remember like reading it probably at least 1015 years ago. But his whole concept is just in general technology needs to go through this playing phase, where people don't really know what it's going to be used for. But they know that they enjoy it. And that, you know, that it might have a usefulness in the future, but it just kind of remains to be seen. And it's a really good book. And it's it's really about in a lot of ways, you know, how he built Linux, because it really was sort of just a fun project, not really knowing where it was gonna go. And, of course, he you know, basically pushed a version, you know, up, Hey, I got this new this, this kernel thing that I built, and all that stuff just completely took off like wildfire. When I published this podcast, I'll have notes and stuff like that. And I'll mention that book to our listeners. But yeah, I would highly recommend checking it out. Totally agree. So AI dungeon is like your main focus? I don't know. Are you guys exploring other other games in this field? As a company? Are you guys I guess, looking at some other things too, as well.

Yeah. So we're still focused on AI dungeon, but I think AI dungeon is still a small percent of what it can be. So a lot of traditional games studios have kind of a design a game, build it, release it, maybe add some small polishing, but they don't really change the core gameplay at all. Right? It's very much a waterfall release cycle. But we develop very differently. We're very much like an iterative. So like, we're constantly adding new things to a dungeon that kind of fundamentally changed the gameplay. And I think that's how you have to develop to build successful AI powered games. Because if you have a three year waterfall cycle, and you use the AI from three years ago, like if we were using the language API from three years ago, a dungeon would not be a thing. It would be like these sentences that maybe look grammatically correct, but aren't really interesting. It's like, rather than building a car where you design every system, and then subsystem and then component, and then you put it all together, it's much more like gardening, or like growing a plant where you like cut places here, you you encourage you to grow here, but you can't fully control. And you have to do a lot of exploration. And so even now, internally, as a company, we do regular hackathons, because it really helps us to explore new ideas and new concepts and just play around because there's such a wide open field of AI generate games that no one has explored yet.

Great. Yeah, that's cool. So how big is your team?

So we're at about 12 people full time now.

Cool. Are you guys, I'll put links to your guys's website and all that all that stuff as well. Are

you guys currently hiring? We just finished a hiring round. So we're probably not going to hire for the next few months. But we will probably be hiring starting again in like January. So beginning of next

year. Yeah, I mean, I know a lot of our listeners are in this field are looking to get into this field and have lots of different experiences. I mean, obviously, having maybe experience with some game design would be preferable, but maybe not required, right.

Yeah, I think we're looking for people who have interesting skills that are related, but also people who are Packers, I would say like people who can build cool things quickly, and don't need to like make it perfect before they release and don't need to like have this really slow traditional game development cycle. Because that is one danger. I think with people from traditional game companies is I think this this kind of cycle and way of working is very different. But yeah, definitely, definitely interested in in talking to people who are excited about AI or have experienced with AI or game design or art or some combination. Yeah, I expect will grow a lot over the next year to our long term ambitions are pretty are pretty huge. And so we'll continue to grow as we build a team that can build the games of our dreams.

That's good. That's good. Are you a gamer at heart? I guess Have you always been a gamer? Yeah, I

would say so. I especially love open world games. And so I think that's one thing that like, kind of led me to make a dungeon is I love games where you can start it in a new place and kind of explore and make your own story and make your own choices. So like Skyrim is an example I often use of like, the kind of game I love. But also it's a good way to point out what the limitations of current game development are. So like Skyrim has this awesome open world. But everything is handcrafted. So it takes like 100 developers years to build and also it's the same every time so like There's only so many times I can start Skyrim again, and start in the same cart and go through the same cutscenes, you know, and do the same things. And also the characters. Like, there's only so much you can add the characters. So you in Skyrim, you see all these guards that have kind of the same lines, they say, or you very limited dialogue with certain characters. And so it kind of in some ways, takes away the feeling that it's a real character. Whereas if you can say anything to a character, and they respond in a way that seems realistic, that's where it feels like they're like a real character, right? rather than a pre programmed NPC.

Yeah, for sure. If you saw the movie, her HDR, this guy falls in love with this with this AI voice. But there's just there's a It feels so real to him, obviously. And I think it's, it's really good. And I guess that's where my mind was sort of like going to, there's a scene in there, where he's like, how many other people are you talking to? Like, how many other people are you in love with? And she's like, Oh, you know, 3580. And so the guy is like, Oh, I'm not the only one. And it's really, in some ways, it's sort of this idea of scale of like, hey, this AI's can can create such a whole different experience on such a different level that one human could never do. Yeah. And so I see games, like you said, sort of like being able to feed these infinite combinations of experiences that people can can go ahead and do just by the sheer fact that they have so much built up knowledge, and it's not a bunch of if then else statements.

Mm hmm. Yeah, yeah. And it enables a lot of really cool things like characters that feel alive, are worlds that actually change with respect to your actions, right. So like, you may take one action in a world and have a completely different result than another action. Whereas that's just not really possible with today's game development, because you would have to create each world that changes based on the actions and that's just so much work. Whereas with an AI, you can let the AI fill in the details and extrapolate what happens if you do X, or what happens if you do y? Absolutely.

What's the best way for people to get a hold of you?

If it's people who are interested in working in latitude, just careers that latitude? die? Oh,

sounds good, cool. Any other things that you wanted to talk about? I mean, we covered a fair amount of ground here, but I always sort of like, leave it open for the guest.

Anything specific, I think I think we're just really excited about kind of the future of AI and games and how that will change things. And I think we're gonna see that over the next five to 10 years that AI is going to massively change how we build games, and what those experiences look like. And so we're really excited for that.

Very good. Well, Nick, I appreciate your time. Thanks for being on the applied AI podcast and we look forward to following you more and more as you and the company grow. And you guys create more and more games in this exciting new world. Awesome. Thanks.

AI Announcer  27:36  
You've listened to another episode of the conversations on applied AI podcast. We hope you are eager to learn more about applying artificial intelligence and deep learning within your organization. You can visit us at applied to keep up to date on our events and connect with our amazing community. Please don't hesitate to reach out to Justin at applied if you are interested in participating in a future episode. Thank you for listening