AI Town: A Virtual Town Where AI Characters Live, Chat, and Socialize

--- type: transcript date: 2023-08-22 youtube_id: -xqhyAge6J4 --- # Transcript: AI Town: A Virtual Town Where AI Characters Live, Chat, and Socialize in this video I'm gonna be showing you AI town which is a new open source project that allows you to create open source characters where the characters will go amongst their day and have conversations amongst one another so if you see here you can see that Bob started a conversation with Pete and you can just go through and see the conversation that they're having right now so you can see other characters are walking around and looking for conversations to be had so it's sort of interesting because you can Define your own characters and their Persona and then sort of set them free within the little AI town and see what conversations they have so say if your favorite movies like Harry Potter or Lord of the Rings or something imagine if you plugged in those characters in here it'd be interesting to see the conversations that they have right is it accurate is it closed is it what you think it'd be sort of an interesting experiment right so the first thing you're going to do is you're just going to head over to the open source repo here so it's a project by Andreessen Horowitz so if you just go down to the repo pull the code down now the instructions are really great I encourage you if you miss anything within the video I'll be showing you step by step the different pieces to do but you can also just reference this as a good place to just run through everything so once you've pulled it down make sure you npm install and then the first time that you actually npm run Dev in your vs code once it's all pulled down it is going to fail and so what you're going to have to do is you're going to have to set up your convex account and then once you have your convex account you can go to the dashboard here and then you should see an AI Town project here so once you go within the project this is going to be where we put in a handful of things we're going to be putting in a handful of environment variables and then this is where the project and the back end is all stored so this is being updated in real time as the conversations are coming in and then it's also going right back to the front end of the application so it's really nice in how it's set up where it's in essentially you can think of it as in real time as it's updating in the back end it's streaming right to the front end all the different responses here so what we're going to be doing within convex is I'm just going to have you go and go over to the settings tab here and if you scroll to the bottom you'll see the environment variables so we're going to have two places of where to put the environment variables and I'm just going to be running through these relatively quickly but they're pretty straightforward so so long as you know where to get them you'll be able to easily get an API key like you just have to make an account for these Services most of them you're actually all of them you're not going to have to put in a credit card off the bat if you haven't used them before and you'll be able to set this up and play around with it for free now the one thing to note if you do leave this running in the background like I have right now you are going to be incurring increasing llm token usage from open AI but as soon as you turn this off it's all going to stop and whatnot so the first thing that I'm going to have you do is we'll just go ahead and go over to the openai website and get an API key so it's pretty simple just go log in make an account go to the top right hand corner click API Keys grab a key and then we're going to put it within convex just like this now the next thing we're going to grab is the pine cone environment variables that we need so if you haven't used pine cone before it's a vector database now the one thing to note when you're setting this up is if you just call it let's say AI town you're going to have to set the dimensions to 15 36 so that's the dimension of the open AI embeddings endpoint so depending on different endpoints they can be a different dimension size so just make sure when you're setting up the index that you specify that number here so once that once that's set up the index will take a minute or two to set up you can head over to your API keys and then you can plug all those in within the convex dashboard here so you'll have your index name which would be AI town you have your API key and then you have the environment itself so next the project uses clerk to be able to have authentication so if you just went to the local host or once it's deployed you actually need to log in to be able to see the the different conversations so it has that ability to have different users and clerk is one of those authentication providers that you can use to have all your user authentication flow set up so once you've made an account on clerk you can go over and create a project if I just head over here nope let me just go back to home rather so this is my or rather AI 10. so I'm not as familiar with Clark so once you're within the project itself you can go down to the API Keys like I showed and then once you're within the API Keys there's a handful of things that you're going to grab here now the thing with these different API keys so just make sure that you actually show it I'm not going to show mine here but we're going to put this one within our vs code here so I've broken these down these are the API keys that we're going to have within convex and these ones at the top are going to be the ones that we have within our local project now once you have vs code all open and everything make sure to put them in the env.local I just have a DOT EnV just to show you and hide some of my secret secret Keys here with a vs code extension that I have so when you get started you should have your convex deployment you should have your next public convex URL and you're just going to be pasting in those two values here that you just grabbed so those are the two that you're going to need in your Dot env.local and then the rest are going to be the ones that you plug within the convex interface so I just put these here just for reference just to have them so I don't have to constantly reach for them and so we've grabbed our openai API key put that into convex we are going to grab our clerk issuer URL next and then replicate is an optional one for if you want to generate different background music so what we're going to do is now within clerk if you just go over to JWT templates you can make a new template I just called mine or if you just click convex here you'll see that you can create a name so I just named mine convex I'll just go back to mine here and then what you're going to be grabbing here is the jwks endpoint so you're going to be taking that and putting that within the convex dashboard just like this and then optionally you can make an account for replicate if you want to have that background music like I mentioned so again super simple you can just go over to replicate create an API key and then copy it in and paste it in just like you see here and that's pretty much it so it takes a moment just to boot up to get all the convex functions running in the background but once you have that set up you can essentially go in and start to make tweaks to your project so the one thing to note is a project like this with the 2D characters it is using Sprite sheets so if you wanted to create your own characters you know maybe you could look into something like mid-journey or a different image generator and try and get your own Sprite sheet where you have your own different characters so I tried with mid-journey I didn't get exactly the template of this where you have the arms moving and the different positions that is sort of rotating through as the character is walking you know up or down or sideways and all of that but I'm sure with some refined prompts you could generate some pretty good templates for different characters that you wanted so just a thought on that if you wanted to edit the characters themselves now if you want to edit the data is you can go within this typescript file here and you can just go in and change the characters so if I say Harry Potter and you can say you are a fictional character whose name is Harry Potter they're friends are Etc so you know there's my GitHub a co-pilot being a little helpful here so your Harry Potter their friends are Ron and Hermione you are a wizard you go to Hogwarts Etc so you can go ahead and save that you can Define different relationships um you can go in and set their initial position and stuff like that so you can just go through this array of all the different characters and you can plug in the different features and sort of persona of all of them so those that's sort of the main piece of where you want to edit that sort of core initial logic before they start to have conversations with one another but if you want to go ahead and sort of make this your own you can just uh search the the directory so I encourage you for open source projects if you're just looking to tweak different pieces and just sort of you know show off like look here's a little AI town that I built and you just want to make it your own sort of with some lipstick so to speak you can just go in you know change the title you can say developers digest for the title and you can say this is this is Hogwarts and then if you just go back to your AI town here and if we refresh so that was another thing to note is if you're looking to actually edit the characters and you've run it already there are some commands to go in here and clear the different database pieces that you need to be able to do that so I'm not going to be showing you all that in this but you can just look through here it's near the bottom I believe where you can clear and reinstand reinstantiate all those initial relationships within the back end so if I go back to my AI Town let's just have a look so you can see now it's uh developers digest this is Hogwarts you can click the different characters and if I click around enough there's Harry Potter right so there he was he doesn't quite look like Harry Potter but you can you know start to get an idea on how you can play around with something like this so in terms of other things within the project I just encourage you to go ahead and explore it if you're using vs code I found you know just command shift F the different pieces that you're looking to find so say if you're trying to change AI town like I had there you can go in and tweak it within the title or like I had within the H1 here so just wanted to show you a quick one interesting project so I cover a lot of you know text generation llm projects on this channel but I thought this was a new interesting approach where you have that text generation between characters and then you also have that visual component so really interesting idea and I encourage you to check this out and pull down the repo and play with it if you're interested but as always if you found this video useful please like comment share and subscribe and otherwise until the next one