How I Built a Customer Support Chatbot in React with Bee Agent Framework

--- type: transcript date: 2024-12-23 youtube_id: hH7vY04LRZ4 --- # Transcript: How I Built a Customer Support Chatbot in React with Bee Agent Framework in this video I'm going to be showing you how to build out a customer service chatbot within this video we're going to be using the B agent framework this is a project that was started by the team over at IBM and what's unique with this framework is that it's built natively for typescript if you're familiar with the AI agent framework landscape you'd probably know that a lot of these tools are generally speaking python based as a typescript developer myself I was really eager to try out this framework to see how well it would work for various use cases and honestly this is probably of the easiest ways on how you can get started with building AI agents if you're a typescript developer now the other thing that I really like about this is that you can use the Llama series of models you can use the brand new llama 3.3 which I'm going to be showing you in this video we're going to be leveraging Gro in this demonstration we're going to be using the new llama 3.3 70b speculative decoding model and this is going to make our agent incredibly fast in terms of some of the key features there are a number of built-in tools that you can leverage there's a code in interpreter within the framework there's a number of different implementations of memory as well and you also have the ability for serialization if you want to have complex workflows say if you want to pause particular workflows and then resume them at a later time you'll be able to do that you also have instrumentation what's nice with this is you have the full visibility on what the agent is doing and that's one thing that I really love about this framework because what it allows you to do is if I look at the example that I have here I built out this capability to see all of the details of what is happening before it gets to the final response there here we see that it has a thought the user wants to know the location of the Seattle store I can use the get store locations function to find this information then based on the tool name we're going to get the argument for that tool then we have the tool output what the location is and then we have the final thought I have the location of the Seattle store and then we have the final example here the Seattle store is located at the following the nice thing with this framework is it really is batteries included you can Define the functions on what you want to have the agent do and then it already has all of the pieces set up for it to Think Through reflect it has all of those mechanisms that you would have otherwise had to set up it passes the context from the tool it also has built-in capability to pass the tool output to have that thought and all tied in within the context of where you get the final answer basically does all of the plumbing of connecting the different pieces of the tool input output all the way to having that be a coherent answer at the end of it in this video I'll show you how you can create these conditionally rendered UI components and also how you'll be able to get answers just to show a couple of the demonstrations if I say I need help right now who can I call there's a function within here which it will determine if there are Urgent requests go to a phone number if it's not as urgent send it to an email so if I say how much is shipping beans to Europe I can send that through and then within here we'll see okay here's the shipping cost we also have how long it takes and you can ask it about various regions as well another feature within here is I can say I want to join the coffee club and if I submit that it renders this modal join the coffee club and what you could have this model as is it could be something that gets sent to an endpoint for your mailing list you can have things like the frequency their roast preference and all of that while this does a number of different sort of custom functionality I promise you that this is much easier than you think that it's going to be what we're going to be using in this video is I'm going to be using the groc implementation the other nice thing with this framework is you can use AMA so if you want to use local models you will be able to use that as well for the framework itself they also have a ton of really great examples in here if you'd like to check those out also if you're looking for some full stock examp by leveraging the B agent framework they also have this Bui where it has a number of different capabilities that highlight the features that are built within the framework I'll also put the link to all of this within the description of the video if you're interested in checking any of this out further finally I'm also going to be showing you how you can leverage a hosted postgress provider if you want to have a lot of the pieces within a managed database instead of maybe having all of the pieces that you might tweak like prices and shipping time and all of that being hardcoded within the back end I'm going to show you how it can be a little bit more manageable by leveraging a simple postrest setup as well the other thing to not with this is everything that I'm showing you with in this video you can try completely for free you'll be able to leverage the free tier on grock to use this you can use the free tier on neon and then of course the framework is obviously free as well what we're going to be doing is I'm going to set up the database really quickly we're just going to set up the schema database and then we're going to be setting up the back end of our agent architecture and then we're going to go through the front end and I'm going to show you some tricks on how you can set up some of those things like the conditionally rendered UI components and what have you the first thing that we can do is we can set up a simple project on neon here so I'm just going to call this IMB demo now any secret keys that you do see on screen here don't worry about it I'm going to be deleting these after the video goes live the first thing that we're going to do as soon as we set up our database is we're just going to go over to the schema here and we're going to set that up once that is set up if we go over to our tables we should be able to see all of the different tables here once we have that we'll go back to the editor here we'll make a new query and then we'll just seed this table with some information and I'll go through this in just a moment here now that we see that that was executed successfully as well we can just quickly go through some of the things that we have here within here we have a handful of things that the customer service chatbot can answer things like Brewing methods if you ask a question about the Arrow Press like I had asked it will give you the water temperature as well as the Brew time grind size and all of that for the Brewing ratios we also have a handful of other tables with in year so information about the coffee club benefits being able to see the delivery times as well as the cost for the different regions we also have our store locations here we have things like the hours and times and address you can see their specialty if a user has a particular question on any of these different things you can basically see how you can leverage your own proprietary data and tie this into custom tools that you can plug into the B agent finally if you do have support questions as well that are specific you can give a bit more information there are a handful of other categories within here but largely these can really be swapped out to anything that you want these can be external apis or what have you they don't have to be about coffee obviously or they can really be about any service that you want to leverage the first thing that we're going to set up is we have a standalone Express server with this demo you'll be able to find all of the code within the description of the video I'll put this within a repo where you can pull down and do whatever you'd like with it the first thing that we're going to do is we're going to import a handful of things now all of these will become a bit more clear as we go through the examples we have things from the B agent framework we're also going to be using Express Zod we have things like cores to be able to get the requests in as well as the pool from our database file the first thing that we're going to set up is you can remove the dot example if you pull this repo down and within the EnV you just have to get your grock API key as well as your database just to show you where those are if we head on on over to Gro you go to API Keys you can generate your key and just like you saw when I set up the neon database to get the postgress key you can just go to the dashboard and then you can copy your connection key and paste them in here so once you have that just make sure you save out your EnV and then we're good to go from there we only really have two files that we need to concern oursel with we have the db. TS and then we also have the index.ts within here we're just establishing the connection to our postgress database and then within the index.ts this this is going to be where we have everything that we've set up within the IMB framework within here the first thing that we're going to do we're going to import a number of different dependencies this is going to be an Express server we're also going to be using Zod we're going to be using cores to be able to have the react application on a different server and be able to interact with it from the front end and then we're also going to be connecting to that pool so that DB file that we had just set up the first thing we're going to do is we're going to initialize our Express app with some middleware from there we're going to go and configure our grock model we're going to set the temperature to be really low so it's a bit more deterministic and then from here you can grab your model ID so like I mentioned in this one I'm going to be using the Llama 3.3 70b speculative decoding model what we're going to do is we're going to declare a handful of dynamic tools Dynamic tools are effectively you can think of them as functions that are mapped with natural language so llms understand how to use them within our first Dynamic tool this this is going to be a tool to get the Brewing information as you saw in the example that I showed at the beginning of the video I asked about specific brewing instructions what this function will be asking as an input schema is going to be okay is the user asking for an espresso por a cold brew like what type of coffee and the question are they asking about also what type of strength are they asking for light medium or strong and then if you ask for the number of servings for instance if you say I have an press and I want to make servings for four people it will be able to handle a request like that as well basically to implement this what we're going to be doing is we're going to be making an SQL query based on the different Brewing methods that I had shown within our neon database within here we're going to be making our postrest request we're going to be getting the Brewing method as well as the Brewing ratios the thing to know with this is you can really use anything it doesn't have to be postgress these can be external apis that you tie into or using other data sources as well now we're also going to be calculating the measurement so depending on what the payload that the llm extracts from the query like if we say okay there's going to be four cups that we want it's going to go ahead and make that calculation based on the output of how many servings that the llm detected if you say I'm brewing for four people or two people or I want a liter of coffee or what have you it's going to go ahead and take that serving ratio perform the calculation to show you how many coffee Gams that you need as well as how many mill obviously this is just an example it can really be anything within here these are all just for demonstration sake finally within the tool we're going to be returning this as a string when it has that final reflection of the response from the tool we're going to see that the recipe is for this many servings so we're going to see The Grind size the water coffee recipe temperature Brew time and basically everything and all of this ultimately gets rendered as those conditionally rendered UI components on the screen there like you saw on the outside of the video it it showed a nice little model with all of the different information for the coffee within our next tool we're going to be setting up one to get a shipping estimate the description is provid shipping estimates as well as delivery information for different regions it's only ever going to be these different region it's going to be USA West USA Central Europe Asia South America other that's one thing that I really like about Zod in combination with function calling is it allows you to write this really succinct block of code where you can specify okay I have an en um of these particular things if you're going to be putting in a region it's going to absolutely be one of those things when using things like enums can often be a good tactic for preventing hallucinations as well then if there is some urgency that's detected with the shipping request we can specify okay this might need to be express shipping or what have you this is an optional parameter we don't need the user to specify the shipping method that they want for instance because typically speaking they probably wouldn't but there could be some edge cases where that could be the case to indicate that you can just specify that it is optional with that method at the end there next similar to the previous example we're going to be getting the shipping regions from our postgress database and then what we're going to be doing next is we're going to be processing the shipping details depending on what the input was if we did have an input method that specified whether it's Express or standard or what have you that's going to inform the context that we're ultimately going to return to the llm just like this within here we're going to say okay the shipping time to this region has the delivery time of this with the shipping cost of that and go through all of the other information so it has the context that it can feedback to our AI agent and ask about particular shipping queries next we're going to be defining our modal that renders on the screen for if the user wants to sign up for the coffee Club program they can put in their name email as well as the frequency as well as their preferred blend and what have you so similar to our other tool calls we're going to be making a query to our neon post database to get the information specifically about the coffee club benefits with this setup you can go and change whatever you have within your database and have that information that gets pulled into the agent again within the Handler here you could reach for an external service you could even hardcode things if you wanted I find for the most part it is nice to have some of those pieces especially pieces that might change and grow over time like benefits for a coffee club or shipping rates or what have you it's nice to probably have those separated with within a database somewhere because then you can just go over to Neon or build a simple little admin UI and be able to update those different pieces of your customer service chat bot next we're going to be defining a tool and what this tool does is we're going to handle these support inquiries with some routing if a user has an urgent request we're going to determine whether it's a low priority medium priority or a high priority request if they're saying I need to talk to someone right away it it will detect that that's a high priority request and then we also have the to as well as the categories for the inquiries if it's a product shipping technical or a wholesale related query finally we have the optional information for the customer name and the order number now one thing to note with how I set this up is I made the intention of not needing pieces that require unique customer service information now with that being said what you could do is if you do have an authenticated app is you very well could pass in the customer's information and have it query specifically things related to their account let's say you have an authenticated user and an ID that's associated with the user you could use that within your query for postgress to get the information about products that a user might have ordered for instance or maybe more order related queries that are unique to that particular user in this case I wanted to keep it relatively broad but if you added authentication and you authenticated on the route for where the agent architecture is you could very well have an agent that is able to ask questions specific to whatever that user has within their account for instance we're going to be setting up a tool where we have these different levels of urgency so low medium and high so this is going to largely be what this tool does and within here if the query is an urgent request like let's say someone's typing in all caps and they're very animated in what they're asking for we might send them to this support number alternatively if it's a lowlevel urgency we could send them to a self-served portal or something like an FAQ page and then if Falls with in the middle we could say okay please email at this email and we'll respond in this particular timeline finally if there are any errors from this response we can send back a blanket statement to say okay there's some technical difficulties if you have an urgent query you can call this particular number finally for our tool call we're going to have a get store locator this is one where you could do quite a bit in terms of conditional UI in the past for things like this I have set up where you could get potentially the coordinates of a store and then based on those coordinates you could send them into something like a map component you could also do things like even reference the coordinates within a Google map URL and just route them out to Google maps for instance but in this one what we're going to do is we're going to provide information about the specific stores in terms of where the stores are located we only have a handful we have San Francisco Portland Seattle Chicago and New York once we have that again we're going to reach to our postrest table for our store locations where all the information is going to to be held and then if there is no store at that location we can just say okay we don't have a store please check back we're expanding then finally if we get those results back based on the particular coffee shop we we can send back to the llm and the agent all of those particular details here's the location the hours the address the specialty as well as things like parking now this can really be whatever you'd like you can swap this out to other fields additional fields or what have you from there to initialize a new agent all that we have to do is declare the variable of agent and then we can reference a new B agent and within here we can specify the llm in this case we specified groc we can specify that we do want to have memory and then finally we can specify the different Tools in this case these are all of the tools that I just showed you and this is as simple as it gets if you want to Define some other tools that they have built in like Duck ducko or weather tools or what have you or the code interpreter you can just add these in the tools array just like that that's as simple as it gets in terms of using the B agent framework in terms of setting up an agent this is probably just about as easy as it gets in terms of setting up your own custom agent architecture without really needing to know too much about how to handle the thinking process assessing retrying all of the different pieces that the B agent framework has figured out next we're going to be setting up a Handler for our endpoint within this we're going to destructure the question that's going to be whatever the user puts within the chatbot next we're going to keep track of an array of the different steps that we ultimately send to to the front end and that's going to be what we have within the details here these are effectively what I'm calling steps so we have our thought our tool name the tool input the tool output the final answer and what heute the other cool thing that I want to point out is it handles complicated queries with ease within this if you ask where is the NYC location can you tell me how to use it for two people I want to join the coffee Club what it will do here is it will go through and it will specify all of the different tools that it needs to know and then finally it will give you that final answer there the NYC is this to use the Arrow Press for two people but is really robust right all that we really needed to do is Define those tools Define our agent and it does all of the hard work behind the scenes within the framework all those different steps that we have hidden by default they're going to live within here then finally to actually invoke the agent we call this run method we can send in our prompt and we can specify what the agent does we're going to say you're a coffee expert a customer service representative if the customer asks X you do not have the ability for orders but you can answer general questions about the company as well as coffee in terms of some of the more advanced settings you can also set the max retries per step the total Max retries for the loop of when you send in a message as well as the max iterations of the different steps that the agent takes now the other really nice thing with the agent is you can call this observe method what it will do is depending on the event you can emit those different steps there are a number of different steps that it will take if it's an error you can push that onto the state of that array that we have there or if it's retry we can push that as well finally if it's an update we can push those to that array that we have there just a circle back here essentially all that this is that's mapping out to the UI it's going to be all of those actions that are within those steps whether it's thinking performing a tool call or what have you in this case we're showing it within the UI next what we're going to do is we're going to return the response the payload that we get back to the front end we're going to have the answer and then we're also going to have the steps so that's going to be that array of all of those different options that we had like I just shown you and then finally within our application if we have any errors we're going to log out all of those errors and then we just have to finally initialize our endpoint itself in this case I set up the endpoint to be at the route of ask and then we're going to be passing in that Handler for all those requests after that all that we have to do is we just have to initialize our Express server in this case I set it up to 38 you can swap this out to whatever you'd like right so for the UI portion I'm not going to be going over the page itself this is all just a placeholder page that I spun up for this demonstration but instead what we're going to have here is this chat component this customer service chat bot like we've seen on some other websites essentially everything that I'm going to be going through here and everything that you saw that I demonstrated from the details to the components to the queries themselves all of that is going to be within this file I'm going to be focused on this chat widget TSX here if you pull down the repo that's going to be the line share of where we map the requests to our agent architecture and where we ultimately consume and render whether it's the messages or those conditionally rendered UI components or what have you those are all going to be within this file now you can break this out but for Simplicity sake for the demo I just wanted to show them within here the first thing that we're going to do is we're going to import a handful of dependencies a lot of these are going to become a lot more clear as we go through within this application I'm first going to show you some of the conditionally rendered UI components within here the way that this is set up is if it detects that a tool call has been invoked that's related to the Brewing guidelines for instance if it's a query about asking how to use an Aro press or a pourover coffee machine what that will do is it's going to take the information and extract that from the response back that we get back from the agent and we're going to make this nice little component based on all of the information that we got back so depending on what we have returned from that Dynamic tool that we declared within the agent framework that's going to ultimately map to this UI component here we have things like the serving the strength as well as the number of grams within the coffee water all of those different pieces like we saw when we asked about the specific piece there effectively it will render this nice little component out and it will give you different instructions based on whether you ask things like how do I use my pourover coffee machine or my espresso machine or what have you that's going to be how it renders that so next we have our shipping estimate displayed and again just to tie this all together that's going to be this component here so depending on what the user asks if they ask for like I live in America in Los Angeles or something it will show you the specific one related to that region so you don't need to necessarily specify what is shipping to Europe you could specify a country and the tool call will be smart enough to Leverage The enums that we declared to know which one to use that's where it can be sort of Handy another thing to note while I'm here is if you wanted to take this a step further what you could try is to actually add these different products to a cart for instance from this chat component you could imagine if a user types in I want to order the Brooklyn dark Rose for instance and you can have that action actually take place and put that within their cart that's a bit of an aside on how you can potentially build on this if you're interested next we have our store locator component which is what this looks like here now you could also have like a link here if you wanted to get the coordinates and reference it out to something like Google Maps for instance but this is how you can leverage that and again if we take a look at how it was determined we have the tool input that was Seattle and based on that we made that request to postest we got that information about our Seattle location and and that's what we ultimately get in order to render this UI component next here's an example of a further component that you could set up what you could do on the back end is let's say you want a conditionally rendered UI component for coffee information on if they ask really specific questions about say the coffee you could render something like this this component I left in as a bit of a challenge where if you want to set up from the back end the tool call to make this work and have the coffee info displayed you should be able to do that by just adding a new tool call that's a bit of a challenge you can give this a shot if you want to try and have this particular component rendered next we have our main display component So within here we're going to Loop through and determine based on the payload that we get back from the endpoint what are the tool names that we have as well as the tool outputs this is effectively how we're going to set up the different state within the application to determine what to actually show within our chat history next we're going to be parsing the response within here this is going to be how we determine what gets mapped to those UI components if it's the Brewing guide or if it's the shipping rates we're going to go ahead and take that payload and that key and effectively just render it in conditionally rendered UI components we're just going to split up the response that we got back from the llm and that's going to be how we map to the data that we ultimately show within the chatbot there is a number of different ways that you could potentially do this we're going to be breaking out the different lines that we have from that response within the tool call you could also look at potentially including structured outputs as well if you want to ask the llm to have a more deterministic output like Json or structured outputs is something that's becoming a bit more popular from structured outputs that is a potential Next Step that you could leverage as well but effectively what is going to happen here is based on the tool name we're going to parse the data and then we're going to give things like the region name delivery time the cost and all of those different functions that's going to be what ultimately gets mapped into that visual component there and then finally we have our typical message so if a user is just going back and forth and there isn't a UI component that needs to be rendered that's going to be where we render the message here next we're going to be setting up a simple interface for our message it's going to be relatively straightforward then here what we're going to set up is that modal for that coffee Club input now what you can do here is you can have this UI component submit to an endpoint that might sign them up to an email list right right now this handle submit just console logs but what you could do is have this actually log the results of what the user is setting to go to a form for your email signup list here we just have some form features the rose preference as well as the frequency and what have you that's basically this whole component here another thing is if you're interested in building on top of this I would just recommend breaking out those UI components into their own file that will make it a lot easier to manage I I did set up everything within as few files as I could just for demonstration sake within the video this is going to be where we declare a number of things like whether the chat model is open whether it's expanded what the messages are as well as the initial message we have a handful of other things that's going to become a little bit more clear as we go through the rest of this we're going to be setting up a helper function to extract these store details next what we're going to do is we're going to specify the user message that's going to be how we track the state on the front end this is going to keep the order of whether it's an AI assistant message a user message and then here is going to be how we query our Express server in this example I have it set to our Local Host 3008 at that ass route but this could very well be an external endpoint you could have it on the same server depending on how you set up and deploy your react application in combination with your Express server but that's just one thing to be mindful when you set all of this up the other thing to know with this is I set up the payload to come back immediately with everything you could also set up streaming but given that I am using an inference provider like grock which has that really fast inference I'm going to just be sending the whole payload because it happens honestly within just a second or two for all of the different responses and even the different steps next we're going to be setting up how to trigger our coffee modals and then finally we're going to set the messages as they come in we're going to determine that it's a bot message and then we're going to have both the content as well as the steps that we have set in our state here next we're going to have this function to determine our step Styles all this is doing is we're adding Tailwind classes dependent on the different steps if we go back to the UI here we have all of these different colors here dependent on the steps that's going to just show within the UI based on this switch case statement here next we have the toggle steps function this is going to be where we hide and show the details and then from there we just have a simple Handler to actually send that coffee Club submission if the user fills out that form we can send that submission just like that from there we're going to render whether the chat button is open or closed next we have the widget size here we can expand it or close it just like that from there we can render our main widget this is going to be all of the different jsx which effectively makes this component here within here we have the top of our component there sort of like the header of the chat interface the main portion within this is we have our messages area and then from there we're just going to map out all of the different messages whether they're from the AI or the user and then from there since we are showing these steps within the chat conversation we can just render them out like this now this is optional you don't need to show the steps I do personally like it to see what the agent architecture is thinking the different tools it will definitely help with being able to set up your agent and also debug it but I think frankly it is a nice feature just to share with users in general where users can just see how those answers were ultimately gotten to what tools were used and what have you you could swap out some language potentially to make it a little bit more user friendly but this is really just to give you an idea on how you can leverage those different pieces on whether you want to render the thinking piece or the output you don't necessarily need to render them all it might just be interesting to render the thought as well as the message potentially that's something that 01 does that I think could be an interesting UI you could add in a similar functionality into something like this and then finally within this we have the expanded step state it will show all of those colorful steps within here and then this is going to be where we have those condition rendered UI components from there we're going to be setting up our input State and then finally we're also going to show where all the messages are added the input the button that's going to be what ultimately gets entered and sent to the user we also have where we can map through and detect on if there are steps that do have tool names to render those components as well and then finally within here you can determine based on the state of if a tool called the text that something like a modal needs to render or one of those other conditionally rendered UI components you can set that up just like this otherwise that's pretty much it for this video if you found this video useful please like comment share and subscribe otherwise until the next one