OpenAI AgentKit in Production: An Honest Builder's Review

Last updated: October 18, 2026

Updated October 18, 2026: The ChatKit sample previously rendered <ChatKit agentId apiKey theme onToolCall /> with a "publishable" browser key. That is not how ChatKit works: your server creates a ChatKit session with your secret key and returns a client secret, and the React component uses useChatKit({ api: { getClientSecret } }) with <ChatKit control={control} />. The sample and the ChatKit FAQ now follow OpenAI's ChatKit guide. OpenAI has also announced that Agent Builder is deprecated and scheduled to shut down on November 30, 2026; ChatKit remains available, and new ChatKit apps should use your own server-side agent. See OpenAI Agent Builder and Evals Are Shutting Down for the migration path.

Official Sources#

What AgentKit Actually Bundles#

OpenAI's AgentKit launch was three products dressed up as one announcement. If you treat it as a single thing, you will be confused. If you split it apart, each piece has a clear job:

For the design side of the same problem, read OpenAI Codex: Cloud AI Coding With GPT-5.3 with OpenAI vs Anthropic in 2026 - Models, Tools, and Developer Experience; they show how agent-generated interfaces fail and how to give coding agents better visual constraints.

• Agent Builder - a visual canvas (think Figma for agents) where nodes are LLM calls, tools, branches, and human-in-the-loop checkpoints. You version flows, fork them, and run them.
• Connector Registry - a managed catalog of authenticated connectors (Gmail, Slack, GitHub, Notion, Linear, etc.) that handles OAuth, token refresh, and scope management. You stop writing OAuth code.
• ChatKit - an embeddable React widget that renders a chat UI talking to your agent flow. Think "Intercom but it is your agent." Streaming, tool-call rendering, file uploads, all included.

The shared value prop is: stop writing the boring 60% of every agent app - auth, UI, glue code - and concentrate on the actual logic. The question is whether the visual canvas is a feature or a tax.

I rebuilt my newsletter-research agent on AgentKit over a weekend. Below is what worked, what did not, and the decision tree I now use.

Building a Real Workflow Visually#

My newsletter agent does four things: pull RSS feeds, read new articles, cluster by topic, draft a digest. Here is the Agent Builder flow I ended up with:

[Trigger: Webhook] -> [Tool: RSS Fetch] -> [LLM: Filter Relevance, gpt-5.5]
   -> [Branch: relevance_score > 0.7?]
       -> yes -> [Tool: Fetch Article] -> [LLM: Summarize, gpt-5.3]
              -> [Tool: Embedding] -> [Tool: Cluster] -> [Human Approval]
              -> [LLM: Draft Newsletter] -> [Tool: Send via Resend]
       -> no -> [End]

Three things became immediately obvious in the visual canvas:

Branching is dramatically clearer than code. When I had this in TypeScript with nested ifs, the relevance branch was buried 80 lines deep. On the canvas it is a single yellow diamond. New collaborators understand the flow in 30 seconds.

Versioning is built-in. Every save creates a numbered version. I can fork v12 to test a new prompt, run it side-by-side with prod v11, and promote when evals pass. Doing this in code means git branches plus a feature-flag system. Builder gives it to you free.

Debugging is a timeline, not a log file. When a run fails, you click the failed node and see the exact prompt, the model response, the token count, and the tool I/O. No more console.log archaeology.

For a side-by-side comparison of how this looks in a Claude Code-flavored designer, see Subagent Studio - same visual-first thesis, different model ecosystem.

When the Canvas Saves Time#

After two weeks I have a clear pattern. The canvas wins when:

1. The flow has more than 3 branches. Visual branching beats nested code every time.
2. You have non-engineering reviewers. A PM can read the canvas. They cannot read your TypeScript.
3. You version prompts often. The built-in versioning is genuinely good.
4. You need human approval steps. AgentKit's approval node hands you a real review UI, not a Slack message hack.
5. Your tools are in the Connector Registry. If Gmail, Slack, GitHub, Notion are 80% of your tools, you save days of OAuth plumbing.

That last one is the sleeper feature. I was about to write Gmail OAuth for the newsletter agent. I deleted that ticket and used the Connector Registry's Gmail node. Token refresh, scope upgrade flow, error handling - all done.

When I Dropped Back to Code#

Three places I bailed:

Custom embedding logic. My clustering uses a non-OpenAI embedding model (Voyage) plus a custom HDBSCAN. AgentKit's "custom tool" node lets you call an HTTP endpoint, but the round trip added 400ms per call and cost me a node on the canvas for what was a 20-line function. I exposed a single /cluster endpoint on my existing API and called it as one node. Canvas stayed clean, performance stayed good.

Tight loops. AgentKit nodes have per-execution overhead - roughly 100-200ms - that adds up if you are looping 50 times per run. My RSS fetch processes ~80 feeds. Doing that as 80 canvas iterations was wasteful. I batched the entire fetch into one custom-tool call and let my own code handle the loop.

Streaming token-level logic. If you need to react to tokens as they stream (e.g. to cut off generation early on a stop sequence), AgentKit's node abstraction hides that. Drop to the Responses API directly for those.

The pattern: Builder for the workflow, code for the hot loops and custom math. Same instinct as React server components - render the structure visually, push the heavy compute to a function.

ChatKit: Embed in Under 30 Minutes#

ChatKit is the one I expected the least and got the most from. It is a chat embed that handles streaming, tool-invocation display, file attachments and widgets, and it outlives Agent Builder: OpenAI's docs keep ChatKit available after the Agent Builder shutdown and point new apps at an advanced integration backed by your own server.

The integration has a server half and a client half. On the server, create a ChatKit session with your OpenAI API key and return only the client secret. This is adapted from the JavaScript example in OpenAI's ChatKit guide, which uses an Agent Builder workflow ID; pass the authenticated user's stable ID as user:

export async function createChatKitClientSecret(userId: string) {
  const response = await fetch("https://api.openai.com/v1/chatkit/sessions", {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "OpenAI-Beta": "chatkit_beta=v1",
      Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
    },
    body: JSON.stringify({
      workflow: { id: "wf_your_workflow_id" },
      user: userId,
    }),
  });
  if (!response.ok) {
    throw new Error(`Failed to create a ChatKit session: ${response.status}`);
  }
  const { client_secret } = await response.json();
  return client_secret;
}

On the client, install @openai/chatkit-react, load the ChatKit script (https://cdn.platform.openai.com/deployments/chatkit/chatkit.js) on the page, and fetch the client secret from your endpoint:

import { ChatKit, useChatKit } from "@openai/chatkit-react";

export function NewsletterChat({ getAppAuthToken }: { getAppAuthToken: () => Promise<string> }) {
  const { control } = useChatKit({
    api: {
      async getClientSecret(existing) {
        if (existing) {
          // implement session refresh
        }
        const res = await fetch("/api/chatkit/session", {
          method: "POST",
          headers: {
            Authorization: "Bearer " + (await getAppAuthToken()),
            "Content-Type": "application/json",
          },
        });
        const { client_secret } = await res.json();
        return client_secret;
      },
    },
  });

  return <ChatKit control={control} className="h-[600px] w-[320px]" />;
}

The gotcha that matters: never put your OPENAI_API_KEY in the browser. The client secret is the credential the frontend uses to open or refresh the session, and your server should authenticate your own user before minting one. For theming, widgets and actions, see OpenAI's ChatKit customization, widgets and actions guides.

AgentKit vs. Rolling Your Own: My Decision Tree#

Is this a one-off internal automation?
  -> Yes: AgentKit. The connector and approval nodes alone pay for themselves.
  -> No: continue.

Will non-engineers review or edit the flow?
  -> Yes: AgentKit. The canvas is the artifact they read.
  -> No: continue.

Do you need bare-metal control over streaming or model parameters?
  -> Yes: roll your own with the Responses API.
  -> No: AgentKit, drop to code only for hot paths.

Is your orchestration multi-tenant, multi-region, or > 100 RPS?
  -> Probably your own infra. AgentKit is fine for the first 90%  -  see
    [DD Orchestrator](https://orchestrator.developersdigest.tech) for when
    you need to own the runtime.

The honest answer for most builders shipping agent features in 2026: start in AgentKit, escape to code where it hurts. The "all visual" maximalists will hit walls; the "all code" purists are leaving days of OAuth plumbing on the table. The blended pattern wins.

For the full screen-recording walkthrough of building this newsletter agent on the canvas, the DevDigest YouTube channel has the AgentKit deep-dive. The canvas is one of those things where seeing it move beats reading about it.

AgentKit will not replace your code. It will replace the boring 60% of your code. That is enough.

Frequently Asked Questions#

What is OpenAI AgentKit and what does it include?#

AgentKit is three products in one: Agent Builder (a visual canvas for designing agent workflows), Connector Registry (managed OAuth integrations for Gmail, Slack, GitHub, Notion, and more), and ChatKit (an embeddable React component for agent UIs). Together they eliminate the repetitive 60% of agent development - auth, UI, and orchestration glue code.

Is AgentKit free or does it have separate pricing?#

AgentKit uses your existing OpenAI API credits. You pay per token for LLM calls (same rates as the Responses API), plus execution overhead for managed connectors. There is no separate AgentKit subscription. The Connector Registry connectors have rate limits tied to your API tier.

When should I use Agent Builder vs writing code?#

Use Agent Builder when your workflow has multiple branches, when non-engineers need to review the flow, when you version prompts frequently, or when you need human approval steps. Drop to code for tight loops (50+ iterations), custom embedding logic, streaming token-level control, or when per-node latency (100-200ms overhead) matters.

How does Agent Builder versioning work?#

Every save creates a numbered version automatically. You can fork any version to test changes, run multiple versions side-by-side, and promote a tested version to production. This replaces the need for git branches plus feature flags for workflow iteration.

What connectors are available in the Connector Registry?#

The launch registry includes Gmail, Slack, GitHub, Notion, Linear, Google Drive, and Calendar. Each connector handles OAuth, token refresh, and scope management. You configure credentials once in the dashboard and reference the connector in your flow nodes.

How do I integrate ChatKit into my React app?#

Create a ChatKit session on your server with your OpenAI API key (POST to /v1/chatkit/sessions with a user ID and, for hosted workflows, a workflow ID) and return only the client secret. In React, install @openai/chatkit-react, load the ChatKit script, call useChatKit({ api: { getClientSecret } }) to fetch that secret from your endpoint, and render <ChatKit control={control} />. Never ship your API key to the browser.

Is Agent Builder still available?#

OpenAI has deprecated Agent Builder. Existing users can keep using it during a transition window, and it is scheduled to shut down on November 30, 2026. ChatKit is still available; for new work, OpenAI recommends an advanced ChatKit integration with your own server-side agent.

Can I call external APIs from Agent Builder?#

Yes. Agent Builder has a "custom tool" node that calls any HTTP endpoint. For complex logic, expose a single endpoint on your own API and call it as one node. This keeps the canvas clean while letting your code handle heavy compute, custom models, or third-party integrations.

What are the performance limits of AgentKit?#

Each node adds 100-200ms execution overhead. For high-frequency loops, batch operations into a single custom-tool call. For multi-tenant or high-RPS deployments (100+ requests per second), you may need your own orchestration infrastructure. AgentKit works well for internal automations and moderate-scale production use.

Continue Reading#

• OpenAI Agent Builder and Evals Are Shutting Down: Move the Agent Stack Into Code