Building an AI Agent with ChatGPT: What Works, What Breaks in Production

You can build something that looks like a working AI agent in an afternoon. Open a Custom GPT, paste in your knowledge base, write instructions, connect an action to your calendar API. It demos beautifully.

Then you point real customers at it and discover the gap between a demo and a system. I have watched a dozen of these builds go from "this is incredible" to "we turned it off" in under six weeks, and the failure causes are remarkably consistent.

Here is what genuinely works, where these builds break, and how to decide what to build yourself versus what to buy.

The Three Build Paths, Honestly Compared

Custom GPTs. Zero code. You write instructions, upload files, optionally connect Actions to external APIs. Genuinely excellent for internal tools — a proposal drafter, an onboarding assistant, a research helper your team uses inside ChatGPT.

The hard limit: your users must have a ChatGPT account and go find your GPT. That makes it a poor fit for customer-facing anything. You are not going to send a lead a link and ask them to sign up for ChatGPT before they can book an appointment.

The Assistants/Agents API. Real programmatic control. Persistent threads, function calling, file retrieval, code execution. This is where an actual customer-facing agent lives. You need a developer, a backend to host it, and somewhere to put the conversation state.

Agent frameworks on top. Orchestration layers for multi-step, multi-tool workflows. Powerful, and the place where most projects over-engineer. If your agent has one job, you do not need an orchestration framework to do it.

For 80% of business use cases, the honest recommendation is: Custom GPT for internal tools, API for anything a customer touches, framework only when you have proven you need multi-agent coordination.

The Five Things That Break in Production

1. It makes things up under pressure.

An agent asked for a price four times will eventually produce one. In a demo you ask nicely once. In production, customers push. Any factual claim — price, availability, policy, timeline — must come from a system of record via a function call, with an explicit instruction to refuse rather than estimate. This is the failure that costs real money, because a quoted price functions as a promise.

2. Latency destroys the experience.

A 4-second response feels thoughtful in a chat window and catastrophic on a phone call. If your agent chains three tool calls before answering, you have built something too slow for voice and borderline for chat. Measure time-to-first-token, not total response time, and design around it.

3. Context management gets expensive and lossy.

Long conversations mean growing context, which means rising cost per turn and degrading instruction-following. Real systems summarize aggressively, cap conversation length, and store durable facts in a database rather than trusting the thread to remember.

4. There is no handoff.

Nearly every DIY agent I audit lacks a real escalation path. When the agent cannot help, it apologizes in a loop. Define hard triggers — anger, legal language, safety issues, an explicit request for a human, two consecutive turns of confusion — and route to a person or a guaranteed callback. This is a product decision, not a prompt.

5. Nobody reads the transcripts.

This is the quiet killer. Teams ship an agent, watch a dashboard, and never read what it actually said. Every meaningful improvement we have ever made to a production agent came from reading conversations, not from staring at metrics.

An agent you do not read is not deployed. It is abandoned in public.

What a Production Build Actually Requires

Beyond the model call, the components nobody budgets for:

That list is the difference between a weekend project and a system. It is also, roughly, six to twelve weeks of engineering for a competent developer who has not built one before.

The Cost Reality

Model calls are usually the smallest line item. A text agent handling a few thousand conversations a month typically costs tens of dollars in inference. What costs money:

The build-versus-buy math usually turns on maintenance rather than build cost. A build is a one-time expense you can stomach. The weekly discipline of reading transcripts, updating prompts, re-running evals, and handling platform changes is the part that quietly does not happen once the person who built it moves on.

Where We Land

We build on this stack, and we still tell people honestly when they should not.

Build it yourself if the agent is internal, the failure cost is low, and you have a developer who will own it past launch. A Custom GPT for your ops team is a genuinely great afternoon.

Buy or partner if the agent talks to customers and revenue depends on it. Not because the technology is hard to start — it is not — but because the operational layer around it is where the value and the risk both live.

Our AI callers respond to every inbound lead within 90 seconds, run a real qualifying conversation, book onto a calendar, and write the transcript and score into a GoHighLevel pipeline. The model call is maybe 5% of that system. The other 95% — the telephony, the barge-in handling, the grounding, the escalation logic, the CRM sync, the weekly transcript review — is what makes it produce revenue rather than a demo.

The Model Choice Question

One decision that comes up in every build and gets overthought: which model.

For agent work, the answer is usually the most capable model available, not the cheapest. The reasoning is economic rather than technical. Inference is a small fraction of total system cost, and a weaker model produces more instruction-following failures, which produce more bad conversations, which produce the one thing that actually costs money — a lost customer or a support escalation.

Where a cheaper model earns its place is in narrow, well-defined subtasks: classifying an intent, extracting a phone number from a transcript, summarizing a completed conversation. Those run at high volume and tolerate occasional error. The customer-facing reasoning turn should not.

Two practical rules from production:

Do not pin yourself to one provider's API shape. Put a thin abstraction between your application and the model call. Providers deprecate endpoints and change behavior between versions, and you want that to be a one-file change rather than a rewrite.

Version-lock your model in production and upgrade deliberately. A model version change is a behavior change. Run it against your eval set before it touches customers. Teams that auto-upgrade discover regressions from customer complaints, which is the most expensive possible test environment.

If You Are Building It Yourself, Do This Order

Steps 6 through 8 are the ones that separate the agents still running a year later from the ones quietly switched off.

If you would rather deploy something that has already been through all eight steps a hundred times, [book a free strategy call](/book) and we will show you what a production agent looks like on a live account.