THINXSTER
Blog/AI Automation
AI Automation9 min readAugust 5, 2026

How to Learn AI Infrastructure: A 90-Day Path That Ends With Something Running

Most AI infrastructure learning stops at tutorials that work on the first try. Here's a 90-day curriculum built around failure modes, evaluation, and cost — the three things that actually get tested.

RK
Ryan Korsz
Founder & CEO, Thinxster

TL;DR

Most AI infrastructure learning stops at tutorials that work on the first try. Here's a 90-day curriculum built around failure modes, evaluation, and cost — the three things that actually get tested.

→ See how this applies to your business (free 30-min call)

The reason most people who "learn AI infrastructure" can't get hired or ship anything is that they learned the happy path. They followed tutorials where the API returned in 200 milliseconds, the document parsed cleanly, and the agent finished in three steps.

Production is the other path. The API times out. The document is a scanned PDF at an angle. The agent loops eleven times and burns forty dollars. The retrieval quietly starts returning garbage because someone changed the chunking.

Everything valuable in this field lives in that second category. So here's a 90-day curriculum organized around failure rather than features, where every phase ends with something deployed.

The Prerequisite Nobody States

Before any of this: you need to be comfortable with one programming language, HTTP, JSON, and the command line. If you're not, spend three weeks there first. Python is the pragmatic default; JavaScript is fine and increasingly common at the edge.

You do not need machine learning theory, linear algebra, or a background in data science. AI infrastructure is a systems discipline. The people who do it best usually came from backend, platform, or SRE work, not from research.

Days 1–15: The Loop and the Cost

Goal: understand exactly what happens in a single model call, and what it costs.

Read the primary API documentation from a major model provider — actual docs, not a summary. Learn what a token is, how context windows work, what streaming does, and how tool calling is structured at the wire level.

Then build the smallest possible thing: a script that takes an input, calls a model with one tool available, and returns a result. Print the raw request and response. Look at them. Most people never do this and it costs them for years.

The exercises that matter:

  • Calculate the cost of a single call by hand from token counts. Then calculate what 10,000 calls a day costs.
  • Run the same prompt against three model sizes and measure quality, latency, and price. This builds the routing intuition you'll use constantly.
  • Deliberately exceed a context window and watch what happens.
  • Ship: a command-line tool that does one useful thing for you, with cost logged per run.

    Days 16–30: Tools and Failure

    Goal: learn that the model is rarely the problem.

    Add three tools to your agent — one that hits an external API, one that reads a database, one that writes somewhere. Then break each one on purpose:

  • Make one return a 500 error.
  • Make one hang for 90 seconds.
  • Make one return valid JSON with the wrong shape.
  • Make one succeed but return an empty result.
  • Handle every case. Add timeouts, retries with backoff, and idempotency so a retried write doesn't duplicate. Add a hard cap on loop iterations and total spend per run.

    This fortnight is the single highest-value block in the entire curriculum, and it's the one every tutorial skips.

    Ship: the same agent, but it survives every failure you can throw at it and reports what went wrong.

    Tutorials teach the happy path. Everything you get paid for lives in the failure path.

    Days 31–45: Retrieval, Properly

    Goal: understand why retrieval quality — not model quality — is usually the bottleneck.

    Build a retrieval system over a real document set you care about. Then measure it, which is the part almost nobody does.

  • Write 30 questions with known correct answers from your corpus.
  • Measure how often the right chunk is retrieved at all. This number will be worse than you expect.
  • Change chunk size and overlap. Re-measure.
  • Add keyword search alongside vector search and combine the results. Re-measure. Hybrid retrieval usually beats pure vector search and it's a thirty-line change.
  • Add a re-ranking step. Re-measure.
  • You'll finish with an intuition that separates you from most practitioners: when someone says "the AI gave a wrong answer," your first question will be whether it ever saw the right information.

    Ship: a question-answering system over your own documents with a measured retrieval accuracy number you can state out loud.

    90s
    the latency budget a production voice agent has to live inside

    Days 46–60: Evaluation

    Goal: be able to prove a change made things better.

    This is the phase that makes you employable. Almost nobody does it well.

  • Build a test set of 50 real inputs with expected outputs, including 10 that should escalate or refuse rather than answer.
  • Write an automated runner that executes all 50 and reports a pass rate.
  • Add a model-graded evaluation for outputs that aren't exact-match — and then check the grader against your own judgment on 20 cases, because an unvalidated grader is just a second source of error.
  • Make a prompt change and watch the score move. Make a change you're sure is an improvement and discover it isn't. That moment is the whole point.
  • Wire the eval suite to run automatically whenever the prompt or model changes.
  • Ship: a regression suite that would catch it if tomorrow's change broke last week's behavior.

    Days 61–75: Deploy and Observe

    Goal: run it somewhere real, and see everything it does.

  • Deploy to a serverless platform or a small container host. Learn cold starts, timeouts, and concurrency limits the hard way.
  • Add structured logging: every input, tool call, latency, token count, cost, and output, with a trace ID linking them.
  • Add tracing so you can reconstruct a single multi-step run end to end.
  • Set up alerts on the three things that actually matter: error rate, p95 latency, and spend per hour.
  • Add a kill switch. You want one before you need one.
  • Ship: the system running on a schedule or a webhook, with a dashboard you check daily.

    Days 76–90: Cost, Scale, and Security

    Goal: make it cheap and safe enough to run without supervision.

  • Route by difficulty: cheap model for classification and extraction, expensive model only for hard reasoning. Measure the quality delta with your eval suite — you'll often find no measurable loss and a 60–80% cost reduction.
  • Add caching for repeated inputs.
  • Add a fallback provider and test it by simulating an outage.
  • Test prompt injection against your own system: put instructions inside a document your agent will read and see whether it obeys them. It probably will. Fix it.
  • Review secrets handling, data retention, and what personally identifiable information ends up in your logs.
  • Ship: a documented write-up of what you built, what broke, and what it costs per thousand runs. This document is worth more in a job search than any certificate.

    Where to Learn From

    Categories rather than links, because specific resources rot fast:

  • Primary provider documentation. The model providers' own guides on tool use, prompt caching, structured outputs, and agent patterns are the highest-quality free material available, and they're updated as things change. Read them before anything else.
  • Engineering blogs from companies running this in production. Post-mortems and architecture write-ups from teams with real traffic teach more than any tutorial, because they describe what broke.
  • The source code of one orchestration library. Reading how a well-built agent framework handles retries, state, and tool dispatch is worth a month of videos.
  • Your own logs. Once you have something running, this becomes the best teacher you have access to.
  • Skip the aggregator newsletters and the model release commentary. They're entertainment, not education, and they consume the attention that should go to building.

    What to Skip

  • Framework tourism. Learn one orchestration approach deeply. The patterns transfer; the APIs expire.
  • Fine-tuning, initially. Nearly every problem people try to solve with fine-tuning is a retrieval or prompting problem. Learn those first.
  • Building your own vector database. Educational, not useful.
  • Benchmark obsession. Public benchmarks correlate weakly with your specific task. Your own eval set is worth more than every leaderboard combined.
  • Waiting for the field to settle. It won't. The fundamentals — retrieval, evaluation, failure handling, cost — have been stable for years while the tooling churned.
  • How This Gets Tested

    If you're doing this to get hired, interviews for these roles converge on four questions:

    1.

    "Your provider starts returning errors on 30% of requests. What happens to your system?" They're testing fallbacks and graceful degradation.

    2.

    "How would you cut inference cost in half without hurting quality?" Routing, caching, prompt compression, smaller models for subtasks.

    3.

    "How do you know your last change didn't make things worse?" Evaluation. This is where most candidates have nothing.

    4.

    "Walk me through debugging a quality regression." Traces, eval sets, retrieval metrics, isolating the layer.

    Notice none of them are about models.

    Where the Learning Actually Compounds

    The engineers who get good fastest are the ones running something with real users, because real users generate failure modes no test suite invents. If you don't have a system with real traffic, find one — an internal tool at work, a friend's business, a volunteer project.

    That's the same reason our systems improve: AI callers responding to every inbound lead within 90 seconds generate thousands of real conversations, and every week someone reads them and tunes what's awkward. The loop between production and improvement is the entire discipline.

    62%
    average lead qualification rate across client accounts

    If you'd rather have the outcome than the education — a system that qualifies and books leads without you building it — [book a free strategy call](/book).

    Free Weekly Briefing

    One AI Marketing Tactic.
    Every Tuesday. Free.

    What's actually working across our client accounts right now — ROAS moves, follow-up sequences, creative angles. The stuff that isn't in any blog post yet.

    No spam. Unsubscribe anytime. 1,200+ business owners already in.

    Ready to Deploy

    SEE THIS IN
    YOUR BUSINESS.

    30 minutes. We scope the exact systems that apply to your situation and give you a plan.

    ★★★★★ Trusted by 47+ local service businesses

    BOOK A STRATEGY CALL →