The map
- Not started
- In progress
- Done
- Builds on
- 01How models behaveTokens, sampling, prompts, shapes0/4
- 02Models in your systemTools, documents, streams, budgets0/4
- 03Shipping with confidenceTests, agents, attacks, trade-offs0/4
- 04Next: system designSame ideas, at service scale
- Retry with backoff and jitterLibrary ↗Model APIs fail in ordinary ways: rate limits, overloads, timeouts. Retry them with backoff and jitter, and only when the request is safe to repeat.
- Circuit breakerLibrary ↗When a model provider is down or slow, stop sending it traffic for a while and fall back to a smaller model, a cached answer or a plain message.
- Rate limitingLibrary ↗Providers limit you by requests and tokens per minute, and you should limit your own users too: one runaway loop can spend a month's budget.
- Durable workflowsLibrary ↗A long agent run is a workflow with many slow, failing steps. Persist each step so a crash resumes instead of starting over.
The examples are vendor-neutral: API code uses placeholder names (llm, MODEL) rather than a real SDK, and the Try it blocks use small fake models so they run in your browser with no network. Provider documentation is linked where a detail differs between vendors.