# Tracking and cutting token costs in Go > Source: https://learn-go.org/ai/tokenomics/ > Part of Learn Go, free to read. The economics are language-independent — [what tokens cost and where the money goes](https://codelearningdojo.com/token-economics/) is the model. This page is the Go implementation, and Go usually occupies a specific position in these systems: the gateway, the proxy, the worker pool. The service that sits between many callers and a metered API. That position is where cost control belongs, and Go is unusually well suited to it: contexts propagate cancellation, `sync/atomic` makes counters cheap, and `errgroup` bounds fan-out in one line. ## Carry the budget in the context The idiomatic Go answer to "every call site needs this and I do not want it in every signature". ```go internal/budget/budget.go package budget import ( "context" "errors" "sync/atomic" ) var ErrExceeded = errors.New("budget exceeded") // Budget is safe for concurrent use. type Budget struct { capMicroUSD int64 maxTurns int64 spentMicros atomic.Int64 turns atomic.Int64 } func New(capMicroUSD int64, maxTurns int) *Budget { return &Budget{capMicroUSD: capMicroUSD, maxTurns: int64(maxTurns)} } // Reserve is called before a request leaves. It is deliberately conservative: // it charges the estimate up front and refunds the difference afterwards, so // concurrent callers cannot all pass the check and then collectively overspend. func (b *Budget) Reserve(estimateMicros int64) error { if b.turns.Add(1) > b.maxTurns { return errors.Join(ErrExceeded, errors.New("turn limit")) } if b.spentMicros.Add(estimateMicros) > b.capMicroUSD { b.spentMicros.Add(-estimateMicros) return ErrExceeded } return nil } func (b *Budget) Settle(estimateMicros, actualMicros int64) { b.spentMicros.Add(actualMicros - estimateMicros) } func (b *Budget) Spent() int64 { return b.spentMicros.Load() } type ctxKey struct{} func With(ctx context.Context, b *Budget) context.Context { return context.WithValue(ctx, ctxKey{}, b) } func From(ctx context.Context) (*Budget, bool) { b, ok := ctx.Value(ctxKey{}).(*Budget) return b, ok } ``` The reserve-then-settle pattern is the part worth copying. A naive "check, then call, then add" lets ten concurrent goroutines all pass the check against the same stale total and blow the cap tenfold. Reserving up front and refunding the difference is correct under concurrency, which in a Go gateway is the only case that matters. :::warn Integer micro-dollars, not float `int64` micro-dollars are exact and atomic. A `float64` accumulator loses fractions across millions of rows, and there is no atomic float. This is the same reasoning as [money in any language](/review/failure-modes/), with the extra constraint that you need `atomic` here. ::: ## Cancellation is a cost control If the caller goes away and you keep streaming from the provider, you keep paying. In Go this is nearly free to get right, and it is the most common thing generated gateway code omits. ```go internal/handler/chat.go func (h *Handler) Chat(w http.ResponseWriter, r *http.Request) { ctx := r.Context() // cancelled when the client disconnects b := budget.New(500_000, 12) ctx = budget.With(ctx, b) flusher, _ := w.(http.Flusher) var usage llm.Usage stream, err := h.llm.Stream(ctx, req) // ctx MUST reach the outbound request if err != nil { http.Error(w, err.Error(), http.StatusBadGateway) return } defer func() { h.record(ctx, "chat", usage, ctx.Err() != nil) }() for chunk, err := range stream { if err != nil { return } if _, werr := io.WriteString(w, chunk.Text); werr != nil { return // client gone; ctx cancels upstream } usage = chunk.Usage if flusher != nil { flusher.Flush() } } } ``` The whole fix is that `r.Context()` reaches the outbound HTTP request. A handler that accepts a context and then builds its upstream call with `context.Background()` has silently disabled cancellation — `contextcheck` in `golangci-lint` catches most instances, and it is worth turning on for exactly this reason. See [the Go failure modes](/review/failure-modes/). Record on cancellation too. You still paid for input tokens and whatever output was generated before the abort, so dropping those records understates spend. ## Bound the fan-out Unbounded `go` statements against a rate-limited API is the classic way a Go gateway turns a batch job into a 429 storm plus a retry bill. ```go func ClassifyAll(ctx context.Context, c llm.Client, inputs []string, limit int) ([]Category, error) { g, ctx := errgroup.WithContext(ctx) g.SetLimit(limit) // the whole fix out := make([]Category, len(inputs)) for i, in := range inputs { g.Go(func() error { b, _ := budget.From(ctx) est := int64(len(in)/4) * 3 // rough micros; conservative if b != nil { if err := b.Reserve(est); err != nil { return fmt.Errorf("classify %d: %w", i, err) } } res, usage, err := c.Complete(ctx, promptFor(in)) if b != nil { b.Settle(est, cost(usage)) } if err != nil { return fmt.Errorf("classify %d: %w", i, err) } out[i] = res return nil }) } return out, g.Wait() } ``` `errgroup.WithContext` also cancels every sibling on the first failure — so a budget exhaustion stops the remaining work rather than letting it run to completion and overspend. ## Make the cache hit Same rule as everywhere: stable prefix first, volatile last. The Go-specific trap is **map iteration order**, which is deliberately randomised. ```go // BAD: tool definitions from a map iterate in a different order every run, // so the prefix differs every call and the cache never hits. for name, tool := range h.tools { defs = append(defs, tool.Definition(name)) } // GOOD: deterministic order for _, name := range slices.Sorted(maps.Keys(h.tools)) { defs = append(defs, h.tools[name].Definition(name)) } ``` This one is genuinely easy to miss because nothing fails — the responses are correct, the tests pass, and the only symptom is a cache hit rate near zero. It is the Go flavour of a mistake every language has a version of. ```go func TestPromptPrefixIsStable(t *testing.T) { a := BuildMessages("q1", []string{"doc-b", "doc-a"}) b := BuildMessages("q2", []string{"doc-a", "doc-b"}) if diff := cmp.Diff(a[:len(a)-1], b[:len(b)-1]); diff != "" { t.Errorf("prefix is not stable, cache will miss (-a +b):\n%s", diff) } } ``` ## Retries, bounded ```go func withRetry(ctx context.Context, attempts int, fn func() error) error { var err error for n := 0; n < attempts; n++ { if err = fn(); err == nil { return nil } var apiErr *llm.APIError if !errors.As(err, &apiErr) || !apiErr.Retryable() { return err // never retry a 400 } delay := time.Duration(1<