Foundations View as Markdown

Goroutines and channels

Go's headline feature, and the one place where the compiler stops protecting you.

go doSomething()      // runs concurrently. that is the whole syntax.

Goroutines are cheap — a few kilobytes of stack, grown as needed — so having thousands is normal. The difficulty is not starting them; it is knowing when they finish and how they communicate.

Waiting for goroutines#

var wg sync.WaitGroup
for _, url := range urls {
    wg.Add(1)                        // before the go statement, not inside it
    go func(u string) {
        defer wg.Done()
        fetch(u)
    }(url)
}
wg.Wait()

Three ways this goes wrong, all of them common: wg.Add(1) inside the goroutine (a race with Wait), a missing defer wg.Done() (hangs forever), and passing the WaitGroup by value (each goroutine gets a copy).

Since Go 1.22 the loop variable is per-iteration, so passing url as a parameter is no longer strictly required — but it remains clearer.

Channels#

ch := make(chan int)        // unbuffered: send blocks until someone receives
buf := make(chan int, 10)   // buffered: send blocks only when full

ch <- 42        // send
v := <-ch       // receive

close(ch)
for v := range ch {   // ranges until the channel is closed
    fmt.Println(v)
}

v, ok := <-ch   // ok is false if the channel is closed and drained

Rules that prevent most channel bugs:

• The sender closes, never the receiver. Closing twice panics; sending on a closed channel panics.
• An unbuffered send blocks until a receiver is ready. ch <- 1 on a channel nobody is reading is a deadlock.
• Receiving from a nil channel blocks forever. So does sending to one.

select#

select {
case v := <-ch1:
    fmt.Println("ch1", v)
case ch2 <- 42:
    fmt.Println("sent")
case <-time.After(time.Second):
    fmt.Println("timeout")
case <-ctx.Done():
    return ctx.Err()
}

select waits on several channel operations and takes whichever is ready first. Adding a default case makes it non-blocking.

Context: how goroutines get told to stop#

func worker(ctx context.Context, jobs <-chan Job) error {
    for {
        select {
        case <-ctx.Done():
            return ctx.Err()          // cancelled or timed out — clean exit
        case job, ok := <-jobs:
            if !ok {
                return nil
            }
            process(job)
        }
    }
}

ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()                        // always. a missing cancel leaks the timer.

Convention: ctx is the first parameter of any function that does I/O, and it gets passed down. A function that accepts a context and ignores it is a bug — cancellation stops working silently.

errgroup: the version you will actually use#

import "golang.org/x/sync/errgroup"

g, ctx := errgroup.WithContext(ctx)
results := make([]Result, len(urls))

for i, url := range urls {
    i, url := i, url
    g.Go(func() error {
        r, err := fetch(ctx, url)
        if err != nil {
            return fmt.Errorf("fetch %s: %w", url, err)
        }
        results[i] = r
        return nil
    })
}
if err := g.Wait(); err != nil {
    return err
}

WaitGroup plus error propagation plus cancellation on first failure. This is the right default for "do these N things concurrently", and it removes most of the ways to leak a goroutine.

Add g.SetLimit(10) to bound concurrency.

Mutexes#

Channels are for passing ownership; mutexes are for protecting shared state. Both are idiomatic.

type Counter struct {
    mu sync.Mutex
    n  int
}

func (c *Counter) Inc() {
    c.mu.Lock()
    defer c.mu.Unlock()
    c.n++
}

Put the mutex directly above the fields it guards. Never copy a struct containing a mutex — go vet catches this.

Leaks#

func leak(urls []string) []Result {
    ch := make(chan Result)
    for _, u := range urls {
        go func(u string) { ch <- fetch(u) }(u)   // blocks forever if unread
    }
    return []Result{<-ch}                          // reads one, leaks the rest
}

Every goroutine must have a guaranteed way to exit. Buffer the channel, drain it, or give the goroutine a context to be cancelled by.

func TestMain(m *testing.M) { goleak.VerifyTestMain(m) }

Common questions#

Channels or mutexes?#

Channels when you are transferring ownership of data between goroutines; a mutex when several goroutines need to read and write the same state. Reaching for a channel to guard a counter is a common over-application of the idiom.

Why does my program exit before the goroutines finish?#

When main returns, the process exits and all goroutines are killed regardless of what they are doing. Use a WaitGroup or an errgroup to wait.

Do I need -race if my tests pass?#

Yes. A data race is a race — it may not manifest on your machine, on this run, at this scheduling. The race detector finds them deterministically by instrumenting memory access, which is not something a passing test can tell you.