Lesson 17 of 20 · Part 5: Packages and concurrency
Mutexes and concurrency patterns
Go concurrency patterns for beginners. Race conditions and the race detector, sync.Mutex, RWMutex, atomic counters, sync.Once, worker pools, limiting concurrency and context cancellation.
5 min read · Updated · Beginner friendly
What you'll learn
- Recognise race conditions and find them with the race detector
- Protect shared data with sync.Mutex and sync/atomic
- Build a worker pool and limit how much runs at once
- Cancel work with context and timeouts
Channels are great for passing data between goroutines. But sometimes several goroutines need to update the same variable, like a counter, a cache or a map of results. Doing that safely needs a few extra tools. This lesson also shows the patterns you’ll use most in real concurrent programs.
Race conditions
What does this print? It starts 1,000 goroutines, and each adds one to a counter:
package main
import (
"fmt"
"sync"
)
func main() {
counter := 0
var wg sync.WaitGroup
for range 1000 {
wg.Add(1)
go func() {
defer wg.Done()
counter++ // read, add one, write back: three steps!
}()
}
wg.Wait()
fmt.Println("counter:", counter)
}counter: 947You’d expect 1000, but you’ll often get a smaller, different number on each run. counter++ is really three steps: read the value, add one, write it back. When two goroutines read the same old value at the same moment, one of the updates is lost.
This bug is called a data race: two goroutines access the same variable at the same time, and at least one of them writes. Data races cause wrong results that appear randomly, which makes them very hard to debug.
The race detector
Go has a built-in tool that finds data races while your program runs. Add -race to go run, go build or go test:
go run -race .==================
WARNING: DATA RACE
Read at 0x00c000012148 by goroutine 8:
main.main.func1()
/home/gopher/race/main.go:16 +0x6c
Previous write at 0x00c000012148 by goroutine 7:
main.main.func1()
/home/gopher/race/main.go:16 +0x7e
...
Found 1 data race(s)It points straight at line 16, counter++. Run your tests with -race regularly; it’s one of the best habits in Go. (On Windows the race detector needs a C compiler such as gcc installed; on macOS and Linux it works out of the box.)
sync.Mutex: one at a time
A mutex (short for mutual exclusion) is a lock. Only one goroutine can hold it at a time; the others wait at Lock() until it’s released with Unlock():
package main
import (
"fmt"
"sync"
)
type SafeCounter struct {
mu sync.Mutex
count int
}
func (c *SafeCounter) Inc() {
c.mu.Lock()
defer c.mu.Unlock()
c.count++
}
func (c *SafeCounter) Value() int {
c.mu.Lock()
defer c.mu.Unlock()
return c.count
}
func main() {
var c SafeCounter
var wg sync.WaitGroup
for range 1000 {
wg.Add(1)
go func() {
defer wg.Done()
c.Inc()
}()
}
wg.Wait()
fmt.Println("counter:", c.Value())
}counter: 1000Now the result is always 1000. Good habits with mutexes:
- Keep the mutex next to the data it protects, in the same struct.
- Use
defer mu.Unlock()right afterLock()so you can’t forget to unlock. - Hold the lock for as short a time as possible.
- Never copy a struct that contains a mutex; use pointer receivers (
go vetwarns you).
RWMutex: many readers, one writer
When data is read far more often than it’s written, a sync.RWMutex lets many readers in at once, but writers get exclusive access:
package main
import (
"fmt"
"sync"
)
type Cache struct {
mu sync.RWMutex
data map[string]string
}
func (c *Cache) Get(key string) (string, bool) {
c.mu.RLock() // shared lock: other readers may hold it too
defer c.mu.RUnlock()
v, ok := c.data[key]
return v, ok
}
func (c *Cache) Set(key, value string) {
c.mu.Lock() // exclusive lock
defer c.mu.Unlock()
c.data[key] = value
}
func main() {
c := &Cache{data: map[string]string{}}
c.Set("lang", "Go")
v, ok := c.Get("lang")
fmt.Println(v, ok)
}Go trueMaps are not safe for concurrent use
Writing to a normal map from several goroutines without a lock can crash your program with fatal error: concurrent map writes. Protect shared maps with a mutex, as above.
Atomic counters
For a simple counter, the sync/atomic package is lighter than a mutex. Its operations happen in a single, indivisible step:
package main
import (
"fmt"
"sync"
"sync/atomic"
)
func main() {
var requests atomic.Int64
var wg sync.WaitGroup
for range 500 {
wg.Add(1)
go func() {
defer wg.Done()
requests.Add(1)
}()
}
wg.Wait()
fmt.Println("requests:", requests.Load())
}requests: 500Use atomics for single numbers and flags; use a mutex when you need to update several things together.
sync.Once: do something exactly once
sync.Once runs a function only the first time it’s called, even if many goroutines call it at once. It’s perfect for lazy initialisation, such as loading configuration:
package main
import (
"fmt"
"sync"
)
var (
once sync.Once
config map[string]string
)
func loadConfig() {
fmt.Println("loading config...")
config = map[string]string{"env": "production"}
}
func getConfig() map[string]string {
once.Do(loadConfig)
return config
}
func main() {
var wg sync.WaitGroup
for range 5 {
wg.Add(1)
go func() {
defer wg.Done()
_ = getConfig()
}()
}
wg.Wait()
fmt.Println("env:", getConfig()["env"])
}loading config...
env: production“loading config…” prints once, even though getConfig was called six times.
Pattern: a worker pool
Starting one goroutine per task is fine for a handful of tasks. For thousands of tasks, or tasks that hit a database or an API, you usually want a fixed number of workers that take jobs from a shared channel:
package main
import (
"fmt"
"slices"
"sync"
"time"
)
type Result struct {
Job int
Square int
}
func worker(id int, jobs <-chan int, results chan<- Result, wg *sync.WaitGroup) {
defer wg.Done()
for j := range jobs { // each worker takes jobs until the channel is closed
time.Sleep(10 * time.Millisecond) // simulate work
results <- Result{Job: j, Square: j * j}
}
}
func main() {
jobs := make(chan int)
results := make(chan Result)
var wg sync.WaitGroup
// 1. Start 3 workers
for id := 1; id <= 3; id++ {
wg.Add(1)
go worker(id, jobs, results, &wg)
}
// 2. Send 9 jobs, then close the jobs channel
go func() {
for j := 1; j <= 9; j++ {
jobs <- j
}
close(jobs)
}()
// 3. Close results once every worker has finished
go func() {
wg.Wait()
close(results)
}()
// 4. Collect the results (they arrive in any order)
var all []Result
for r := range results {
all = append(all, r)
}
slices.SortFunc(all, func(a, b Result) int { return a.Job - b.Job })
for _, r := range all {
fmt.Printf("%d^2 = %d\n", r.Job, r.Square)
}
}1^2 = 1
2^2 = 4
3^2 = 9
4^2 = 16
5^2 = 25
6^2 = 36
7^2 = 49
8^2 = 64
9^2 = 81The four steps are the shape of almost every worker pool: start the workers, feed the jobs, close the results when the workers are done, and collect. We sort at the end because results arrive in whatever order the workers finish.
Pattern: limiting concurrency with a semaphore
Sometimes you’re happy to start a goroutine per task but want at most N running at once, for example to stay within an API’s rate limit. A buffered channel makes a simple semaphore:
package main
import (
"fmt"
"sync"
"sync/atomic"
"time"
)
func main() {
const limit = 2
sem := make(chan struct{}, limit)
var running, peak atomic.Int32
var wg sync.WaitGroup
for range 6 {
wg.Add(1)
go func() {
defer wg.Done()
sem <- struct{}{} // take a slot (waits if all slots are taken)
defer func() { <-sem }() // give the slot back
now := running.Add(1)
if now > peak.Load() {
peak.Store(now)
}
time.Sleep(20 * time.Millisecond)
running.Add(-1)
}()
}
wg.Wait()
fmt.Println("never more than", peak.Load(), "at once")
}never more than 2 at onceCancellation with context
Long-running work should stop when it’s no longer needed: when the user cancels, a request times out, or another part of the program fails. The standard way to signal this in Go is a context.Context:
package main
import (
"context"
"fmt"
"time"
)
func slowTask(ctx context.Context, steps int) error {
for i := 1; i <= steps; i++ {
select {
case <-ctx.Done(): // the context was cancelled or timed out
return fmt.Errorf("stopped at step %d: %w", i, ctx.Err())
case <-time.After(100 * time.Millisecond): // one step of work
}
}
return nil
}
func main() {
ctx, cancel := context.WithTimeout(context.Background(), 250*time.Millisecond)
defer cancel() // always release the context's resources
if err := slowTask(ctx, 1); err != nil {
fmt.Println(err)
} else {
fmt.Println("short task finished")
}
if err := slowTask(ctx, 10); err != nil {
fmt.Println(err)
}
}short task finished
stopped at step 2: context deadline exceededConventions you’ll see in every Go codebase:
ctx context.Contextis the first parameter of functions that do I/O or long work.- Pass the context down to everything that function calls (HTTP requests, database queries).
- Always
defer cancel()afterWithTimeoutorWithCancel. - Every HTTP request in a Go web server carries a context that is cancelled when the client disconnects.
Choosing the right tool
| Situation | Reach for |
|---|---|
| Pass data or results between goroutines | channels |
| Wait for a group of goroutines | sync.WaitGroup |
| Several goroutines update shared state | sync.Mutex (or RWMutex) |
| A shared counter or flag | sync/atomic |
| Initialise something once, lazily | sync.Once |
| Many tasks, fixed number of workers | worker pool |
| Stop work on timeout or cancel | context |
Exercises
1. Safe word counter. Several goroutines each count the words of one sentence and add them to a shared map[string]int protected by a mutex. Print the count for "go".
Show solution
package main
import (
"fmt"
"strings"
"sync"
)
func main() {
sentences := []string{
"go is fun",
"go is fast",
"learn go today",
}
var mu sync.Mutex
counts := map[string]int{}
var wg sync.WaitGroup
for _, s := range sentences {
wg.Add(1)
go func() {
defer wg.Done()
for _, w := range strings.Fields(s) {
mu.Lock()
counts[w]++
mu.Unlock()
}
}()
}
wg.Wait()
fmt.Println("go:", counts["go"], "is:", counts["is"])
}go: 3 is: 22. Worker pool for lengths. Use 2 workers to compute the length of each word in []string{"gopher", "channel", "mutex", "go"} and print the total of all lengths.
Show solution
package main
import (
"fmt"
"sync"
)
func main() {
words := []string{"gopher", "channel", "mutex", "go"}
jobs := make(chan string)
lengths := make(chan int)
var wg sync.WaitGroup
for range 2 {
wg.Add(1)
go func() {
defer wg.Done()
for w := range jobs {
lengths <- len(w)
}
}()
}
go func() {
for _, w := range words {
jobs <- w
}
close(jobs)
}()
go func() {
wg.Wait()
close(lengths)
}()
total := 0
for n := range lengths {
total += n
}
fmt.Println("total length:", total)
}total length: 203. Give up after a deadline. Write wait(ctx context.Context, d time.Duration) error that waits for d or returns ctx.Err() if the context ends first. Call it with a 50ms timeout for 10ms and for 1s.
Show solution
package main
import (
"context"
"fmt"
"time"
)
func wait(ctx context.Context, d time.Duration) error {
select {
case <-time.After(d):
return nil
case <-ctx.Done():
return ctx.Err()
}
}
func main() {
for _, d := range []time.Duration{10 * time.Millisecond, time.Second} {
ctx, cancel := context.WithTimeout(context.Background(), 50*time.Millisecond)
err := wait(ctx, d)
cancel()
fmt.Printf("wait %v: %v\n", d, err)
}
}wait 10ms: <nil>
wait 1s: context deadline exceededRecap
- A data race happens when goroutines share a variable and at least one writes;
-racefinds them. sync.Mutexlets one goroutine at a time touch shared data;RWMutexallows many readers.sync/atomichandles simple counters;sync.Onceruns initialisation once.- Worker pools and semaphores control how much work runs at the same time.
contextcarries cancellation and deadlines; it’s the first parameter of long-running functions.
That completes Part 5. In Part 6 you’ll put Go to work on real tasks, starting with a tour of the standard library.
Finished this lesson?
Your progress is saved in this browser only.