Lesson 20 of 20 · Part 6: Real-world Go
Project: build a word-count CLI
Build a complete Go project step by step. A word-frequency command-line tool with packages, flags, file reading, maps, sorting, error handling and table-driven tests.
6 min read · Updated · Beginner friendly
What you'll learn
- Plan and structure a small Go project as a module with packages
- Combine maps, slices, structs, methods, errors and io.Reader in real code
- Add command-line flags and read from files or standard input
- Test the core logic and build a shareable executable
Time to put everything together. In this lesson you’ll build wordfreq, a command-line tool that reads text and reports how many lines, words and characters it has, plus its most frequent words with a little bar chart:
lines: 3 words: 37 characters: 176
Top 5 words:
1. the 7 ███████
2. code 3 ███
3. and 2 ██
4. dig 2 ██
5. gopher 2 ██Along the way you’ll use modules and packages (lesson 15), io.Reader (lesson 13), maps (lesson 7), structs and methods (lessons 8 and 12), sorting (lesson 6), errors (lesson 11), flags and files (lesson 18) and tests (lesson 19).
Step 1: Plan the program
Before writing code, decide what the tool should do:
- Read text from one or more files given as arguments, or from standard input if there are none, so it works with pipes:
cat book.txt | wordfreq. - Count lines, words and characters.
- Count how often each word appears, ignoring case and punctuation, so “The” and “the,” are the same word.
- Print the top N words; N comes from a
-topflag with a default of 5.
Then split the work:
- A
statspackage does the counting. It knows nothing about files, flags or printing, which makes it easy to test. - The
mainpackage handles the outside world: flags, opening files, printing and exit codes.
wordfreq/
├── go.mod
├── main.go package main: flags, files, output
└── stats/
├── stats.go package stats: the counting logic
└── stats_test.go tests for the counting logicStep 2: Create the module
mkdir wordfreq
cd wordfreq
go mod init example.com/wordfreq
mkdir statsStep 3: Count lines, words and characters
The heart of the program is a Count function. It accepts an io.Reader, not a file name, so the same function works with files, standard input, network connections and, in tests, plain strings.
// Package stats counts lines, words and word frequencies in text.
package stats
import (
"bufio"
"cmp"
"io"
"slices"
"strings"
"unicode"
)
// Stats holds the counts for a piece of text.
type Stats struct {
Lines int
Words int
Chars int
Freq map[string]int
}
// WordCount is one word and how often it appeared.
type WordCount struct {
Word string
Count int
}
// Count reads all of r and returns its statistics.
// Words are lower-cased, and punctuation around them is ignored.
func Count(r io.Reader) (Stats, error) {
s := Stats{Freq: make(map[string]int)}
scanner := bufio.NewScanner(r)
for scanner.Scan() {
line := scanner.Text()
s.Lines++
s.Chars += len([]rune(line))
for _, w := range splitWords(line) {
s.Words++
s.Freq[strings.ToLower(w)]++
}
}
if err := scanner.Err(); err != nil {
return Stats{}, err
}
return s, nil
}
// splitWords splits a line into words made of letters, digits and apostrophes.
func splitWords(line string) []string {
return strings.FieldsFunc(line, func(r rune) bool {
return !unicode.IsLetter(r) && !unicode.IsDigit(r) && r != '\''
})
}
// Top returns the n most frequent words, most frequent first.
// Words with the same count are sorted alphabetically.
func (s Stats) Top(n int) []WordCount {
list := make([]WordCount, 0, len(s.Freq))
for w, c := range s.Freq {
list = append(list, WordCount{Word: w, Count: c})
}
slices.SortFunc(list, func(a, b WordCount) int {
if a.Count != b.Count {
return cmp.Compare(b.Count, a.Count) // higher counts first
}
return cmp.Compare(a.Word, b.Word)
})
if n < len(list) {
list = list[:n]
}
return list
}Let’s look at the key decisions:
bufio.Scannerreads the input one line at a time, so even a huge file uses little memory.len([]rune(line))counts characters, not bytes, so “café” counts as 4 (lesson 4). Line breaks aren’t counted.strings.FieldsFuncsplits a line wherever the function returnstrue. Anything that isn’t a letter, digit or apostrophe separates words, so"tunnel,"becomestunneland"don't"stays one word.s.Freq[...]++relies on a missing map key giving0(lesson 7).Topis a method onStats. It turns the map into a slice of structs, then sorts by count, from high to low, and alphabetically for ties, so the output is always the same. Remember: map iteration order is random, so sorting is essential here.splitWordsis lowercase: an internal helper that other packages can’t call.
Step 4: Test the logic
Before building the command, make sure the counting is right. Table-driven tests (lesson 19) make it easy to cover edge cases:
package stats
import (
"slices"
"strings"
"testing"
)
func TestCount(t *testing.T) {
tests := []struct {
name string
input string
lines, words, chars int
}{
{"empty", "", 0, 0, 0},
{"one line", "Go is fun", 1, 3, 9},
{"two lines", "Hello, world!\nHello again.", 2, 4, 25},
{"punctuation only", "... !!!", 1, 0, 7},
{"apostrophes", "don't stop", 1, 2, 10},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := Count(strings.NewReader(tt.input))
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if got.Lines != tt.lines || got.Words != tt.words || got.Chars != tt.chars {
t.Errorf("Count(%q) = %d lines, %d words, %d chars; want %d, %d, %d",
tt.input, got.Lines, got.Words, got.Chars, tt.lines, tt.words, tt.chars)
}
})
}
}
func TestTop(t *testing.T) {
s, _ := Count(strings.NewReader("the cat and the dog and THE bird"))
got := s.Top(2)
want := []WordCount{{"the", 3}, {"and", 2}}
if !slices.Equal(got, want) {
t.Errorf("Top(2) = %v, want %v", got, want)
}
if all := s.Top(100); len(all) != 5 {
t.Errorf("Top(100) returned %d words, want 5", len(all))
}
}strings.NewReader turns a string into an io.Reader. This is the payoff for accepting io.Reader in Count: no test files needed.
go test ./...? example.com/wordfreq [no test files]
ok example.com/wordfreq/stats 0.421sStep 5: The command
Now the main package. It parses the -top flag, opens the files (or uses standard input), calls stats.Count and prints the report:
// Command wordfreq prints line, word and character counts for text,
// followed by the most frequent words.
//
// Usage:
//
// wordfreq [-top n] [file ...]
//
// With no files, wordfreq reads from standard input.
package main
import (
"flag"
"fmt"
"io"
"os"
"strings"
"example.com/wordfreq/stats"
)
func main() {
top := flag.Int("top", 5, "number of most frequent words to show")
flag.Parse()
if err := run(os.Stdout, flag.Args(), *top); err != nil {
fmt.Fprintln(os.Stderr, "wordfreq:", err)
os.Exit(1)
}
}
func run(out io.Writer, files []string, top int) error {
var input io.Reader = os.Stdin
if len(files) > 0 {
readers := make([]io.Reader, 0, len(files))
for _, name := range files {
f, err := os.Open(name)
if err != nil {
return err
}
defer f.Close()
readers = append(readers, f)
}
input = io.MultiReader(readers...)
}
s, err := stats.Count(input)
if err != nil {
return fmt.Errorf("reading input: %w", err)
}
fmt.Fprintf(out, "lines: %d words: %d characters: %d\n", s.Lines, s.Words, s.Chars)
if s.Words == 0 {
return nil
}
words := s.Top(top)
fmt.Fprintf(out, "\nTop %d words:\n", len(words))
for i, wc := range words {
bar := strings.Repeat("█", wc.Count)
fmt.Fprintf(out, "%2d. %-10s %3d %s\n", i+1, wc.Word, wc.Count, bar)
}
return nil
}Design notes:
mainis tiny. It parses flags, callsrun, and turns an error into a message on standard error and a non-zero exit code. Command-line tools report failure through exit codes, so scripts can detect it.runreturns an error instead of callingos.Exititself, and writes to anio.Writerinstead of straight toos.Stdout. Both make it easy to test later.io.MultiReaderjoins several readers into one, so several files are counted as one long text.defer f.Close()closes each file whenrunreturns.- The doc comment at the top describes the command;
go docshows it.
Step 6: Run it
Create a text file to try it on:
The gopher dug a tunnel, and the tunnel led to code.
The code was fast, the code was clear, and light became the load.
So if you dig, then dig with Go: the gopher knows the road.go run . poem.txtlines: 3 words: 37 characters: 176
Top 5 words:
1. the 7 ███████
2. code 3 ███
3. and 2 ██
4. dig 2 ██
5. gopher 2 ██Use the flag to change how many words are shown:
go run . -top 3 poem.txtlines: 3 words: 37 characters: 176
Top 3 words:
1. the 7 ███████
2. code 3 ███
3. and 2 ██Pipe text in through standard input. Here there are only 4 different words, so only 4 are shown:
echo "to be or not to be" | go run .lines: 1 words: 6 characters: 18
Top 4 words:
1. be 2 ██
2. to 2 ██
3. not 1 █
4. or 1 █And see what happens with a file that doesn’t exist:
go run . missing.txtwordfreq: open missing.txt: no such file or directory
exit status 1The error goes to standard error and the exit status is 1. (On Windows the message ends with “The system cannot find the file specified.”)
The -h flag prints help generated by the flag package. Here it is for the built program from step 7; with go run, the first line shows a temporary file path instead of wordfreq:
./wordfreq -hUsage of ./wordfreq:
-top int
number of most frequent words to show (default 5)Step 7: Build and share it
go buildThis creates a single executable, wordfreq (or wordfreq.exe on Windows), that you can run from anywhere and copy to any machine with the same operating system. Go can even cross-compile for other systems by setting two environment variables:
GOOS=linux GOARCH=amd64 go build -o wordfreq-linux
GOOS=windows GOARCH=amd64 go build -o wordfreq.exe
GOOS=darwin GOARCH=arm64 go build -o wordfreq-mac(On Windows PowerShell, set them with $env:GOOS="linux" first.) If you publish the code on GitHub, anyone can install it with go install github.com/yourname/wordfreq@latest.
What you used
| Concept | Where it appears |
|---|---|
| Modules and packages (lesson 15) | example.com/wordfreq with a separate stats package |
| Exported and unexported names | Count, Top are public; splitWords is private |
| Structs and methods (8, 12) | Stats, WordCount, Stats.Top |
| Maps (7) | Freq map[string]int with ++ counting |
| Slices and sorting (6) | slices.SortFunc with cmp.Compare |
| Interfaces (13) | io.Reader input, io.Writer output |
| Errors (11) | wrapping with %w, exit codes |
| Closures (9) | the function passed to strings.FieldsFunc |
| Files and flags (18) | os.Open, io.MultiReader, flag.Int |
| Tests (19) | table-driven tests with strings.NewReader |
Challenges: make it your own
Try extending the tool. Each challenge practises something from the tutorial:
- Ignore common words. Add a
-skipflag (for example-skip=the,and,a) and leave those words out of the top list. (strings.Split, maps) - Minimum length. Add
-min 4to only count words with at least four letters. (flags, runes) - JSON output. Add
-jsonto print the result as JSON instead of text. (encoding/json, struct tags) - Test
run. Write a test that callsrunwith astrings.Builderas the writer and a temporary file, and checks the output. (testing, io.Writer) - Many files at once. Count each file in its own goroutine and combine the results. (goroutines, channels or a mutex)
- Serve it. Wrap
stats.Countin an HTTP handler that accepts text in a POST request and returns JSON. (net/http, httptest)
Where to go next
Congratulations, you’ve finished the tutorial! 🎉 You’ve gone from “Hello, World!” to a tested, structured command-line tool. Here are good next steps:
- A Tour of Go, the official interactive tour, is a great quick review.
- Effective Go and the Go Code Review Comments explain idiomatic style.
- pkg.go.dev to explore the standard library and third-party packages.
- Build things. A REST API with
net/httpand a database, a CLI for a task you do every week, or a concurrent web crawler. Building real projects is the fastest way to grow.
There are also practical articles on the t2run blog, such as running Go on AWS Lambda. Happy coding, Gopher!
Finished this lesson?
Your progress is saved in this browser only.