Skip to main content
  1. Articles/

Go Tutorial: Iterators

·2086 words·10 mins· loading · loading ·
Marko Milojevic
Author
Marko Milojevic
Software engineer and architect. Golang and LLM enthusiast. Awful chess player, gym rat, harmonica newbie and cat lover.
Iterators in Go - This article is part of a series.
Part 1: This Article

Go 1.22 introduced range over functions, and Go 1.23 brought the iter package to go with it. Together they gave iterators a proper place in the language. Before that, iterating over custom data structures meant either returning slices upfront — loading everything into memory — or writing callback-based helpers that nobody could agree on naming. I have seen both approaches and neither felt right.

The core idea behind iterators is straightforward: instead of computing all values upfront and handing them back as a list, you compute each value on demand and yield it to the caller one at a time. The caller controls when to stop. This matters any time you are working with large or potentially infinite sequences.

This article walks through why iterators exist in Go, how the yield-based pattern works, what the iter package provides, and where the current limits of the feature sit.

Why do we need Iterators?
#

The simplest case for iteration is a slice of numbers. You range over it, print each value, move on.

func main() {
    numbers := []int{1, 2, 3, 4, 5}
    for _, i := range numbers {
        fmt.Println(i)
    }
}
// OUT:
// 1
// 2
// 3
// 4
// 5

That works fine until the collection gets large. If you need to generate a million numbers, you have to allocate memory for all of them before you can even start ranging.

func main() {
    n := 1_000_000
	numbers := make([]int, n)

	for i := range numbers {
		numbers[i] = i * 2
	}
	for _, i := range numbers {
		fmt.Println(i)
	}
}
// OUT:
// 1
// 2
// ...

You can always add a break once you hit your threshold, but the damage is already done — the entire slice was allocated upfront. In other cases, you might not even know how many items you will need. The for range loop can iterate for some time, until it reaches the breakpoint, depending on some value provided in the item. In such cases, the size of such a list must be not just too big, but absolutely unpredictable.

func main() {
    n := 1_000_000
	numbers := make([]int, n)

	for i := range numbers {
		numbers[i] = i * 2
	}
	for _, i := range numbers {
		fmt.Println(i)
		if i > 10 {
		    break
		}
	}
}
// OUT:
// 1
// 2
// 4
// 8
// 10
// 12

In a real application, the decision about when to stop often happens dynamically — driven by user input, a timeout, or a condition that evaluates to true before the fifth item. Allocating a million items and then breaking on the fifth is wasteful. This is exactly the problem iterators solve.

Iterators in Go
#

An iterator in Go is a function that accepts a yield function as its argument. For each item in the sequence, it calls yield with that item. If yield returns false — which happens when the caller breaks out of the loop or returns from the enclosing function — the iterator should stop too.

func iterateNumbers(yield func(number int) bool) {
	numbers := []int{10, 20, 30, 40, 50}
	for _, number := range numbers {
		if !yield(number) { // returns false on  break, return or no more items
			break
		}
	}
}

func main() {
	for i := range iterateNumbers {
		fmt.Println(i)
	}
}
// OUT:
// 10
// 20
// 30
// 40
// 50

The yield function returns true as long as the caller wants to continue, and false the moment they stop. Checking that return value and breaking out of the internal loop is what makes the iterator cooperate with us in the first place.

func main() {
	for i := range iterateNumbers {
		fmt.Println(i)
		break
	}
}
// OUT:
// 10

Break works exactly as expected. The iterator gets the signal on the next yield call and stops cleanly.

func main() {
    for i := range iterateNumbers {
        if i%20 == 0 {
            continue
        }
        fmt.Println(i)
    }
}
// OUT:
// 10
// 30
// 50

Here continue does not interrupt the loop — it skips the current iteration body and moves to the next value. The iterator keeps running.

One thing worth noting: yield is not a reserved word in Go. You can name the yield function parameter whatever you like, like in the example below.

func iterateNumbers(continueIteration func(number int) bool) {
	numbers := []int{10, 20, 30, 40, 50}
	for _, number := range numbers {
		if !continueIteration(number) { // returns false on  break, return or no more items
			break
		}
	}
}

func main() {
	for i := range iterateNumbers {
		fmt.Println(i)
	}
}
// OUT:
// 10
// 20
// 30
// 40
// 50

The name yield has become the convention because it communicates intent clearly, known from other programming languages, but that is not the case in Go. In the example above, the yield function is called continueIteration, which clearly indicates that you can call it whatever you like.

How to fix memory allocation?
#

The real power of iterators shows up when you generate values lazily — computing each item only when the caller asks for it, with no upfront allocation.

func iterateNumbersDynamically(yield func(number int) bool) {
	number := 0
	for {
		if !yield(number) { // returns false on  break, return or no more items
			break
		}
		number += 2
	}
}

func main() {
	for i := range iterateNumbersDynamically {
		fmt.Println(i)
		if i > 10 {
			break
		}
	}
}
// OUT:
// 0
// 2
// 4
// 6
// 8
// 10
// 12

This iterator has no internal collection at all. It computes the next value from the previous one and yields it. You can iterate over as many items as you need without allocating memory for items you will never see. As you can see, the million-number problem from earlier is gone.

Introduction to iter package
#

Go 1.23 formalized the iterator pattern by introducing the iter package and two named types.

package iter

type Seq[K any] func(yield func(K) bool)

Here Seq is a generic function type that represents a sequence of single values. Any function that matches this signature can be used directly in a for loop. It wraps the pattern we built manually in the previous sections into a named, reusable type.

func iterateWithIter() iter.Seq[int] {
	return func(yield func(int) bool) {
		number := 0
		for {
			if !yield(number) { // returns false on  break, return or no more items
				break
			}
			number += 3
		}
	}
}

func main() {
	for i := range iterateWithIterator() {
		fmt.Println(i)
		if i > 10 {
			break
		}
	}
}
// OUT:
// 0
// 3
// 6
// 9
// 12

For sequences of pairs — like key-value, index-value, or value-error — there is Seq2.

package iter

type Seq2[K, V any] func(yield func(K, V) bool)

So the example with Seq2 might look like this:

func iterateWithIter2() iter.Seq2[int, string] {
	return func(yield func(int, string) bool) {
		number := 0
		text := fmt.Sprintf("number: %d", number)
		for {
			if !yield(number, text) {
				break
			}
			number += 3
			text = fmt.Sprintf("number: %d", number)
		}
	}
}

func main() {
	for i, v := range iterateWithIter2() {
		fmt.Println(i, v)
		if i > 10 {
			break
		}
	}
}
// OUT:
// 0 number: 0
// 3 number: 3
// 6 number: 6
// 9 number: 9
// 12 number: 12

A practical use of Seq2 is reading a file line by line. Instead of loading the entire file into memory, you yield one line at a time alongside any error that occurred while scanning.

type FileReader struct {
	filePath string
}

func (f *FileReader) ReadLine() iter.Seq2[string, error] {
	return func(yield func(string, error) bool) {
		file, err := os.Open(f.filePath)
		if err != nil {
			yield("before lines", err)
			return
		}
		defer file.Close()
		scanner := bufio.NewScanner(file)
		for scanner.Scan() {
			line := scanner.Text()
			err := scanner.Err()
			if err != nil {
				yield("", err)
				return
			}
			if !yield("new line: "+line, nil) {
				return
			}
		}
	}
}

func main() {
	reader := FileReader{filePath: "./file_reader/main.go"}
	for s, e := range reader.ReadLine() {
		fmt.Println(s, e)
	}
}
// new line: type FileReader struct { <nil>
// new line: 	filePath string <nil>
// new line: } <nil>
// new line:  <nil>
// new line: func (f *FileReader) ReadLine() iter.Seq2[string, error] { <nil>
// new line: 	return func(yield func(string, error) bool) { <nil>
// ...

The first value is the line content, the second is any error that surfaced during the scan. The caller handles both in the same loop where they consume the lines, which keeps the error handling close to the data. This example is one concrete useful application of Seq2 and iterators in general. During the execution of the application, you can’t predict how many lines there will be in the file - and there can be very many of them. The Seq2 type is a perfect fit for this, because it allows you to yield each line, and break at any point when you find the line which might match your regex pattern, for example.

Functions Pull and Pull2 from iter package
#

Types Seq and Seq2 are push-based: the iterator drives the loop by calling yield. Sometimes you want the opposite — a pull-based model where you ask for the next value explicitly. The iter package provides iter.Pull and iter.Pull2 for this.

func iterateWithIter() iter.Seq[string] {
	return func(yield func(string) bool) {
		counter := 0
		number := 0
		text := fmt.Sprintf("number: %d", number)
		for {
			if !yield(text) { // returns false on  break, return or no more items
				break
			}
			number += 3
			text = fmt.Sprintf("number: %d", number)
			counter++
			if counter == 5 {
				yield(text)
				return
			}
		}
	}
}

func main() {
	next, stop := iter.Pull(iterateWithIter())
	defer stop()

	for {
		k, ok := next()
		if !ok {
			break
		}
		fmt.Println(k)
	}
}
// OUT:
// number: 0
// number: 3
// number: 6
// number: 9
// number: 12
// number: 15

Here iter.Pull returns two functions: next and stop. Each call to next returns the next value in the sequence and a boolean indicating whether returned value is an actual item. When the sequence is exhausted, next returns the zero value of type V and false. Function stop stops the iterator and releases all resources associated with it.

func iterateWithIter2() iter.Seq2[int, string] {
	return func(yield func(int, string) bool) {
		counter := 0
		number := 0
		text := fmt.Sprintf("number: %d", number)
		for {
			if !yield(number, text) { // returns false on  break, return or no more items
				break
			}
			number += 3
			text = fmt.Sprintf("number: %d", number)
			counter++
			if counter == 5 {
				yield(number, text)
				return
			}
		}
	}
}

func main() {
	next2, stop2 := iter.Pull2(iterateWithIter2())
	defer stop2()

	for {
		k, v, ok := next2()
		if !ok {
			break
		}
		fmt.Println(k, v)
	}
}
// OUT:
// 0 number: 0
// 3 number: 3
// 6 number: 6
// 9 number: 9
// 12 number: 12
// 15 number: 15

With iter.Pull2 it is the same idea applied to Seq2. Its next function returns three values: the two from the pair and the boolean.

Can we use iterator for yielding triplets?
#

The iter package stops at pairs. There is no Seq3. Sure, you can define the type yourself:

type Seq3[K, V, T any] func(yield func(K, V, T) bool)

But you cannot use it in a for loop. The language spec for range loops only recognises functions whose yield takes one or two arguments. Three is outside that boundary.

func iterateWithIterator3() Seq3[int, string, string] {
	return func(yield func(int, string, string) bool) {
		/// ...
	}
}

Honestly, this is not much of a limitation in practice. If you need to yield three or more values together, wrap them in a struct and use Seq or Seq2. A struct is a cleaner holder for multiple related values than a growing list of type parameters anyway.

Conclusion
#

Go’s iterator model is a small surface area with meaningful depth. The yield-based push model integrates cleanly with for loops, and iter.Pull covers the cases where you need explicit control over when the next value is fetched. The standard library types Seq and Seq2 give you named anchors to build on. The memory allocation problem that motivated this feature — allocating everything upfront just to break early — has a clean solution now.

Useful Resources
#

Iterators in Go - This article is part of a series.
Part 1: This Article

Related

Golang Release 1.22: version

·823 words·4 mins· loading · loading
With the release of Go 1.22, the Go standard library introduced several new features. As you might have noticed in articles related to the previous release, here we mostly concentrate on the new exciting packages and features that they give us. This article will start this journey, by providing a deeper look into the implementation of the version package in Go. Lang # The first function we are ready to examine is the Lang function. This function provides a cleaned, valid Go version as a string. In case it can’t determine the actual version, due to an invalid state of the string value, it will return an string as a result. Lang function func Lang(x string) string As we can see the function signature above, function expects one argument, a string, that represents a Go version. An output should be also one value, a string, as a cleaned Go version. Lang function examples package main import ( "fmt" "go/version" ) func main() { fmt.Println(version.Lang("go1.0")) // go1 fmt.Println(version.Lang("go1")) // go1 fmt.Println(version.Lang("go1.22.4")) // go1.22 fmt.Println(version.Lang("go1.22.3")) // go1.22 fmt.Println(version.Lang("go1.22.2")) // go1.22 fmt.Println(version.Lang("go1.22.rc1")) // fmt.Println(version.Lang("go1.22rc1")) // go1.22 fmt.Println(version.Lang("1.22")) // fmt.Println(version.Lang("wrong")) // fmt.Println(version.Lang("")) // } In the example above, we can see how the Lang function adapt the Go version string. It removes all minor versions and appearance of “release candide” phrase, and present them in the end as an official Go versions that we experienced in the past (and we might experience in the future). In cases where we provided an invalid, or empty string, the ending result will be also an empty string, as the Lang function can’t find the actual version name. One interesting point, not just for the Long function, but, as you will see, for all functions in this package, to consider some string as a valid Go version, it needs to have a prefix go. IsValid # The next function we are examining is the IsValid function. This function checks a string with a potential Go version and returns a boolean result that tells us if the version is valid or not. IsValid function func IsValid(x string) bool As we can see the function signature above, function expects one argument, a string, that represents a Go version. An output should be a bool value, which tells us if the Go version is valid or not.

Golang Release 1.21: slices - Part 2

·2301 words·11 mins· loading · loading
With the release of Go 1.21, the Go standard library introduced several new features. While we’ve already discussed some of them in previous articles, in this episode, we’ll dive into more advanced enhancements. Naturally, we’ll focus on the new functions designed for sorting slices, which are part of the new slices package. This article will provide a deeper look into the implementation of these three new functions and touch on benchmarking as well. Sort # The Sort function is the first one we’d like to explore. This implementation is built upon the enhanced Pattern-defeating Quicksort, positioning it as one of the best-known unstable sorting algorithms. Don’t worry; we will discuss this “instability” aspect in this article. But first, let’s take a look at the function’s signature: Sort function func Sort[S ~[]E, E cmp.Ordered](x S) As we’ve seen in some other articles, nearly all improvements in the Go standard library are built upon generics, a feature introduced in Go version 1.18, almost three years ago. Similar to other functions, the Sort function also expects a slice of a generic type as an argument, where each item must adhere to the Ordered constraint. The function doesn’t return a new value but sorts the original slice in place. Below, you’ll find some basic examples: Sort function examples ints := []int{1, 2, 3, 5, 5, 7, 9} slices.Sort(ints) fmt.Println(ints) // Output: // 1 2 3 5 5 7 9 ints2 := []int{9, 7, 5, 5, 3, 2, 1} slices.Sort(ints2) fmt.Println(ints2) // Output: // 1 2 3 5 5 7 9 floats := []float64{9, 3, 5, 7, 1, 2, 5} slices.Sort(floats) fmt.Println(floats) // Output: // 1 2 3 5 5 7 9 strings := []string{"3", "9", "2", "5", "1", "7", "5"} slices.Sort(strings) fmt.Println(strings) // Output: // 1 2 3 5 5 7 9 In the example above, we can observe the result of the Sort method. All the outputs consist of sorted slices, arranged in ascending order. However, what makes this function particularly intriguing is its ability to handle various data types using a single function, distinguishing it from the implementations we already possess in the sort package. Now that we’ve examined the results, let’s proceed to compare the performance benchmarks with the existing package. Benchmark # In this section, we aim to evaluate the performance of the new function by comparing it to the already existing sort package. Below, you’ll find the benchmark test results:

Golang Release 1.21: slices - Part 1

·1939 words·10 mins· loading · loading
As part of the new Go release, several exciting changes have been introduced to the Go ecosystem. While we’ve explored some of these changes in other articles about the maps package and the cmp package, there’s much more to discover beyond these two packages. In this article, we’ll focus on the first part of the slices package, specifically its new search functionality. Like many other updates and newly introduced packages, this one is also built upon the foundation of generics, which were introduced in Go 1.18. BinarySearch and BinarySearchFunc # Let’s start by exploring the first pair of functions designed for efficiently searching a target value within sorted slices. In this context, we’re referring to the well-known Binary Search algorithm, which is renowned as one of the most significant algorithms and is frequently used in coding interviews. Below, you’ll find the signatures of both of these functions: BinarySearch function func BinarySearch[S ~[]E, E cmp.Ordered](x S, target E) (int, bool) BinarySearchFunc function func BinarySearchFunc[S ~[]E, E, T any](x S, target T, cmp func(E, T) int) (int, bool) Looking at the signatures of both functions, we can identify some small differences between them, and these differences serve specific purposes. The first function, BinarySearch, expects two arguments. The first argument should be a slice of sorted items, and it must adhere to the Ordered constraint. When the items are ordered, the algorithm can efficiently compare them using the Compare function from the cmp package. On the other hand, the second function, BinarySearchFunc, is more versatile. It allows searching within slices where the items don’t necessarily conform to the Ordered constraint. This flexibility is achieved by introducing a third argument, the comparison function. This function is responsible for comparing items and determining their order. It will be called by the BinarySearchFunc itself to make comparisons. Both functions return two values. The first value is the index of the item within the slice, and the second is a boolean value indicating whether the item was found in the slice or not. Let’s explore some examples below: BinarySearch examples fmt.Println(slices.BinarySearch([]int{1, 3, 5, 6, 7}, 5)) // Output: // 2 true fmt.Println(slices.BinarySearch([]int{1, 3, 5, 6, 7}, 9)) // Output: // 5 false fmt.Println(slices.BinarySearch([]int{1, 3, 5, 6, 7}, -5)) // Output: // 0 false fmt.Println(slices.BinarySearch([]string{"1", "3", "5", "6", "7"}, "5")) // Output: // 2 true fmt.Println(slices.BinarySearch([]string{"1", "3", "5", "6", "7", "8"}, "9")) // Output: // 6 false fmt.Println(slices.BinarySearch([]string{"1", "3", "5", "6", "7"}, "4")) // Output: // 2 false Take a close look at the results returned by the BinarySearch function, especially when the item doesn’t exist in the slice. In our examples, we encountered four such cases where the function returned 0, 2, 5, and 6. When the requested item isn’t present in the slice, the function indicates where it should be positioned if it were to be added to the slice. Since the slice is sorted, it’s possible to determine the appropriate position for the item within the slice.