In Go is it possible to iterate over a custom type?

GoIterator

Go Problem Overview


I have a custom type which internally has a slice of data.

Is it possible, by implementing some functions or an interface that the range operator needs, to iterate (using range) over my custom type?

Go Solutions


Solution 1 - Go

The short answer is no.

The long answer is still no, but it's possible to hack it in a way that it sort of works. But to be clear, this is most certainly a hack.

There are a few ways you can do it, but the common theme between them is that you want to somehow transform your data into a type that Go is capable of ranging over.

Approach 1: Slices

Since you mentioned that you have a slice internally, this may be easiest for your use case. The idea is simple: your type should have an Iterate() method (or similar) whose return value is a slice of the appropriate type. When called, a new slice is created containing all of the elements of the data structure in whatever order you'd like them to be iterated over. So, for example:

func (m *MyType) Iterate() []MyElementType { ... }

mm := NewMyType()
for i, v := range mm.Iterate() {
    ...
}

There are a few concerns here. First, allocation - unless you want to expose references to internal data (which, in general, you probably don't), you have to make a new slice and copy all of the elements over. From a big-O standpoint, this isn't that bad (you're doing a linear amount of work iterating over everything anyway), but for practical purposes, it may matter.

Additionally, this doesn't handle iterating over mutating data. This is probably not an issue most of the time, but if you really want to support concurrent updates and certain types of iteration semantics, you might care.

Approach 2: Channels

Channels are also something that can be ranged over in Go. The idea is to have your Iterate() method spawn a goroutine that will iterate over the elements in your data structure, and write them to a channel. Then, when the iteration is done, the channel can be closed, which will cause the loop to finish. For example:

func (m *MyType) Iterate() <-chan MyElementType {
    c := make(chan MyElementType)
    go func() {
        for _, v := range m.elements {
            c <- v
        }
        close(c)
    }()
    return c
}

mm := NewMyType()
for v := range mm.Iterate() {
    ...
}

There are two advantages of this method over the slice method: first, you don't have to allocate a linear amount of memory (although you may want to make your channel have a bit of a buffer for performance reasons), and second, you can have your iterator play nicely with concurrent updates if you're into that sort of thing.

The big downside of this approach is that, if you're not careful, you can leak goroutines. The only way around this is to make your channel have a buffer deep enough to hold all of the elements in your data structure so that the goroutine can fill it and then return even if no elements are read from the channel (and the channel can then later be garbage collected). The problem here is that, a) you're now back to linear allocation and, b) you have to know up-front how many elements you're going to write, which sort of puts a stop to the whole concurrent-updates thing.

The moral of the story is that channels are cute for iterating, but you probably don't want to actually use them.

Approach 3: Internal Iterators

Credit to hobbs for getting to this before me, but I'll cover it here for completeness (and because I want to say a bit more about it).

The idea here is to create an iterator object of sorts (or to just have your object only support one iterator at a time, and iterate on it directly), just like you would in languages that support this more directly. What you do, then, is call a Next() method which, a) advances the iterator to the next element and, b) returns a boolean indicating whether or not there's anything left. Then you need a separate Get() method to actually get the value of the current element. The usage of this doesn't actually use the range keyword, but it looks pretty natural nonetheless:

mm := MyNewType()
for mm.Next() {
    v := mm.Get()
    ...
}

There are a few advantages of this technique over the previous two. First, it doesn't involve allocating memory up-front. Second, it supports errors very naturally. While it's not really an iterator, this is exactly what bufio.Scanner does. Basically the idea is to have an Error() method which you call after iteration is complete to see whether iteration terminated because it was done, or because an error was encountered midway through. For purely in-memory data structures this may not matter, but for ones that involve IO (e.g., walking a filesystem tree, iterating over database query results, etc), it's really nice. So, to complete the code snippet above:

mm := MyNewType()
for mm.Next() {
    v := mm.Get()
    ...
}
if err := mm.Error(); err != nil {
    ...
}

Conclusion

Go doesn't support ranging over arbitrary data structures - or custom iterators - but you can hack it. If you have to do this in production code, the third approach is 100% the way to go, as it is both the cleanest and the least of a hack (after all, the standard library includes this pattern).

Solution 2 - Go

No, not using range. range accepts arrays, slices, strings, maps, and channels, and that's it.

The usual sort of idiom for iterable things (for example a bufio.Scanner) seems to be

iter := NewIterator(...)
for iter.More() {
    item := iter.Item()
    // do something with item
}

but there's no universal interface (wouldn't be very useful given the type system anyway) and different types that implement the pattern generally have different names for their More and Item methods (for example Scan and Text for a bufio.Scanner)

Solution 3 - Go

joshlf gave an excellent answer, but I'd like to add a couple of things:

Using channels

A typical problem with channel iterators is that you have to range through the entire data structure or the goroutine feeding the channel will be left hanging forever. But this can be quite easily circumvented, here's one way:

func (s intSlice) chanIter() chan int {
	c := make(chan int)
	go func() {
		for _, i := range s {
			select {
			case c <- i:
			case <-c:
				close(c)
				return
			}
		}
		close(c)
	}()
	return c
}

In this case writing back to the iterator channel interrupts the iteration early:

s := intSlice{1, 2, 3, 4, 5, 11, 22, 33, 44, 55}
c := s.chanIter()
for i := range c {
	fmt.Println(i)
	if i > 30 {
		// Send to c to interrupt
		c <- 0
	}
}

Here it is very important that you don't simply break out of the for loop. You can break, but you must must write to the channel first to ensure the goroutine will exit.

Using closures

A method of iteration I often tend to favour is to use an iterator closure. In this case the iterator is a function value, which, when called repeatedly, returns the next element and indicates whether the iteration can continue:

func (s intSlice) cloIter() func() (int, bool) {
	i := -1
	return func() (int, bool) {
		i++
		if i == len(s) {
			return 0, false
		}
		return s[i], true
	}
}

Use it like this:

iter := s.cloIter()
for i, ok := iter(); ok; i, ok = iter() {
    fmt.Println(i)
}

In this case it's perfectly ok to break out of the loop early, iter will eventually be garbage collected.

Playground

Here's the link to the implementations above: http://play.golang.org/p/JC2EpBDQKA

Solution 4 - Go

There is another option that wasn't mentioned.

You can define a Iter(fn func(int)) function which accepts some function that will be called for each item in your custom type.

type MyType struct {
    data []int
}

func (m *MyType) Iter(fn func(int)) {
    for _, item := range m.data {
	    fn(item)
    }
}

And it can be used like this:

d := MyType{
	data: []int{1,2,3,4,5},
}

f := func(i int) {
	fmt.Println(i)
}
d.Iter(f)

Playground
Link to working implementation: https://play.golang.org/p/S3CTQmGXj79

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionKoala3View Question on Stackoverflow
Solution 1 - GojoshlfView Answer on Stackoverflow
Solution 2 - GohobbsView Answer on Stackoverflow
Solution 3 - GoLemurFromTheIdView Answer on Stackoverflow
Solution 4 - GoLu.nemecView Answer on Stackoverflow