Multiple goroutines listening on one channel
GoConcurrencyGoroutineGo Problem Overview
I have multiple goroutines trying to receive on the same channel simultaneously. It seems like the last goroutine that starts receiving on the channel gets the value. Is this somewhere in the language spec or is it undefined behaviour?
c := make(chan string)
for i := 0; i < 5; i++ {
go func(i int) {
<-c
c <- fmt.Sprintf("goroutine %d", i)
}(i)
}
c <- "hi"
fmt.Println(<-c)
Output:
goroutine 4
EDIT:
I just realized that it's more complicated than I thought. The message gets passed around all the goroutines.
c := make(chan string)
for i := 0; i < 5; i++ {
go func(i int) {
msg := <-c
c <- fmt.Sprintf("%s, hi from %d", msg, i)
}(i)
}
c <- "original"
fmt.Println(<-c)
Output:
original, hi from 0, hi from 1, hi from 2, hi from 3, hi from 4
NOTE: the above output is outdated in more recent versions of Go (see comments)
Go Solutions
Solution 1 - Go
Yes, it's complicated, But there are a couple of rules of thumb that should make things feel much more straightforward.
- prefer using formal arguments for the channels you pass to go-routines instead of accessing channels in global scope. You can get more compiler checking this way, and better modularity too.
- avoid both reading and writing on the same channel in a particular go-routine (including the 'main' one). Otherwise, deadlock is a much greater risk.
Here's an alternative version of your program, applying these two guidelines. This case demonstrates many writers & one reader on a channel:
c := make(chan string)
for i := 1; i <= 5; i++ {
go func(i int, co chan<- string) {
for j := 1; j <= 5; j++ {
co <- fmt.Sprintf("hi from %d.%d", i, j)
}
}(i, c)
}
for i := 1; i <= 25; i++ {
fmt.Println(<-c)
}
http://play.golang.org/p/quQn7xePLw
It creates the five go-routines writing to a single channel, each one writing five times. The main go-routine reads all twenty five messages - you may notice that the order they appear in is often not sequential (i.e. the concurrency is evident).
This example demonstrates a feature of Go channels: it is possible to have multiple writers sharing one channel; Go will interleave the messages automatically.
The same applies for one writer and multiple readers on one channel, as seen in the second example here:
c := make(chan int)
var w sync.WaitGroup
w.Add(5)
for i := 1; i <= 5; i++ {
go func(i int, ci <-chan int) {
j := 1
for v := range ci {
time.Sleep(time.Millisecond)
fmt.Printf("%d.%d got %d\n", i, j, v)
j += 1
}
w.Done()
}(i, c)
}
for i := 1; i <= 25; i++ {
c <- i
}
close(c)
w.Wait()
This second example includes a wait imposed on the main goroutine, which would otherwise exit promptly and cause the other five goroutines to be terminated early (thanks to olov for this correction).
In both examples, no buffering was needed. It is generally a good principle to view buffering as a performance enhancer only. If your program does not deadlock without buffers, it won't deadlock with buffers either (but the converse is not always true). So, as another rule of thumb, start without buffering then add it later as needed.
Solution 2 - Go
Late reply, but I hope this helps others in the future like https://stackoverflow.com/q/19802037/143225
Effective Go explains the issue:
> Receivers always block until there is data to receive.
That means that you cannot have more than 1 goroutine listening to 1 channel and expect ALL goroutines to receive the same value.
Run this Code Example.
package main
import "fmt"
func main() {
c := make(chan int)
for i := 1; i <= 5; i++ {
go func(i int) {
for v := range c {
fmt.Printf("count %d from goroutine #%d\n", v, i)
}
}(i)
}
for i := 1; i <= 25; i++ {
c<-i
}
close(c)
}
You will not see "count 1" more than once even though there are 5 goroutines listening to the channel. This is because when the first goroutine blocks the channel all other goroutines must wait in line. When the channel is unblocked, the count has already been received and removed from the channel so the next goroutine in line gets the next count value.
Solution 3 - Go
It is complicated.
Also, see what happens with GOMAXPROCS = NumCPU+1. For example,
package main
import (
"fmt"
"runtime"
)
func main() {
runtime.GOMAXPROCS(runtime.NumCPU() + 1)
fmt.Print(runtime.GOMAXPROCS(0))
c := make(chan string)
for i := 0; i < 5; i++ {
go func(i int) {
msg := <-c
c <- fmt.Sprintf("%s, hi from %d", msg, i)
}(i)
}
c <- ", original"
fmt.Println(<-c)
}
Output:
5, original, hi from 4
And, see what happens with buffered channels. For example,
package main
import "fmt"
func main() {
c := make(chan string, 5+1)
for i := 0; i < 5; i++ {
go func(i int) {
msg := <-c
c <- fmt.Sprintf("%s, hi from %d", msg, i)
}(i)
}
c <- "original"
fmt.Println(<-c)
}
Output:
original
You should be able to explain these cases too.
Solution 4 - Go
I've studied existing solutions and created simple broadcast library https://github.com/grafov/bcast.
group := bcast.NewGroup() // you created the broadcast group
go bcast.Broadcasting(0) // the group accepts messages and broadcast it to all members
member := group.Join() // then you join member(s) from other goroutine(s)
member.Send("test message") // or send messages of any type to the group
member1 := group.Join() // then you join member(s) from other goroutine(s)
val := member1.Recv() // and for example listen for messages
Solution 5 - Go
For multiple goroutine listen on one channel, yes, it's possible. the key point is the message itself, you can define some message like that:
package main
import (
"fmt"
"sync"
)
type obj struct {
msg string
receiver int
}
func main() {
ch := make(chan *obj) // both block or non-block are ok
var wg sync.WaitGroup
receiver := 25 // specify receiver count
sender := func() {
o := &obj {
msg: "hello everyone!",
receiver: receiver,
}
ch <- o
}
recv := func(idx int) {
defer wg.Done()
o := <-ch
fmt.Printf("%d received at %d\n", idx, o.receiver)
o.receiver--
if o.receiver > 0 {
ch <- o // forward to others
} else {
fmt.Printf("last receiver: %d\n", idx)
}
}
go sender()
for i:=0; i<reciever; i++ {
wg.Add(1)
go recv(i)
}
wg.Wait()
}
The output is random:
5 received at 25
24 received at 24
6 received at 23
7 received at 22
8 received at 21
9 received at 20
10 received at 19
11 received at 18
12 received at 17
13 received at 16
14 received at 15
15 received at 14
16 received at 13
17 received at 12
18 received at 11
19 received at 10
20 received at 9
21 received at 8
22 received at 7
23 received at 6
2 received at 5
0 received at 4
1 received at 3
3 received at 2
4 received at 1
last receiver 4
Solution 6 - Go
Quite an old question, but nobody mentioned this, I think.
First, the outputs of both examples can be different if you run the codes many times. This is not related to the Go version.
The output of the 1st example can be goroutine 4, goroutine 0, goroutine 1,... actually all the goroutine can be a one who sends the string to the main goroutine.
Main goroutine is one of the goroutines, so it's also waiting for data from the channel. Which goroutine should receive the data? Nobody knows. It's not in the language spec.
Also, the output of the 2nd example also can be anything:
(I added the square brackets just for clarity)
// [original, hi from 4]
// [[[[[original, hi from 4], hi from 0], hi from 2], hi from 1], hi from 3]
// [[[[[original, hi from 4], hi from 1], hi from 0], hi from 2], hi from 3]
// [[[[[original, hi from 0], hi from 2], hi from 1], hi from 3], hi from 4]
// [[original, hi from 4], hi from 1]
// [[original, hi from 0], hi from 4]
// [[[original, hi from 4], hi from 1], hi from 0]
// [[[[[original, hi from 4], hi from 1], hi from 0], hi from 3], hi from 2]
// [[[[original, hi from 0], hi from 2], hi from 1], hi from 3]
//
// ......anything can be the output.
This is not magic, nor a mysterious phenomenon.
If there are multiple threads being executed, no one knows exactly which thread will acquire the resource. The language doesn't determine it. Rather, OS takes care of it. This is why multithread programming is quite complicated.
Goroutine is not OS thread, but it behaves somewhat similarly.