ruby working on array elements in groups of four
RubyRuby Problem Overview
I have a ruby script array when each element needs processing :
threads = []
elemets.each do |element|
threads.push(Thread.new{process(element)}}
end
threads.each { |aThread| aThread.join }
how ever due to resource limitations, the script works in an optimal way if no more the four elements are processed at a time.
no I know I can dump the each loop and use a variable to count 4 elements and then wait but is there a cooler ruby way to do it ?
Ruby Solutions
Solution 1 - Ruby
You can enumerate in groups of 4 for an array:
>> [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12].each_slice(4) {|a| p a}
[1, 2, 3, 4]
[5, 6, 7, 8]
[9, 10, 11, 12]
So you can try something like
elements.each_slice(4) do | batch |
batch.each do | element |
threads.push(Thread.new{process(element)}}
end
(do stuff to check to see if the threads are done, otherwise wait )
end
Its may not be what you need, though - I have been up since 3 AM and I only had a couple of hours sleep. :/
Solution 2 - Ruby
If I read you right, you want to have no more than 4 threads processing at a time.
Sounds to me like you should launch only 4 threads, and have them all read from a shared Queue (part of the standard thread lib) to process the elements.
You can have the threads end when the queue is empty.
Slicing the array into 4 equal arrays, and having each thread process 1/4 of the elements assumes that each element processes in the same time. If some take longer than others, some of your threads will finish early.
Using a queue, no thread stops until the shared queue is empty, so it is I think a more efficient solution.
Here is a working program based on your code to demonstrate:
require 'thread'
elements = [1,2,3,4,5,6,7,8,9,10]
def process(element)
puts "working on #{element}"
sleep rand * 10
end
queue = Queue.new
elements.each{|e| queue << e }
threads = []
4.times do
threads << Thread.new do
while (e = queue.pop(true) rescue nil)
process(e)
end
end
end
threads.each {|t| t.join }
Solution 3 - Ruby
In rails(not Ruby) a more readble form can be used in_groups_of
arr= [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
arr.in_groups_of(4, false) {|a| p a}
result:
[1, 2, 3, 4]
[5, 6, 7, 8]
[9, 10, 11]
The last row has only 3 elements as we have specified false as second argument in in_group_of
. If you want nil or any other value you can replace false with that value.
Solution 4 - Ruby
Not sure if the following variant counts as just using a "variable to count 4 elements", or could be considered cool, but it gives you an array in slices of size no greater than 4 elements:
x = (1..10).to_a
0.step(x.size - 1, 4) do |i|
# Choose one
p x.slice(i, 4)
p x[i, 4]
end
Solution 5 - Ruby
Yes, but you need to do some method overriding. Usual approach is to override '/' for Array
like so:
class Array
def / len
a = []
each_with_index do |x,i|
a << [] if i % len == 0
a.last << x
end
a
end
end
And with that defined you can now easily do:
foo = [1,2,3,4,5,6]
foo / 2
# Result is [[1,2], [3,4], [5,6]]