What use can I provide for Ruby threads if they are not parallel?

Question

When I first discovered the threads, I tried to verify that they actually worked as expected by invoking sleep in many threads, depending on normal sleep. It worked and I was very happy.

But then my friend told me that these flows were not really parallel, and this dream was supposed to pretend.

So now I wrote this test to do the actual processing:

class Test ITERATIONS = 1000 def run_threads start = Time.now t1 = Thread.new do do_iterations end t2 = Thread.new do do_iterations end t3 = Thread.new do do_iterations end t4 = Thread.new do do_iterations end t1.join t2.join t3.join t4.join puts Time.now - start end def run_normal start = Time.now do_iterations do_iterations do_iterations do_iterations puts Time.now - start end def do_iterations 1.upto ITERATIONS do |i| 999.downto(1).inject(:*) # 999! end end end 

And now I am very sad because run_threads () not only does not work better than run_normal, but even slower!

Then why should I complicate my application with threads if they are not parallel?

** UPDATE **

@ fl00r said that I could use streams if I used them for I / O, so I wrote two more do_iterations:

 def do_iterations # filesystem IO 1.upto ITERATIONS do |i| 5.times do # create file content = "some content #{i}" file_name = "#{Rails.root}/tmp/do-iterations-#{UUIDTools::UUID.timestamp_create.hexdigest}" file = ::File.new file_name, 'w' file.write content file.close # read and delete file file = ::File.new file_name, 'r' content = file.read file.close ::File.delete file_name end end end def do_iterations # MongoDB IO (through MongoID) 1.upto ITERATIONS do |i| TestModel.create! :name => "some-name-#{i}" end TestModel.delete_all end 

The performance results are the same: normal> threads.

But now I'm not sure if my VM can use all the cores. Will be back when I checked this.

Answer 1

Streams can be faster only if you have a slow IO.

In Ruby, you have Global Interpreter Lock, so only one Thread can work at a time. So, Ruby spends a lot of time to control which thread should be launched in an instant (thread scheduling). So in your case, when there is no I / O, it will be slower!

You can use Rubinius or JRuby to use real threads.

Example with IO:

 module Test extend self def run_threads(method) start = Time.now threads = [] 4.times do threads << Thread.new{ send(method) } end threads.each(&:join) puts Time.now - start end def run_forks(method) start = Time.now 4.times do fork do send(method) end end Process.waitall puts Time.now - start end def run_normal(method) start = Time.now 4.times{ send(method) } puts Time.now - start end def do_io system "sleep 1" end def do_non_io 1000.times do |i| 999.downto(1).inject(:*) # 999! end end end Test.run_threads(:do_io) #=> ~ 1 sec Test.run_forks(:do_io) #=> ~ 1 sec Test.run_normal(:do_io) #=> ~ 4 sec Test.run_threads(:do_non_io) #=> ~ 7.6 sec Test.run_forks(:do_non_io) #=> ~ 3.5 sec Test.run_normal(:do_non_io) #=> ~ 7.2 sec 

I / O jobs are 4 times faster in threads and processes, while non-IO jobs in processes are twice as fast as threads and synchronization methods.

Ruby also offers Fiber lightweight "corutines" and awesome em-synchrony gem for processing asynchronous processes.

Answer 2

fl00r is right, the global locking of the interpreter prevents the simultaneous execution of several threads in the ruby, with the exception of IO.

The parallel library is a very simple library that is useful for truly parallel operations. Install using gem install parallel . Here is your example rewritten to use it:

 require 'parallel' class Test ITERATIONS = 1000 def run_parallel() start = Time.now results = Parallel.map([1,2,3,4]) do |val| do_iterations end # do what you want with the results ... puts Time.now - start end def run_normal start = Time.now do_iterations do_iterations do_iterations do_iterations puts Time.now - start end def do_iterations 1.upto ITERATIONS do |i| 999.downto(1).inject(:*) # 999! end end end 

On my computer (4 cpus), Test.new.run_normal takes 4.6 seconds and Test.new.run_parallel 1.65 seconds.

Answer 3

The behavior of the threads is determined by the implementation. For example, JRuby implements threads with JVM threads, which in turn use real threads.

Global Interpreter Lock exists only for historical reasons. If Ruby 1.9 simply presented real threads from nowhere, backward compatibility would be compromised, and this would further slow its adoption.

This answer from Jörg W Mittag provides an excellent comparison between stream models of various Ruby implementations. Choose the one that suits your needs.

With that said, threads can be used to wait for a child process to complete:

 pid = Process.spawn 'program' thread = Process.detach pid # Later... status = thread.value.exitstatus 

Answer 4

Even if threads are not executed in parallel, they can be a very efficient and easy way to perform certain tasks, for example, in work tasks such as cron. For instance:

 Thread.new{ loop{ download_nightly_logfile_data; sleep TWENTY_FOUR_HOURS } } Thread.new{ loop{ send_email_from_queue; sleep ONE_MINUTE } } # web server app that queues mail on actions and shows current log file data 

I also use Threads on a DRb server to handle long-term computing for one of my web applications. The web server starts computing in the stream and immediately continues to respond to web requests. He can periodically look at the state of the task and see how he progresses. For more information, read DRb Server for long-term web processes .

Answer 5

For an easy way to see the difference, use Sleep instead of IO, which also relies on too many variables:

 class Test ITERATIONS = 1000 def run_threads start = Time.now threads = [] 20.times do threads << Thread.new do do_iterations end end threads.each {|t| t.join } # also can be written: threads.each &:join puts Time.now - start end def run_normal start = Time.now 20.times do do_iterations end puts Time.now - start end def do_iterations sleep(10) end end 

it will matter between the threaded solution even on MRB, and GIL