RuntimeError (detected cyclic dependency on startup of persistent multithreading applications

Question

I get this error: RuntimeError (detected cyclic dependency on autoload of persistent applications

when i am multithreading. Here is my code below. Why is this happening? The reason I'm trying to use multithreading is because I am writing an HTML cleanup application. The Nokogiri :: HTML call (open ()) is a synchronous blocking call that returns 1 second and I have 100,000 + pages to visit, so I'm trying to run multiple threads to solve this problem. Is there a better way to do this?

class ToolsController < ApplicationController def getWebsites t1=Thread.new{func1()} t2=Thread.new{func1()} t3=Thread.new{func1()} t4=Thread.new{func1()} t5=Thread.new{func1()} t6=Thread.new{func1()} t1.join t2.join t3.join t4.join t5.join t6.join end def func1 puts Thread.current apps = Apps.order("RANDOM()").where("apps.website IS NULL").take(1) while apps.size == 1 do app = apps[0] puts app.name puts app.iTunes doc = Nokogiri::HTML(open(app.iTunes)) array = doc.css('div.app-links a').map { |link| url = link['href'] url = Domainatrix.parse(url) url.domain + "." + url.public_suffix } array.uniq! if (array.size > 0) app.website = array.join(', ') puts app.website else app.website = "NONE" end app.save apps = Apps.order("RANDOM()").where("apps.website IS NULL").take(1) end end end 

Answer 1

"require" is not thread safe

Change your methods so that everything that needs to be “done” is done before the threads begin.

For instance:

 def get_websites # values = Apps.all # try uncommenting this line if a second-try is required ar = Apps.where("apps.website IS NULL") t1 = Thread.new{ func1(ar) } t2 = Thread.new{ func1(ar) } t1.join t2.join end def func1( ar ) apps = ar.order("RANDOM()").limit(1) while (apps.size == 1) puts Thread.current end end 

But, as someone remarked, multithreading inside the controller is not recommended.