Changeset 279

Show
Ignore:
Timestamp:
12/16/07 04:04:20 (8 months ago)
Author:
gethema..@gmail.com
Message:

check in code that handles threads and also implements memcache clusters

Files:

Legend:

Unmodified
Added
Removed
Modified
Copied
Moved

  • trunk/README

    r272 r279  
    77from http request/response cycle. 
    88 
    9 This new release of BackgrounDRb is also modular and can be used without 
    10 Rails. So any Ruby program or framework can use it. 
     9This new release of BackgrounDRb is also modular and can be used without Rails so that any Ruby program or framework can use it. 
    1110 
    1211Copyright (c) 2006 Ezra Zygmuntowicz,skaar[at]waste[dot]org, 
     12 
    1313Copyright (c) 2007 Hemant Kumar (mail[at]gnufied[dot]org) 
    1414 
    …… 
    5050   |       :trigger_args: */10 * * * * * * 
    5151 
    52 Above sample configuration file would schedule worker methods 'foobar' and 'barbar' 
    53 to be executed at different trigger periods. Also, it would load production rails environment. 
    54 If you skip the :environment option, development env will be loaded. 
    55  
    56 NOTE: Please note that, because of addition of this feature, format of backgroundrb.yml 
    57 has changed slightly and hence modify your config file according to this new option. 
     52The above sample configuration file would schedule worker methods 'foobar' and 'barbar' from within FooWorker to be executed at different trigger periods. 
     53Also, it would load production rails environment. If you skip the :environment option, development environment will be loaded by default. 
     54 
     55NOTE: Because of the addition of this feature the format of backgroundrb.yml has changed slightly and you must modify your config file according to this new option. 
    5856 
    5957 
    …… 
    8381 
    8482- Cron Scheduling 
    85     You can use configuration file for cron scheduling of workers. Method specified in configuration 
    86     file would be called periodically. You should take care of the fact that, time gap between periodic 
    87     invocation of a method should be more than the time thats actually required to execute the method. 
    88     If a method takes longer time than the time window specified, your method invocations would lag 
    89     perpetually. 
     83 
     84  You can use a configuration file for cron scheduling of workers. The method specified in the configuration 
     85  file would be called periodically. You should accommodate for the fact that the time gap between periodic 
     86  invocation of a method should be more than the time that is actually required to execute the method. 
     87  If a method takes longer time than the time window specified, your method invocations will lag 
     88  perpetually. 
     89 
    9090 
    9191- Normal Scheduler 
    …… 
    9393 
    9494- add_periodic_timer method 
    95     A third and very basic form of scheduling that you can use is, "add_periodic_timer" method. You can call 
     95    A third and very basic form of scheduling that you can use is, "add_periodic_timer" method. You can call this 
    9696    method from anywhere in your worker. 
    9797 
    …… 
    9999             add_periodic_timer(5) { say_hello } 
    100100           end 
    101  
    102 Above snippet would register the proc for periodic execution at every 5 seconds. 
     101The above snippet would register the proc for periodic execution at every 5 seconds. 
    103102 
    104103=== A Word about Cron Scheduler 
    105104 
    106   Note that the initial field in the BackgrounDRb cron trigger, specifies 
     105  Note that the initial field in the BackgrounDRb cron trigger specifies 
    107106  seconds, not minutes as with Unix-cron. 
    108107 
    109108  The fields (which can be an asterisk, meaning all valid patterns) are: 
    110109 
    111     sec[0,59] min[0,59], hour[0,23], day[1,31], month[1,12[, weekday[0,6], year 
     110    sec[0,59] min[0,59], hour[0,23], day[1,31], month[1,12], weekday[0,6], year 
    112111 
    113112  The syntax pretty much follows Unix-cron. The following will trigger 
    …… 
    116115    0 30 1 * * * * 
    117116 
    118   Following trigger will trigger specified method every 10 seconds: 
     117  The following will trigger the specified method every 10 seconds: 
    119118 
    120119     */10 * * * * * * 
    121120 
    122   Following trigger will trigger specified method every 1 hour: 
     121  The following will trigger the specified method every 1 hour: 
    123122 
    124123     0 0 * * * * * 
    125124 
    126125  For each field you can use a comma-separated list. The following would 
    127   trigger on the fifth, sixteenth and twenty-third minute every hour: 
     126  trigger on the 5th, 16th and 23rd minute every hour: 
    128127 
    129128    0 5,16,23 * * * * * 
    130129 
    131130  Fields also support ranges, using a dash between values. The following 
    132   triggers the eighth through the seventeenth hour, five past the hour: 
     131  triggers from 8th through the 17th hour, at five past the hour: 
    133132 
    134133    0 5 8-17 * * * * 
    …… 
    139138    0 */5 6/2 * * * * 
    140139 
    141   At last a more contrived example: months 0,2,4,5,6,8,10,12, every day 
    142   and hour, minutes 1,2,3,4,6,20, seconds: every fifth second counting 
    143   from the twenty-eighth second plus the fifty-ninth second: 
     140  Here is a more complex example: months 0,2,4,5,6,8,10,12, every day 
     141  and hour, minutes 1,2,3,4,6,20, seconds: every 5th second counting 
     142  from the 28th second plus the 59th second: 
     143 
    144144 
    145145    28/5,59 1-4,6,20 */1 * 5,0/2 * * 
    …… 
    153153* Generate a Worker 
    154154 
    155   Install the plugin, and run setup task. Create a worker, using worker generator. 
    156  
    157           ./script/generatr worker bar 
    158  
    159   You will have a bar_worker.rb in your RAILS_ROOT/lib/workers/( called WORKER_ROOT henceforth ). 
    160   Generated code will look like this: 
     155  Install the plugin and run the setup task (rake backgroundrb:setup). Now create a worker using worker generator. 
     156 
     157          ./script/generate worker bar 
     158 
     159  This will create a bar_worker.rb in your RAILS_ROOT/lib/workers/ (called WORKER_ROOT henceforth). The generated code will look like this: 
    161160 
    162161   class BarWorker < BackgrounDRb::MetaWorker 
    …… 
    169168   end 
    170169 
    171   All the workers inside WORKER_ROOT directory will be automatically loaded and forked into a 
    172   separate process. If you don't want to start one particular worker automatically, then you can 
    173   use following class method to disable that behaviour: 
     170  All the workers inside WORKER_ROOT directory will be automatically loaded and forked into a separate process. 
     171  If you don't want to start one particular worker automatically you can use following class method (set_no_auto_load) to disable that behaviour: 
     172 
    174173 
    175174    class DynamicWorker < BackgrounDRb::MetaWorker 
    …… 
    178177    end 
    179178 
    180   'create' method gets called, when worker is loaded and created. Each worker runs in its 
     179  The 'create' method gets called when a worker is loaded and created. Each worker runs in its 
    181180  own process and you can use 'create' for initializing worker specific stuff. 
    182181 
    183   Following code snippet, would ask bdrb to execute method 'add_values' in 'foo_worker' with 
     182  The following code snippet would ask bdrb to execute method 'add_values' in 'foo_worker' with 
    184183  arguments '10+10' and return the result. 
    185184 
    186185       MiddleMan.send_request(:worker => :foo_worker, :worker_method => :add_values,:data => "10+10") 
    187186 
    188   When you are using "send_request" method, you are expecting a result back and hence, above code 
    189   will block until, your worker invokes a send response. The worker code, for handling above 
     187 When you are using the 'send_request' method, you are expecting a result back.  As such, the above code 
     188 will block until your worker invokes a send response. The worker code for handling the above 
     189 
    190190  method would look like 
    191191 
    …… 
    207207       MiddleMan.ask_work(:worker => :foo_worker, :worker_method => :add_values, :data => "10+10") 
    208208 
    209   You can also use register_status as described in following snippet to register status of 
     209  You can also use register_status as described in the following snippet to register the status of 
    210210  your worker with master, which can be directly queried from rails. 
    211211 
    212212       register_status(some_status_data) 
    213213 
    214   From rails, you can query status object using following code: 
     214  From rails, you can query status of your worker object using following code: 
    215215 
    216216       MiddleMan.ask_status(:worker => :foo_worker) 
    217217 
    218   Above code would return status object of 'foo_worker'. When you call register_status 
    219   from a worker, it replaces older state of the worker with master. Since, master process 
    220   stores status of the worker, all the status queries are served by master itself. It can be 
     218  The above code would return status object of 'foo_worker'. When you call register_status 
     219  from a worker, it replaces the older state of the worker with master. Since master process 
     220  stores the status of the worker, all the status queries are served by master itself. It can be 
     221 
    221222  used to store result hashes and stuff. 
    222223 
    223224* Starting and stopping a worker from Rails : 
    224225 
    225   All the workers can be dynamically started and stopped from rails. You can also use separate job_keys 
     226  All workers can be dynamically started and stopped from rails. You can also use separate job_keys 
    226227  to run more than one copy of a worker at a time. 
    227228 
    228   For example, following code in a rails controller will start "error_worker" and schedule to run according to trigger arguments. 
     229  For example, the following code in a rails controller will start "error_worker" and schedule it to run according to the arguments associated with trigger_args. 
    229230 
    230231    MiddleMan.new_worker(:worker => :error_worker, :job_key => :hello_world,:data => "wow_man",:schedule => { :hello_world => { :trigger_args => "*/5 * * * * * *",:data => "hello_world" }}) 
    231232 
    232   NOTE: Please note that first data argument would be passed to create method inside your worker, however 
    233   one specified under :schedule heading would be used by worker method, when its schedule comes. 
     233  NOTE: The first data argument will be passed to the create method inside your worker. However, 
     234  one specified under the :schedule heading would be used by the worker method when its schedule comes. 
     235 
    234236 
    235237  To stop a worker, you can use: 
    236238    MiddleMan.delete_worker(:worker => :error_worker, :job_key => :hello_world) 
    237239 
    238   If not job_key is specified, general worker name itself becomes job_key. You should create job_keys with 
    239   care, so as for one worker, they are never the same. 
     240  If no job_key is specified the general worker name itself becomes job_key. 
     241  You should create job_keys with care so they are never the same for one worker class. 
     242 
    240243 
    241244* Starting and stopping from CLI : 
    …… 
    249252* Query Status/Result of a worker : 
    250253 
    251   All Workers, can log their results with master, using 'register_status' method, this status can be queried from 
    252   rails using ask_status. For example: 
     254  All Workers can log their results with master, using the 'register_status' method. 
     255  This status can be queried from rails using ask_status. For example: 
     256 
    253257 
    254258   class ProgressWorker < BackgrounDRb::MetaWorker 
    …… 
    264268   end 
    265269 
    266   And using MiddleMan proxy, you can keep queering status of your progress bar: 
     270  And using MiddleMan proxy, you can keep querying the status of your progress bar: 
    267271 
    268272     MiddleMan.ask_status(:worker => :progress_worker) 
    …… 
    270274* Query status of All workers : 
    271275 
    272   You can also, query status of all currently running workers in one shot. 
     276  You can also query the status of all currently running workers in one shot. 
    273277 
    274278    def ask_status 
    …… 
    278282    end 
    279283 
    280   Currently, when a worker is deleted/exits, its result/status is also gone and hence 
    281   you can't query status of a worker, which is not running.This behaviour is expected to 
    282   change. 
     284  Currently, when a worker is deleted/exits, its result/status is also gone (i.e. you can't 
     285  query the status of a worker which is not running).  This behaviour is expected to change in future releases. 
     286 
    283287 
    284288* Important difference between MiddleMan.ask_work and MiddleMan.send_request : 
    285289 
    286   As noted previously ask_work is used, when you want one shot execution of a worker method, 
    287   without waiting for results in rails. So, there aren't any explicit return statement is required. 
    288   But, when you use MiddleMan.send_request, you are asking BDRB, ok please execute this method 
    289   on worker and I would wait for results until method returns. Hence in this case, you must return 
     290  As noted previously ask_work is used when you want one shot execution of a worker method 
     291  without waiting for results in rails. So an explicit return statement is not required. 
     292  But when you use MiddleMan.send_request, you are asking BDRB, "ok please execute this method 
     293  on worker and I will wait for results until the method returns". Hence in this case, you must return 
    290294  the value you want to get back in rails. 
    291295 
    292   Since, not all objects can be dumped in ruby, and if you are trying to send an object, which 
    293   can't be dumped, you will get error messages logged in your log file and will get error string in 
    294   controller too. 
    295  
    296   For example, lets say you are invoking method "hello_world" from 'foo_controller' like this: 
     296  Not all objects can be dumped in ruby.  If you are trying to send an object which 
     297  can't be dumped, you will get error messages logged in your log file and will get an error string in your 
     298  controller, too. 
     299 
     300 
     301  For example, let's say you are invoking method "hello_world" from 'foo_controller' like this: 
    297302 
    298303    worker_response = MiddleMan.send_request(:worker => :foo_worker, :worker_method => :hello_world) 
    …… 
    305310    end 
    306311 
    307   Now, since a lambda can't be dumped, the worker_response that you will receive in controller will be, 
    308   'invalid_result_dump_check_log' and appropriate error will be logged in backgroundrb.log file too. 
    309   Now, originally this error could have potentially aborted BDRB worker hence make sure that you 
     312  Now since a lambda can't be dumped, the worker_response that you will receive in your controller will be, 
     313  'invalid_result_dump_check_log' and an appropriate error will also be logged in the backgroundrb.log file. 
     314  Now, such an error could potentially abort the BDRB worker.  Hence, make sure that you 
    310315  avoid such cases. 
    311316 
     317* Running BackgrounDRb clusters and storing of results in Memcache cluster 
     318 
     319  New version allows access to worker status objects even after a worker has died/exited. By default, 
     320  this data would be held in Master Process memory. Those of you, who want to run, BackgrounDRb in 
     321  a cluster, and if you run a BackgrounDRb server on each node and would rather want results to be 
     322  stored in MemCache, you can use following option for storing results in MemCache: 
     323 
     324      # backgroundrb.yml 
     325 
     326      | :backgroundrb: 
     327      |   :port: 11006 
     328      |   :ip: 0.0.0.0 
     329      |   :log: foreground 
     330      |   :result_storage: 
     331      |     :memcache: "10.10.10.2:11211,10.10.10.6:11211" 
     332 
     333 
     334* Using Threads inside BackgrounDRb 
     335 
     336  Remember BackgrounDRb follows event model of network programming, but sad truth of life is 
     337  not all networking libraries follow this model and hence they make use of blocking IO and threads. 
     338  But you need not fear, BackgrounDRb allows you to run all such tasks concurrently in threads 
     339  which are internally managed by BackgrounDRb thread pool. 
     340 
     341  Each worker has access to object "thread_pool" which can be used to run task in threads concurrently. 
     342 
     343    thread_pool.defer(wiki_scrap_url) { |wiki_url| scrap_wikipedia(wiki_url) } 
     344 
     345  So whatever task you specify within scrap_wikipedia() is going to run concurrently. 
     346 
     347  WARNING: You shouldn't try to use +register_status+ method from within the block supplied to +defer+. Because, if you do that, 
     348  you can get corrupted result hashes. However, if you are confident, you should wrap your status_hash ( or whatever data type, you 
     349  are going to store as a status ) in a mutex and then use +register_status+ . It would make sure that, only one thread 
     350  resisters status at a time. 
     351 
     352 
    312353* Internal Server and Unhandled Exception Logging on console : 
    313354 
    314   Sometimes, you may want all the internal error messages and unhandled exceptions to appear on console. 
    315   For that, you can start backgroundrb with config option : 
     355  Sometimes you may want all the internal error messages and unhandled exceptions to appear on the console. 
     356  For that, you can start backgroundrb with the following config option : 
     357 
    316358 
    317359   # backgroundrb.yml 
    …… 
    328370 
    329371=== Testing 
    330 * where will you be without test cases Phaedrus? New version comes with a baked in mechanism to write test cases. 
    331   First make sure that, you have bdrb_test_helper.rb in test directory of your rails app ( run 
    332   rake backgroundrb:setup, if you dont have one ). 
     372 
     373* where will you be without test cases Phaedrus? This new version comes with a baked in mechanism to write test cases. 
     374  First make sure that you have bdrb_test_helper.rb in the test directory of your rails app (run 
     375  rake backgroundrb:setup, if you dont have one). 
     376 
    333377  Just put your worker test cases in test/unit directory of your rails application and require the helper. 
    334378  Now, you should be good to go. 
    …… 
    347391=== Legacy and deprecated stuff 
    348392 
    349    Although, You need to wrap your head a bit for understanding "evented" model of network programming, 
    350    but it gets easier once you get hang of it. Much of the older stuff is deprecated. Here is a brief list: 
     393   Although You need to wrap your head a bit to understanding the "evented" model of network programming, 
     394   it gets easier once you get hang of it. Much of the older stuff is deprecated. Here is a brief list: 
     395 
    351396 
    352397   - ACL : gone, trust to thy firewalls. 
    …… 
    354399=== Exciting new stuff 
    355400  * Rock solid stable ( or will be , after few bug reports ) 
    356   * Each worker comes with Event loop of its own and can potentially do lots of fancy stuff. Two noteworthy methods are: 
     401  * Each worker comes with an Event loop of its own and can potentially do lots of fancy stuff. Two noteworthy methods are: 
    357402 
    358403         connect(ip,port,Handler) 
    359404         start_worker(ip,port,Handler) 
    360405 
    361     If you are familiar with EventMachine or Twisted style of network programming, above methods allow you to 
    362     start tcp servers inside your workers or lets you connect to external tcp servers. For Each accepted client or 
    363     connected socket a instance of Handler class would be created and integrated with main event loop. 
     406    If you are familiar with the EventMachine or Twisted style of network programming, the above methods allow you to 
     407    start tcp servers inside your workers or let you connect to external tcp servers. For Each accepted client or 
     408    connected socket, an instance of Handler class would be created and integrated with main event loop. 
    364409    This can be used for worker to worker communication between backgroundrb servers running on two machines. 
    365410 
    366     You are encouraged to look into framework directory, and see the code that implements all this stuff.Guts of 
    367     new bdrb is based on this library, which would be released soon as separately. 
     411    You are encouraged to look into framework directory and see the code that implements all this stuff.  The guts of 
     412    this new version of bdrb is based on this library which will be released soon as a separate entity. 
     413 
    368414 
    369415== Online Resources 

  • trunk/framework/core.rb

    r275 r279  
    144144        loop do 
    145145          check_for_timer_events 
     146          user_thread_window #=> let user level threads run for a while 
    146147          ready_fds = select(@read_ios,@write_ios,nil,0.005) 
    147148          #next if ready_fds.blank? 
    …… 
    158159          end 
    159160        end 
     161      end 
     162 
     163      def user_thread_window 
     164        run_user_threads if respond_to?(:run_user_threads) 
    160165      end 
    161166 

  • trunk/framework/packet_master.rb

    r275 r279  
    1414    def self.run 
    1515      master_reactor_instance = new 
    16       master_reactor_instance.result_hash = {} 
     16      # master_reactor_instance.result_hash = {} 
    1717      master_reactor_instance.live_workers = DoubleKeyedHash.new 
    1818      yield(master_reactor_instance) 
    …… 
    2020      master_reactor_instance.start_reactor 
    2121    end # end of run method 
     22 
     23    def set_result_hash(hash) 
     24      @result_hash = hash 
     25    end 
    2226 
    2327    def update_result(worker_key,result) 

  • trunk/server/master_worker.rb

    r278 r279  
    123123 
    124124  class MasterProxy 
     125    attr_accessor :config_file 
    125126    def initialize 
    126       config_file = YAML.load(ERB.new(IO.read("#{RAILS_HOME}/config/backgroundrb.yml")).result) 
    127       debug_logger = DebugMaster.new(config_file[:backgroundrb][:log]) 
     127      raise "Running old Ruby version, upgrade to Ruby >= 1.8.5" unless check_for_ruby_version 
     128      @config_file = YAML.load(ERB.new(IO.read("#{RAILS_HOME}/config/backgroundrb.yml")).result) 
     129      debug_logger = DebugMaster.new(@config_file[:backgroundrb][:log]) 
    128130 
    129       load_rails_env(config_file) 
     131      load_rails_env 
    130132      Packet::Reactor.run do |t_reactor| 
     133        enable_memcache_result_hash(t_reactor) if @config_file[:backgroundrb][:result_storage] && @config_file[:backgroundrb][:result_storage][:memcache] 
    131134        t_reactor.start_worker(:worker => :log_worker) 
    132         t_reactor.start_server(config_file[:backgroundrb][:ip],config_file[:backgroundrb][:port],MasterWorker) { |conn|  conn.debug_logger = debug_logger } 
     135        t_reactor.start_server(@config_file[:backgroundrb][:ip],@config_file[:backgroundrb][:port],MasterWorker) { |conn|  conn.debug_logger = debug_logger } 
    133136      end 
    134137    end 
    135138 
    136     def load_rails_env(config_file) 
     139    def load_rails_env 
    137140      db_config_file = YAML.load(ERB.new(IO.read("#{RAILS_HOME}/config/database.yml")).result) 
    138       run_env = config_file[:backgroundrb][:environment] || 'development' 
     141      run_env = @config_file[:backgroundrb][:environment] || 'development' 
    139142      ENV["RAILS_ENV"] = run_env 
    140143      RAILS_ENV.replace(run_env) if defined?(RAILS_ENV) 
    …… 
    142145      ActiveRecord::Base.allow_concurrency = true 
    143146    end 
    144   end 
     147 
     148    def enable_memcache_result_hash(t_reactor) 
     149      require 'memcache' 
     150      memcache_options = { 
     151        :c_threshold => 10_000, 
     152        :compression => true, 
     153        :debug => false, 
     154        :namespace => 'backgroundrb_result_hash', 
     155        :readonly => false, 
     156        :urlencode => false 
     157      } 
     158      cache = MemCache.new(memcache_options) 
     159      cache.servers = @config_file[:backgroundrb][:result_storage][:memcache].split(',') 
     160      t_reactor.set_result_hash(cache) 
     161    end 
     162 
     163    def check_for_ruby_version; return RUBY_VERSION >= "1.8.5"; end 
     164 
     165  end # end of module BackgrounDRb 
    145166end 
    146167 

  • trunk/server/meta_worker.rb

    r277 r279  
    1414    end 
    1515  end 
     16 
     17  class WorkData 
     18    attr_accessor :data,:block 
     19    def initialize(*args,&block) 
     20      @data = args 
     21      @block = block 
     22    end 
     23  end 
     24 
     25  class ThreadPool 
     26    attr_accessor :size 
     27    attr_accessor :threads 
     28    attr_accessor :work_queue 
     29    def initialize(size) 
     30      @size = size 
     31      @threads = [] 
     32      @work_queue = Queue.new 
     33      @running_tasks = Queue.new 
     34      @size.times { add_thread } 
     35    end 
     36 
     37    # can be used to make a call in threaded manner 
     38    # passed block runs in a thread from thread pool 
     39    #   def fetch_url(url) 
     40    #     puts "fetching url #{url}" 
     41    #     thread_pool.defer(url) do |url| 
     42    #       begin 
     43    #         data = Net::HTTP.get(url,'/') 
     44    #         File.open("#{RAILS_ROOT}/log/pages.txt","w") do |fl| 
     45    #           fl.puts(data) 
     46    #         end 
     47    #       rescue 
     48    #         logger.info "Error downloading page" 
     49    #       end 
     50    #     end 
     51    #   end 
     52    # you can invoke above method from rails as: 
     53    #   MiddleMan.ask_work(:worker => :rss_worker, :worker_method => :fetch_url, :data => "www.example.com") 
     54 
     55    def defer(*args,&block) 
     56      @work_queue << WorkData.new(args,&block) 
     57    end 
     58 
     59    def add_thread 
     60      @threads << Thread.new do 
     61        while true 
     62          task = @work_queue.pop 
     63          @running_tasks << task 
     64          if task.data && !task.data.empty? 
     65            task.block.call(*(task.data)) 
     66          else 
     67            task.block.call 
     68          end 
     69          @running_tasks.pop 
     70        end 
     71      end 
     72    end 
     73 
     74    # method ensures exclusive run of deferred tasks for 2 seconds, so as they do get a chance to run. 
     75    def exclusive_run 
     76      if @running_tasks.empty? && @work_queue.empty? 
     77        return 
     78      else 
     79        puts "going to sleep for a while" 
     80        sleep(2) 
     81        return 
     82      end 
     83    end 
     84  end 
     85 
    1686  # == MetaWorker class 
    1787  # BackgrounDRb workers are asynchrounous reactors which work using events 
    …… 
    78148  class MetaWorker < Packet::Worker 
    79149    attr_accessor :config_file, :my_schedule, :run_time, :trigger_type, :trigger 
    80     attr_accessor :logger 
     150    attr_accessor :logger, :thread_pool 
    81151 
    82152    # does initialization of worker stuff and invokes create method in 
    83153    # user defined worker class 
    84154    def worker_init 
     155      @thread_pool = ThreadPool.new(20) 
     156 
    85157      @config_file = YAML.load(ERB.new(IO.read("#{RAILS_HOME}/config/backgroundrb.yml")).result) 
    86158      # load_rails_env 
    …… 
    93165      end 
    94166      if respond_to?(:create) 
    95         create(@worker_options[:data]) 
     167        create_arity = method(:create).arity 
     168        (create_arity == 0) ? create : create(@worker_options[:data]) 
    96169      end 
    97170      @logger.info "#{worker_name} started" 
    …… 
    166239    end 
    167240 
     241    # probably this method should be made thread safe, so as a method needs to have a 
     242    # lock or something before it can use the method 
    168243    def register_status p_data 
    169244      status = {:type => :status,:data => p_data} 
    …… 
    229304    end 
    230305 
     306    # method would allow user threads to run exclusively for a while 
     307    def run_user_threads 
     308      @thread_pool.exclusive_run 
     309    end 
     310 
    231311    #     we are overriding the function that checks for timers 
    232312    #     def check_for_timer_events