Changeset 323

Show
Ignore:
Timestamp:
03/11/08 18:30:43 (5 months ago)
Author:
gethema..@gmail.com
Message:

commit new code changes

Files:

Legend:

Unmodified
Added
Removed
Modified
Copied
Moved

  • trunk

    • Property svn:ignore set to
      rdoc

  • trunk/doc/content/bugs/bugs.txt

    r319 r323  
    66You can also submit potential patches that fix known bugs or add exciting features to the above trac url. 
    77 
    8 Also, if you are using older version of _BackgrounDRb_ and having a problem, why not upgrade to latest code from git, your bug  
     8Also, if you are using older version of _BackgrounDRb_ and having a problem, why not upgrade to latest code from git, your bug 
    99might have been fixed there. 
     10 
     11%(entry-title)<a name="known_bugs">Known Problems </a>% 
     12 
     13Since _BackgrounDRb_ has feature of thread pools, which allows users to execute tasks 
     14concurrently, we are using @allow_concurrency = true@ in @ActiveRecord@ model objects. 
     15This has been known to create some issues with Oracle database Adapters. If you are sure of 
     16what you are doing and don't need thread pool feature, you can go ahead and remove @allow_concurrency = true@ 
     17from _BackgrounDRb_ code and it should work. 
    1018 
    1119</div> 

  • trunk/doc/content/bugs/bugs.yaml

    r319 r323  
    33# Custom 
    44title: "Report Bugs" 
    5 sidebar_items: [["Trac", "#trac"]] 
     5sidebar_items: [["Trac", "#trac"], ["KnownBugs", "#known_bugs"]] 

  • trunk/doc/content/community/community.txt

    r319 r323  
    2222Its a perfect place for discussing new ideas, asking questions, submitting patches and stuff. 
    2323 
     24%(entry-title)<a name="irc"> IRC Help </a>% 
     25 
     26You can also try #backgroundrb on freenode for help. 
     27 
     28%(entry-title)<a name="contribute"> Contribute </a>% 
     29 
     30Your contributions/feedback is most welcome. We have very simple rule for giving new coders 
     31commit access. If you submit one patch which is accepted for inclusion in the _BackgrounDRb_ 
     32code repository, you are elligible for commit access. Create an account on "Gitorious":http://gitorious.org/ 
     33and let me (gethemant at gmail dot com) know your login handle. 
     34 
    2435</div> 
    2536 

  • trunk/doc/content/community/community.yaml

    r319 r323  
    33# Custom 
    44title: "Join the BackgrounDRb Community" 
    5 sidebar_items: [["People", "people"], ["MailingList", "#mailing_list"]] 
     5sidebar_items: [["People", "people"], ["MailingList", "#mailing_list"], ["IRC", "#irc"], ["Contribute", "#contribute"]] 

  • trunk/doc/content/content.txt

    r319 r323  
    1111%(entry-title)<a name="installation"> Installation </a>% 
    1212 
     13*Update::* 
     14There was a small error in _BackgrounDRb_ startup script, because of which you may not be 
     15able to start _BackgrounDRb_ server. Its been fixed in git and svn trunks. Hence, make sure 
     16that you are running latest code from git or svn trunks. Also, after checking out code from repo, 
     17don't forget to remove old 'backgroundrb' script from script directory of rails application 
     18and run @rake backgroundrb:setup@ 
     19 
    1320p(sub-title). Installing the dependencies : 
    1421 
    …… 
    1926 
    2027Please note that, this version of _BackgrounDRb_ needs packet version 0.1.5 or greater, so make 
    21 sure you have that.  
     28sure you have that. 
    2229 
    2330p(sub-title). Getting the code from Subversion : 
    …… 
    4148 
    4249After getting the plugin, you must configure it for use. _BackgrounDRb_ comes with a rake task 
    43 for automating plugin configuration. Run, following command from root directory of your rails 
    44 application, after installation: 
     50for automating plugin configuration. Before running rake task, remove if any old 'backgroundrb' 
     51script is there in script folder of your rails app after that run, following command from 
     52root directory of your rails application, after installation: 
    4553 
    4654<pre class="boxed">rake backgroundrb:setup </pre> 
    …… 
    6775:backgroundrb: 
    6876  :port: 11006 
    69   :ip: 0.0.0.0  
     77  :ip: 0.0.0.0 
    7078  :environment: production </pre> 
    7179 
    …… 
    8088  :environment: production # rails environment to load 
    8189  :log: foreground # foreground mode,print log messages on console 
    82   :lazy_load: true # do not load models eagerly 
    83   :debug_log: false # disable log workers and other logging </pre> 
     90  :lazy_load: false # load models eagerly 
     91  :debug_log: false # disable log workers and other logging 
     92:schedules: # optional task scheduling 
     93  : # look in scheduling section </pre> 
    8494 
    85 @lazy_load@ option should be true if you want to pass @ActiveRecord@ model 
    86 objects around, However, this option is generally not encouraged to use,  
    87 because if your model makes use of some other ActiveRecord plugin and  
    88 plugin is not available during load, loading of model will fail. In  
    89 new version of BackgrounDRb its generally discouraged to pass model objects around,  
     95@lazy_load@ option should be false if you want to pass @ActiveRecord@ model 
     96objects around, However, this option is generally not encouraged to use, 
     97because if your model makes use of some other ActiveRecord plugin and 
     98plugin is not available during load, loading of model will fail. In 
     99new version of BackgrounDRb its generally discouraged to pass model objects around, 
    90100since they are harder to serialize and deserialize. 
     101 
     102It is possible to set options depending on the environment: 
     103 
     104<pre class="multiline"> 
     105:backgroundrb: 
     106  :ip: 0.0.0.0 
     107 
     108:development: 
     109  :backgroundrb: 
     110    :port: 11111     # use port 11111 
     111    :log: foreground # foreground mode,print log messages on console 
     112 
     113:production: 
     114  :backgroundrb: 
     115    :port: 22222      # use port 22222 
     116    :lazy_load: true  # do not load models eagerly 
     117    :debug_log: false # disable log workers and other logging </pre> 
     118 
     119This will cause @script/backgroundrb@ to use port 11111 and to log to foreground under 
     120development environment and @script/backgroundrb -e production@ to use port 22222, 
     121lazy_load and no debug_log under production environment. The ip 0.0.0.0 is used in all 
     122environments. 
     123 
    91124 
    92125%(entry-title)<a name="worker"> Workers </a>% 
    …… 
    102135create  lib/workers/billing_worker.rb </pre> 
    103136 
    104 And generated worker will look like:  
     137And generated worker will look like: 
    105138 
    106139<pre class="multiline">class BillingWorker < BackgrounDRb::MetaWorker 
    107140  set_worker_name :billing_worker 
    108141  def create(args = nil) 
    109     # method gets called, when new instance of worker is created.                       
     142    # method gets called, when new instance of worker is created. 
    110143   end 
    111144  end </pre> 
    …… 
    120153    # this method is called, when worker is loaded for the first time 
    121154  end 
    122    
     155 
    123156  def charge_customer(customer_id = nil) 
    124157    logger.info 'charging customer now' 
    …… 
    159192 
    160193 
    161    
    162194 
     195 

  • trunk/doc/content/rails/rails.txt

    r319 r323  
    1616p(sub-title). Invoke an asynchronous task on a worker : 
    1717 
     18Let's say, you have following worker code: 
     19 
     20<pre class="multiline">class FooWorker < BackgrounDRb::MetaWorker 
     21  set_worker_name :foo_worker 
     22  def create(args = nil) 
     23    # this method is called, when worker is loaded for the first time 
     24  end 
     25 
     26  def some_task args 
     27    # perform a long running task 
     28  end 
     29end 
     30</pre> 
     31 
     32And you want to invoke @some_task@ method with appropriate arguments from rails. 
    1833Following snippet will invoke method @some_task@ with argument @data@ in @foo_worker@. Also, method will 
    1934be invoked asychrounously and Rails won't wait for result from BackgrounDRb server. 
    …… 
    2237worker.some_task(data) </pre> 
    2338 
    24 Here, I would like to illustrate  a point that contrary to general percenption, since @some_task@ method is being 
    25 executed asyhcrounously, don't expect any meaningful return values from second line. 
     39It should be noted that, since @some_task@ method is being 
     40executed asyhcrounously, don't expect any meaningful return values from method invocation. 
     41If you want to invoke a method on worker and collect results returned by it, you 
     42should read next section (Invoke method and collect results). 
    2643 
    2744When you invoke @MiddleMan.worker(:foo_worker)@ it returns a worker proxy, hence you can combine above two lines in 
    …… 
    3552p(sub-title). Invoke a method on worker and get results : 
    3653 
    37 Following snippet will invoke method @some_task@ with argument @data@ in @foo_worker@. Also, method block 
     54Following snippet will invoke method @some_task@ with argument @data@ in @foo_worker@. Also, method will block 
    3855until BackgrounDRb server returns a result. 
    3956 
    4057<pre class="multiline">worker = MiddleMan.worker(:foo_worker) 
    4158result = worker.some_task(data,true) </pre> 
     59 
     60Since, now you are expecting a return value from your worker method, new worker code will look like: 
     61 
     62<pre class="multiline">class FooWorker < BackgrounDRb::MetaWorker 
     63  set_worker_name :foo_worker 
     64  def create(args = nil) 
     65    # this method is called, when worker is loaded for the first time 
     66  end 
     67 
     68  def some_task args 
     69    billing_result = UserPayment.bill! 
     70    return billing_result 
     71  end 
     72end 
     73</pre> 
    4274 
    4375As illustrated above, you can use @job_key@ or make them in single line too. When you are invoking a method 
    …… 
    4779p(sub-title). Fetch Status/Result Objects of a worker : 
    4880 
    49 If you are using @register_status@ in your worker code to store status/result objects, you can retrieve them from  
     81If you are using @register_status@ in your worker code to store status/result objects, you can retrieve them from 
    5082rails using: 
    5183 
    5284<pre class="boxed">status_obj = MiddleMan.worker(:foo_worker).ask_status </pre> 
    5385 
    54 You can as usual use @job_key@ if *worker was started with a job_key*.  
     86You can as usual use @job_key@ if *worker was started with a job_key*. 
    5587 
    5688You can query status/result objects of all workers in one shot. For example, in your controller: 
    5789 
    58 <pre class="multiline"> def ask_status 
     90<pre class="multiline">def ask_status 
    5991  t_response = MiddleMan.query_all_workers 
    6092  running_workers = t_response.map { |key,value| "#{key} = #{value}"}.join(',') 
    …… 
    75107Important thing to be kept in mind is, when you are creating a worker using above approach, you 
    76108must use a unique @job_key@ while starting the worker. Also, while invoking any of the other methods 
    77 like @ask_status@, @worker_info@ or one of the worker methods, you must use @job_key@. 
     109like @ask_status@, @worker_info@ or one of the worker methods, you must user same @job_key@. 
    78110 
    79111Also another complicated example of starting a worker will be: 
    …… 
    85117 
    86118Above code will start @error_worker@ with @job_key@ and will pass argument @:data@ to @create@ 
    87 method of the worker. Worker will be scheduled to run @hello_world@ method every 5 seconds with argument  
     119method of the worker. Worker will be scheduled to run @hello_world@ method every 5 seconds with argument 
    88120specified in @:data@. 
    89121 

  • trunk/doc/content/scheduling/scheduling.txt

    r319 r323  
    33%(entry-title)<a name="simple_schedule"> Timer Based Scheduling </a>% 
    44 
    5 Simple tasks in the workers can be scheduled using @add_timer@ and @add_periodic_timer@ methods. 
     5Simple tasks in the workers can be scheduled using @add_timer@ or @add_periodic_timer@ methods. 
    66For example: 
    77 
    88<pre class="multiline">class HelloWorker < BackgrounDRb::MetaWorker 
    99  set_worker_name :hello_worker 
    10    
     10 
    1111  def create(args = nil) 
    1212    # time argument is in seconds 
    1313    add_periodic_timer(10) { expire_sessions } 
    1414  end 
    15    
     15 
    1616  def expire_sessions 
    1717    # expire user sessions 
    …… 
    1919end </pre> 
    2020 
    21 Similar one can use @add_timer@ to fire oneshot task execution. 
     21Similarly one can use @add_timer@ to fire oneshot task execution. 
    2222 
    2323%(entry-title)<a name="unix_scheduler"> Unix Scheduler </a>% 
    …… 
    2626from @backgroundrb.yml@ file. A sample configuration looks like: 
    2727 
    28 <pre class="multiline">:backgroundrb:                            
     28<pre class="multiline">:backgroundrb: 
    2929  :ip: 0.0.0.0 
    3030  :port: 11006 
    31   :schedules: 
    32     :foo_worker: 
    33       :foobar: 
    34         :trigger_args: 
    35           :start: <%= Time.now + 5.seconds %> 
    36           :end: <%= Time.now + 10.minutes %> 
    37           :repeat_interval: <%= 1.minute %> </pre> 
     31:schedules: 
     32  :foo_worker: 
     33    :foobar: 
     34      :trigger_args: 
     35        :start: <%= Time.now + 5.seconds %> 
     36        :end: <%= Time.now + 10.minutes %> 
     37        :repeat_interval: <%= 1.minute %> </pre> 
    3838 
    3939Above scheduler option schedules method @foobar@ defined inside @foo_worker@ to start 
    …… 
    4444%(entry-title)<a name="cron_scheduling"> Cron Scheduling </a>% 
    4545 
    46 _BackgrounDRb_ also supports Cron based ccheduling.    
     46_BackgrounDRb_ also supports Cron based ccheduling. 
    4747You can use a configuration file for cron scheduling of workers. The method specified in the configuration 
    4848file would be called periodically. You should accommodate for the fact that the time gap between periodic 
    …… 
    5353A Sample Configuration file for Cron based Scheduling looks like: 
    5454 
    55 <pre class="multiline">:schedules: 
     55<pre class="multiline"> 
     56:backgroundrb: 
     57  :ip: 0.0.0.0 
     58  :port: 11006 
     59:schedules: 
    5660  :foo_worker: 
    5761    :barbar: 
    …… 
    6165 
    6266Above scheduler will schedule invocation of @barbar@ method inside @foo_worker@ at every 10 seconds. 
     67You can also schedule invocation of multiple methods in same worker at different intervals, just use 
     68following as an example configuration file. 
    6369 
    64 p(sub-title). A Word about Cron Scheduler  
     70<pre class="multiline"> 
     71:backgroundrb: 
     72  :ip: 0.0.0.0 
     73  :port: 11006 
     74:schedules: 
     75  :foo_worker: 
     76    :barbar: 
     77      :trigger_args: */10 * * * * * 
     78      :data: Hello World 
     79    :some_task: # execute some_method in foo_worker every 2nd hour 
     80      :trigger_args: 0 * */2 * * * 
     81      :data: Hello World </pre> 
     82 
     83 
     84p(sub-title). A Word about Cron Scheduler 
    6585 
    6686Note that the initial field in the BackgrounDRb cron trigger specifies 
    …… 
    109129it will trigger every second for the subsequent match. 
    110130 
    111 p(sub-title). Loading Workers on demand  
     131p(sub-title). Loading Workers on demand 
    112132 
    113133Usually when your worker is scheduled to execute at longer intervals, it 
    114134doesn't make sense to have worker around, when its doing nothing. Since, scheduling 
    115135via configuration file requires that your worker must be loaded when _BackgrounDRb_ starts, 
    116 your worker is always around, even when doing nothing.  
     136your worker is always around, even when doing nothing. 
    117137 
    118 You can reuse worker in processing requests from rails, but if its not possible  
     138You can reuse worker in processing requests from rails, but if its not possible 
    119139and you rather want worker to start afresh each time, scheduler detects a firetime, you can use 
    120140following syntax to autostart workers on scheduled time: 
    …… 
    123143  set_worker_name :hello_worker 
    124144  reload_on_schedule true 
    125    
     145 
    126146  def create(args = nil) 
    127147    # this method is called, when worker is loaded for the first time 

  • trunk/lib/backgroundrb/bdrb_config.rb

    r321 r323  
    33 
    44class BackgrounDRb::Config 
    5   def self.parse_cmd_options 
     5  def self.parse_cmd_options(argv) 
    66    require 'optparse' 
    77    options = { :environment => (ENV['RAILS_ENV'] || "development").dup } 
    …… 
    2020      opts.on("-v","--version", 
    2121              "Show version.") { $stderr.puts "1.0.3"; exit } 
    22     end.parse! 
     22    end.parse!(argv) 
    2323 
    2424    ENV["RAILS_ENV"] = options[:environment] 

  • trunk/script/backgroundrb

    r322 r323  
    1010 
    1111require RAILS_HOME + '/config/boot.rb' 
     12 
     13# parse CLI options and set RAILS_ENV before loading RoR environment 
     14require "bdrb_config.rb" 
     15BackgrounDRb::Config.parse_cmd_options ARGV 
     16 
     17require RAILS_HOME + '/config/environment' 
    1218require "rubygems" 
    1319require "yaml" 
    …… 
    1622require "packet" 
    1723require "backgroundrb_server" 
    18 require "bdrb_config.rb" 
    1924 
    20 BackgrounDRb::Config.parse_cmd_options 
    2125CONFIG_FILE = BackgrounDRb::Config.read_config("#{RAILS_HOME}/config/backgroundrb.yml") 
    2226pid_file = "#{RAILS_HOME}/tmp/pids/backgroundrb_#{CONFIG_FILE[:backgroundrb][:port]}.pid" 

  • trunk/server/lib/master_worker.rb

    r319 r323  
    156156      raise "Running old Ruby version, upgrade to Ruby >= 1.8.5" unless check_for_ruby_version 
    157157      @config_file = BackgrounDRb::Config.read_config("#{RAILS_HOME}/config/backgroundrb.yml") 
    158        
     158 
    159159      log_flag = CONFIG_FILE[:backgroundrb][:debug_log].nil? ? true : CONFIG_FILE[:backgroundrb][:debug_log] 
    160160      debug_logger = DebugMaster.new(CONFIG_FILE[:backgroundrb][:log],log_flag) 
    161161 
    162162      load_rails_env 
    163        
     163 
    164164      find_reloadable_worker 
    165        
     165 
    166166      Packet::Reactor.run do |t_reactor| 
    167167        @reactor = t_reactor 
    …… 
    172172      end 
    173173    end 
    174      
     174 
    175175    def gen_worker_key(worker_name,job_key = nil) 
    176176      return worker_name if job_key.nil? 
    …… 
    178178    end 
    179179 
    180      
     180 
    181181    # method should find reloadable workers and load their schedule from config file 
    182182    def find_reloadable_worker 
    …… 
    196196      end 
    197197    end 
    198      
     198 
    199199    def load_reloadable_schedule(t_worker) 
    200200      worker_method_triggers = { } 
    201201      worker_schedule = CONFIG_FILE[:schedules][t_worker.worker_name.to_sym] 
    202        
     202 
    203203      worker_schedule && worker_schedule.each do |key,value| 
    204204        case value[:trigger_args] 
    …… 
    220220        value.delete_if { |key,value| value[:trigger].respond_to?(:end_time) && value[:trigger].end_time <= Time.now } 
    221221      end 
    222        
     222 
    223223      worker_triggers.each do |worker_name,trigger| 
    224224        trigger.each do |key,value| 
    …… 
    232232      end 
    233233    end 
    234      
     234 
    235235    # method will load the worker and invoke worker method 
    236236    def load_and_invoke(worker_name,p_method,data) 
    …… 
    241241        worker_name_key = gen_worker_key(worker_name,job_key) 
    242242        data_request = {:data => { :worker_method => p_method,:data => data[:data]}, 
    243           :type => :request, :result => false  
     243          :type => :request, :result => false 
    244244        } 
    245      
     245 
    246246        exit_request = {:data => { :worker_method => :exit}, 
    247           :type => :request, :result => false  
     247          :type => :request, :result => false 
    248248        } 
    249      
     249 
    250250        @reactor.live_workers[worker_name_key].send_request(data_request) 
    251251        @reactor.live_workers[worker_name_key].send_request(exit_request) 
    252252      rescue LoadError 
    253         puts "no such worker #{worker_name}"  
     253        puts "no such worker #{worker_name}" 
    254254      rescue MissingSourceFile 
    255         puts "no such worker #{worker_name}"  
     255        puts "no such worker #{worker_name}" 
    256256        return 
    257257      end 
    …… 
    264264      RAILS_ENV.replace(run_env) if defined?(RAILS_ENV) 
    265265      require RAILS_HOME + '/config/environment.rb' 
    266       load_rails_models unless CONFIG_FILE[:backgroundrb][:lazy_load] 
     266      lazy_load = CONFIG_FILE[:backgroundrb][:lazy_load].nil? ? true : CONFIG_FILE[:backgroundrb][:lazy_load].nil? 
     267      p lazy_load 
     268      load_rails_models unless lazy_load 
    267269      ActiveRecord::Base.allow_concurrency = true 
    268270    end 

  • trunk/tasks/backgroundrb_tasks.rake

    r319 r323  
    88    FileUtils.chmod 0774, script_src 
    99 
    10     defaults = {:backgroundrb => {:ip => 'localhost',:port => 11006 } } 
     10    defaults = {:backgroundrb => {:ip => '0.0.0.0',:port => 11006 } } 
    1111 
    1212    config_dest = "#{RAILS_ROOT}/config/backgroundrb.yml"