Better Sidekiq Classes
Sidekiq is pretty much the go-to solution for enqueuing jobs for background processing when working on a Ruby-based project. It’s simple to implement, has a clear DSL, and is well-supported by common testing frameworks like RSpec. That being said, and I may be nitpicking, but the standard implementation examples do add a little extra overhead if you wish to be more flexible with your job business logic and additional test setup is required.
The “normal” way
Given we want a little background job that updates a user’s name and any other business logic needed (to keep the example simple), we could create a standard Sidekiq job with that logic and call it from our code as such:
class UpdateUserJob include Sidekiq::Job def perform(user_id, name) user = User.find(user_id) user.update!(name: name) # .. additional business logic here end end UpdateUserJob.perform_async(1, "bob") # perform the job as soon as possible UpdateUserJob.perform_in(5.minutes, 1, "bob") # delay the processing by 5 minutes
This is simple enough. But what if we want to test the logic being performed inside this job? Firstly, we’d need to update our
spec_helper to let it know what to do with a job. If you’re using RSpec, you might even need to add the rspec-sidekiq gem for additional configuration and matcher options. A classic-looking spec for this job might look something like this:
it "updates the user" do UpdateUserJob.perform_async(1, "bob") expect(UpdateUserJob.jobs.size).to eq(1) UpdateUserJob.drain expect(User.find(1).name).to eq("bob") end
We need to
drain the job before we can test the result of its work. Alternatively, we could wrap the test inside one of the Sidekiq testing helper methods:
it "updates the user" do Sidekiq::Testing.inline! do UpdateUserJob.perform_async(1, "bob") expect(User.find(1).name).to eq("bob") end end
This does work as expected however it does leave a little to be desired. We have to remember to do this whenever testing job logic (the number of times I’ve had to search through previous specs to remind myself of the syntax), it adds a few extra lines to the tests which are not necessary and there is a spec performance hit as well.
The “use case” way
So, what options do we have? Well, the first option would be to extract the logic performed by the job into a class which we would then pass the job attributes to for processing. That way we can independently test the business logic without having to mess with background workers and the like. I have used various names for these classes, Use Case, Resource, or Library to name a few (use whatever best suits you and your conventions). Our new job would look something like this:
class UpdateUserJob include Sidekiq::Job def perform(user_id, name) UpdateUserUseCase.new(user_id, name).call end end class UpdateUserUseCase def initialize(user_id, name) @user_id = user_id @name = name end def call user = User.find(@user_id) user.update!(name: @name) # .. additional business logic here end end
Now, when it comes to testing, we could just forgo testing the actual job class as it does nothing except pass on some arguments to a plain old Ruby object. We can just test the class directly. Sidekiq is also already extensively tested so there is little point in testing whether the jobs on the queue have increased or not unless you are expecting new jobs to be created as a knock-on effect of your business logic.
it "updates the user" do UpdateUserUseCase.new(1, "bob").call expect(User.find(1).name).to eq("bob") end
The added benefit to doing it this way is that you have a plain old Ruby object that you can call independently and inline in your code elsewhere should the need arise. For instance, you may want to defer the processing to the background when handling a mass import of data, however, when being performed as a single HTTP request to your API, you could wish to do it inline.
The downside to this is that we now, however, have had to create an additional class to provide this flexibility. That means, if we wish to keep this convention going, for each
Job class, there would be a corresponding
UseCase class. What if we could take this idea one step further? What if we could turn our Job class into a plain old Ruby object right from the start? Let’s rewrite our
The “runner” way (with POROs)
Let’s pretend you had never heard of Sidekiq or background queues, and you want to write some code to perform a specific task. For the sake of convention, I have ended the following class name with “Job” to match previous examples however by syntax definition, it is no longer what we would call a classic “job”. It is now just a Plain Old Ruby Object (PORO; identical to our previous
class UpdateUserJob def initialize(user_id, name) @user_id = user_id @name = name end def call user = User.find(@user_id) user.update!(name: @name) # .. additional business logic here end end
With that in place, we need to set up some sort of mechanism to have this code be run by Sidekiq in our background queue. We can introduce a simple class to wrap the sidekiq calls, allowing us to inject our custom job class with its arguments. To not conflict with the Sidekiq namespace, I have created the
Runners module which will run this code for me (side note: no matter what project I’ve worked on in the past, I always seem to use these “runner” style classes for various things).
module Runners class Sidekiq include ::Sidekiq::Job sidekiq_options(retry: 3, backtrace: true, queue: "default") def perform(job_class_name, *args) Kernel.const_get(job_class_name).new(*args).call end def run(job_class, *args) self.class.perform_async(job_class.to_s, *args) end def run_in(delay_in_seconds, job_class, *args) self.class.perform_in(delay_in_seconds, job_class.to_s, *args) end end end
And with that, we have everything we need to be able to throw plain old Ruby classes into the background for processing. If we wish to run our business logic inline, we simply run
but if we want to delegate the processing to the background queue, we can send it off immediately or delay it
Runners::Sidekiq.new.run(UpdateUserJob, 1, "bob") # perform the job as soon as possible Runners::Sidekiq.new.run_in(5.minutes, UpdateUserJob, 1, "bob") # delay the processing by 5 minutes
And that’s pretty much it. You can now define your background job classes as plain old Ruby objects, run them inline, or pass them over to Sidekiq to run in the background, and have the classes be easily testable. But when can we have our cake and eat it? There are a couple of caveats to be aware of. Firstly, the Sidekiq jobs will all be of the same type - you will have to use their arguments to identify specific ones; secondly, all the job classes will need to follow the same convention of the
initialize method accepting all the arguments (remember, with background jobs, do not pass in complex objects as arguments) and a
call method (in this example; you could use
process or whatever default you prefer) to perform the business logic. Credit to my friend Kris for the initial concept.