ShouldBeUnique Vs WithoutOverlapping In Laravel 10

Summary

ShouldBeUnique will prevent any new, duplicate jobs from entering the queue while another instance of that job is queued OR processing.

WithoutOverlapping allows duplicate jobs to be queued, but prevents them from being processed at the same time using an atomic lock.

While similar, they are unique, and which you want to use will depend on your use case. Below I'll dive into my experience and which I ended up needing for my application.

Introduction

Laravel's queues are, in my opinion, one of the greatest features that Laravel has to offer. No framework, library or project that I have seen has yet to implement a system so easy to use, or reliable. Each time I encounter an issue that I need to overcome in my business logic, Laravel somehow always has the answer.

For everything queues related - the Laravel docs are a great place to start

A perfect example of this is when I tried to prevent two instances of a job from running concurrently. While Laravel did provide the solution I was looking for, I got a little caught up in the difference between the ShouldBeUnique contract, and the WithoutOverlapping middleware. Both have similar functions, but there are subtle differences that I want to outline here.

A Practical Example

I thought I'd start with my example, as code is commonly best explained anecdotally.

I was working on a feature recently where I needed to calculate the amount of money that a particular user owed to the system. That calculation was expensive, so I'd decided I needed to run it as a background process.

There were several different scenarios that could trigger the need to calculate this value. To name a few:

• When the user updated their payment method
• When the user incurred a new charge
• When the user's billing cycle was reached

Each time one of these occurred, a queued job was immediately dispatched to calculate the updated total for a statement.

This introduced a new problem in the system however. It became possible for two (or more) of these situations to arise at the same time, and cause multiple jobs to be dispatched concurrently. In this, we create a small race condition. It was possible that one of the jobs might be dispatched, only for the data to become outdated immediately after by another event. Then, it was a race between the two jobs to see which would update the total soonest.

So, I now had a task: ensure that the queue can only process one of those jobs at the same time.

After some quick googling, I found ShouldBeUnique and WithoutOverlapping pretty quickly. So what is the difference, and which should I have used?

The ShouldBeUnique Contract

As per laravel's documentation:

Sometimes, you may want to ensure that only one instance of a specific job is on the queue at any point in time. You may do so by implementing the ShouldBeUnique interface on your job class.

This means that if the application tries to queue this job while another is in the queue OR is processing, it will simple not be queued.

This sounds pretty close to what I need. The only caveat was I need to add the uniqueId method to make sure that in my application, we did not allow only one of the jobs to process across ALL users.

There's one problem here. I still want new jobs to be queued, I just don't want them to be executed at the same time.

Now let's take a look at WithoutOverlapping middleware.

The WithoutOverlapping Middleware

From the Laravel documentation:

[...] middleware that allows you to prevent job overlaps based on an arbitrary key. This can be helpful when a queued job is modifying a resource that should only be modified by one job at a time.

For example, let's imagine you have a queued job that updates a user's credit score and you want to prevent credit score update job overlaps for the same user ID. To accomplish this, you can return the WithoutOverlapping middleware from your job's middleware method:

This sounds perfect. In my scenario of needing to update the total amount due, I want to make sure that only one job is writing to my resources at a time.

One thing that I found confusing here is that when you have a job that overlaps, and that job is released back to the queue. That counts as a retry. So in the event that you have a low number of retries, be aware that it might fail permanently if it gets released too many times.

The Solution

So, now I know that we have two options for achieving the outcome I need. Either we prevent new jobs from being queued with the ShouldBeUnique contract. Or we ensure that duplicate jobs cannot be run using the WithoutOverlapping middleware. Here is what I settled on:

namespace App\Jobs;  
  
use Illuminate\Bus\Queueable;  
use Illuminate\Contracts\Queue\ShouldQueue;  
use Illuminate\Foundation\Bus\Dispatchable;  
use Illuminate\Queue\InteractsWithQueue;  
use Illuminate\Queue\Middleware\WithoutOverlapping;  
use Illuminate\Queue\SerializesModels;  
  
class UpdateUserStatement implements ShouldQueue  
{  
    use Dispatchable;  
    use InteractsWithQueue;  
    use Queueable;  
    use SerializesModels;  
  
    public function __construct(protected User $entry)  
    {  
        //  
    }  
  
    public function handle(): void  
    {  
        // ...  
    }  
  
    public function middleware(): array  
    {  
        // I'm expecting that the UpdateUserStatement job
        // shouldn't take longer than 5 seconds. So we
        // should be able to safely retry the subsequent
        // job after that amount of time.
        return [(new WithoutOverlapping($this->user->id))->releaseAfter(5)];  
    }  
}