Anthony Calandra

Queues in Postgres

2025-03-01T00:00:00Z

# Queues in Postgres

Problem: We've got a queue of tasks that workers can consume and work on. The database stores all the tasks, a worker will come along to take the oldest task available, assign itself to it then begin its work.

Let's create a simple schema to represent the list of tasks and workers:

CREATE TABLE task {
  id SERIAL PRIMARY KEY,
  worker_id INTEGER
};

CREATE TABLE worker {
  id SERIAL PRIMARY KEY  
};

Assumptions:

Assume other external systems are responsible for inserting tasks and managing workers.
Assume task ids are implicitly ordered by their task id; i.e. older tasks have smaller ids.
Assume workers eventually delete the task from the queue when it's done.
Generally, there are enough workers around that the size of the queue will be small, so we won't add an index.

How do we assign workers to tasks?

# Naive Implementation

Here's a simple implementation:

UPDATE task
SET worker_id = ?
WHERE id = (
  SELECT MIN(id) AS task_id
  FROM task
  WHERE worker_id IS NULL
  LIMIT 1
);

The query selects the oldest task id currently in the queue that isn't assigned to a worker. If we find a task, we set the worker to be the current one executing the query (using the ? as a placeholder).

On the surface this query is correct, however, how would it behave if there are multiple workers competing for tasks?

# Implementation 2: Fix Concurrency Bugs

Try and spot some concurrency bugs in this implementation. Can you find any?

...

Here's one: two workers work on the same task. But how? Let's see if we can reproduce this scenario.

Worker 0	Worker 1
`BEGIN;`	`BEGIN;`
`UPDATE task SET worker_id = 0 WHERE id = ( SELECT MIN(id) AS task_id FROM task WHERE worker_id IS NULL LIMIT 1 );`	`UPDATE task SET worker_id = 1 WHERE id = ( SELECT MIN(id) AS task_id FROM task WHERE worker_id IS NULL LIMIT 1 );`
`COMMIT;`
	`COMMIT;`

Result: the task is assigned to worker 1!

What we experienced here is a lost update. Implicit in this query, is that it's running at the READ COMMITTED transaction isolation level -- the default in Postgres ^[1].

When the UPDATE query for Worker 1 is executed, it's blocked on the same row that Worker 0 updated until that worker commits. Then once the change is committed, Worker 1's UPDATE query rereads the row, rechecks all of its conditions in the WHERE clause, and successfully executes the UPDATE and commits.

You can find this in the full set of the documentation on READ COMMITTED here.

The change is pretty simple: just add an extra condition on the WHERE clause to ensure the worker_id is still NULL before doing the UPDATE:

UPDATE task
SET worker_id = ?
WHERE id = (
    SELECT MIN(id) AS task_id
    FROM task
    WHERE worker_id IS NULL
    LIMIT 1
  )
  AND worker_id IS NULL;

# Implementation 3: Better Performance

We're still faced with a subtle (but important) problem: blocking. As indicated in the documentation, when a transaction is making an update (or delete, etc.) to a row that has been changed by another pending transaction, it's required to block on that transaction until completion and load the new row if it had changed. If the pending transaction had rolled back, the UPDATE simply locks the row and changes it as usual.

For a highly-contended task queue, this scales poorly when more workers are competing for tasks to work on.

Is there a way to prevent the worker from waiting?

Postgres provides explicit locking clauses that let us choose certain behaviors when locking on rows. Two of them (as of Postgres 17) are NOWAIT and SKIP LOCKED.

# NOWAIT

Using this clause, if a row-level lock cannot be acquired the moment it is needed, Postgres will abort the transaction an issue an error to the client.

This may be useful for client-sided retry logic and error-handling, but it's likely for most queues we'll want the second option.

# SKIP LOCKED

This option will skip a row if the row-level lock cannot be acquired.

For queueing systems, this is likely the behavior that would be desired; the worker will simply give up on that task and attempt to find the next one to work on (if there are any). Since only one worker will be able to lock a row at a given time, it's not possible for a lost update to occur because any other competing workers would have lost the race and moved on.

Here's what the new query looks like:

UPDATE task
SET worker_id = ?
WHERE id = (
  SELECT MIN(id) AS task_id
  FROM task
  WHERE worker_id IS NULL
  FOR UPDATE OF task SKIP LOCKED
  LIMIT 1
);

Three things to observe with these changes:

FOR UPDATE grants the transaction an exclusive lock on the row for the task that's selected. With the SELECT .. FOR UPDATE command, if a row had been changed by a concurrently committed transaction (i.e. after it committed and released its lock, and while the current transaction still sees the old row in its snapshot), it will reload and lock the latest version of the row to ensure it still satisfies the WHERE clause's conditions.
SKIP LOCKED will skip this row if the transaction is unable to acquire a lock.
The condition in the UPDATE for worker_id is gone since we will be the exclusive reader of the row that's to be updated. No other transaction can see this row in their task selection query.

To summarize, this query won't cause transactions to block on a task that may have been taken by another. So for example, if 50 workers had tried to access this task at the same time, the winner will get to take ownership of the task, while the other 49 will move on to the next oldest task to take, and so on.

# Isolation Levels

While the condition to take a task is pretty simple, in your production system you may have more requirements on how workers should take a task to work on.

Keep in mind the complexity of your task selection, for you might need the stronger guarantees that other levels such as REPEATABLE READ and SERIALIZABLE provide. At these levels, transaction rollbacks are common and may come at a performance cost for your application.

At work, I had seen hundreds of thousands of transaction rollbacks per day when our task selection query was both expensive and at the REPEATABLE READ isolation level. Once we refactored the query and were able to drop down to READ COMMITTED, our average task queue times dropped significantly. A task queue time is defined by how long a task takes to be picked up and owned by a worker.

Now because we dropped down to this level, our task selection logic had to accept stale reads in certain circumstances, therefore affecting how workers accepted tasks. This is a change we were fine with for our use case.

So to put it simply: if your task selection process is complicated, the tradeoff in correctness by choosing a stronger isolation level is performance cost, and that tradeoff may be quite significant. If your application can retry and do it quickly for potential rollbacks, the cost of using a higher isolation level may be acceptable. On the contrary, if transaction rollbacks are too slow to be acceptable, dropping down to READ COMMITTED and using a locking strategy like in Implementation 3 is likely to be more beneficial.

# Other Implementations

Postgres also provides LISTEN and NOTIFY for alerting workers to new tasks. This provides an opportunity for a different task selection implementation.

const in C++

2024-12-15T00:00:00Z

# const in C++

In this article I'm going to go over all the cases of const in C++ I see regularly.

# Why `const`?

First I want to briefly touch on why we should bother having to qualify as many variables, member functions, etc. whenever we can.

The topic of compiler optimizations have been written many times before. Please take a look at this GotW article for when const is effective and also when it may be ineffective at optimizing code. Another practical example can be found here by Matt Godbolt, specifically when talking about const in testFunc.

Besides compiler optimizations, const is also important for developers, arguably more so. When you something is qualified as const, it's effectively shrinking the set of runtime state you need to manage in your mental model of the code you're looking at. In other words, the fewer variables that are changing in a function, the more you can focus on the code that does change state.

void foo(int a, int b) {
    bool b = ...;

    T t = ...;
    U u = ...;

    ...
}

In foo, because nothing is qualified as const, you need to examine every single local variable and determine if you need to keep track of their state throughout the body of this function. However, with bar:

void bar(const int a, const int b) {
    bool b = ...;

    const T t = ...;
    U u = ...;

    ...
}

In this fictitious function, I qualified three local variables with const. Now for a future developer who needs to understand what this function does, step through in a debugger, and so on, we only need to observe the values of a, b, and t once, and focus on the runtime state of u and b only.

# When `const`?

I like to make it a habit to proactively use const, and while I continue working on code if I find I have to change the state of something and it makes sense to do so, I'll remove it. Due to having to write out an extra few keystrokes, I think it's more cumbersome than using a language like Rust which took the opposite direction (const by default rather than opt-in). Although it's annoying, the benefits are worth it.

# `const` with variables

Use const when you want to essentially mark the value of this variable immutable. When used with objects, you can only call special member functions (such as constructors, destructors) and const member functions.

const int& x = 123;
const bool b = true;
const T t;

# Lifetime extension

An interesting feature of using const references like in the first line above is you get lifetime extension: the literal value 123 can be bound to a reference because the compiler extends the lifetime of this value simply because we use const to tell the compiler we aren't changing the value. This would not compile if it was a mutable reference. In practice, don't worry about this too much.

# `const` with functions

Consider const in the return type when you're returning a pointer or reference that you want don't intend to modify. Don't use const for value types, like int, some T type you create, etc. as this doesn't provide any practical benefit in modern C++.

// BAD:
const int get() { ... }

// GOOD:
int get() { ... }

Consider const for parameters that you don't intend to modify. This advice also includes value types.

// BAD: `n` doesn't change, so it could be `const`.
int square(int n) {
    return n * n;
}

// GOOD: no intention to change `n`.
int square(const int n) {
    return n * n;
}

Do not add const qualifiers to value types in the parameter list of function declarations (such as those in header files), as these are redundant.

For example:

void foo(const int x); // BAD: no need for `const` here.
void foo(const int x) {
    // do something with x
}

void foo(int x); // GOOD
void foo(const int x) {
    // do something with x
}

Same applies to member functions.

But why?

const matters for things like references and pointers, but for value types these are copied anyway. It doesn't matter if the thing passed in is const from a declaration perspective, what matters when we use const is how we're using it in the definition (i.e. the body of a function). From the interface, a value type that gets copied doesn't need to advertise its "const-ness"; it's irrelevant to the caller.

The two declarations cannot be overloaded together, as they're the same declaration:

struct myfoo {
    void foo(int x);
    void foo(const int x);
};

void myfoo::foo(const int x) {}

Using GCC 14.2:

<source>:10:6: error: 'void myfoo::foo(int)' cannot be overloaded with 'void myfoo::foo(int)'
10 | void foo(const int x);
    |      ^~~
<source>:9:6: note: previous declaration 'void myfoo::foo(int)'
    9 | void foo(int x);
    |      ^~~

# `const` with member functions

const can be applied to member functions like so:

class MyClass {
    int x = ...;
public:
    int get() const {
        return x;
    }
};

The advice for using const in regular functions apply here too.

Consider const when the member function doesn't change the state of the class. In the above example, since we simply read the value of x, we're not modifying the class in any way so it's appropriate to mark get as const-qualified.

The this pointer also becomes const-qualified by the compiler in the body of a const member function.

# `mutable`

There is one edge-case to be remindful of when it comes to const member functions. I'll illustrate with a common example:

class MyClass {
    int x = ...;
    mutable mutex mx = ...;
public:
    int get() const {
        int xValue;

        mx.lock();
        xValue = x;
        mx.unlock();

        return xValue;
    }
};

This is a simplified example where we use a mutex to safely get the value of x since there is concurrency involved. Notice that we call lock and unlock which changes the state of mx -- also changing the state of MyClass. We can use the mutable keyword to allow these calls in a const-qualified member function.

But why is this useful? The idea is if we have an interface that semantically makes sense to be const (think of this as read-only), we can use mutable as an escape hatch to keep our interface sensible. The only reason we needed to use mutable was due to an implementation detail hidden from the user (i.e. locking/unlocking the mutex). Remember, we only did this because the mutex is a side-effect of safe concurrency -- it is not part of the semantic state of our class and so it's acceptable to keep it logically separated from our interface.

# `const` with member variables

Member variables const-qualified are generally not useful except for view types (classes which are used as read-only containers for data -- think of std::string_view semantically).

Additionally, const-qualified members may prevent implicitly-defined special member functions such as default constructors.

For class constants, consider making them static.

# `const` with raw pointers

const has a few forms with raw pointers, so use the form most appropriate for your use case:

int* - pointer to int;
int const * - pointer to const int;
int * const - const pointer to int;
int const * const - const pointer to const int.

# `const` with smart pointers

const also shares a few forms with smart pointers, so use the form most appropriate for your use case:

std::unique_ptr<int> - exclusive pointer to int;
std::unique_ptr<const int> - exclusive pointer to const int;
const std::unique_ptr<int> - const exclusive pointer to int;
const std::unique_ptr<const int> - const exclusive pointer to const int.

# `const` with containers

Using const prevents reassignment of a container and also keeps immutable elements:

const std::vector v {1, 2, 3};
v = std::vector{3, 2, 1}; // ERROR

const std::vector v {1, 2, 3};
v[0] = 123; // ERROR

Some containers also propagate const because the operator[] or at member functions return const references:

struct Foo {
    void f() {}
};

const std::vector<Foo> v {Foo{}, Foo{}, Foo{}};
v[0].f(); // ERROR: f is not `const`-qualified

However, not all containers do this!

int arr[] = {1, 2, 3};
const std::span s {arr};
s[0] = 3; // OK

With std::span you'd need a span of const int to make this work.

# `const` with iterators

Most people are familiar with begin and end on containers to express the begin and end iterators respectively. The standard library also provides their const equivalents: cbegin and cend. These return the const-equivalent iterators into the container.

These are useful when you want to express algorithms or ranges where you don't intend to mutate container elements.

std::vector v {1, 2, 3};
const auto sum = std::accumulate(v.cbegin(), v.cend(), 0, std::plus{});
// sum == 6

# Lambda parameters

Lambdas don't work differently when it comes to parameters than with functions, but I often see confusion when they are passed to algorithms. Here's an example:

std::vector v {1, 2, 3};
const auto sum = std::accumulate(v.cbegin(), v.cend(), 0, 
    [](auto x, auto y) { return x + y; });

This function returns the same sum, but while this example compiles, x and y are not const. Since we're taking a copy of a simple type like an int, this works just fine, but these parameters should be const for the reasons in the opening section (see: "Why const?"). We would get a compiler error if we tried to take a mutable reference:

const auto sum = std::accumulate(v.cbegin(), v.cend(), 0, 
    [](auto& x, auto& y) { return x + y; }); // ERROR: int& is not `const`

So in general, match up the iterator qualification with the lambda parameters unless you're doing non-trivial work (in that case, it probably makes sense to move this out into a free/member function).

# `const` with ranges

Ranges with const is messy and complicated. In C++20: In Detail by Nicolai Josuttis, he goes into this topic in much more depth than I can here. Based on my own conclusion from reading his book, const is probably too dangerous to use safely with ranges until a bunch of problems get fixed in versions later than C++20, which is a shame because of the benefits of const.

It's okay to suffer a little bit

2024-08-18T00:00:00Z

# It's okay to suffer a little bit

Sometimes it's good to suffer for a bit, whether it be a problem you don't know how to solve, a bug you can't figure out, or even trying to find a job. Problem-solving is one of those core life skills we should be developing endlessly. If you always look for others when you get stuck, you become dependent on them. Struggling is when we start thinking outside the box, trying anything we can to make it work. You know you're in the trenches when you need to stop what you're doing, clear your mind, and sink deep into thought about how to approach next steps. Commonly for me this is when I'm recompiling some code and thinking "if this isn't the fix then I don't know what is..." When I say that to myself then I know I'm really in deep.

Often I end up figuring it out. Other times that's when I had to learn to ask others for help.

An example of this in my life is when I was a university student looking for my first internship. It was Spring 2014 and my skills as a software developer were mostly PHP, SQL, JavaScript and some HTML/CSS. I developed these skills working in mostly open-source developing forum software plugins, and even did a few freelancing gigs on the side. So given my experience, I was extremely confident that I would be hired by a prestigious company in the first round.

"Everyone has a plan until they get punched in the mouth." - Mike Tyson

Instead I didn't get any job in my field, spending the summer working in customer service. I was devastated.

Despite feeling at my lowest it was a big boost for my growth. Because of this situation, I reassessed my skills and decided to learn some of the "hot" new tech at the time which was server-side JavaScript (NodeJS). I spent all summer developing my skills and working on side-projects to make myself more employable. During the next co-op cycle––Spring 2015––I interviewed with a few companies and chose to begin my career as a full-stack developer intern at SugarCRM in the Bay Area. Looking back, I'm glad I didn't get a job during that first year.

It's okay to suffer a little bit.