Haskell Bounded Concurrency for Background Workers
A worker that drains a job queue is the first Haskell service most teams ship, and it is also the first one that gets killed by the kernel.

A worker that drains a job queue is the first Haskell service most teams ship, and it is also the first one that gets killed by the kernel. The code looks harmless: pull the pending rows, hand the list to mapConcurrently, write the results back. It passes review because the types check and the logic is four lines. Then a backlog builds up overnight, the next run pulls 20,000 rows instead of 40, resident memory climbs from 90 MB to 2.1 GB in under a minute, and Kubernetes terminates the pod with exit code 137.
What makes this painful to debug is that nothing in the code looks concurrent in a dangerous way. There is no explicit forkIO, no mutable state, no unsafe call. The fault is an absent number: nobody said how many jobs may be in flight at once. Haskell bounded concurrency is the discipline of always answering that question, and it costs about six lines to do properly.
What actually breaks when nothing is bounded
Green threads are cheap, which is exactly why people assume unbounded spawning is fine. A GHC thread starts with a 1 KB stack chunk, so ten thousand of them cost around 10 MB of stack. That number is true and it is also beside the point.
The expensive part is everything a running job holds onto. If each job decodes a 40 KB JSON payload, builds an intermediate map and keeps a response body in memory, ten thousand simultaneous jobs retain on the order of a gigabyte of live data that the garbage collector cannot touch. With the copying collector, peak RSS is roughly twice the live set, so a 1 GB live heap wants 2 GB of memory. Your container limit is 512 Mi.
Three other failures arrive at the same time, usually in this order:
- The database pool runs dry. If every job needs a connection and the pool holds 10, the other 9,990 threads queue on an MVar inside the pool. Acquisition waits that were 0.3 ms go to several seconds, and the health check endpoint starts timing out because it shares the pool.
- Downstream services return 429 or just stop answering. A payment API that allows 50 requests per second does not care that your concurrency was accidental.
- Garbage collection stops being incremental. A heap that keeps growing means more major collections, each scanning a larger live set. The GHC report from the statistics RTS flag shows productivity dropping below 60 percent, which is the clearest signal that the problem is memory pressure and not CPU.
Sizing the pool correctly is a related problem and worth doing first; the arithmetic is in our guide to Haskell connection pooling for Postgres services.
The advice to just use mapConcurrently, and why I argue with it
Search for concurrent work in Haskell and the answer is almost always mapConcurrently from the async package. It is a fine function. It is also a loaded gun, because its concurrency equals the length of the list you pass it, and the length of that list is decided by production data rather than by you.
The usual rebuttal in code review is that the list is small. It is small today. Queue processors specialise in getting large exactly once: after an incident, after a migration backfill, after a consumer was paused for maintenance. The cost of the bounded version is one extra argument, so there is no trade worth discussing.
The second piece of common advice is to set the number of capabilities and assume that bounds the work. It does not. Capabilities bound how many Haskell threads execute CPU work at the same instant. Threads blocked on a socket or a database round trip are not using a capability, and in a worker that is 95 percent of them. A service running on 4 cores can easily hold 8,000 jobs in flight, all of them waiting, all of them retaining memory.
Three ways to put a ceiling on it
Pick based on how long the process lives and how much control you want over failure.
| Approach | Package | Good for | Watch out for |
|---|---|---|---|
| Semaphore around each job | base, QSemN | Adding a limit to code you cannot restructure | Must use bracket so a dying thread returns its token |
| Pooled traversal | unliftio, pooledMapConcurrentlyN | Batch work where you want the results back | Still builds the whole result list in memory |
| Fixed workers reading a bounded queue | stm, TBQueue | Long running consumers, Kafka or Postgres queues | You own startup, restart and drain logic |
For most batch jobs pooledMapConcurrentlyN is the honest default. It takes the limit as its first argument, spawns that many workers, and feeds them from the list, so the change from the unbounded version is a single token on the line. Memory still scales with the result list, so if you are processing 50,000 rows and only need a count, fold as you go rather than collecting.
For a service that runs for weeks, the fixed worker pattern is better. Create a TBQueue with a capacity around twice your worker count, start N workers in a loop, and let the producer block on a full queue. That back pressure is the feature: a slow consumer automatically slows the reader instead of buffering the backlog into the heap. Bounded queues are the one piece of the design I have never regretted adding.
Choosing the number instead of guessing it
The limit follows from throughput and latency, not from core count. If a job spends 120 ms waiting on an HTTP call plus 15 ms of CPU, and you need to finish 200 jobs per second, then in flight work is 200 times 0.135, which is 27. Round to 30 and you have a defensible number.
Then check the number against every shared resource it touches. With a pool of 10 connections and jobs that each hold a connection for most of their life, a limit of 30 just moves the queue from the scheduler into the pool, so the real ceiling is 10 and the extra 20 threads only waste memory. If the downstream API allows 50 requests per second and each call takes 200 ms, your side of the contract is 10 concurrent calls.
Multiply the limit by the peak memory of one job to sanity check the heap. Thirty jobs at 6 MB each is 180 MB live, which wants roughly 360 MB of RSS under the copying collector, which fits in a 512 Mi container with room for the rest of the program. This is the calculation I wish I had done before the first time I woke up to an OOM alert.
Exceptions, restarts and draining
A bounded pool is only safe if a failing job cannot leak its slot. Acquire with bracket or withQSemN rather than a plain acquire and release pair, because an async exception between the two silently shrinks your pool by one until the process restarts. I have seen a worker degrade from 20 slots to 3 over a week this way, with no error in the logs beyond the original timeouts.
Decide explicitly what one bad job does to the batch. With pooledMapConcurrentlyN, an exception in any job cancels all the others, which is right for a transactional backfill and wrong for a mail queue. If you want the rest to continue, wrap each job in try and return an Either, then count the failures at the end. Unhandled exceptions in a bare forkIO thread, by contrast, kill only that thread and print to stderr, which is how jobs vanish without anyone noticing.
Shutdown needs the same care. On SIGTERM, stop reading new work, let in flight jobs finish within a deadline of around 25 seconds, and only then close the pool. The ordering and the signal handling are covered in Haskell graceful shutdown for long running services.
What to put on a dashboard
Four numbers tell you whether the limit is right: jobs in flight as a gauge, queue depth, job duration as a histogram, and slots waiting for a semaphore token. If in flight sits pinned at the limit while duration stays flat, you are throughput bound and can raise the limit. If duration rises as in flight rises, something downstream is saturating and raising the limit makes everything slower. Queue depth growing over hours means the limit cannot keep up at all and you need more replicas rather than more threads per replica. More patterns for this kind of service live in the systems section.
Frequently asked questions
What is a sensible default concurrency limit for a Haskell worker?
For jobs that mostly wait on network or database calls, start between 10 and 30 and verify it against your connection pool size. For CPU heavy jobs, start at the number of capabilities and measure, since more threads than cores only adds scheduling overhead.
Does mapConcurrently create one OS thread per job?
No. Jobs run as lightweight GHC threads multiplexed onto a small number of OS threads. That is why the memory cost comes from retained job data rather than from stacks, and why the limit still matters.
Should I use QSemN or a TBQueue for bounding work?
Use a semaphore when you are adding a ceiling to an existing traversal. Use a bounded queue with fixed workers when the process runs continuously and you want back pressure to reach the producer.
How do I tell that unbounded concurrency caused an out of memory kill?
Compare peak resident memory against the number of jobs in the batch. If memory tracks batch size almost linearly and the statistics report shows a large live set with low productivity, the concurrency is unbounded rather than the heap being leaky.
More in Systems
Systems
Haskell Health Check Endpoints for Kubernetes Probes
A student of mine shipped her first Haskell service to a small Kubernetes cluster last spring, and it restarted itself 41 times...
Systems
Haskell Structured Logging for Production Services
Your Haskell service crashed at 03:12, and the only clue is a wall of putStrLn output with no request id, no timestamp...
Systems
Haskell Retry With Exponential Backoff for Flaky Services
Your Haskell service calls a payment API, a Postgres replica, or an S3 bucket, and roughly one request in two hundred fails...