ThreadPoolExecutor Internals & Pool Sizing

#Introduction

Every Executors factory method (except work-stealing and virtual-thread executors) returns a ThreadPoolExecutor configured in a particular way. Building one directly gives you full control — and avoids the unbounded queues and thread counts hidden in the factories.

#The Constructor

ThreadPoolExecutor pool = new ThreadPoolExecutor(
        4,                                    // corePoolSize
        16,                                   // maximumPoolSize
        60, TimeUnit.SECONDS,                 // keepAliveTime for threads above core
        new ArrayBlockingQueue<>(500),        // workQueue — bounded
        namedThreadFactory("order-worker"),   // threadFactory
        new ThreadPoolExecutor.CallerRunsPolicy()   // rejection handler
);

#How a Task Is Accepted

This algorithm surprises most developers:

Example

Core 2, max 4, queue capacity 3 — tasks are long-running:

#Work Queues

How the Factory Methods Are Built

#Rejection Policies

When the queue is full and the pool is at maximumPoolSize — or after shutdown() — the handler decides what happens:

RejectedExecutionHandler handler = (task, executor) -> {
    rejectedCounter.increment();
    log.warn("Task rejected; active={} queue={}", executor.getActiveCount(), executor.getQueue().size());
    throw new RejectedExecutionException("Order pool saturated");
};

#A Custom ThreadFactory

static ThreadFactory namedThreadFactory(String prefix) {
    AtomicInteger counter = new AtomicInteger(1);
    return runnable -> {
        Thread t = new Thread(runnable, prefix + "-" + counter.getAndIncrement());
        t.setDaemon(false);
        t.setUncaughtExceptionHandler((th, ex) -> log.error("Uncaught in {}", th.getName(), ex));
        return t;
    };
}

Java 21 also offers Thread.ofPlatform().name("order-worker-", 1).factory(). Guava's ThreadFactoryBuilder and Spring's CustomizableThreadFactory do the same.

#Sizing a Thread Pool

There's no universal number — it depends on the work.

CPU-Bound Work

threads ≈ number of cores   (often cores + 1)

More threads than cores only add context switching.

I/O-Bound Work

Threads spend most of their time waiting (database, HTTP), so more threads keep the CPU busy:

threads ≈ cores × (1 + wait time / compute time)

Example: 8 cores, each request waits 90 ms on a database and computes for 10 ms → 8 × (1 + 90/10) = 80 threads.

Little's Law

threads needed = arrival rate × average time per task

200 requests/second × 0.25 seconds each = 50 concurrent tasks in flight.

Separate Pools for Different Workloads (Bulkheads)

Don't run slow calls to a flaky third-party API on the same pool as fast internal work. If the API hangs, it consumes every thread and everything else starves. Give each kind of work its own pool — the bulkhead pattern (as in Resilience4j).

#Hooks and Monitoring

ThreadPoolExecutor pool = new ThreadPoolExecutor(4, 16, 60, TimeUnit.SECONDS, new ArrayBlockingQueue<>(500)) {
    @Override
    protected void beforeExecute(Thread t, Runnable r) {
        MDC.put("pool", "orders");
    }

    @Override
    protected void afterExecute(Runnable r, Throwable t) {
        MDC.clear();
        if (t != null) log.error("Task failed", t);   // only for execute(); submit() wraps exceptions in the Future
    }
};

Useful metrics: getActiveCount(), getPoolSize(), getQueue().size(), getCompletedTaskCount(), getLargestPoolSize(). Micrometer's ExecutorServiceMetrics exports them; Spring Boot does this automatically for its own task executors.

Pools can be tuned at runtime with setCorePoolSize(), setMaximumPoolSize() and allowCoreThreadTimeOut(true), and warmed up with prestartAllCoreThreads().

#Key Takeaways

• ThreadPoolExecutor is configured by core size, max size, keep-alive, queue, thread factory and rejection handler.
• New threads beyond core are created only when the queue is full.
• The Executors factories use unbounded queues or unbounded threads — avoid them for production services.
• Rejection policies define behaviour under overload; CallerRunsPolicy gives back-pressure.
• CPU-bound pools ≈ cores; I/O-bound pools are larger and must match downstream capacity.