Severe Memory Leak (Heap & Off-Heap) via ThreadLocal in ParallelGZIPOutputStream

[QiuYucheng2003]

Bug Description

There is a critical memory leak in ParallelGZIPOutputStream.java. The class uses a ThreadLocal<State> to cache Deflater instances and large byte[] buffers to avoid allocation overhead, but it lacks any cleanup mechanism.

Root Cause

In ParallelGZIPOutputStream.java, the STATE ThreadLocal is initialized but never cleared:

private static final ThreadLocal<State> STATE = new ThreadLocal<State>() { ... };

Inside Block.call(), worker threads retrieve the state via STATE.get(). However, STATE.remove() is never invoked. Since the tasks are submitted to an ExecutorService, the worker threads are long-lived. These threads retain strong references to the State objects in their ThreadLocalMap indefinitely.

Impact
This misuse leads to two severe issues:

1. Heap Exhaustion (OOM): ByteArrayOutputStreamExposed retains a dynamically expanding byte[] (starting at 72KB) per thread. In an application with heavy thread pool usage, this causes linear heap memory growth and eventual OutOfMemoryError.

2. Off-Heap Memory Exhaustion (OOM Killer): The Deflater object allocates significant native (off-heap) memory via zlib. Because the State object is anchored by the long-lived thread, it is never garbage collected. Consequently, Deflater.end() is never called, leading to severe native memory leaks and potentially triggering the OS-level OOM Killer.

Proposed Fix
·Deprecate ThreadLocal usage: In an ExecutorService environment, ThreadLocal is dangerous without strict lifecycle management.

·Implement an Object Pool: Replace ThreadLocal with a bounded thread-safe object pool (e.g., using ConcurrentLinkedQueue<State>). Worker threads can borrow a State at the start of call(), use it, and return it in a finally block, ensuring safe reuse and enabling explicit def.end() cleanup upon pool shutdown.