HTTP Gzip, Deflate, and Brotli: Compression Encoding for Web Developers

Quick answer

💡HTTP response compression is negotiated via the Accept-Encoding request header and declared via Content-Encoding in the response. Use Content-Encoding for compressed response bodies (gzip, br, deflate) and Transfer-Encoding for hop-by-hop chunked transfer. Brotli (br) achieves 15-20% better compression than gzip on text. In Node.js, use zlib.createGzip() or zlib.createBrotliCompress() as Transform streams.

Encode/Decode Base64 →

Error symptoms

  • ✕Response body is garbled or binary-looking in browser devtools despite readable JSON on the server
  • ✕Content-Encoding: gzip response fails to decompress with 'incorrect header check' error
  • ✕Double-compressed responses — gzip applied twice because middleware and CDN both compress
  • ✕Node.js server sends compressed response but client receives 'ERR_CONTENT_DECODING_FAILED'
  • ✕Pre-compressed .gz static files served with wrong Content-Encoding header, appearing as downloads
  • ✕Extremely small response expanding to gigabytes on decompression — a gzip bomb

Common causes

  • •Confusing Content-Encoding (message body encoding, end-to-end) with Transfer-Encoding (transport hop-by-hop)
  • •Compressing already-compressed data like JPEG, PNG, or ZIP files, which increases size instead of reducing it
  • •Forgetting to check Accept-Encoding before compressing, sending gzip to clients that did not request it
  • •Applying gzip at both the application layer and a reverse proxy (nginx, Cloudflare), causing double compression
  • •Not setting Content-Length after compression since the compressed size differs from the original
  • •Missing Vary: Accept-Encoding header causing CDNs to cache the compressed response and serve it to all clients

When it happens

  • •When adding compression middleware to an Express or Fastify server for the first time
  • •When deploying behind a CDN or reverse proxy that also applies compression
  • •When serving pre-compressed static assets from disk with a file server like serve-static
  • •When implementing a file upload endpoint that receives gzip-compressed payloads from clients
  • •When parsing HTTP archives (HAR files) or proxy logs and encountering compressed response bodies

Examples and fixes

Apply gzip compression to an HTTP response using Node.js zlib streams, with correct Accept-Encoding negotiation.

Node.js gzip response stream with proper headers

❌ Wrong

const http = require('http');
const zlib = require('zlib');
http.createServer((req, res) => {
  const data = JSON.stringify({ message: 'Hello World'.repeat(1000) });
  // Wrong: always compresses regardless of Accept-Encoding
  // Wrong: Content-Length is set to uncompressed size
  res.setHeader('Content-Length', Buffer.byteLength(data));
  res.setHeader('Content-Type', 'application/json');
  const gz = zlib.createGzip();
  gz.pipe(res);
  gz.write(data);
  gz.end();
}).listen(3000);

✅ Fixed

const http = require('http');
const zlib = require('zlib');
http.createServer((req, res) => {
  const data = JSON.stringify({ message: 'Hello World'.repeat(1000) });
  const acceptEncoding = req.headers['accept-encoding'] || '';
  res.setHeader('Content-Type', 'application/json');
  res.setHeader('Vary', 'Accept-Encoding');
  if (acceptEncoding.includes('br')) {
    res.setHeader('Content-Encoding', 'br');
    zlib.createBrotliCompress().pipe(res).end(data);
  } else if (acceptEncoding.includes('gzip')) {
    res.setHeader('Content-Encoding', 'gzip');
    const gz = zlib.createGzip();
    gz.pipe(res);
    gz.end(data);
  } else {
    res.end(data); // no compression for unsupported clients
  }
}).listen(3000);

The broken version always applies gzip compression without checking whether the client supports it. This breaks clients that do not send Accept-Encoding: gzip, which includes some HTTP/1.0 clients, certain API testing tools, and programmatic HTTP clients that do not auto-decompress. It also sets Content-Length to the uncompressed size, which causes clients to hang waiting for more bytes than will actually arrive since the compressed body is shorter. The fixed version negotiates encoding from the Accept-Encoding header, preferring Brotli when available, sets the Vary header so CDNs cache separate versions for compressed and uncompressed clients, and omits Content-Length since chunked transfer encoding handles framing for streamed responses.

Handle clients that send gzip-compressed request bodies, such as IoT devices or batch upload clients.

Decompressing a gzip-encoded request body in Express

❌ Wrong

const express = require('express');
const app = express();
// express.json() does not handle compressed request bodies
app.use(express.json());
app.post('/api/data', (req, res) => {
  // If client sends Content-Encoding: gzip, body will be empty
  // or JSON.parse will throw on the compressed binary data
  console.log(req.body);
  res.json({ received: true });
});

✅ Fixed

const express = require('express');
const zlib = require('zlib');
const app = express();
// Decompress gzip request bodies before JSON parsing
app.use((req, res, next) => {
  if (req.headers['content-encoding'] === 'gzip') {
    const gunzip = zlib.createGunzip();
    req.pipe(gunzip);
    const chunks = [];
    gunzip.on('data', chunk => chunks.push(chunk));
    gunzip.on('end', () => {
      req.body = JSON.parse(Buffer.concat(chunks).toString());
      next();
    });
    gunzip.on('error', err => res.status(400).json({ error: 'Invalid gzip' }));
  } else {
    next();
  }
});
app.use(express.json());
app.post('/api/data', (req, res) => res.json({ received: true }));

Express's built-in body parsers (express.json(), express.urlencoded()) do not handle compressed request bodies by default. When a client sends a POST request with Content-Encoding: gzip, the raw compressed bytes arrive at the JSON parser, which cannot parse binary gzip data as UTF-8 JSON and either throws or silently produces an empty body. The fix intercepts requests with Content-Encoding: gzip before the JSON middleware, decompresses them using zlib.createGunzip() as a transform stream, and reconstructs the body string. The error handler on the gunzip stream returns a 400 response for malformed compressed data, protecting against decompression errors from corrupted payloads.

Content-Encoding vs Transfer-Encoding: key differences

HTTP defines two separate headers for encoding: Content-Encoding and Transfer-Encoding, and confusing them is a common source of bugs. Content-Encoding describes a transformation applied to the message body before transmission and is end-to-end — it travels with the message all the way to the final recipient. When a server sends Content-Encoding: gzip, the client must decompress the body to get the original content. Proxies and CDNs must preserve and forward this header unchanged.

Transfer-Encoding describes a transformation applied for a single network hop and is removed by each node that processes it. Transfer-Encoding: chunked, for example, is used to stream a response of unknown length. It is stripped by the first HTTP proxy or load balancer and never reaches the browser. Transfer-Encoding: gzip (as a hop-by-hop encoding) is almost never used in practice — nearly all HTTP compression uses Content-Encoding.

The practical difference matters when you have a reverse proxy in front of your application. If your application sets Transfer-Encoding: gzip instead of Content-Encoding: gzip, nginx will strip the Transfer-Encoding header and forward the raw compressed bytes to the browser without the Content-Encoding header that tells the browser how to decompress them. The browser receives garbled binary data instead of readable JSON or HTML.

Brotli (the 'br' encoding value) is a modern compression algorithm developed by Google, standardized in RFC 7932. Brotli achieves 15-20% better compression ratios than gzip on typical web assets (HTML, CSS, JavaScript) while maintaining similar or better decompression speed. It is supported in all modern browsers. However, Brotli is computationally more expensive to compress at maximum quality, making it more suitable for pre-compressed static assets than for on-the-fly compression of dynamic API responses where you would use a lower quality level (quality 4-6 instead of the maximum 11).

Diagnosing HTTP compression problems

The fastest way to diagnose compression issues is with curl: curl -v -H 'Accept-Encoding: gzip, br' https://yourapi.com/endpoint --output /tmp/response. The -v flag shows the response headers, and --output saves the body to a file. Check the response headers for Content-Encoding. If it shows 'gzip' or 'br', the body is compressed. If you see garbled characters in the response body when piping to a terminal, that confirms binary compressed data is being received without decompression.

For double-compression issues, compare the response body size with and without the Accept-Encoding header. If the response is larger when Accept-Encoding is included, the body may be getting compressed twice. Verify by decompressing once: if the result is still binary rather than readable text, decompress again. Double-compressed responses are typically caused by a reverse proxy (nginx, Cloudflare, AWS ALB) compressing a response that your application already compressed.

For 'incorrect header check' errors in zlib.gunzip(), the compressed data may have been corrupted in transit, or the encoding may be deflate rather than gzip. Gzip and deflate both use the deflate algorithm internally, but gzip wraps it in an additional header and CRC checksum. The two are not interchangeable. Node.js zlib.createGunzip() handles gzip; zlib.createInflate() or zlib.createInflateRaw() handles deflate. The 'deflate' Content-Encoding value in HTTP actually uses the zlib format (RFC 1950), not raw deflate — a historical confusion in the HTTP specification.

For the Vary header issue, use your CDN's cache inspection tool or curl with --header 'Accept-Encoding:' (empty) vs '--header 'Accept-Encoding: gzip' and compare the responses. If both return the same compressed content, the CDN is not respecting Vary: Accept-Encoding and is serving compressed responses to clients that did not request compression.

Implementing HTTP compression correctly in Node.js

For Express applications, use the compression npm package (maintained by the Express team). It automatically handles Accept-Encoding negotiation, sets Content-Encoding, and adds Vary: Accept-Encoding. Install with npm install compression and add app.use(compression()) before your routes. The default threshold is 1024 bytes — responses smaller than 1 KB are not compressed. For Brotli support in Express, use the shrink-ray-current or express-static-gzip packages.

For Node.js core HTTP servers, use zlib.createGzip() or zlib.createBrotliCompress() as Transform streams. The key pattern is: check req.headers['accept-encoding'] for the supported encoding, create the appropriate compressor, pipe the compressor to res (the response stream), and write your data to the compressor. Do not set Content-Length when using streaming compression because the compressed size is not known in advance — let the HTTP chunked transfer encoding handle framing.

For pre-compressing static assets, generate both gzip (.gz) and Brotli (.br) versions during your build process. With webpack, use CompressionWebpackPlugin with both gzip and brotli configurations. With Vite, the vite-plugin-compression plugin supports both formats. Serve them with nginx using the gzip_static and brotli_static directives, which serve .gz and .br files directly when the client supports them, avoiding runtime compression CPU overhead entirely.

For Fastify, use the @fastify/compress plugin which supports gzip, deflate, and Brotli out of the box with content-type filtering (you typically do not want to compress already-compressed formats like JPEG). The plugin automatically adds Vary: Accept-Encoding and handles content negotiation according to RFC 7231 quality values in the Accept-Encoding header.

Gzip bombs, already-compressed formats, and streaming

A gzip bomb (also called a zip bomb or decompression bomb) is a small compressed file that expands to an enormous size when decompressed, causing memory exhaustion or disk fill-up. A classic example is a 1 MB gzip file that expands to 1 GB or more of repeated null bytes. Protect against gzip bombs in Node.js by limiting the decompressed size: create a counter that increments in the 'data' event handler and close the stream if it exceeds your limit (typically 10-50 MB for API requests). The gunzip stream's error event fires if the compressed data is malformed but not if it is simply enormous.

Never attempt to compress already-compressed data formats. JPEG, PNG, WebP, AVIF, MP3, MP4, ZIP, PDF, and WebAssembly files are already compressed internally. Applying gzip to them typically increases the file size by 1-5% due to gzip's own headers and the entropy of compressed data being near-random. The compression middleware in Express uses a content-type filter to avoid this: it only compresses text/html, application/json, text/css, application/javascript, and similar text-based types.

Streaming large responses requires special handling. If you are streaming a database query result as JSON (using a streaming JSON serializer like @fastify/response-serialization or jsonstream), pipe it through a gzip compressor before the response. The zlib streams are Node.js Transform streams and work naturally in the pipe chain: dbStream.pipe(jsonStringifier).pipe(zlib.createGzip()).pipe(res). Set Transfer-Encoding: chunked (which is the default for streamed responses in HTTP/1.1) and Content-Encoding: gzip, but do not set Content-Length.

HTTP/2 changes the picture slightly. HTTP/2 uses binary framing and HPACK header compression for headers, which is separate from Content-Encoding compression of the body. HTTP/2 still supports the same Content-Encoding values (gzip, br, deflate) for body compression. However, HTTP/2 cannot use Transfer-Encoding: chunked — it uses its own DATA frame mechanism for streaming. Most HTTP/2 implementations handle this transparently, but if you are writing low-level HTTP/2 code, be aware that Transfer-Encoding is not a valid HTTP/2 header.

Common HTTP compression mistakes in production

Compressing small responses is a common premature optimization that wastes CPU. Compressing a 200-byte JSON error response takes more CPU time than the savings from network transfer, especially on localhost or local network connections where bandwidth is not the bottleneck. The compression middleware default threshold of 1024 bytes is a reasonable starting point. Profile your response size distribution before adjusting the threshold.

Not invalidating the CDN cache after changing compression settings is a production incident waiting to happen. If you enable gzip on a server and the CDN does not have Vary: Accept-Encoding in the cached response, the CDN may serve the uncompressed cached version to new clients, ignoring the new compression. Similarly, if you disable compression, the CDN may serve stale compressed responses. Always add Vary: Accept-Encoding before enabling compression, and purge the CDN cache when changing compression settings.

Using the wrong Accept-Encoding quality values in client code. The Accept-Encoding header supports quality values (q=0-1) to indicate preference order: Accept-Encoding: br;q=1.0, gzip;q=0.8, identity;q=0.6. If you set a quality value of 0 for identity (uncompressed), some servers interpret this as refusing to serve uncompressed responses and may return a 406 Not Acceptable error. Avoid setting identity;q=0 unless you are certain all servers support the required encoding.

Assuming all clients support decompression is incorrect for programmatic HTTP clients. curl decompresses gzip when you pass the --compressed flag but not by default. Python's requests library decompresses gzip automatically. Go's net/http client decompresses gzip automatically but removes the Content-Encoding header afterward, which surprises developers expecting to check the header. Always test your compression implementation with both a browser and at least one programmatic HTTP client.

HTTP compression best practices for production

Use Brotli for pre-compressed static assets and gzip for dynamic API responses. Brotli at quality level 11 (maximum) achieves the best compression ratios but is 100x slower to compress than gzip. For build-time pre-compression of JavaScript bundles and CSS, the extra compression time is acceptable. For on-the-fly compression of API responses, use gzip (zlib.Z_DEFAULT_COMPRESSION) or Brotli at quality 4-6, which compresses in milliseconds.

Always set Vary: Accept-Encoding when serving compressed content. This tells CDNs and intermediate caches to store separate versions of the response for different Accept-Encoding values. Without this header, a CDN that caches a gzip-compressed response may serve it to a client that did not include Accept-Encoding: gzip, resulting in undecodable binary data.

Measure compression ratios for your actual content. Typical compression ratios: JSON API responses achieve 3-5x compression (gzip) and 3.5-6x (Brotli). HTML with inline CSS achieves 5-10x. Minified JavaScript achieves 2-3x. Database exports in CSV format can achieve 10-20x. Pre-compressed binary formats (JPEG, ZIP, WebAssembly) achieve less than 1x (actually increase in size). Use these ratios to calculate the bandwidth savings vs CPU cost tradeoff for your workload.

For Content-Security-Policy and compression, be aware of the BREACH attack (Browser Reconnaissance and Exfiltration via Adaptive Compression of Hypertext). BREACH exploits HTTP compression to infer parts of a secret (like a CSRF token) in a response by observing compressed response sizes. Mitigations include: length hiding (padding responses to fixed sizes), CSRF token rotation, or disabling compression for responses that include user-controlled content alongside secrets. The Express csrf protection pattern of using synchronizer tokens is generally sufficient.

Quick fix checklist

  • ✓Use Content-Encoding for response body compression, not Transfer-Encoding
  • ✓Check Accept-Encoding request header before compressing — never compress unconditionally
  • ✓Add Vary: Accept-Encoding to all compressed responses to prevent CDN cache pollution
  • ✓Do not compress JPEG, PNG, ZIP, or other already-compressed formats
  • ✓Use zlib.createGzip() for gzip and zlib.createBrotliCompress() for Brotli in Node.js streams
  • ✓Set a decompressed size limit when gunzipping request bodies to prevent gzip bomb attacks
  • ✓Pre-compress static assets at build time with Brotli quality 11 for maximum ratio
  • ✓Purge CDN cache after changing compression settings to avoid stale compressed responses

Related guides

Frequently asked questions

What is the difference between Content-Encoding and Transfer-Encoding?

Content-Encoding is an end-to-end header declaring how the message body was compressed before transmission. It travels with the message to the final recipient and the client must decompress the body. Transfer-Encoding describes a hop-by-hop transformation (like chunked streaming) that is removed by each proxy. For HTTP response body compression, always use Content-Encoding: gzip or Content-Encoding: br, never Transfer-Encoding for compression.

How much better is Brotli compared to gzip?

Brotli achieves approximately 15-20% better compression ratios than gzip on typical web content (HTML, CSS, JavaScript, JSON). The improvement varies: highly repetitive text like HTML templates can see 25% improvement; already-minified JavaScript typically sees 10-15%. Brotli decompresses at similar speeds to gzip but compresses significantly slower at maximum quality, making it best suited for pre-compressed static assets rather than on-the-fly dynamic response compression.

Why does my gzip response show as binary in browser devtools?

The browser is receiving a compressed response body but either the Content-Encoding header is missing or set incorrectly, so the browser does not know it needs to decompress. Check the Network tab in devtools: click the response, go to Headers, and verify Content-Encoding: gzip is present. If Content-Encoding is missing, your server is sending compressed bytes without telling the client how to decode them. Add the Content-Encoding header alongside the compressed body.

How do I prevent a gzip bomb attack in Node.js?

Limit the decompressed size in your gunzip stream handler. In the 'data' event callback, accumulate the total bytes received and abort the stream if they exceed your threshold (typically 10-50 MB for API requests): gunzip.on('data', chunk => { total += chunk.length; if (total > MAX_SIZE) { gunzip.destroy(new Error('Payload too large')); } }). Return HTTP 413 Payload Too Large when the limit is exceeded.

What does the Vary: Accept-Encoding header do?

Vary: Accept-Encoding tells CDNs and HTTP caches to store separate cache entries for each distinct Accept-Encoding value. Without it, a CDN that caches a gzip-compressed response might serve it to a client that sent no Accept-Encoding header. That client receives binary compressed data it cannot decode. Adding Vary: Accept-Encoding ensures the cache returns the right version (compressed or not) based on what each client supports.

Why is 'deflate' Content-Encoding unreliable in HTTP?

The HTTP specification defines Content-Encoding: deflate as using the zlib format (RFC 1950), which wraps raw deflate with a 2-byte header and Adler-32 checksum. However, some servers historically sent raw deflate (RFC 1951) without the zlib wrapper. This ambiguity caused many clients to fail on deflate responses. Gzip has no such ambiguity and is universally supported correctly. Avoid Content-Encoding: deflate in new implementations; use gzip or Brotli instead.

Does compressing already-compressed images help?

No — applying gzip or Brotli to JPEG, PNG, WebP, WebAssembly, ZIP, or other pre-compressed formats typically increases file size by 1-5%. These formats already compress their data internally, and gzip's overhead (headers, checksum) plus the near-random entropy of compressed bytes makes further compression counterproductive. Configure your compression middleware to skip these content types using a content-type filter.

How do I serve pre-compressed files with nginx?

Enable the gzip_static and brotli_static directives in your nginx configuration. With gzip_static on, nginx checks for a .gz file alongside each static asset and serves it directly when the client sends Accept-Encoding: gzip. Similarly, brotli_static on serves .br files. Generate these files during your build with gzip --keep or brotli --keep. This eliminates runtime compression CPU overhead entirely for static assets.

All tools run in your browser. Your data never leaves your device. Last updated: 2026-05-06.