Complete URL percent encoding guide based on RFC 3986

Percent encoding is defined in RFC 3986 (Uniform Resource Identifier: Generic Syntax). The mechanism is simple: any byte can be represented as % followed by two uppercase hexadecimal digits representing the byte value. The character space (ASCII 32, hex 0x20) encodes as %20. The ampersand (ASCII 38, hex 0x26) encodes as %26. Non-ASCII characters are first encoded as UTF-8, then each byte is percent-encoded: the euro sign € (U+20AC) encodes to three UTF-8 bytes E2, 82, AC, which produces %E2%82%AC.

RFC 3986 divides characters into three categories. Unreserved characters (A-Z, a-z, 0-9, -, _, ., ~) may appear anywhere in a URI without encoding and should not be encoded. Reserved characters serve as delimiters: gen-delimiters (: / ? # [ ] @) separate major URI components, and sub-delimiters (! $ & ' ( ) * + , ; =) serve as delimiters within components. Reserved characters must be percent-encoded when they appear as data in a component where they would otherwise be interpreted as delimiters.

The context where a value appears determines which characters must be encoded. In a path segment, the / character must be encoded as %2F when it appears in data (to distinguish from the path separator). The ? and # characters must also be encoded in path data. In a query string value, both & and = must be encoded to prevent them from being interpreted as parameter delimiters. In a fragment, no characters are sent to the server so encoding is only necessary for parsing consistency.

The percent sign itself (%) must be encoded as %25 when it appears as data, not as the first character of a percent-encoded sequence. A URL validator must verify that every % is followed by exactly two hexadecimal characters. A bare % or a % followed by non-hex characters is a malformed URL that different parsers handle inconsistently — some reject it, some pass it through unchanged, and some attempt to encode it on the fly.

The fastest diagnostic is to use the browser DevTools Network tab to inspect the actual URL sent in a request. Click on the request, view the Headers tab, and look at the Request URL field. This shows the exact URL including encoding. Compare this against what your code constructed. If the URL was modified by the browser or framework, the difference is visible here.

In Node.js, log the URL at the point of construction before passing it to fetch, axios, or http.request. console.log(url) shows the string as JavaScript sees it. Inspect whether special characters are already percent-encoded or still raw. For incoming requests, log req.url (raw, with encoding) and req.path (decoded by Express) to see both forms.

To check whether a string is valid percent-encoded, test it with decodeURIComponent wrapped in try/catch. If it throws, the string has malformed percent sequences. To check whether a query string is correctly structured, parse it with new URLSearchParams(queryString) and inspect the resulting entries. Parameters that should be distinct but share a name indicate an unencoded & in a value.

For internationalized domain name issues, check whether the HTTP client is handling IDN automatically. Node.js's built-in http and https modules (and fetch) use the WHATWG URL parser which converts IDN to punycode: new URL('https://münchen.de').host returns 'xn--mnchen-3ya.de'. Some older HTTP client libraries do not perform this conversion and require the caller to provide the punycode hostname manually using the 'punycode' npm package (deprecated) or the 'tr46' package.

For query string construction, use URLSearchParams as the primary API. It correctly encodes all values and handles & separators. For API clients that need specific query string formats (such as PHP-style array notation), use the qs package: qs.stringify({ filters: ['a', 'b'] }) produces filters%5B0%5D=a&filters%5B1%5D=b with properly encoded brackets.

For path segments with dynamic data, encode each segment individually: '/files/' + encodeURIComponent(filename). If the path includes multiple levels constructed from user data, map over the segments: [baseSegment, ...userSegments.map(encodeURIComponent)].join('/'). Never apply encodeURIComponent to an entire path that includes / delimiters.

For complete URL construction with both path and query components, the URL constructor is the most robust approach. const u = new URL(path, baseUrl); u.searchParams.set('q', query); return u.toString(). This handles encoding, normalizes the path, and produces a correctly formatted URL regardless of what the individual values contain.

For Content-Disposition headers in file download responses, the filename must be percent-encoded using a specific encoding format. The modern approach (RFC 5987 extended notation) is: Content-Disposition: attachment; filename*=UTF-8''${encodeURIComponent(filename)}. The filename* parameter with the UTF-8'' prefix supports non-ASCII characters correctly in all modern browsers. The legacy filename parameter should also be included for compatibility with older clients, with non-ASCII characters transliterated or stripped.

Internationalized domain names (IDN) use Punycode encoding at the DNS level. The domain münchen.de has the code point U+00FC (ü) which is not valid in a DNS label. The IDN encoding converts this to the ACE (ASCII Compatible Encoding) form xn--mnchen-3ya.de using the Punycode algorithm defined in RFC 3492. Modern browsers and the WHATWG URL parser perform this conversion automatically. However, older Node.js code that manually constructs hostnames from user input may need to apply Punycode conversion explicitly. The built-in url module (not WHATWG URL) does not perform IDN conversion; use the WHATWG URL class (new URL()) instead.

URL fragments (the # portion) are never transmitted to the server in HTTP requests. They are processed entirely by the browser. Despite this, percent-encoding rules still apply to fragment content for consistency with URL parsing. Fragment values can contain any Unicode character percent-encoded as UTF-8. If a server-side rendered page needs to set a fragment value that contains special characters, it should percent-encode the value in the HTML href.

The plus sign + has ambiguous status in URL encoding. In application/x-www-form-urlencoded (HTML form submission), + represents a space. In RFC 3986 URI syntax, + is a sub-delimiter that is allowed in path and query components without encoding. When a URL is parsed by a server, + in the path remains as a literal plus sign. + in a query string may be decoded as a space (by frameworks that implement form decoding on query strings) or kept as + (by frameworks that implement strict RFC 3986 decoding). This ambiguity is why %20 is safer than + for spaces in URLs that may be processed by different parsers.

Data URIs use percent encoding for the data portion when the media type is not base64. A data URI for an SVG image with non-ASCII characters must percent-encode those characters. Data URIs that use base64 encoding (data:image/png;base64,...) do not percent-encode the base64 data, but the base64 alphabet (A-Z a-z 0-9 + /) is safely allowed in URI context without encoding.

Encoding the entire URL string with encodeURIComponent is the most disruptive encoding mistake. This encodes the : and // in the protocol, producing https%3A%2F%2F which is not a valid URL. The fix is always to encode individual components (path segments, query values) and combine them with unencoded structural characters.

Not encoding the # character in redirect URLs is a common security-adjacent mistake. If an OAuth redirect_uri contains a # (such as a single-page application hash route), the server-side OAuth handler must encode the # as %23 in the redirect_uri parameter. If it does not, the authorization server may interpret the # as the beginning of the fragment and truncate the redirect_uri parameter value.

Using the wrong encoding for Content-Disposition filename fields breaks file downloads in non-ASCII languages. The legacy filename="file.pdf" field does not support non-ASCII characters. Using UTF-8 bytes directly in this field produces garbled filenames on Windows and is technically invalid. The correct approach is to use the RFC 5987 format: filename*=UTF-8''percent-encoded-name for non-ASCII filenames, with a fallback ASCII filename for older clients.

Ignoring the encoding of matrix parameters (semicolon-separated path parameters like /path;param=value) is less common but causes problems with some Java and .NET frameworks. Matrix parameters use the same percent-encoding rules as query strings. If a matrix parameter value contains a semicolon or equals sign, these must be encoded as %3B and %3D respectively.

Build a URL abstraction layer in your application that accepts structured inputs and produces correctly encoded URLs, rather than allowing ad-hoc string concatenation throughout the codebase. The URL and URLSearchParams classes are the foundation for this layer in browser and Node.js environments. For server-side URL construction in Express, use the path module for path manipulation and url.format with an options object for full URL construction.

Document the encoding contract for every URL field in your API. In OpenAPI specification, use the parameter encoding object to specify the encoding style (form, spaceDelimited, pipeDelimited) for each parameter. This generates correct encoding in SDK clients and clarifies expectations in documentation.

For URL parameters that contain sensitive data (such as authentication tokens in OAuth flows), prefer encoding the entire value with encodeURIComponent even if it does not strictly require encoding. A value like a JWT access token may contain base64url characters that do not need encoding, but explicitly encoding it prevents future bugs if the token format changes.

Monitor your web server logs for requests containing %2F in path segments. A URL like /files/%2F../../../etc/passwd uses an encoded slash to attempt a path traversal attack. Your URL decoder must handle this by rejecting paths that resolve outside the intended directory even after decoding. Normalize paths with path.resolve and verify the result starts with the expected base directory before serving files.