Vulnerability Research

NGINX CVE-2026-90439: one handshake, two extension counts

NGINX's HTTP/3 compatibility path could allocate a TLS extension array under one configuration and traverse it under another after SNI selection, causing a limited heap overflow; affected QUIC deployments should update NGINX and, while waiting, remove QUIC listeners while retaining TCP HTTPS.

A hand-drawn desk on warm paper, with two rotating configuration cards above a divided card drawer, introducing a change of state during a handshake.
In this article

It is quite reasonable to enable HTTP/3 for some sites on an NGINX instance while leaving others on ordinary HTTPS. The difficulty is that these sites give different TLS configurations to the same library, and a single handshake can use more than one of them. CVE-2026-90439 occurs between those configurations. OpenSSL has already allocated an extension array using the first configuration when NGINX selects another one for the hostname supplied by the client. OpenSSL then uses the second configuration to decide how many entries to visit. It can ask for one more entry than the array contains.

NGINX released fixes in 1.30.5 and 1.31.6 on September 15, 2026. F5 describes a limited, non-deterministic heap overflow that can restart a worker or cause limited data corruption. The useful response is to identify the actual build and TLS configuration, then update the appropriate NGINX branch. If that cannot happen immediately, remove QUIC listeners while keeping existing TCP HTTPS service. A heap-overflow label alone does not establish that the server has been taken over; a Medium score is equally insufficient reason to overlook an affected public-facing instance.

This explanation comes from public patches and pinned source. We reviewed the introducing NGINX release, affected and fixed snapshots on both release branches, and OpenSSL 3.5.0 handshake processing. We did not reproduce the vulnerability in a complete NGINX instance. The source explains the array mismatch and what the patch changes; it does not give us a production trigger rate or exploit success rate.

1 Which configuration does a connection use?

When an HTTPS handshake begins, the server has not received an HTTP Host header. The client normally provides the intended server name earlier, through SNI in its TLS ClientHello. NGINX starts with the listening address's default server configuration, then uses SNI to select the appropriate virtual server's certificate, protocols and other TLS settings. This is normal behavior: it is how multiple names can share an address.

In OpenSSL, an SSL_CTX holds TLS configuration that connections can use. The relevant part here is its registered custom extensions. HTTP/3 runs over QUIC, whose transport parameters are exchanged in a TLS extension. When NGINX uses its compatibility layer for older OpenSSL versions, it registers handlers for this extension. Its type is 0x39, and the registration appears in NGINX 1.31.5's ngx_quic_compat_init(). This is part of the handshake, not an HTTP header or URL parameter.

Before the fix, NGINX registered it only for virtual servers reachable over QUIC. The initialization branch in ngx_http_ssl_init() is conditional on addr[a].opt.quic; a compatibility initializer then visits the servers on that address. Two TLS contexts in the same process can therefore have different custom-extension counts.

These details also limit the affected population. F5's CNA record identifies ngx_http_v3_module, OpenSSL 3.5.0 or earlier, and particular configurations. HTTP/3 needs build support, and QUIC listeners must be configured. The http3 on default in the module documentation does not mean that every NGINX installation automatically exposes HTTP/3. Check the running binary, its actual TLS library, and the effective configuration after all include files have been read.

OpenSSL 3.5.1 is an important boundary here. The QUIC header in NGINX 1.31.5 selects the native OpenSSL QUIC API from that version onward. Older OpenSSL uses the compatibility path discussed here; BoringSSL, LibreSSL and QuicTLS have separate branches. The official build guidance accordingly recommends OpenSSL 3.5.1 or later. That number identifies an API boundary, not a recommendation that every deployment install exactly 3.5.1 today. Use a currently maintained TLS package from the deployment's supplier, and update NGINX itself.

2 The array stays put; the count changes

Follow one ClientHello through OpenSSL. Extension collection and extension parsing are separate stages. During collection, OpenSSL reserves a RAW_EXTENSION slot for each extension it recognizes, storing raw data and status there. It does not simply allocate one slot for each extension the client happened to send.

In OpenSSL 3.5.0, tls_collect_extensions() adds the built-in extension count to the current custom-extension count before allocating memory. Call the built-in count B. With no QUIC extension registered in the initial context, the allocation has B slots. With that extension registered, it needs B + 1. The letter is explanatory notation; the actual built-in count depends on the OpenSSL build.

/* OpenSSL 3.5.0: excerpt from extension collection */
num_exts = OSSL_NELEM(ext_defs) + (exts != NULL ? exts->meths_count : 0);
raw_extensions = OPENSSL_zalloc(num_exts * sizeof(*raw_extensions));

Only after storing this array in clienthello->pre_proc_exts does OpenSSL call the application's ClientHello callback. Both operations are visible in statem_srvr.c, lines 1680–1733. This is where NGINX reads SNI. Its ngx_ssl_client_hello_callback() validates the name's encoding, then calls ngx_http_ssl_servername() to select a virtual server.

After finding that server, NGINX calls SSL_set_SSL_CTX() at line 983. The call changes more than the certificate displayed to a client. OpenSSL duplicates the destination context's CERT configuration and replaces the connection's sc->cert. The custom-extension method table belongs to that configuration too. The relevant function body does not reallocate the previously created pre_proc_exts array.

The copy does not preserve the old count. ssl_cert_dup() copies the destination context's custom-extension table and count. The subsequent custom_exts_copy_flags() transfers status flags for extensions common to both contexts without restoring the old number of entries. The connection now holds objects created under different configurations: the array belongs to the original selection, while the method table belongs to the SNI-selected server.

OpenSSL continues the handshake and reaches the full-extension parsing call at line 1963. Instead of consuming the length captured during collection, tls_parse_all_extensions() reads the current s->cert->custext.meths_count and calculates its loop bound again.

/* OpenSSL 3.5.0: excerpt from extension parsing */
numexts += s->cert->custext.meths_count;
for (i = 0; i < numexts; i++) {
    if (!tls_parse_extension(s, i, context, exts, x, chainidx)) {
        return 0;
    }
}

If the custom-extension count changes from 0 → 1, the loop attempts to visit entry B + 1 in an array containing only B entries. A change in the other direction, 1 → 0, visits fewer entries. With zero in both contexts, or the same extension registered in both, this count difference does not produce an overrun. The failing direction matters; an SNI switch alone is not enough.

The actual out-of-bounds access occurs in tls_parse_extension(). It obtains &exts[idx], reads present and parsed, and conditionally writes parsed = 1. Visiting one extra slot is therefore different from reliably writing an arbitrary client-chosen byte. The status being read is already outside the array; whether processing reaches the write and subsequent parser depends on that memory. This source path is consistent with F5's description of non-deterministic behavior and limited corruption.

A TLS handshake allocates B extension slots under its initial configuration, then visits B plus 1 after SNI selection; after the fix, both contexts register the QUIC extension and allocation and traversal use B plus 1.

Scroll sideways to read the diagram.

Figure 1: the same extension array through a handshake. B denotes the built-in count; the diagram isolates the one-entry difference introduced by the QUIC custom extension. It illustrates the execution path, not a captured crash.

Normal error handling can terminate this path. Invalid name encoding, an internal error during virtual-server lookup, ssl_reject_handshake, or incompatible protocols or ciphers can end the handshake before the full-extension loop. A name that simply has no match can instead continue with the default server. These checks do not reconcile the allocated length with the new method count; a connection that passes the relevant checks still reaches that inconsistent assumption.

The source also argues against limiting the investigation to UDP. A TCP-only default server can participate in TLS virtual-server selection alongside a server that is also available over QUIC. The latter's context already has the extension registered, so SNI selection can change the count. HTTP/3 configuration creates the difference; TLS handshakes that use those contexts also need examination. We did not execute this configuration as a trigger and do not present it as a tested attack case. It nevertheless means that blocking UDP alone cannot establish that the in-process inconsistency has disappeared.

3 The patch keeps SNI and makes registration consistent

Why does the affected range begin with NGINX 1.29.2? The introducing change moved SNI-based server selection into the early ClientHello callback. It addressed a real ordering problem: select the server before deciding session resumption and protocol version, so client-certificate verification and per-server ssl_protocols settings are applied in the appropriate order. The 1.29.1 SSL module does not register that early callback; the same location in 1.29.2 does. Improving this order placed the existing registration difference between extension collection and parsing.

The main September 15 fix retains early server selection. It first checks whether any QUIC listener exists in the HTTP configuration. If one does, it registers the same transport-parameter extension in the relevant TLS contexts, including those of servers that cannot be reached over QUIC. The 1.31.6 initialization code moves registration outside the former QUIC-address-only branch. Both the initial and selected contexts now reserve a slot for this extension.

Registering a handler remains separate from enabling a protocol. The new compatibility initializer separates extension registration from the keylog callback used for QUIC, keeping the latter on QUIC addresses. The fix does not silently open a UDP listener for an ordinary HTTPS site or turn a TCP connection into QUIC.

A small compatibility change shipped alongside it. If an ordinary TCP TLS connection receives this extension, the parse callback now succeeds and ignores it, avoiding a handshake rejection caused by the newly registered handler. The add callback still returns 0 on non-datagram connections and does not send QUIC parameters. Return values serve different purposes in the two callbacks; reducing the patch to “0 became 1, so validation was removed” would miss the behavior being repaired.

The stable branch received the same repair. We compared the old branch in 1.30.4 with the new initialization in 1.30.5, and inspected the corresponding 1.31.5 and 1.31.6 implementations. We did not runtime-test every intermediate release. The continuous affected range comes from the NGINX advisory; the source snapshots establish the introduction, representative affected code and both fixed branches.

4 Update NGINX; keep TCP HTTPS while waiting

As of September 22, 2026, the NGINX release page lists 1.30.5 as the latest stable release and 1.31.6 as the latest mainline release. These are also the first fixes for this CVE. The affected range starts at 1.29.2 and runs through 1.31.5, with 1.30.5 explicitly listed as fixed. A flat numerical comparison that ignores branches can incorrectly classify the repaired stable release as vulnerable.

Product and branchVersions listed as affectedRepair target
NGINX Open Source1.29.2–1.31.5; stable 1.30.4 listed separately, with 1.30.5 fixedstable 1.30.5; mainline 1.31.6
NGINX Plus 37.0 / 37.137.0.0–37.0.6; 37.1.0–37.1.137.0.6.2 LTS; 37.1.1.2 CR
F5 NGINX Ingress Controller, 2026-lts2026-lts-r1–2026-lts-r72026-lts-r8
F5 NGINX Ingress Controller, 5.x5.0.0–5.6.25.6.3
F5 NGINX Ingress Controller, 4.x / 3.x4.0.0–4.0.1; 3.7.0–3.7.2No repair in these branches; move to a fixed branch
NGINX Gateway Fabric, 2.x2.2.2–2.7.12.7.2
NGINX Instance Manager, 2.x2.21.1–2.23.02.23.1

This table follows F5's September 18 product table. It includes Ingress Controller, Gateway Fabric and Instance Manager, which are absent from the September 16 CNA record; synchronizing the CVE JSON alone would miss these delivery forms. The Plus advisory summarizes affected versions with three components. Check the full package number when installing: the current release page explicitly lists 37.0.6.2 LTS and 37.1.1.2 CR. F5 evaluates only versions still within technical support, so an old release absent from the table cannot be classified as unaffected on that basis.

Ingress Controller here means F5's product, not the community ingress-nginx project. The Controller, Gateway and Instance Manager have their own versioning and upgrade paths. Do not replace an embedded NGINX executable in isolation: packages, container images, dynamic modules and management components need to remain compatible. Verify distribution backports against their package advisories. Similar version numbers cannot substitute for product identity.

First establish which installation you are examining on each host or container. These are local inspection commands, not vulnerability requests to a website:

nginx -V
nginx -T

-V reports the executed binary's version and build parameters; the QUIC troubleshooting guide also uses it to check the TLS library. -T validates and expands configuration, helping identify listen ... quic, default servers and SNI names. Its output can include internal addresses or sensitive configuration values, so keep it local. If the service uses a custom -c, -p, container or binary path, match the actual startup configuration. A different nginx found in your shell cannot establish what the running service uses.

If an immediate update is unavailable, use the workaround in F5's current advisory: remove every QUIC listener while retaining existing TCP HTTPS. The earlier CNA record also listed enabling QUIC for all servers; the current advisory no longer offers that option. It changes sites and network exposure that previously did not offer HTTP/3 and should not remain a default alternative in current guidance.

Apply the workaround to the full configuration. If a server already has both listen 443 quic reuseport; and listen 443 ssl;, remove the former and retain the latter. Merely deleting the word quic from the first line can leave an unintended TCP listener or conflict with an existing one. Check IPv4, IPv6, other ports and included server blocks; editing one visible server is insufficient if QUIC remains elsewhere.

# For a site that already has TCP HTTPS, remove its QUIC listener:
# listen 443 quic reuseport;

# Retain the existing TCP HTTPS listener and certificate configuration.
listen 443 ssl;

Stop advertising the unavailable HTTP/3 service through Alt-Svc, and verify client fallback to TCP. Updating advertisements reduces attempts to use the old endpoint but does not close a listener. Setting http3 off is also different from this workaround: the directive controls protocol negotiation, without removing QUIC addresses or their effect on context registration. Temporary removal costs HTTP/3 support and can change connection behavior for some clients. Continued HTTP/1.1 or HTTP/2 over TLS service depends on the existing configuration and business-path tests.

Run nginx -t before applying the configuration through the deployment's service manager, then inspect the result. NGINX keeps the old configuration if the new one cannot be applied. A command having run, or a process remaining alive, does not prove QUIC is disabled. If the update fails, retain the verified workaround. If an old image must be restored, keep that containment in place rather than restoring the old binary and QUIC configuration together to public service. For this CVE, the lowest repaired releases that can resume the original QUIC configuration are stable 1.30.5, mainline 1.31.6 and the corresponding Plus releases above. Other vulnerabilities and module compatibility may impose stricter rollback limits.

5 Verify the handshake and the running program

The most useful test population is the different virtual servers on a listening address. Opening a homepage once usually covers one name, one protocol and one fresh connection. An affected release can appear normal when its counts happen to agree. Keep the original server relationships and use ordinary clients to check the default server, each SNI name, and the established handling of missing or unknown names. Cover both TCP TLS and the QUIC service you intend to restore. Where certificate selection, protocol versions, session resumption or client-certificate verification differ by server, verify those policies too. The memory fix should preserve the reason early configuration selection was introduced.

Three concrete outcomes make a useful acceptance check:

  • The correct program is running. Every node and container loads the repaired binary, TLS library and compatible modules. Old workers and instances have exited when the rollout finishes. Reloading configuration and upgrading an executable are distinct operations; a reload alone does not prove that a process switched to new code.
  • The required service still works. Ordinary TCP HTTPS, required HTTP/3, and each name's certificate and authentication policy behave as intended. During containment, QUIC listeners are absent from the effective configuration and TCP fallback has been checked. Restore the original QUIC scope and advertisements only after the permanent update passes.
  • Source regressions exercise the changed relationship. Teams maintaining a custom build or patch branch should check consistent extension registration across context switches in an isolated environment they own, including the behavior that ignores QUIC extensions on ordinary TLS. Keep the TLS library, configuration and build options unchanged when comparing the old and fixed code so the comparison can identify what removed the mismatch.

These are acceptance recommendations for the patch, not claims that we executed them against a complete NGINX deployment. Source review identifies the relationship that must hold; the running version, configuration and actual tests establish whether a service satisfies it. Teams using vendor packages should prioritize the first two outcomes and the supplier's verification method. Sending potentially crashing input to production is unnecessary to demonstrate that a repair has been taken seriously.

When investigating existing anomalies, correlate worker exits, restarts, memory errors and handshake failures. The problem occurs before HTTP request processing, so an access log need not contain a corresponding URL. Existing error logs, service-manager records and protected crash records may be more useful. Worker exits also have other causes; one exit is insufficient to attribute exploitation to this CVE. Core dumps can contain connection data and key material and should be handled as sensitive files, not uploaded to a public issue.

At our September 22, 2026 UTC check, CISA's 2026.09.21 KEV snapshot did not contain CVE-2026-90439. That is a statement about that catalog at that time. NVD remained Awaiting Analysis; its CVSS 3.1 score of 6.5 and CVSS 4.0 score of 6.9 both came from F5, not an additional NIST assessment. F5 limits the reported effects to denial of service or limited data corruption in the data plane. The available evidence establishes neither reliable remote code execution nor a reason to rotate every certificate and key solely because this CVE exists.

The engineering lesson is specific: when moving a configuration switch earlier, inspect objects that have already been created under the previous configuration. An array, cache, session or callback table may outlive the assumptions that produced it. NGINX kept the useful SNI ordering and made extension registration consistent across that switch. For operators, the task is similarly concrete: identify the TLS path, run the appropriate repair, preserve the HTTPS service people need, and remove the temporary changes only when their replacement has been verified.

Research basis

Research basisStatic review of relevant NGINX 1.29.1, 1.29.2, 1.30.4, 1.30.5, 1.31.5 and 1.31.6 source and the introducing and fixing commits. Traced ClientHello processing, SSL_CTX replacement and extension arrays in OpenSSL 3.5.0. No complete NGINX vulnerability reproduction, deployment-prevalence measurement, crash-probability measurement or code-execution test was performed. No external service was probed. Official status checked through 2026-09-22 UTC.

SourceNGINX and F5 advisories, pinned release source and patches; OpenSSL 3.5.0 source; CVE CNA, NVD and CISA KEV

Evidence confidence High

6Evidence and sources

6.1Timeline

  1. Earlier SNI selection

    The early ClientHello callback enters NGINX; release 1.29.2 contains the behavior.

  2. Disclosure and fixes

    NGINX releases 1.30.5 and 1.31.6. F5 publishes CVE-2026-90439 and credits reporter Banny Liao.

  3. Products and mitigation updated

    The CNA record changes on September 16; F5's September 18 advisory includes additional products and recommends removing QUIC listeners as containment.

6.2Sources and material

  1. NGINX official security advisories and fixed releaseshttps://nginx.org/en/security_advisories.html
  2. F5 advisory K000162604https://my.f5.com/manage/s/article/K000162604
  3. F5's original CNA recordhttps://cveawg.mitre.org/api/cve/CVE-2026-90439
  4. NGINX: register the extension consistently across SSL contextshttps://github.com/nginx/nginx/commit/7c7363266d54bc3836c1d13d1ed1e1d93ee9ed98
  5. NGINX: ignore the QUIC extension on ordinary TLShttps://github.com/nginx/nginx/commit/22ba5662440d2865153ac1ff7de5f2b171414346
  6. OpenSSL 3.5.0 extension collection and parsinghttps://github.com/openssl/openssl/blob/636dfadc70ce26f2473870570bfd9ec352806b1d/ssl/statem/extensions.c
  7. SSL_set_SSL_CTX in OpenSSL 3.5.0https://github.com/openssl/openssl/blob/636dfadc70ce26f2473870570bfd9ec352806b1d/ssl/ssl_lib.c#L5453-L5504
  8. NGINX QUIC build, configuration and troubleshooting guidancehttps://nginx.org/en/docs/quic.html
  9. NGINX configuration reloads and executable upgradeshttps://nginx.org/en/docs/control.html
  10. NVD record and change historyhttps://nvd.nist.gov/vuln/detail/CVE-2026-90439