OpenSSL: how a handshake retry can expose other memory

1 Find the applications that actually use DTLS

OpenSSL supplies encrypted communication to many applications. DTLS provides TLS-like protection for datagram communication, usually over UDP. UDP does not guarantee that handshake messages arrive in order, so DTLS handles fragmentation, waiting and retransmission itself. This vulnerability affects that path; ordinary HTTPS over TCP/TLS is not automatically affected. Both clients and servers can send large handshake messages, so both sides need consideration. The FIPS cryptographic modules listed by OpenSSL are unaffected because the faulty code lies outside them; applications using those modules can still call the affected DTLS code.

The failure also requires a write to pause part-way through. OpenSSL reports SSL_ERROR_WANT_WRITE when the underlying transport cannot accept more data yet, asking the application to continue later. The connection remains alive, and its retransmission timer can expire first. Rebuilding a handshake message at that moment changes shared send state that the paused write still needs. The explanation below follows dtls1_retransmit_message() and dtls1_do_write() in version 3.5.8.

For a deployment already on a supported upstream branch, install its fix: 4.0.3, 3.6.5, 3.5.9 or 3.4.8. As checked on October 1, these are also each branch's latest stable release. Distributions may backport the repair under a lower upstream version number; Section 4 gives concrete packages. Where an immediate update is unavailable, disabling the DTLS entry point through a supported product setting or restricting it to necessary peers can reduce exposure. This interrupts dependent traffic, and allowed peers can still reach the old code. A permanent repair requires the fixed library or the product that contains it.

OpenSSL rates the issue High and assigns CWE-125, an out-of-bounds read. At 01:20 UTC on October 1, CISA's latest KEV catalog had no entry for this CVE and NVD was awaiting analysis; the displayed CVSS 3.1 score of 8.2 came from CISA-ADP. Services confirmed to use the affected DTLS code should still update.

2 Message A replaces the buffer; B leaves its position behind

During an ordinary send, init_buf holds the handshake message, init_off identifies the current position and init_num tracks the length still to send. For a large message, dtls1_do_write() in 3.5.8 chooses fragments that fit the MTU, the packet size the path can carry. A successful fragment advances the position and reduces the remaining length. If the next write cannot proceed, the function returns so the application can resume from the saved position.

OpenSSL also keeps handshake messages in a retransmission queue. Its name, sent_messages, can be misleading: a message is queued as construction finishes, before all its fragments have necessarily gone out. Queue membership does not mean that the peer acknowledged it. Sending and retransmission can therefore encounter the same unfinished message or different messages within one handshake flight.

The upstream server test supplies a useful concrete sequence. A short ServerHello goes out, followed by the first fragment of a larger Certificate message, and then writing pauses. The shared buffer still contains Certificate and the position lies part-way through it. When the timer expires, retransmission fetches an earlier queued message, copies it to the beginning of the buffer and resets the length to send. These old lines omit a corresponding reset of init_off:

memcpy(s->init_buf->data, frag->fragment,
    frag->msg_header.msg_len + header_length);
s->init_num = frag->msg_header.msg_len + header_length;

The state now describes two different operations. The buffer's beginning and the length belong to retransmitted message A, while the position comes from paused message B. The next call to dtls1_do_write() rewrites a fragment header at that position and passes &s->init_buf->data[s->init_off] to the record layer. That layer uses the supplied pointer and length to build its outgoing record. The message label belongs to A, but its body can come from bytes B left farther along, potentially reaching beyond the allocated buffer.

OpenSSL confirms possible plaintext disclosure of heap contents to the peer and denial of service if the read reaches unmapped memory. The available bytes and quantity depend on message and memory state. The advisory supplies no basis for concluding that a particular private key was disclosed.

3 Resetting the position still leaves a write to resume

The first change is small. Fix commit 906cf0ef adds s->init_off = 0; after copying the message and setting its length, making retransmission start at the proper beginning. The original Certificate message is still waiting to resume. What happens to its saved progress?

The old function contains code described as saving and restoring state. It saves the record-layer object and method, allowing transmission under the state associated with the queued message. It does not also save the paused write's init_buf, init_off, init_num or handshake header. When retransmission completes, the send function clears the position and remaining length. The next SSL_accept(), SSL_connect(), SSL_read() or SSL_write() still needs to finish the original handshake message. In the Certificate example, retransmission has already changed its contents, position and remaining length.

The handshake state machine still records that it stopped during sending. Its WRITE_STATE_SEND branch invokes the writer again, but the message length and header no longer agree. OpenSSL describes a resulting assertion and process abort in debugging builds. This consequence is separate from an out-of-bounds read: a retransmission can damage the original write even when its own read happens to stay within the buffer.

The second change sits in dtls1_handle_timeout(). If the handshake remains in its writing flow and has not returned to the transition state, the current timer event leaves retransmission alone:

if (s->statem.state == MSG_FLOW_WRITING
    && s->statem.write_state != WRITE_STATE_TRANSITION)
    return 0;

The timer has already been rearmed before this check. The application can resume its paused write on its next call. Retransmission waits until the active send releases the shared state, and starts at position zero when it is allowed to proceed.

The two regression tests added with the patch focus on this handoff. One enlarges the client's ClientHello; the other fragments the server's Certificate. A controlled input/output layer pauses each write with WANT_WRITE. The tests then remove that layer's write restriction, handle three timer expirations and check that its write-call counter has not increased. Removing the restriction before the timer checks matters: the new guard prevents retransmission while the test apparatus is already willing to send.

Finally, the tests resume the original handshake call and exclude SSL_ERROR_SSL and SSL_ERROR_SYSCALL. They check that the paused state survives and the next call can continue; they do not assert completion of the entire handshake or measure disclosed data.

4 A lower version number can still contain the fix

Upstream confirms affected releases in the 4.0, 3.6, 3.5, 3.4, 3.0, 1.1.1 and 1.0.2 series. The table gives the first fixed versions on currently publicly supported branches. On October 1, 2026, they are also those branches' latest stable releases. Updating within an existing supported branch will usually make compatibility easier to control than an emergency major-version migration.

Branches 3.4 and 3.6 are close to the end of upstream support. Patching the current deployment before scheduling migration separates this security repair from the later compatibility work. A new deployment requiring a long maintenance window can evaluate 3.5 LTS first. The advisory also lists 3.0.23, 1.1.1zj and 1.0.2zs, supplied through Premium support. Distribution-maintained older series follow their own support and backport arrangements, so the public upstream download table alone cannot settle their status.

Ubuntu's live package table makes the distinction concrete. Its fixed openssl package for 26.04 is 3.5.5-1ubuntu3.6; for 24.04 it is 3.0.13-0ubuntu3.16; and for 22.04 it is 3.0.2-0ubuntu1.30. Each upstream number is below the advisory's first-fixed release, while the distribution revision contains the backport. Check the complete package version, including its suffix, and apply the result only to that component's row. Other applications embedding OpenSSL on the same system can have a separate status.

Debian's tracker shows why the repository matters too. The trixie security package 3.5.7-1~deb13u3 is fixed, while 3.5.7-1~deb13u2 listed in the regular repository remains vulnerable. The bookworm security package 3.0.22-1~deb12u1 was also still marked vulnerable at this check. These states are anchored to October 1 at 01:20 UTC. For deployment, use the current status and complete package version from the relevant repository.

5 Check the library inside the running process

After a package update, an old process is an easy omission. openssl version identifies the version used by that command. The business application may still have an old shared library loaded or contain a separate, statically linked copy. Check the service's actual library or product build and restart or redeploy as the product requires. Container images, appliance firmware and applications with bundled dependencies need their own treatment; replacing one host package may leave their executing code unchanged.

Validate with the product's normal peers: complete a DTLS handshake and an application operation, then check recovery after packet loss, fragmentation and paused writes. Product maintainers can reuse the upstream regression scenarios described above to check that repeated timeouts leave the paused message's send progress intact. If the upgrade fails, keep the necessary access restrictions in place; a rollback needs a supported build with an equivalent fix before the entry point can reopen.

Determining whether earlier connections disclosed sensitive data needs further evidence from logs, crashes or traffic. An ordinary handshake failure cannot answer that question, and a successful patch cannot reconstruct bytes sent before it. If abnormal evidence exists, investigate the data available to the affected process to determine which secrets need revocation or replacement.