Researchers at genua GmbH and Fraunhofer AISEC completed 126500 virtual private network handshakes across 1296 different network configurations to assess the impact of large post-quantum keys on current security protocols. The work demonstrates that no proposed mechanism fully compensates for the performance hit caused by these keys within the IKEv2 framework. Specifically, tests with Classic McEliece show handshakes taking up to nearly two minutes under challenging conditions, congesting links or rendering the system unusable. For ML-KEM and FrodoKEM, the team identifies two potential paths forward: standardized extensions or delegating transport.

Classic McEliece Performance Under Stress

Testing across 1296 different network configurations revealed that current proposals fail to fully compensate for the performance impact of large keys used in post-quantum cryptography. Measurements across different bandwidths, latencies and packet loss rates, and with ML-KEM, FrodoKEM and Classic McEliece, total 126500 handshakes in 1296 configurations and show that no mechanism fully compensates for the effect of the key size on performance. For Classic McEliece, no approach handles challenging conditions acceptably.

The approaches either congest the link or take up to nearly two minutes per handshake on average. The research highlights a critical bottleneck for Classic McEliece. Existing approaches struggle to handle high latency or packet loss without significant delays. These delays stem from the sheer size of the keys, which exceed the capacity of the initial key exchange within the IKEv2 protocol and cannot be easily fragmented. The team identified two specification gaps in current IETF drafts related to acknowledgment mechanisms.

IKEv2 Adaptation for Post-Quantum KEMs

The structure of classical key agreements, such as those using Diffie-Hellman, relies on exchanging key shares with similar structural properties in both directions, a task largely handled by implementation details. However, post-quantum key encapsulation mechanisms introduce a significant difference: key sizes. Measurements reveal that the sheer scale of public keys used in post-quantum cryptography presents a fundamental challenge for the IKEv2 virtual private network handshake protocol, which was not designed to accommodate such large data volumes.

This disparity is not merely a matter of bandwidth. The implications extend beyond theoretical concerns, as the inability to fragment the initial exchange creates a bottleneck for larger post-quantum keys. This work, alongside colleagues, highlights that the protocol’s limitations are not easily overcome by simply adjusting parameters or employing standard optimization techniques.

The team systematically screened IETF documents for proposed adjustments, even adding three of their own, one using forward error correction and two using IKEv2 with QUIC, but found no single solution fully mitigates the performance impact. The research demonstrates that even with these proposed adjustments, the performance penalty remains substantial.

While the specific impact varies between key encapsulation mechanisms, the fundamental constraint of a non-fragmentable initial exchange limits the effectiveness of any workaround. The study’s findings suggest that either standardized extensions to IKEv2 or a delegation of the transport layer are necessary to effectively integrate these larger post-quantum keys without unacceptable performance degradation.

Impact of Large KEM Keys on VPN Handshakes

The testing, encompassing 126500 handshakes, systematically evaluated eight different mechanisms, including three novel approaches using forward error correction and variations of IKEv2 with QUIC. Despite these efforts, substantial performance penalties persist under realistic network conditions, demonstrating the difficulty of retrofitting existing protocols for significantly larger key sizes. These gaps, alongside the inability to fragment the initial key exchange, create bottlenecks that limit scalability and reliability.

The team’s implementation work, built on the open-source codebase, allowed for direct comparison of approaches, revealing that Classic McEliece consistently exhibited unacceptable performance under challenging conditions. For ML-KEM and FrodoKEM, the research suggests two potential avenues for improvement: building on already standardized extensions to IKEv2 or delegating the transport layer.

QUIKE-Composite for Faster, Resilient Key Exchange

Luby codes, a refinement of earlier layering techniques, offer a speed advantage over Reed-Solomon codes despite requiring more symbols, demonstrating a trade-off between efficiency and recovery potential in forward error-correcting code design. The research team’s findings indicate that the size of the key encapsulation mechanism, or KEM, is the primary determinant of handshake performance, necessitating adjustments to transmission approaches. Specifically, with Classic McEliece-sized KEMs, no tested mechanism successfully completed handshakes under challenging network conditions, revealing a critical bottleneck for this particular post-quantum cryptographic algorithm.

For ML-KEM and FrodoKEM, a composite KEM combined with a change in transport protocol proved more reliable and faster, while also simplifying implementation, with the QUIKE-composite proposal achieving the best overall performance. For ML-KEM and FrodoKEM, there are two ways forward: building on already standardized extensions or delegating the transport.

Systematic Review of IETF PQC KEM Proposals

We systematically screened the IETF document corpus for such proposals, added three of our own (one using forward error correction and two using IKEv2 with QUIC, namely QUIKE and QUIKE with a composite key exchange), identified two specification gaps in existing acknowledgment drafts, and implemented all eight on the same open-source codebase. For ML-KEM and FrodoKEM, there are two ways forward: building on already standardized extensions or delegating the transport.

The former requires the larger implementation change and needs further optimization, such as acknowledgment mechanisms or error-correcting codes, but grants continuity and compatibility. The latter can use a composite key exchange to avoid these drawbacks and achieve a faster, more resilient handshake and leaner implementation, with QUIKE-composite offering the best overall performance.

FEC Approach to Stabilize Fragmented KEM Keys

The work details a reproducible assessment of eight distinct methods, including three novel proposals, across 1296 unique network configurations, demonstrating that no solution fully mitigates the performance impact of sizable KEM keys. This comprehensive testing, encompassing varying latency, bandwidth and packet loss rates, highlights the difficulty of achieving stable, fast handshakes under realistic conditions.

The team’s approach involved building all eight mechanisms on a shared open-source codebase, allowing for isolated performance assessment without external variables influencing results. The researchers found that the approaches either overwhelmed network links or failed to establish connections within a reasonable timeframe.

ML-KEM and FrodoKEM: Pathways for Optimization

Measurements across 1296 distinct network configurations reveal that no current protocol fully compensates for the performance impact of integrating large post-quantum key encapsulation mechanisms (KEMs) into Internet Key Exchange version 2 (IKEv2) handshakes. For ML-KEM and FrodoKEM, two distinct implementation paths emerge, offering different trade-offs for developers. One path prioritizes compatibility by building upon the already standardized Auxiliary-Detached Key Exchange (ADDKE), allowing for incremental optimization while maintaining backwards compatibility.

This approach, however, results in a more complex design and does not provide the same benefits for smaller KEMs. The other path focuses on maximizing performance, but requires accepting an additional dependency on external code. The researchers emphasize that an implementation prioritizing continuity over maximum performance would likely choose the datagram-based path, using the established ADDKE standard. The established path offers a simpler mechanic and fewer protocol interactions, but at the cost of peak performance.

Specification Gaps in IKEv2 Acknowledgment Drafts

These gaps, identified through systematic screening of IETF documents and implementation on a shared open-source codebase, highlight potential flaws even in advanced proposals before widespread adoption. The team’s work extends beyond simply identifying these issues. They also propose solutions within the existing framework and through integration with QUIC. The research involved a comprehensive review of IETF documentation to pinpoint proposals for post-quantum cryptography key encapsulation mechanisms (KEMs) within IKEv2.

This choice, detailed in Table VIII of their work, represents a pragmatic approach to stabilizing fragmented key transmission. Future research will focus on the impact of these approaches on authentication methods, particularly the migration to post-quantum public keys and certificate chains. Plans are underway to discuss the identified gaps within the IETF working group. The work’s findings are intended to inform standardization efforts and provide a comprehensive view of IKEv2’s post-quantum migration.

Performance Evaluation Across Network Conditions

Classic McEliece, with its particularly large key size, proved especially problematic, with no tested approach achieving acceptable handshake times under challenging network conditions. This finding emphasises the need for careful consideration when selecting post-quantum algorithms for deployment in environments with limited bandwidth or high latency. For ML-KEM and FrodoKEM, two potential pathways emerged. The researchers found that using standardized extensions or delegating transport functions offered viable improvements.

👉 More information
🗞 Embedding Large PQC KEMs in IKEv2
✍️ Daniel Herzinger, Marco Thomas, Tobias Handirk, Stefan-Lukas Gazdag, Daniel Loebenberger and Marian Margraf
🧠 ArXiv: https://arxiv.org/abs/2610.32356

More like this