Snvme-of: Secure Disaggregated Storage Achieves Confidential Computing with Integrity and Freshness Guarantees, Reducing Performance Compromises by 2%

Modern data centres increasingly rely on disaggregated storage using NVMe-over-Fabrics to boost performance and efficiency, while confidential computing establishes stronger security for sensitive data. However, existing methods struggle to secure this advanced storage without sacrificing speed or robust protection. Marcin Chrapek and Marcin Copik from ETH Zurich, alongside Meni Orenbach, Ahmad Atamli, and Fritz Alder from NVIDIA, address this challenge with sNVMe-oF, a new storage management system that extends the NVMe-oF protocol to guarantee confidentiality, integrity, and data freshness. This innovative system optimises performance by utilising NVMe metadata and a novel disaggregated Hazel Merkle Tree, while also leveraging the power of smart NIC accelerators to deliver line-rate speeds with minimal performance impact, achieving as little as 2% degradation in testing with AI training workloads.

The study addresses limitations of traditional confidential computing methods applied to modern disaggregated storage, which often compromise security, scalability, or performance. Researchers recognized that existing solutions struggle with managing both control and data when integrated with high-performance disaggregation, leading to scalability issues and performance degradation. To overcome these challenges, the team pioneered a scalable control path leveraging the disaggregated nature of the storage itself.

Recognizing that Merkel Trees are commonly used to ensure data freshness, scientists innovatively distributed Merkel Tree management and verification to the storage node, minimizing communication overhead and addressing scalability concerns associated with rapidly growing storage capacities. This approach avoids unrealistic synchronizations that would congest the network and reduce throughput. Furthermore, the study introduced a new disaggregated Hazel Merkle Tree to optimize data path performance by leveraging NVMe metadata and avoiding redundant security protections. The team prototyped sNVMe-oF on an NVIDIA BlueField-3 smart NIC, demonstrating that the system achieves as little as 2% performance degradation for both synthetic patterns and AI training workloads, showcasing its practical viability and effectiveness.

Secure NVMe-oF Maintains High Performance

The research team has developed sNVMe-oF, a new storage management system that enhances security for modern data centers without significantly compromising performance. This system extends the NVMe-oF protocol to provide confidentiality, integrity, and freshness guarantees for sensitive data. Crucially, sNVMe-oF achieves this without modifying the underlying NVMe-oF protocol itself, instead leveraging smart NICs to accelerate security processes. Experiments demonstrate that sNVMe-oF introduces minimal performance overhead, achieving only a 2% reduction for both synthetic patterns and artificial intelligence training workloads.

Initial tests reveal substantial optimization, with overhead decreasing to just 1-2% for sequential reads and 5% for random reads. Freshness adds approximately 1% performance overhead. The system requires 4-5 processor cores to match bare metal performance, with CPU usage averaging 6% per core. Detailed analysis of latency shows overheads remain within 30% of bare metal performance, reaching around 10% at higher percentiles, primarily due to encryption processes. The team also investigated the impact of metadata cache pollution, finding that throughput remains largely unaffected, while input/output operations per second and latency are considerably impacted by contention in the metadata cache. Finally, the team deployed an ext4 file system and evaluated sNVMe-oF’s impact on machine learning training pipelines using ResNet50 and UNet3D models, indicating minimal overhead for these workloads.

Secure Disaggregated Storage with Minimal Overhead

Scientists have developed sNVMe-oF, a secure disaggregated storage solution designed for modern data centers. This system addresses the challenge of protecting sensitive data while maintaining high performance in environments utilizing NVMe-over-Fabrics technology. The research introduces a novel control path that separates freshness checks, enabling scalability to extremely large storage systems, and modifies the data path to incorporate concepts like metadata caching and eventual consistency. The team demonstrates that sNVMe-oF achieves minimal performance overhead, as little as 2%, while simultaneously providing confidentiality, integrity, and data freshness guarantees.

This is accomplished through careful optimization, including leveraging smart NIC accelerators and avoiding redundant security protections. The researchers implemented sNVMe-oF using SPDK and tested it on BlueField-3 hardware, demonstrating its potential as a foundational component for scalable confidential computing in data centers. This research represents a significant step towards securing large-scale disaggregated storage, paving the way for more robust and trustworthy data management systems.