As part of Microsoft’s ongoing commitment to providing a confidential cloud — where hardware and software can give data owners control over how their data is shared and used — there’s a new announcement relating to Nvidia.
Indeed, Microsoft has shared in a blog post with plans and proposals for how it wants to power the next generation of AI using Nvidia’s GPUs. The announcement is very technical in detail, but it comes down to expanding confidential computing services to include support for Nvidia GPUs.
Right now, these services are all limited to CPUs, which can be challenging since they’re demanding in performance, especially for AI workloads. Trusted execution environments (TEE technology) is what is typically used in this case, and the vision is to extend this trust boundary to GPUs, allowing code running in the CPU TEE to securely offload computation and data to GPUs.
Microsoft aims to use a new feature called Ampere Protected Memory (APM) in the NVIDIA A100 Tensor Core GPUs to accomplish this. It will extend the GPU with these capabilities, in order to protect GPUs from attacks.
- A new mode where all sensitive state on the GPU, including GPU memory, is isolated from the host
- A hardware root-of-trust on the GPU chip that can generate verifiable attestations capturing all security sensitive state of the GPU, including all firmware and microcode
- Extensions to the GPU driver to verify GPU attestations, set up a secure communication channel with the GPU, and transparently encrypt all communications between the CPU and GPU
- Hardware support to transparently encrypt all GPU-GPU communications over NVLink
- Support in the guest operating system and hypervisor to securely attach GPUs to a CPU TEE, even if the contents of the CPU TEE are encrypted
This technology is now in private preview with Azure Confidential GPU VMs. Those VMs are designed in collaboration with NVIDIA, Azure, and Microsoft Research, feature up to four A100 GPUs with 80 GB of HBM and APM technology. Heading into the future, Microsoft also hopes to work with Nvidia on its Hopper technology, which it says can “protect both the confidentiality and integrity of data and AI models in use.”
