NVIDIA DGX Cloud

NVIDIA DGX Cloud: A Unified Cloud Platform for AI Development and Deployment

Overview

NVIDIA DGX Cloud is a comprehensive cloud platform developed by NVIDIA, designed to enable organizations to access advanced computational power for AI training and inference without the need for complex on-premises infrastructure.

Technical Specifications

• Graphics Processing Units (GPUs):
  Each DGX Cloud instance includes 8 NVIDIA H100 or A100 80GB Tensor Core GPUs, providing a total of 640 GB of GPU memory.
• Networking and Connectivity:
  Utilizes NVIDIA NVLink and NVSwitch technologies to deliver high-bandwidth inter-GPU communication, combined with RDMA networking to minimize latency and maximize data transfer speeds.
• Storage:
  Equipped with high-performance NVMe storage optimized for demanding AI workloads.

Infrastructure Components of DGX Cloud

• Management Software:
  NVIDIA’s Base Command Platform is employed to orchestrate and monitor AI training workloads efficiently.
• High-Performance Networking:
  DGX Cloud leverages low-latency, high-bandwidth networking to enable seamless scalability for distributed AI tasks.

Key Features and Services

• DGX Cloud Create:
  A Kubernetes-based AI workload orchestration platform that facilitates training and fine-tuning of models. This fully managed service includes both a user-friendly graphical interface and command-line tools for interacting with GPU clusters.
• Serverless Inference:
  Enables scalable and efficient deployment of AI models without the need to manage infrastructure, leveraging automatic GPU resource optimization.
• Benchmarking Capabilities:
  Offers ready-to-use templates and guidelines to benchmark AI models such as Llama 3.1, Grok1, and NeMo Megatron at scale, supporting configurations up to 2,048 GPUs.

Collaborations with Cloud Service Providers

NVIDIA DGX Cloud is integrated with leading cloud service providers including Oracle Cloud Infrastructure (OCI), Microsoft Azure, Google Cloud, and Amazon Web Services (AWS), offering global-scale access to high-performance computing resources.

Use Cases and Clients

• Amgen:
  Utilizes DGX Cloud in conjunction with the BioNeMo software suite to accelerate biologics drug discovery.
• Cerence:
  Trains automotive-specific large language models to enhance in-vehicle user experiences.
• ServiceNow:
  Develops intelligent virtual assistants and customer service agents using DGX Cloud infrastructure.
• Deloitte:
  Employs DGX Cloud and NVIDIA BioNeMo to advance large language models in pharmaceutical research and drug development.

Key Advantages

• Access to NVIDIA Expertise:
  Clients benefit from direct collaboration with NVIDIA engineers to optimize model performance and deployment.
• Multi-Cloud Support:
  DGX Cloud is available across various cloud platforms, including Oracle Cloud Infrastructure, Microsoft Azure, and Google Cloud, offering flexibility and resilience.

Pricing

Each DGX Cloud instance, equipped with 8 GPUs and 640 GB of GPU memory, starts at $36,999 per month.
This pricing includes hardware, software, storage, and 24/7 expert support.

Conclusion

NVIDIA DGX Cloud provides a robust end-to-end solution for organizations aiming to accelerate and scale their AI initiatives.

With state-of-the-art infrastructure, cutting-edge software platforms, and specialized support, DGX Cloud empowers enterprises to fast-track AI innovation across industries.