NVIDIA A100 80GB

Unmatched Acceleration for the World’s Most Powerful Elastic Data Centers

The NVIDIA A100 80GB Tensor Core GPU delivers groundbreaking acceleration across all scales, enabling the highest-performing elastic data centers globally. It supports AI, data analytics, and high-performance computing (HPC) applications with unmatched speed and efficiency. As the cornerstone of NVIDIA’s data center platform, the A100 offers performance gains up to 20 times greater than the previous NVIDIA Volta generation. Utilizing Multi-Instance GPU (MIG) technology, a single A100 can be segmented into seven fully isolated GPU instances, creating a flexible platform that adapts dynamically to changing workload demands. This GPU forms part of NVIDIA’s comprehensive data center ecosystem, combining cutting-edge hardware, networking solutions, advanced software, libraries, and AI models optimized through NVIDIA GPU Cloud (NGC). This all-encompassing AI and HPC platform enables researchers to generate tangible results and deploy scalable production solutions while allowing IT teams to maximize the use of every A100 GPU available.

NVIDIA A100 80GB

Key Specifications

 

CUDA Cores: 6912

Streaming Multiprocessors: 108

Third-Generation Tensor Cores: 432

GPU Memory: 80GB HBM2e with ECC enabled by default

Memory Interface: 5120-bit

Memory Bandwidth: 1555 GB/s

NVLink: 2-Way, 2-Slot with 600 GB/s bidirectional bandwidth

Multi-Instance GPU (MIG) Support: Yes, up to 7 isolated GPU instances

FP64 Performance: 9.7 TFLOPS

FP64 Tensor Core Performance: 19.5 TFLOPS

FP32 Performance: 19.5 TFLOPS

TF32 Tensor Core Performance: 156 TFLOPS (312 TFLOPS with sparsity)

BFLOAT16 Tensor Core Performance: 312 TFLOPS (624 TFLOPS with sparsity)

FP16 Tensor Core Performance: 312 TFLOPS (624 TFLOPS with sparsity)

INT8 Tensor Core Performance: 624 TOPS (1248 TOPS with sparsity)

INT4 Tensor Core Performance: 1248 TOPS (2496 TOPS with sparsity)

Thermal Solution: Passive cooling

vGPU Support: NVIDIA Virtual Compute Server (vCS)

System Interface: PCIe 4.0 x16

Maximum Power Consumption: 300 Watts

 

NVIDIA Ampere-Based Architecture

The A100 GPU is engineered to accelerate workloads ranging from small tasks to large-scale multi-node operations. It offers tremendous flexibility through Multi-Instance GPU (MIG) technology, which partitions the physical GPU into multiple isolated instances tailored for varying workload sizes. For massive workloads, multiple A100 GPUs can be interconnected via NVLink, delivering superior performance in parallel. This scalable architecture enables users to address the full spectrum of computational demands with ease.

Third-Generation Tensor Cores

Tensor Cores debuted in NVIDIA’s Volta architecture, revolutionizing AI training and inference by significantly accelerating computations, reducing AI training times from weeks to hours. The Ampere architecture’s third-generation Tensor Cores advance this innovation by delivering up to 20 times more floating-point operations per second (FLOPS) for AI tasks. They enhance performance across existing precision types and introduce new ones — including TF32, INT8, and FP64 — simplifying AI adoption and extending Tensor Core capabilities to HPC workloads.

TF32 for AI: 20x More Performance Without Code Changes

AI workloads continue to expand rapidly in complexity and size, increasing computational demands dramatically. While lower-precision math formats traditionally provided speed benefits, they often required software modifications. The A100 introduces TF32, a new precision that operates like FP32 but delivers 20 times the AI performance without needing any code changes. Additionally, NVIDIA’s automatic mixed precision feature allows developers to double performance by enabling FP16 precision with minimal code updates. The A100’s Tensor Cores also support BFLOAT16, INT8, and INT4, making it a versatile solution for both AI training and inference scenarios.

Double-Precision Tensor Cores: A Major Leap for HPC

The NVIDIA A100 extends Tensor Core acceleration to double-precision (FP64) operations, marking the most significant advancement for HPC since GPUs adopted FP64 computing. Its third-generation Tensor Cores support fully IEEE-compliant FP64 matrix operations, delivering up to 2.5 times more performance and efficiency for HPC applications requiring high-precision math. These improvements are enhanced by NVIDIA CUDA-X math libraries, benefitting scientific simulations, modeling, and other double-precision intensive workloads.

Multi-Instance GPU (MIG)

Not all AI or HPC workloads need the full power of an entire A100 GPU. With Multi-Instance GPU (MIG) technology, a single A100 can be divided into up to seven fully isolated GPU instances, each with its own dedicated memory, cache, and compute cores. This provides developers with breakthrough acceleration for tasks of any size while guaranteeing consistent quality of service. For IT administrators, MIG enables optimal resource allocation, expanding GPU access across more users and applications for improved overall utilization. MIG is supported in both bare metal and virtualized environments, integrated seamlessly with NVIDIA Container Runtime and compatible with container runtimes such as Docker, LXC, CRI-O, Containerd, Podman, and Singularity. In Kubernetes environments, each MIG instance is recognized as a distinct GPU type, compatible with major Kubernetes distributions like Red Hat OpenShift and VMware Project Pacific, on-premises or in cloud deployments via NVIDIA Device Plugin for Kubernetes. Additionally, MIG supports hypervisor-based virtualization solutions including KVM and VMware ESXi through NVIDIA vComputeServer.

HBM2e Memory

The A100 is equipped with an impressive 80GB of HBM2e high-bandwidth memory, which delivers raw bandwidth of 1.6 TB/s, representing a 1.7 times increase compared to the previous generation. With dynamic random access memory (DRAM) utilization efficiency of 95%, the A100 handles large models and datasets efficiently, reducing bottlenecks and speeding up complex computations.

Structural Sparsity

AI models frequently contain millions or even billions of parameters, but not all contribute equally to accuracy. By converting some parameters to zero, models become “sparse” without losing predictive power. The A100’s Tensor Cores accelerate sparse matrix operations, offering up to double the throughput for sparse models. Although sparsity benefits are more pronounced during AI inference, it also contributes to faster training times.

Next-Generation NVLink

NVIDIA’s NVLink interconnect technology in the A100 provides double the throughput of the previous generation, reaching up to 600 GB/s. This high-speed link enables multiple A100 GPUs to communicate efficiently within a single server, dramatically improving application performance. Two PCIe A100 boards can be connected via NVLink, and multiple NVLink pairs can coexist in a server depending on system design, cooling, and power capabilities.

Compatibility with AI Frameworks and HPC Applications

The NVIDIA A100 Tensor Core GPU supports every major deep learning framework and accelerates over 700 HPC applications. It is widely deployed in desktops, servers, and cloud environments, delivering substantial performance improvements and cost savings.

Virtualization Support

The A100 is well-suited for virtualized compute environments, supporting workloads such as AI, deep learning, and HPC through NVIDIA Virtual Compute Server (vCS). It provides an ideal upgrade path for infrastructures currently using V100 or V100S GPUs.

Summary of Structural Sparsity Benefits

Modern AI networks often contain vast numbers of parameters, many of which can be sparsified (converted to zero) without impacting model accuracy. The A100 leverages this by delivering up to twice the performance for sparse models, with most benefits seen in inference, but also improving training speeds.