NVIDIA A100 Liquid Cooled

NVIDIA Ampere-Based Architecture

• A100 Liquid Cooled accelerates workloads big and small. Whether using MIG to partition an A100 GPU into smaller instances, or NVLink to connect multiple GPUs to accelerate large-scale workloads, the A100 easily handles different-sized application's needs, from the smallest job to the biggest multi-node workload.

TF32 for AI: 20x higher performance, Zero Code Change

• As AI networks and datasets continue to expand exponentially, their computing appetite is similarly growing. Lower precision math has brought huge performance speedups, but they've historically required some code changes. A100 Liquid Cooled brings a new precision, TF32, which works just like FP32 while providing 20x higher FLOPS for AI without requiring any code change. NVIDIA's automatic mixed precision feature enables a further 2x boost to performance with just one additional line of code using FP16 precision. A100 Tensor Cores also include support for BFLOAT16, INT8, and INT4 precision, making A100 an incredibly versatile accelerator for both AI training and inference.

HBM2e

• With 80 gigabytes (GB) of high-bandwidth memory (HBM2e), A100 Liquid Cooled delivers improved raw bandwidth of 1.5 TB/s, as well as higher dynamic random access memory (DRAM) utilization efficiency at 95 percent. A100 delivers 1.7x higher memory bandwidth than the previous generation.

Structural Sparsity

• AI networks are big, having millions to billions of parameters. Not all of these parameters are needed for accurate predictions, and some can be converted to zeros to make the models "sparse" without compromising accuracy. Tensor Cores in A100 Liquid Cooled can provide up to 2x higher performance for sparse models. While the sparsity feature more readily benefits AI inference, it can also improve the performance of model training.

Every Deep Learning Framework, 700+ GPU-Accelerated Applications

• The NVIDIA A100 Liquid Cooled Tensor Core GPU is a key component of the NVIDIA data center platform for deep learning, HPC, and data analytics. It accelerates every major deep learning framework and accelerates over 700 HPC applications. It's available everywhere, from desktops to servers to cloud services, delivering both dramatic performance gains and cost-saving opportunities.

Third-Generation Tensor Cores

• First introduced in the NVIDIA Volta architecture, NVIDIA Tensor Core technology has brought dramatic speedups to AI training and inference operations, bringing down training times from weeks to hours and providing massive acceleration to inference. The NVIDIA Ampere architecture builds upon these innovations by providing up to 20x higher FLOPS for AI. It does so by improving the performance of existing precisions and bringing new precisions—TF32, INT8, and FP64—that accelerate and simplify AI adoption and extend the power of NVIDIA Tensor Cores to HPC.

Multi-Instance GPU (MIG)

• Every AI and HPC application can benefit from acceleration, but not every application needs the performance of a full A100 Liquid Cooled. With Multi-Instance GPU (MIG), each A100 can be partitioned into as many as seven GPU instances, fully isolated at the hardware level with their own high-bandwidth memory, cache, and compute cores. Now, developers can access breakthrough acceleration for all their applications, big and small, and get guaranteed quality of service (QoS). IT administrators can offer right-sized GPU acceleration for optimal utilization and expand access to every user and application.
  
  MIG is available across bare metal and virtualized environments and is supported by NVIDIA's Container Runtime which supports all major runtimes such as LXC, Docker, CRI-O, Containered, Podman, and Singularity. Each MIG instance is a new GPU type in Kubernetes and will be available across all Kubernetes distributions such as Red hat OpenShift, VMware Project Pacific, and others on-premises and on public clouds via NVIDIA Devise Plugin for Kubernetes. Administrators can also benefit from hypervisor-based virtualization, including KVM based hypervisors such as Red hat RHEL/RHV, and VMware ESXi, on MIG instances with NVIDIA AI Enterprise.

Next Generation NVLink

• The NVIDIA A100 Liquid Cooled NVLink implementation delivers 2x higher throughput compared to the previous generation, at up to 600 GB/s to unleash the highest application performance possible on a single server, while promoting energy efficiency. Two NVIDIA A100 Liquid Cooled boards can be bridged via NVLink, and multiple pairs of NVLink connected boards can reside in a single server (number varies based on server enclosure, thermals, and power supply capacity).

Virtualization Capabilities

• Virtualized compute workloads such as AI, Deep learning, and high-performance computing (HPC) with NVIDIA AI Enterprise. The NVIDIA A100 Liquid Cooled is an ideal upgrade path for existing V100/V100s Tensor Core GPU infrastructure.