NVIDIA NIM Accelerates Healthcare and Autonomous Vehicles

Large Language Models (LLMs), such as Llama, are AI systems trained on massive text datasets to understand and generate human-like text. These models power applications ranging from customer service chatbots to content creation tools. However, deploying them in production often requires significant infrastructure expertise and resources, which many organizations lack.

To address this, NVIDIA developed NIM Microservices. NIM abstracts backend complexity with containerized microservices that handle the infrastructure required to run LLMs efficiently. Instead of building and maintaining custom server stacks, developers can focus on building AI-powered applications that work across a variety of environments.

NIM ensures consistent performance and scalability, by packaging components into portable units, whether deployed in the cloud or on-premises. This makes advanced AI accessible to teams without deep infrastructure knowledge.

One example of this is at Mayo Clinic, where NIM is deployed on-premises to run LLMs without sending sensitive patient data to the cloud. When it comes to AI in healthcare, security and speed aren't mutually exclusive. NIM makes it possible to keep patient data protected while accelerating insight.

On-premises AI deployment: security and control

Pairing NVIDIA NIM with on-premises AI deployment allows regulated industries the ability to leverage LLMs without sending sensitive data to external cloud services. For healthcare organizations like Mayo Clinic, this means running AI workloads like medical imaging analysis, while meeting strict security and compliance standards.

Dell PowerEdge servers provide the enterprise-grade performance and reliability needed for these demanding environments. Mayo Clinic is equipped with the NVIDIA DGX Blackwell 200 system, which delivers 1.4 terabytes of GPU memory, ideal for processing large, whole-slide images. This system integrates with Mayo's existing digital platform and autonomous robot labs, enabling advanced diagnostics at unprecedented scale and speed.

Investing in specialized GPUs and on-prem infrastructure enables faster response times and greater scalability, while keeping data within the organization's secure perimeter. This reduces analysis time from hours to minutes while maintaining the highest security standards.

Cloud-based AI deployment: scalability and flexibility

Mayo Clinic turns to on-premises deployment to meet strict privacy and compliance standards. In contrast, organizations like Waymo rely on the cloud to address demands around scale, speed, and geographic flexibility. Autonomous vehicle fleets generate massive volumes of real-time data that must be processed instantly across distributed environments.

Waymo uses NVIDIA NIM to deploy AI across multiple data centers while maintaining high performance. Their autonomous vehicles constantly generate sensor data that powers decision-making on the road and drives ongoing improvements in safety algorithms. It is important to process the sensor data in real-time to maintain safety standards and cloud-based deployment with NIM enables companies to scale their processing capabilities across multiple data centers.

NVIDIA DGX Systems are pre-configured AI computing platforms that combine multiple GPUs with optimized software. NVIDIA NVLink and NVLink Switch technologies connect these GPUs at higher speeds than traditional connection methods.

NVIDIA DGX Systems with NVLink systems deliver bandwidth speeds up to 1.8 TB/s, double that of the previous generation. The infrastructure allows rapid analysis of fleet-wide driving patterns and accelerates validation of new scenarios, helping Waymo evolve its technology faster while maintaining safety at scale.

The infrastructure that powers modern AI

NVIDIA NIM simplifies AI deployment by providing containerized microservices that handle the complex infrastructure requirements of production AI systems. Whether organizations choose on-premises deployment for security requirements or cloud deployment for scalability, NIM provides the foundation for reliable AI applications.

Learn more:

• NVIDIA AI Cloud deployment solutions
• Dell PowerEdge AI Servers
• Continuing to Power the Future of AI with Dell's Cooling and Computing Innovations