NVIDIA NIM Agent Blueprints: Enterprise AI Applications

Posted August 30, 2024 iotric

NVIDIA NIM Agent Blueprints: Revolutionizing Enterprise AI Applications

The initial wave of generative AI has shown incredible possibilities with tools such as ChatGPT, Google Bard, Charisma AI, and Alpha3D. These tools can help people research and write content, generate images, videos, or 3D models from simple prompts, and streamline work processes to make them faster and more efficient than ever before.

However, the second wave of generative AI has arrived, and Nvidia is leading the charge by expanding its capabilities through NIM and NIM Agent Blueprints. With the introduction of advanced foundation AI models and advancements in compatibility with popular libraries and deployment with a single command, Nvidia is enhancing the efficiency and autonomy of AI workflows.

This article provides an in-depth overview of Nvidia NIM and NIM Agent Blueprints, exploring how NVIDIA is taking generative AI to the next level.

What is NVIDIA NIM Microservices?

Nvidia NIM, also known as Nvidia Inference Microservices, is a container of user-friendly microservices designed to simplify and accelerate the deployment of generative AI models across multiple platforms, including the cloud, data centers, and workstations.

The NIM pre-packaged containers can include any model, from open-source to proprietary, enabling seamless integration of pre-trained and customized AI models into applications, development frameworks, and workflows.

With the inclusion of USD (Universal Scene Description), NIM can now not only generate and edit 3D models but also generate code to interact with 3D scenes. This capability leverages the power of OpenUSD, enabling seamless integration and manipulation of complex 3D environments, advancing industries like digital twins and robotics.

These microservices can be deployed with a single command, making them operational anywhere. They automatically expose industry-standard APIs, allowing for quick integration into existing systems and enabling the efficient processing of complex data.

NVIDIA NIM’s New USD Microservices

Nvidia’s NIM microservices now include advanced capabilities to interact with OpenUSD, an open-source 3D scene description framework that facilitates seamless interchange and interaction with 3D data.

USD Code NIM: This NIM is designed to assist developers by generating OpenUSD Python code based on text prompts and answering general knowledge questions related to OpenUSD. This generated code can be visualized in applications like usdview from Pixar or any Nvidia Omniverse Kit-based platform, offering a practical way to explore 3D data.
USD Search NIM: Developers can search through extensive libraries of OpenUSD, 3D, and image data using natural language or image inputs.
USD Validate NIM: This NIM ensures the compatibility of uploaded files with OpenUSD release versions. It also generates RTX-rendered, path-traced images using Nvidia Omniverse Cloud APIs or application programming interfaces.

Nvidia is expanding its NIMs with new OpenUSD-based microservices for scene assembly, material application, mesh generation, physics simulation, and large-scale neural radiance fields.

What are NIM Agent Blueprints?

Nvidia announced NIM Agent Blueprints, which refers to comprehensive pre-trained, customizable AI workflows that help businesses create and manage custom AI applications for generative AI tasks and use cases. Using NIM Agent Blueprints, NIM microservices, and NeMo framework, enterprise developers can quickly build and deploy AI applications that learn and improve from data over time.

NIM Agent Blueprints has everything developers need to get started, including partner microservices, AI agents, reference code, customization documentation, and a Helm chart for easy deployment.

Nvidia recently introduced three NIM Agent Blueprints, Digital Humans for Customer Service, Multimodal PDF Data Extraction for Enterprise RAG, and Generative Virtual Screening for Drug Discovery, which are now available.

Let’s understand their capabilities and how they are helping enterprises build and deploy AI workflows quickly.

Digital Humans for Customer Service

The blueprint Digital Humans for Customer Service allows enterprises to build 3D animated digital humans or avatars to provide a more engaging user experience than traditional customer service options across industries.

This blueprint is powered by the collection of Nvidia NIM inference microservices and Nvidia ACE technologies, along with Omniverse RTX, Audio2Face, and Llama 3.1.

To experience the capabilities of this blueprint, you can interact with an Nvidia digital human named “James.” James uses the Nvidia Product knowledge base, understands Nvidia product details, and provides natural responses to your questions about NVIDIA.

Multimodal PDF Data Extraction for Enterprise RAG

The “Multimodal PDF Data Extraction for Enterprise RAG” blueprint is designed to extract detailed insights from large volumes of PDF documents, including text, tables, and charts.

This workflow enhances generative AI with Retrieval-Augmented Generation (RAG) capabilities, allowing AI systems to retrieve and utilize information from proprietary data sources. This blueprint helps enterprises build high-accuracy, multimodal retrieval pipelines for smarter, more responsive AI applications.

Generative Virtual Screening for Drug Discovery

The generative virtual screening NIM Agent Blueprint for drug discovery accelerates identifying and optimizing promising drug-like molecules, significantly reducing time and cost by generating molecules with favorable properties and a higher probability of success.

Researchers and application developers can quickly customize and deploy AI models for 3D protein structure prediction, small molecule generation, and molecular docking. This blueprint incorporates Nvidia NIM microservices — including AlphaFold2, MolMIM, and DiffDock — to accelerate the virtual screening of small molecules using generative models.

In combination with other tools available in NVIDIA BioNeMo, enterprises can easily connect multiple NIM Agent Blueprints to build increasingly sophisticated AI applications and accelerate their drug discovery work.

NVIDIA Ecosystem Accelerates Enterprise Adoption

Nvidia is partnering with top global system integrators and technology companies such as Accenture, Deloitte, SoftServe, and World Wide Technology (WWT) to bring the NIM Agent Blueprints to enterprises worldwide.

Moreover, companies like Cisco, Dell Technologies, Hewlett Packard Enterprise, and Lenovo also provide full-stack NVIDIA-accelerated infrastructure and solutions to help deploy NIM Agent Blueprints faster.

To maximize the value of generative AI, NIM Agent Blueprints can also integrate with data and storage platforms from Nvidia partners such as Cohesity, Datastax, Dropbox, NetApp, and VAST Data.

The Future of NIM Agent Blueprints

Nvidia is actively developing more NIM Agent Blueprints to create generative AI applications across various industries, including content generation, software engineering, retail shopping advisors, and research and development. According to Justin Boitano, new NIM Agent Blueprints will be released monthly.

Each blueprint is designed for easy access and deployment, making it more straightforward for enterprises to adopt generative AI. Supported by Nvidia’s partners, these NIM Agent Blueprints enable virtually any enterprise to seamlessly integrate generative AI into their operations, driving innovation and efficiency across multiple sectors.