Nvidia Corp. today announced at Computex 2024 in Taipei new artificial intelligence tools that will help developers get their apps optimized and rolled out for desktop PCs and Windows and get AI assistants into their games.

Most pretrained generative AI models are general-purpose and were designed for the datacenter. They will respond generically and without a sense of personality and in a neutral voice. Developers working with consumers in games or an app with some sort of punch need to be able to respond on topic and in a style consistent with what they’re working with and also have it optimized for the hardware that it’s running on.

To handle this, Nvidia announced the RTX AI Toolkit, a collection of tools and a software development kit that will provide developers with everything they need to optimize and deploy AI models for Windows applications and orchestrate inference in the cloud. They can also account for the capabilities of the machine that the AI model is running on and the application, be it a game or a productivity app.

For example, using a generic pretrained large language model for a character in a role-playing game could get reasonable dialogue, but it would be uninspired and “phoned in” and it would run rather slowly even on a powerful RTX 4090 video card. After optimization and fine-tuning with the AI toolkit, it could produce a highly customized experience that would account for the setting, background and vision of the developer while also running faster with less overhead.

“The RTX AI toolkit will help Windows application and game developers close the gap between the powerful new AI models that are available and the needs of Windows app development,” said Jessie Clayton, director of product for AI PC at Nvidia.

Taking this a step further, Nvidia also announced optimized inference microservices, called NIMs, for deploying AI-driven digital humans in games and as avatars for customer interactions. This technology, first introduced the platform, named Avatar Cloud Engine, or ACE, at SIGGRAPH 2022, and demoed the technology at GTC 2024 for in-game narrative design with multiple characters.

Now, Nvidia is allowing developers to build these new digital human AI models and deploy them as microservices that can run locally or in the cloud alongside a new Nvidia AI inference manager that will orchestrate and load balance them dynamically.

The Nvidia AI Inference Manager, or AIM, software development toolkit is now available in early access, which will simplify the process of AI integration for PC application developers running their AI apps on desktops and the cloud. It can preconfigure AI models, engines and dependencies in NIM format and supports all major inference backends — such as TensorRT, DirectML, Llama.cpp and PyTorch-CUDA — across numerous processors, including CPUs, NPUs and CPUs.

In-game chatbots with G-Assist

Gamers and game developers are getting another upgrade soon with a new chatbot service called G-Assist that will provide gamers assistance when and where they need it most: within the gaming experience itself.

Gaming itself is a complex entertainment experience with numerous controls, items, keywords, places and characters to remember. It’s not uncommon for gamers to get overwhelmed in a role-playing game when trying to remember where they need to return to to complete a quest or if they’re returning to a game they haven’t picked up in six months.

To deal with this, most gamers will head to the internet to search for information on forums, wikis or watch videos for minutes on end.

“The amount of information available is immense. There are over 3 trillion words of documented game-related knowledge,” said Guillermo Siman, senior product manager for gaming products at Nvidia. “So understandably, we found that 70% of players complained about wanting easier access to knowledge. Because of this, we’re building tools and AI models for developers to enhance the ecosystem with AI assistants for games and apps.”

This is where G-Assist comes in. It will take a voice or text prompt from the player along with context on screen and provide recommendations on their play and help them in their game so that they don’t have to tab out of their game or pick up a tablet.

The AI assistant can go further than simply offering in-game recommendations by even exploring options for optimizing the player’s gaming rig. For example, it can also provide insights on tweaking graphics settings to take full advantage of visual responsiveness and a display’s capabilities or consume less power on a laptop.

Image: SiliconANGLE/Microsoft Designer