Inference Performance — Entidade

developer.nvidia.com·há 15 dias

Co-Designing AI Model Attention for Fast, Interactive Long-Context Inference

As agentic and long-context workloads become common, the context lengths increase and attention consumes a larger share of inference time (Figure 1). Because...

developer.nvidia.com·há 22 dias

ModelExpress: Distributing Model Artifacts at the Speed of Light

Every byte moved has a cost. As model checkpoints grow to hundreds of gigabytes or even a terabyte, that cost adds up quickly. To make things even worse, moving...

developer.nvidia.com·há cerca de 1 mês

AI Model Co-Design: Hardware-Friendly LLM Design

AI performance comes down to three dimensions: Accuracy: How well the model reasons and produces outputs Throughput: How many tokens per second a...

developer.nvidia.com·há cerca de 1 mês

Hardware-Rooted AI Security That Won’t Slow You Down

AI has transformed how organizations operate, driving unprecedented levels of productivity and innovation. However, AI adoption can be impeded by concerns...

developer.nvidia.com·há cerca de 2 meses

Scaling AI Inference Across Multiple GPUs Using NVIDIA TensorRT with Multi-Device Inference Support

Generative AI workloads are rapidly outgrowing the memory and compute budget of single GPUs. For inference developers building media generation pipelines, the...

developer.nvidia.com·há cerca de 2 meses

Maximize AI Factory Energy Efficiency Through Full-Stack Inference and Training Optimizations

Power can account for 40% of the operating expenses (OpEx) to run an AI factory. Each watt can be spent on overhead, data ingestion, training, or generating...

developer.nvidia.com·há cerca de 2 meses

Boost Inference Performance up to 15x on NVIDIA Blackwell Using DFlash Speculative Decoding

As AI systems move from single-turn interactions to coordinated multiagent workflows, low-latency inference becomes increasingly important. Autoregressive LLMs...

developer.nvidia.com·há 2 meses

NVIDIA Achieves Leading Agentic Coding Performance on First Agentic AI Benchmark

AI agents have fundamentally changed the complexity of inference workloads. Until now, the industry has struggled to define a standard for measuring how...