Accelerate Vision AI with NVIDIA CUDA-Optimized VC-6 for Efficient, Parallel Video Processing

NVIDIA has introduced a CUDA-accelerated implementation of SMPTE VC-6 (ST 2117-1), a next-generation video codec designed for high-performance vision AI workloads. As GPU compute power continues to grow, data pipelines often become the bottleneck due to slow I/O, host-to-device transfers over PCIe, and CPU-bound processing like decoding and resizing. This issue, known as GPU starvation, can severely limit AI training and inference throughput. VC-6’s architecture is uniquely suited to overcome this by enabling efficient, parallel data access directly on the GPU. VC-6 encodes images as a hierarchical, multi-resolution structure called an S-Tree, where each level—known as an echelon—represents a different quality layer. The smallest resolution is encoded first, and each higher layer captures the difference (residuals) between the target and the upsampled lower layer. This approach allows for selective decoding: users can retrieve only the data needed for a specific resolution, region of interest (RoI), or color plane without processing the entire file. This selective data recall drastically reduces I/O. On the DIV2K dataset, decoding at quarter resolution required 37% less data than full resolution, and at lower bit depths, savings reached up to 72%. This directly reduces storage, network, PCIe bandwidth, and GPU memory usage—critical advantages in large-scale AI training. The codec’s design aligns naturally with GPU parallelism. Each echelon, color plane, and image tile can be processed independently, enabling fine-grained parallel execution. Unlike traditional codecs that rely on sequential processing, VC-6’s dual hierarchies operate in orthogonal dimensions, minimizing dependencies and maximizing concurrency—perfect for the SIMT (Single Instruction, Multiple Thread) model of NVIDIA GPUs. V-Nova and NVIDIA collaborated to build a native CUDA implementation of VC-6, replacing the earlier OpenCL version. The new CUDA library integrates seamlessly with AI frameworks like PyTorch and supports GPU-resident output via the CUDA array interface. This means decoded images can be directly consumed by AI models without CPU copies or synchronization overhead. Performance benchmarks on an NVIDIA RTX PRO 6000 Blackwell Server-Edition show significant gains. The CUDA version outperforms both CPU and OpenCL implementations, especially in throughput mode. Profiling with Nsight Systems reveals that single-image decoding underutilizes the GPU due to small grid sizes and kernel launch overhead. However, by leveraging asynchronous, batched decoding across multiple images, GPU utilization improves dramatically. Key optimization opportunities include kernel fusion to reduce overhead in upsampling chains, and using larger grid dimensions to better occupy the GPU’s 188 streaming multiprocessors. While kernel-level parallelism across multiple threads works, it introduces CPU overhead and scheduling inefficiencies. A single, larger GPU grid is far more effective. The current CUDA implementation is in alpha and supports partial decoding, RoI extraction, and selective resolution retrieval. Native batching and further optimizations are planned. Developers can install the VC-6 Python package via pip and begin using it immediately with GPU-accelerated workflows. For AI teams building high-throughput, multimodal vision pipelines, VC-6 on CUDA offers a powerful way to close the data-to-tensor gap. By enabling efficient, selective, and GPU-native data access, it helps ensure that high-performance models are no longer limited by slow data pipelines. The technology is already available in alpha with C++ and Python APIs, and ongoing collaboration with NVIDIA will further enhance its capabilities for future AI workloads.