GPU-accelerated video processing with ffmpeg

I want to use ffmpeg to accelerate video encode and decode with an NVIDIA GPU.

From NVIDIA's website:

> NVIDIA GPUs contain one or more hardware-based decoder and encoder(s) (separate from the CUDA cores) which provides fully-accelerated hardware-based video decoding and encoding for several popular codecs. With decoding/encoding offloaded, the graphics engine and the CPU are free for other operations.

My question is: can I use CUDA cores to encode and decode video, maybe faster?

FFmpeg provides a subsystem for hardware acceleration, which includes NVIDIA: https://trac.ffmpeg.org/wiki/HWAccelIntro

In order to enable support for GPU-assisted encoding with an NVIDIA GPU, you need:

• A ​supported GPU

• Supported drivers for your operating system

• The NVIDIA Codec SDK

• ffmpeg configured with --enable-nvenc (default if the drivers are detected while configuring)

Quick use on ​supported GPU:

CUDA

ffmpeg -hwaccel cuda -i input output

CUVID

ffmpeg -c:v h264_cuvid -i input output

Full hardware transcode with NVDEC and NVENC:

ffmpeg -hwaccel cuda -hwaccel_output_format cuda -i input -c:v h264_nvenc -preset slow output

If ffmpeg was compiled with support for libnpp, it can be used to insert a GPU based scaler into the chain:

ffmpeg -hwaccel_device 0 -hwaccel cuda -i input -vf scale_npp=-1:720 -c:v h264_nvenc -preset slow output.mkv

Source: https://trac.ffmpeg.org/wiki/HWAccelIntro

As Mike mentioned, ffmpeg wraps some of these HW-accelerations. You should use it instead of going for more low-level approaches (official NVIDIA libs) first!

The table shows, that NVENC is probably your candidate.

But: Be careful and do some benchmarking. While GPU-encoders should be very fast, they are also worse than CPU ones in comparison to visual quality.

The thing to check here is: Does a GPU-encoder compete with a CPU-encoder when some quality at some given bitrate is targeted? I would say no no no (except for very high bitrates or very bad quality), but that's something which depends on your use-case. GPU-encoding is not a silver-bullet providing only advantages.

For AMD cards, use these -vcodec options:

Windows:
h264_amf
hevc_amf

Linux:
h264_vaapi
hevc_vaapi

ffmpeg -i input.mp4 -b:v 10400k -vcodec h264_amf -vf crop=1920:848:0:116 -c:a copy output.mp4

ffmpeg -i input.mp4 -b:v 10400k -vcodec hevc_amf -vf crop=1920:848:0:116 -c:a copy output.mp4

ffmpeg -i input.mp4 -b:v 10400k -vcodec h264_vaapi -vf crop=1920:848:0:116 -c:a copy output.mp4

ffmpeg -i input.mp4 -b:v 10400k -vcodec hevc_vaapi -vf crop=1920:848:0:116 -c:a copy output.mp4