FFmpeg Codes – the best encoder for 360 VR Videos

If someone asked me what my favorite encoder is, I would probably answer with FFmpeg. Nobody has done that yet, (I wonder why !? 😀 ) but still I would like to introduce it here.
For half an eternity, I’ve been having a notepad with the best command-line codes scattered around. So I thought I’d bring some order to the whole thing and hope it’d be helpful to anybody.

Why FFmpeg Codes?

Before we go into the nerdy details and we can feel like with hackertyper.com, first a few basic words, why FFmpeg convinced me, even if it doesn’t even have a graphical user interface (GUI) and can seem a bit daunting at first glance:

• FFmpeg supports virtually any audio and video format. You are not bound to presets as with most editing software and can intervene in the encoding process in great detail.
• DIY sound engineer: in the production process, especially in virtual reality, it can save a lot of time to simply convert the video as you would like it to be, without having to ask the editor to again render a new file, which then has to be uploaded and downloaded.
• Also interesting for the editor: Re-rendering is not necessary in most cases. For example, if only the sound needs to be changed, FFmpeg can simply change the audio stream, while the video remains untouched and is therefore not only interesting for 360° videos.
• Most program codes, like here, can simply be copied, pasted, and looked up here again and again.
• FFmpeg is also interesting for streaming conventional and VR videos.

If you want to know which audio format is best used for which platform, you can also check at my blog post right here: Spatial Audio format support for 360° Video Platforms & Players

Installation

The setup of FFmpeg is not quite as easy as you are used to with other software nowadays, but the files can be downloaded here for free.
Fortunately, there are very good instructions on the Internet for Windows and Mac in German, English and other languages, so just ask the search engine of trust.
If you did everything right, you can now open the command prompt (cmd) directly in the explorer with “Ctrl” + “right click” and directly use the codes described below.

Overview of Codes

Die Basics:

• Remove Audio from Video
• Extract Audio from Video
• Add Audio to Video
• Replace Audio.wav with Video.mp4
• Adjust the volume of a video
• Adjust the length of a media file
• Adjust the length to the shortest media file

360° Video specific:

• Halve Framerate
• Scale Video
• Stereoscopic to Monoscopic
• Loop 360° image and add 360° sound

Spatial Audio Encoding:

• Four-channel FOA.wav with Video
• Encode Video for Google Jump Inspector
• Higher Order Ambisonic HOA.wav to Quicktime.mov
• Quad-Binaural

Weitere interessante Codecs:

• H.264 (MPEG-4/AVC: Advanced Video Coding)
• H.265 (HEVC: High-Efficiency Video Coding)
• DNxHD (Digital Nonlinear Extensible High Definition)
• WebM
• TIFF, EXR or DPX sequences
• MPEG-H 3D Audio

Copy and Paste Command Lines

The Basics

Remove Audio from Video

Let’s get started. The process I use most is to completely remove (not just mute) the audio track that is usually attached to a video file. This ensures that no conflict arises during subsequent encoding, for example, if the sound is to be exchanged. A new video file with identical image content is created, the video stream is simply copied but no longer contains an audio track.

ffmpeg -i input.mp4 -c:v copy -an output.mp4

Extract Audio from Video

The same works also the other way round if you want to get the sound out of a video for instance. Most of the time the sound is already compressed by AAC or similar lossy, so it is rather the unpleasant solution to simply return the audio to a wav, but can be practical for testing purposes.

ffmpeg -i video.mp4 -vn -c:a copy -acodec pcm_s16le output.wav

Add Audio to Video

If you want to add sound to a silent video, you can do this with two inputs, moving picture and sound. Here, the sound must be converted to AAC, because mp4 as H264 does not support audio stream as Wav format.

ffmpeg -i input.mp4 -i input.wav -c:v copy -c:a aac -b:a 320k output.mp4

Replace Audio.wav with Video.mp4

If you don’t want to take the detour of first creating a new video file without a sound track, but simply exchange the audio directly, this is how it works:

ffmpeg -i input.mp4 -i newaudio.wav -c:v copy -c:a aac -b:a 320k -map 0:v:0 -map 1:a:0 output.mp4

Adjust the volume of a video

Adjusting the volume is also very practical. This code can provide an even volume for multiple videos without having to render again or even export new sound.

ffmpeg -i input.mp4 -vcodec copy -af "volume=-15dB" -b:a 320k output.mp4

Adjust the length of a media file

If you only want to use a certain part of a video, or audio, for example, for testing, this works as follows. Here, too, no rendering is done, only the unwanted parts of the video are removed.

ffmpeg -i input.mp4 -ss 00:01:00 -t 00:00:10 -c:v copy -c:a copy output.mp4

Adjust length to the shortest media file

Normally, image and sound should have the same length, but it can also be a helpful abbreviation, since “-shortest” sets the length of the final file to the shortest duration of the two inputs.

ffmpeg -i input.mp4 -i audio.aac -shortest -c:v copy -c:a copy output.mp4

360° Video specific examples

Halve Framerate

Convert a video that was recorded with 59.94 frames to 29.97 frames? FFmpeg will automatically select a bitrate which is set too low in most cases. It would have to be set manually with the command -b:v 40m e.g. to half the original bitrate.

ffmpeg -i input.mp4 -r 30000/1001 output.mp4

Scale Video

Videos with 4k or more are usually too big to work with and burden the performance unnecessarily. You can set the height and width of the video manually.

ffmpeg -i input.mp4 -s 1920*960 output.mp4

Alternatively, the video filter can halve the size of the video.

ffmpeg -i input.mp4 -vf scale=iw\*.5:ih\*.5 output.mp4

Stereoscopic to Monoscopic

If you have a 3D 360° video, it certainly looks great on VR glasses, but is less suitable for conventional screens, since one eye is enough for viewing, i.e. 2D. Here the video is halved in the middle, horizontally or vertically.

Over-under (top-bottom):

ffmpeg -i input.mp4 -vf crop=h=in_h/2:y=0 output.mp4

Side-by-side (left-right):

ffmpeg -i input.mp4 -vf crop=w=in_w/2:x=0 output.mp4

Loop 360° image and add 360° sound

If you only have a panoramic 360° image and want to quickly convert the sound into a video, you can simply loop the still image one after the other with the following command.

ffmpeg -loop 1 -i stillframe.jpg -i input.wav -map 1:a -map 0:v -c:a copy -channel_layout 4.0 -c:v libx264 -b:v 50000k -bufsize 50000k -shortest output.mov

Spatial Audio

Four-channel FOA.wav with Video

Useful for e.g. YouTube and SamsungVR. It is also easier with the FB360 Encoder, which injects metadata automatically, which otherwise has to be done manually with the “YouTube Spatial Media Metadata Injector”. Also works for FuMa or Quad, just everything that is four-channel, but mostly it is a First Order Ambisonic in ambiX format.

ffmpeg -i input.mp4 -i input.wav -c:v copy -c:a aac -b:a 512k output.mp4

Encode Video for Google Jump Inspector

Jump Inspector supports Ambisonics first order and Ambisonics third order, but also requires the video in a certain format. In addition, the Google Jump Inspector reads the meta data from the file name, so make sure to adjust it, as suggested here.

ffmpeg -i input.mp4 -c:v libx264 -b:v 40m -vf scale=3840:2160 -r 30 -profile main -pix_fmt yuv420p -an output.360.mono.mp4

Higher Order Ambisonic HOA.wav to Quicktime .mov

Since mp4 does not support 16 audio channels, the video must be converted to Quicktime format. The mov container can play the sound loss-free as a wav stream, which can be interesting for the Google Jump Inspector, GoPro VR Player 3 or VLC Media Player. Sound is a third-order ambisonic wav with 16 channels.

ffmpeg -i input.mp4 -i input.wav -c:a copy -c:v copy output.mov

Quad-Binaural

A small special case is quad-binaural, sometimes also called omni-binaural, which is supported by the SamsungVR player. Here, too, the meta data is read from the file name. Pros and cons of this exotic can be read here: Formats for 360° Sound

ffmpeg -i input.mp4 -i 0.wav -i 90.wav -i 180.wav -i 270.wav -map 0:v -map 1:a -map 2:a -map 3:a -map 4:a -vcodec copy -c:a aac -b:a 512k output_mono360_quadraphonic_binaural.mp4

Various interesting codecs

H.264 (MPEG-4/AVC: Advanced Video Coding)

Probably the most common format for videos is H264, which doesn’t support audio as.wav, but e.g. AAC, so it’s not a lossless sound, which plays a role when working with 3D audio later on.

ffmpeg -i input.mp4 -c:v libx264 -b:v 50M -profile main -pix_fmt yuv420p -c:a aac -b:a 320K output.mp4

H.265 (HEVC: High Efficiency Video Coding)

H265 is becoming more and more popular, but not yet so widespread. It can reduce the file size of a video by half compared to H264 with the same quality, but is correspondingly compute-intensive.

ffmpeg -i input.mp4 -c:v libx265 -b:v 25M -c:a aac -b:a 320K output.mp4

DNxHD (Digital Nonlinear Extensible High Definition)

Most digital audio workstations (DAW) like Avid ProTools can save processing power by bringing the video to a low resolution and converting it to a codec that is less processor intensive than, say, H264.

ffmpeg -i input.mp4 -vf scale=iw\*.5:ih\*.5 -map 0:v -c:v dnxhd -pix_fmt yuv422p -profile:v dnxhr_lb -y output.mov

WebM

WebM is a container for the video codecs VP8 and VP9 introduced by Google, which usually uses the audio format Opus as.ogg, but takes some time to encode.

ffmpeg -i input.mp4 -c:v libvpx -crf 4 -b:v 50M -map 0:v:0 -map 0:a:0 -c:a libvorbis -af "pan=quad|c0=c0|c1=c1|c2=c2|c3=c3" -preset ultrafast output.webm

TIFF, EXR or DPX sequences

Such file types rarely reach audio post-production but are still not uninteresting. Here not only a file is set as input, but also a frame rate for the input must be defined so that the length of the later video is clear. In the output, the frame rate can be readjusted with “-r”.

ffmpeg -i input_0001.dpx -framerate 60 -c:v libx264 -b:v 30M -r 30 -an output.mp4

MPEG-H 3D Audio

I couldn’t find anything about this format regarding FFmpeg yet, but I’m curious how it develops in terms of object-based audio and audio definition model (ADM). I think that FFmpeg can become unwieldy with all the meta data and suspect that special encoders will be used here, but we will see.

Troubleshooting

FFmpeg usually displays error messages in red. These can be googled quickly. Otherwise these little tricks will help spontaneously:

• remove empty spaces from the file name
• file path does not contain the required file(s)
• video codec does not support the format chosen as a variable and FFmpeg cannot find suitable alternatives.

Also Helpful

If you are a Mac user and want to skip all the fun, you can try iFFmpeg, or simply convert your files with droplets from AudioEase360 by drag and drop. Thanks for that!
Windows users can go for myFFmpeg.

In addition, Github has proved helpful in creating this article and also has an overview of the variables. FFmpeg Cheat Sheet for 360° video.

Here the own Wiki from and to FFmpeg: https://trac.ffmpeg.org/wiki

With the command ffmpeg -i input.mp4 you can also analyze files. But for me it is more practical to use another software, like MediaInfo, because files can be controlled faster and clearer by drag and drop.

Thanks also to all forums, which could also answer my questions via search engine. I hope this article is not only helpful to me.

Here you can find a YouTube Tutorial : https://youtu.be/MPV7JXTWPWI?si=0qvnRZoyT75wes4E