2019-12-08

Happy Birthday Matroska

On Friday Matroska turned 17. There was a celebration at the No Time To Wait 4 conference in Budapest with some nice cake with the MKV initials. I wanted to make a small speech for the occasion but didn't find the opportunity (and then I had a train back to France to catch). So here's a few of the things I would have said and more.

NTTW 4 MKV17 Cake


It was nice to be back at No Time To Wait. It's such a fantastic gathering of archivists and developers and the atmosphere is always great. It's nice to feel welcome although I am not an archivist and know little of all the things they have to go through in their job. Although every time after NTTW I feel like I know a bit more and thus can help more.

The work they do with Matroska (combined with FFV1) is exactly one of the main use Matroska was designed for. Except neither me nor the other creators of Matroska ever dreamed it would ever be used professionally for prestigious archival like at the BFI (British Film Institute) and possibly at the Library of Congress in the USA. Of course there was already traction in the "corporate" world as MKV is used for all kinds of video sharing as files and also the basis of WebM. It's officially supported in OSes like Windows 10 or Android. But the archival world is a different thing. It's not only about sharing the latest movie//TV show you got, but it's keeping important content for a long time. IMO it has a deeper impact in the long term and some historical, political and artistic value that can't be beaten. This is also what makes it so special to me as I always try to find an extra bit of "soul" in whatever I do. It's not just about writing some code or documentation. It's also a great motivation knowing it will benefit something bigger.

It's amazing what a hobby project started (forked) 17 years ago has become. In my mind (and probably all other involved) we were driven by the same spirit that was on the Internet during that time. Creating something great for free and possibly challenge the "corporate" world. One of the U in my robUx4 nickname stands for Utopia. That was always part of the goal.

Just talking about why some details of Matroska and how we came to the conclusion of that detail always remind me of the amount of work we put in this, all the challenges we faced (like trying to be as good as ogg for streaming). It's fun having to go back to all these memories when we're doing the IETF specifications and realize the things we got right as complete n00bs and some we got wrong (nanoseconds non rationale timestamps that we called timecodes, to show how n00b we were, the clock was even in floating point because in the analog world clocks aren't perfect). Matroska was designed to last 10 years in a time were there was a new video codec every 3 months. It was still hard to predict the full evolution of things (no VR for example). The challenges posed by long term archival is also interresting. Here we have to support all kinds of sources (analog and digital) with very specific characteristics (and keep everything as does RAWcooked thanks to attachments). There's still plenty of areas to improve (Timecodes, Bayer support). After 17 years, Matroska still has room to grow.

I am very grateful to have meet all kinds of new people who care deeply about the work we do on Matroska and see again familiar faces who care at least as much. You have no idea how great is to have all of your support and that you took a leap of faith choosing Matroska when we just started making proper specifications. From NTTW1 it was not very clear whether people would actually use Matroska at all.

I would like to thank in particular the organizers of the event: Dave, Jerome, Ashley, Alessandra and Zsuzsa. It's a privilege to be welcome in your community.

👓👓

Budapest ShowYourStripes

2019-03-10

Windows Video Playback Performance

As a VLC developer I have spent a lot of time working on video decoding and displaying for Windows, especially with Direct3D11. VLC 3.0 is the result of that work and we keep on improving it.

Despite that, people are still complaining about the performance of VLC compared to other Windows players. I know we did a good job, so I wanted to know where we're at regarding the performance.

I tested the following software with various source files on Windows 64:
All players have been used with their default settings, fresh from installation.

My system is an i7-8700 and I used the integrated Intel 630 GPU connected to a 2560x1440 display at 120Hz connected by DisplayPort. It has 12 logical threads at up to 4 GHz, so CPU decoding is supposedly OK.

Here are the results for the various test files:

Sony Camp HEVC HDR 4K 60 fps


This is the main sample I used when working on HDR. It has a high bitrate that I have a hard time playing over my NAS and even locally it can stutter in the hardware decoder (we only found a fix for that recently). Apart from 8K and AV1 that's pretty much the hardest thing to decode right now. Not only that but the HDR content needs to be handled properly. In my case the screen is not HDR so tone mapping has to be applied in the player (the HDR mode of Windows is not enabled).


PlayerCPU %Memory UsageGPU 3DGPU DecodeGPU ProcessorSmooth PlaybackColoursRealtime
VLC2184040400yesokyes
MPV1008356500stuttersdarkyes
MPC-BE21250603041notoo brightno
MPC-HC11054606053yeswashed outyes
Movies & TV165060400yessaturatedyes
Kodi21045404055yeswashed outyes
MPV Ctrl+H293280450yesdarkyes

The first thing noticeable is that by default MPV doesn't use the GPU to decode this file. You have to manually tell it to do it. The last line adds the performance of MPV with hardware decoding (d3d11va) on.

The second thing is that apart from VLC, no player display the HDR colours/luminance correctly (there is a SDR version of the same file for comparison, but I don't know how official it is, I also compare to what my HDR TV does). It's surprising from MPV as the tone mapping in VLC is inspired by their code.

The third thing is that MPC-BE cannot play this file in real time, even though the CPU and GPU are not maxed out. Maybe a buffering issue. The audio stops every few second and then playback resumes.

The stuttering in MPV means the 60fps of the source is not respected. The frames are either skipped or not displayed at the right time (something we fixed in VLC after some hard work).

DNCE H264 1080i 29.97fps

(found on https://kodi.wiki/view/Samples)

This sample is more simple to decode but the interlacing still needs to be done. Either by the CPU or the GPU.

PlayerCPU %Memory UsageGPU 3DGPU DecodeGPU ProcessorSmooth PlaybackDeinterlaced
VLC2399301218yesyes
MPV7133700yesno
MPC-BE1306601230veryyes
MPC-HC1340401230yesyes
Movies & TV216201127yesyes
Kodi230773928yesyes
MPV Ctrl+H / D1160141135yesyes

The last column shouldn't be there, but by default MPV doesn't deinterlace the file. You have to press the d key to enable deinterlacing. The last line adds the performance of MPV with hardware decoding and deinterlacing on.

MPC-BE seems to double the original frame rate by default and interpolate between frames (soap opera effect). It may be good for sport but this is not a sport sample...

Movies & TV is impressing as it manages to display the content with 0% GPU 3D usage. It likely because they do all the processing in the Video Processing and nothing during display. That's an area we could improve in VLC.

Big Buck Bunny H264 1080p 30fps


This is the most common kind of file people are playing (apart from 720p files).

PlayerCPU %Memory UsageGPU 3DGPU DecodeGPU Processor
VLC1418990
MPV3148800
MPC-BE0280301013
MPC-HC1265301012
Movies & TV195090
Kodi223570625
MPV Ctrl+H01048100

As expected the CPU usage is negligeable. The DirectShow based players seems to take a lot of GPU to display this simple file. And Kodi even more, even though it's using less GPU to decode. Not sure why they need some GPU processing here, maybe color conversion which VLC does in the shader. That would explain the extra GPU processor for the 1080i sample as well.

Freedom '90 Music Video Outtakes VP9 1080p

(from YouTube)

If you watch a lot of YouTube there's a chance you might be decoding VP9 so I tested that as well. This is decoded by the GPU.

PlayerCPU %Memory UsageGPU 3DGPU DecodeGPU ProcessorPicture Quality
VLC1196860normal
MPV1100600normal
MPC-BE021540610macroblocks
MPC-HC218330013macroblocks
Movies & TV076167normal
Kodi328070525macroblocks
MPV Ctrl+H166660normal

In this case MPC-HC, MPC-BE and Kodi show noticeable macroblocks that the other players don't have.

LG 4K Tech Demo HEVC 60 fps 


A more regular 4K file that has no HDR, so should have less to do in the GPU.

PlayerCPU %Memory UsageGPU 3DGPU DecodeGPU ProcessorSmooth PlaybackRealtime
VLC4121518650yesyes
MPV346153000yesyes
MPC-BE2840704060nono
MPC-HC2765605065yesyes
Movies & TV129315700yesyes
Kodi3485604555yesyes
MPV Ctrl+H346933000yesyes

As with the HDR sample, MPC-BE can't play this file in realtime. The audio stops once in a while.

Despite the request to enable hardware decoding, MPV doesn't seem to be using it.

Movies & TV does an impressive job of using little memory.

Conclusion


VLC seems to be the overall best player with Movies & TV for all this content. The main drawback of VLC is currently the memory usage. It's possible to decrease it by using --avcodec-threads=1 but if you set this, you may have problems playing files your GPU can't decode.

We are working on this memory consumption which should be reduced in all cases for VLC 4.0.