What’s new in AVQT

♪ ♪ Ahmed Badr: Hello and welcome. My name is Ahmed and I am part of the Display and Color Technologies team at Apple. Today, I would like to talk about some of the new features and enhancements we made in the Advanced Video Quality Tool, or short AVQT. Let me start with a quick recap. So what's AVQT? AVQT is a command line executable that we first released in WWDC 2021. It's a full reference video quality metric that assesses the quality of videos with potential compression and scaling artifacts. It attempts to mimic how people rate the quality of videos. AVQT supports all AVFoundation-based video formats, both SDR and HDR. This includes HDR10, HLG, and Dolby Vision. There are three key attributes that are unique to AVQT. First, AVQT achieves high correlation with how people rate videos. This applies to all content types such as animation, natural scenes, and sports. Second, AVQT comes with impressive processing speeds, thanks to AVFoundation and Metal that AVQT relies on for video decoding and processing. Third, AVQT is designed to accommodate different viewing setups. The same video can result in different experiences depending on the viewing setup. This can be configured in AVQT through factors such as display resolution, display size, and viewing distance.

To learn more about AVQT, please check our 2021 video titled "Evaluate videos with Advanced Video Quality Tool." In the WWDC 2021 video where AVQT was announced, we presented the impressive processing speeds it achieves on M1 Macs. Since then, Apple processors got more capable and AVQT got faster. Here are the processing speeds on the new family of M1 processors. On M1 Ultra, AVQT evaluates a 2-hour, HEVC-compressed 4K movie in 20 minutes. This is six times faster than real time. For a full HD movie with a similar duration, AVQT processes the video in less than 11 minutes. This is more than ten times faster than real time. You can now evaluate more videos in the same amount of time.

Next, let me introduce the new features and enhancements that we added to AVQT this year. You can now visualize the quality of your video using AVQT without any extra steps. The latest version of AVQT produces HTML-based reports with interactive plots and diagrams showing analysis of the quality of your videos. These reports can easily be generated by adding the new ‘visualize’ flag.

You can easily identify sections in your video where quality did not meet the expectation. You can also share AVQT reports with other colleagues. They don't need to have AVQT or any other tool installed. They can just open the report in Safari.

So let me show you how to create an interactive report and how to use some of its features.

I will start by showing the help menu. We added a new argument, ‘visualize,’ which you can use to create reports. Here's an AVQT command that evaluates the quality of a test video that I created by compressing a segment from last year's WWDC presentation. You can then add the visualize argument and it will also create a report. Once it's done, you will see the location of the HTML report file printed to standard out. This file can simply be opened in Safari.

The table on the left shows information about the analyzed videos. The one on the right includes settings and parameters used while evaluating the videos.

This figure shows both frame-level and segment-level AVQT scores over time.

If you're interested in a specific interval, you can simply zoom in to get more details.

Hovering over the plot will give you more information about that point, such as time, AVQT score, and index of the frame or segment that this score belongs to.

To zoom out to the home view, you can double-click anywhere in the figure.

A pie chart is also included that shows the distribution of the AVQT frame scores of the analyzed video. It shows the percentage and number of frames in each of the five quality categories, "Bad," "Poor," "Fair," "Good," and "Excellent".

The figures on the bottom are similar to those on the top except that they show PSNR scores.

We hope this feature will help summarize the quality of your videos as well as identify issues, if any. Here's another feature that I am excited to tell you about. You can now specify the time windows from your reference and test videos to be evaluated. This enables focusing on a specific scene or multiple scenes in your video. It also allows for comparing videos that are not temporally aligned. We added four new command-line arguments to help you accurately specify the beginning and end of the video sections you are interested in analyzing.

Let me show you an example of how to use the time window feature.

Here's the video I evaluated in the previous demo. This time, I am interested in evaluating a single scene of the video. Here's how I can accomplish this. In Quicktime Player, I will start by seeking to that scene. It's this one. Clicking here shows the frame index. I will use the arrow keys to get to the first frame in the scene. It's 270. I will repeat the same steps to find the last frame in the scene. It's frame 486. Now, let's evaluate this scene in AVQT. The first part of the command is the same as before. I will then add the start and end frame indices I got from QuickTime Player using these four newly-added command line arguments.

I now have the AVQT score for this scene.

Running this command is faster than processing the whole video and then only looking at the scores of that scene. We believe this adds to the flexibility of AVQT. Speaking of flexibility, we also extended the support of raw YUV formats in this version. AVQT supports all AVFoundation video formats. Besides that, it also supports raw YUV videos. This enables scoring videos that were never compressed, such as a raw camera feed. It's also useful when evaluating videos that are compressed and decoded outside the ecosystem. This year, we extended the support for raw YUV to include a total of 20 formats with a wide range of chroma sub-samplings and bit depths. It's worth noting that `reference-fourcc` and `test-fourcc` flags are now deprecated. We replaced them with two separate sets of flags, one for chroma-subsampling and another for bit-depth.

This feature will allow you to evaluate more videos without the need of pre-processing.

Since AVQT was first released in WWDC 2021, there has been a growing interest in the tool. We received numerous comments and requests. One of the most common requests is to add support for Linux. Today, we are glad to announce that AVQT is now available on Linux.

We understand that this will allow people to evaluate content stored in the cloud or compressed in Linux-based servers without the need to move the videos. Together with the macOS version, we are also releasing a beta version of AVQT for Linux that supports a wide range of Linux distributions. It can be easily deployed as it doesn't require any external dependencies. It's essentially plug and play. To learn more about the supported Linux distributions and versions, please refer to the README file in the AVQT package.

The Linux version of AVQT is designed to have the same look and feel of the macOS version. It uses the same naming of the command line flags and produces output files of the same format. The Linux version supports all 20 raw formats that are included in the latest version of the macOS tool. The viewing conditions parameters are not available in the Linux beta version but will be enabled in the future.

To wrap up, here are a few important takeaways. AVQT is a perceptual video quality tool that we released in WWDC 2021. It's available for download on the developers’ portal. If you haven't tried the tool, go ahead and use it. This year, we added several enhancements to AVQT. You can now visualize AVQT scores of your videos in an interactive HTML-based report that you can share with your colleagues. You can also use the new ‘Time Window’ feature to focus on the quality of specific scenes in your video. And finally, for those of you who have videos in the cloud or use Linux-based video encoders, AVQT is now available on Linux. Thank you very much.