Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support for Spatial Audio on macOS Monterey #3444

Open
leolabs opened this issue Jul 9, 2021 · 45 comments · May be fixed by #4909
Open

Support for Spatial Audio on macOS Monterey #3444

leolabs opened this issue Jul 9, 2021 · 45 comments · May be fixed by #4909
Assignees
@low-batt
Labels
progress: pr ready type: feature request

Comments

@leolabs
Copy link

leolabs commented Jul 9, 2021

What you want IINA to do:

It would be awesome if IINA could passthrough surround / Dolby Atmos streams to connected AirPods so they can make use of the new spatial audio features in macOS Monterey.

What IINA does currently:

Currently, IINA downsamples surround streams to stereo when connected to AirPods.

Why you think this should be added:

It would enhance the movie watching experience.

Examples of other projects that have something similar:

Since this is a fairly new feature, I've only seen QuickTime, Safari, and their TV app support this so far.
@tbodt
Copy link

tbodt commented Nov 5, 2021

mpv-player/mpv#9252

@iGerman00
Copy link

Any news on this? I cannot find any app whatsoever that's not an Apple or Catalyst app that can even do spatial audio. Not even Safari can do it with Plex or Netflix, at best its spatialize stereo but not even that is available on most apps...

@noelboss
Copy link

This would be awesome!

@madhavtummala madhavtummala mentioned this issue Dec 26, 2021
Closed
@richtong
Copy link

richtong commented Jan 8, 2022 •
edited

Yes, please do, with the latest HDR branch #3539 which does HDR well, this is the last missing piece. Please :-) The list is small but growing https://www.macrumors.com/guide/apps-support-apples-spatial-audio-feature. And there are no MacOS apps yet (Air Video HD is iDevice only :-)

  • Air Video HD (Turn on Surround in Audio settings)
  • Apple's TV app
  • Netflix
  • Disney+
  • FE File Explorer (DTS 5.1 unsupported)
  • Foxtel Go (Australia)
  • HBO Max
  • Hulu
  • Plex (Enable old video player in Settings)
  • Peacock
  • Discovery+
  • Paramount+
  • Vudu
  • Some TIDAL songs

@mjbaldwin
Copy link

Just wanted to +1 this -- I had to stop using IINA and started using Infuse because it's the only Mac player that will output spatial audio on my M1 Mac with AirPods on my MKV files.

Would love it if IINA supported this.

@mjbaldwin
Copy link

It is just AC3/EAC3, since that's the only stream type Apple spatial audio supports.

It's not TrueHD or AC4. Those would require audio transcoding to EAC3 which doesn't seem to be a reasonable expectation for a video player right now. (If you really want spatial audio for those, you could use ffmpeg to transcode file audio to EAC3.)

@madhavtummala
Copy link

Infuse won me over, it supports spatial audio (even atmos output), can connect to my plex/jellyfin server and can also do trakt sync! And its the only M1 native app that can do this!

@dasti555
Copy link

Yes, please do, with the latest HDR branch #3539 which does HDR well, this is the last missing piece. Please :-) The list is small but growing https://www.macrumors.com/guide/apps-support-apples-spatial-audio-feature. And there are no MacOS apps yet (Air Video HD is iDevice only :-)

  • Air Video HD (Turn on Surround in Audio settings)
  • Apple's TV app
  • Netflix
  • Disney+
  • FE File Explorer (DTS 5.1 unsupported)
  • Foxtel Go (Australia)
  • HBO Max
  • Hulu
  • Plex (Enable old video player in Settings)
  • Peacock
  • Discovery+
  • Paramount+
  • Vudu
  • Some TIDAL songs

Optimus Player does this.

@byGits
Copy link

byGits commented Sep 7, 2022 •
edited

Yes, please do, with the latest HDR branch #3539 which does HDR well, this is the last missing piece. Please :-) The list is small but growing https://www.macrumors.com/guide/apps-support-apples-spatial-audio-feature. And there are no MacOS apps yet (Air Video HD is iDevice only :-)

Infuse Pro on App Store supports both Spatial Audio (head-tracked and fixed) and HDR.

But it is a paid app, $1/m and TBH not as good as IINA.

@dasti555
Copy link

dasti555 commented Sep 7, 2022

Look comment above. And it's a one-time purchase only.

@JulesLePrince JulesLePrince mentioned this issue Dec 27, 2022
Closed
@mjbaldwin
Copy link

Just wanted to check if there's been any progress towards this.

In the MPV issue it looks like another player, KSPlayer, has possibly added the relevant code to support spatial audio? See mpv-player/mpv#9252 (comment)

@uiryuu uiryuu mentioned this issue Mar 25, 2023
Closed
@CoderNoel CoderNoel mentioned this issue Apr 21, 2023
Closed
2 tasks
@brownknightrises23
Copy link

Any update on this?

@brownknightrises23
Copy link

mpv-player/mpv#11955

@Crear12
Copy link

Crear12 commented Feb 8, 2024

Please, Spatial Audio + Vision Pro playing videos with 6/8 channel audio would be next decade experience!

@godly-devotion
Copy link

Since AVKit (and QuickTime Player) supports Spatial Audio I decided to quickly patch together a super basic video player that supports HDR and Spatial Audio playback for multichannel audio as an interim solution for me.

https://github.com/godly-devotion/FrontRow

@kingslay
Copy link

kingslay commented Mar 5, 2024

Can use the TracyPlayer APP. It supports HDR and Spatial Audio.
https://apps.apple.com/us/app/tracyplayer/id6450770064

@koochey
Copy link

koochey commented Mar 14, 2024

+1

@JulesLePrince
Copy link

This commit adds AVFoundation support to mpv. I tested it and spatial audio is working well.

@ShlomoCode
Copy link

mpv now supports spatial audio!
mpv-player/mpv#9252 (comment)

@Rozzemak
Copy link

So,.. if I update mpv via brew to get the latest brew binaries and symlink them to IINA, it should be automagically working ?

Basically very similar to this:
(https://www.svp-team.com/wiki/SVP:IINA)

@ShlomoCode
Copy link

It is now included in official mpv version - https://github.com/mpv-player/mpv/releases/tag/v0.38.0
@low-batt do you know when there will be an iina version that includes the latest mpv version?

@low-batt
Copy link
Contributor

My first reaction is that I am disturbed at the release of 0.38.0 as the mpv issue I entered about audio on the Mac not working for apps that use libmpv and open more than one file (IINA issue #4789) is still open. If that is not fixed then IINA will have to patch mpv which I was hoping IINA would not have to do. I will have to test the release.

Glad all of you have been following the mpv progress and brought this to my attention. I've been too busy with IINA code lately to keep an eye on mpv changes.

I am a junior IINA developer so I can only speak to what I think the release plans are…

The 1.3.4 release was rushed in order to meet an Open Subtitles end of 2023 deadline. Quite a few fixes missed the boat. So the plan is to follow up with a 1.3.5 release. This is intended to be a stabilization release and therefore unlikely to contain major changes. If doable this release will restore support for macOS 10.11 and 10.12. For that reason IINA will not be upgrading libmpv as doing so would require IINA to drop support for macOS 10.11-10.14 as mpv has raised the minimum supported macOS to 10.15. There are a few PRs that still need to be reviewed and merged for 1.3.5. The hard work left is to get IINA working under macOS 10.11. I don't know the status of that work.

After that the plan is to start on the long delayed feature release. This release will include betas. The new plugin system will be enabled for this release. This will also include some fixes that are more risky to make (for example threading problems) that have been postponed for a release that includes betas. There is also a bunch of PRs ready for review that add small features. My own preference would be to put as much as possible into the first beta rather than dribble out features over a series of beta releases. This release will definitely include upgrading mpv and FFmpeg. So IINA will have to raise the minimum supported macOS version to 10.15 for this release.

As for when these releases will come out, I don't have a guess on that. Other than the 10.11 issue I don't see a lot of work to get 1.3.5 out besides the normal work needed to produce a release. I see a lot of work to get the outstanding PRs merged for the first beta. Upgrading dependencies is always "interesting". When that comes out depends upon decisions about what will be included in the feature release. That hasn't been nailed down yet. Need to first get pass 1.3.5.

@ruihe774
Copy link

My first reaction is that I am disturbed at the release of 0.38.0 as the mpv issue I entered about audio on the Mac not working for apps that use libmpv and open more than one file (IINA issue #4789) is still open. If that is not fixed then IINA will have to patch mpv which I was hoping IINA would not have to do. I will have to test the release.

We are discussing a fix for it now (mpv-player/mpv#13902 (comment)). I think we are going to fix it soon. Sad that the fix did not catch up with v0.38 release. You may do some backporting afterwards.

ao_avfoundation is a new ao that supports spatial audio. You may want to test it, and IMO you can make it the new default for IINA giving the lack of a maintainer of old ao_coreaudio.

@low-batt
Copy link
Contributor

I confirmed mpv-player/mpv#13348 still reproduces with the latest mpv master.

I did a very quick test with ao_avfoundation and did not encounter a problem.

low-batt added a commit that referenced this issue May 4, 2024
@low-batt
Add support for Spatial Audio, #3444
87abaff 
This commit will add setting the mpv ao property to avfoundation in the
mpvInit method of the MPVController class. This causes IINA to use the
AVFoundation framework for audio (which supports spatial audio) instead
of the Core Audio framework.
@low-batt low-batt linked a pull request May 4, 2024 that will close this issue
Draft
2 tasks
@low-batt low-batt self-assigned this May 4, 2024
@low-batt low-batt linked a pull request May 4, 2024 that will close this issue
Add support for Spatial Audio, #3444 #4909
Draft
2 tasks
@low-batt
Copy link
Contributor

low-batt commented May 4, 2024

The trivial changes in PR #4909 switches IINA to the new audio driver by setting the mpv ao option to avfoundation. The mpv team did all the work on this feature, IINA merely needed to enable the support.

Many "spatial audio" videos found on the net did not meet Apple's requirements. I found SpatialAudioDemo-Spatialized.mp4 from High Fidelity SPATIAL AUDIO DEMOS worked as seen in this test run:
spatial-audio

On this question from @Rozzemak:

So,.. if I update mpv via brew to get the latest brew binaries and symlink them to IINA, it should be automagically working ?

No. By default mpv will still use Core Audio. To switch to AVFoundation follow these instructions:

  • Start IINA
  • Click on Settings… under the IINA menu
  • The settings panel appears
  • On the left side of the panel click on Advanced
  • Slide the Enable advanced settings toggle button to be on (blue)
  • In the Additional mpv options section click on +
  • A new entry appears in the table
  • Double click on name and replace it with ao
  • Double click on value and replace it with avfoundation
  • Confirm the settings panel looks like the screenshot below
  • Restart IINA to activate the new settings
    coreaudo-settings

This sets the mpv ao option to avfoundation.

With mpv 0.38.0 you will encounter the problem in IINA issue #4789 if you open more than one window at a time. That problem is fixed if you build mpv from the master branch.

Many THANKS to the mpv team for this feature!

@ark-1
Copy link
Contributor

ark-1 commented May 4, 2024

Hi! Thanks for the update.
I have a question about this statement:

Many "spatial audio" videos found on the net did not meet Apple's requirements

Does it mean that the equivalent mp4 won't be spatialized by QuickTime too, and there's nothing to be done from the side of IINA or mpv? Does Infuse Pro handle these videos? (I can check myself if you give me links)

@ruihe774
Copy link

ruihe774 commented May 4, 2024 •
edited

Hi! Thanks for the update. I have a question about this statement:

Many "spatial audio" videos found on the net did not meet Apple's requirements

Does it mean that the equivalent mp4 won't be spatialized by QuickTime too, and there's nothing to be done from the side of IINA or mpv? Does Infuse Pro handle these videos? (I can check myself if you give me links)

@ark-1 If QuickTime Player can spatialize your media, ao_avfoundation can also. They use the same API. You can have a try. (An exception is the support of proprietary codecs like Dolby Atoms; it is another topic.)

@ShlomoCode
Copy link

ShlomoCode commented May 4, 2024 •
edited

Is it possible to build locally with mpv v0.38.0 using the download_libs.sh script? or only by building mpv manually

@low-batt
Copy link
Contributor

low-batt commented May 4, 2024

@ark-1 Good question. I should have explained more but was tired from the search for a usable test video.

As @ruihe774 indicated with the ao setting mpv is using the same lower level framework as QuickTime. So there should not be a problem with the low level code. To answer your question I have recreated some of my search…

I first found YouTube Spatial Audio Test Demo which seemed like exactly what I was looking for. I downloaded it using this yt-dlp command:

yt-dlp --limit-rate 256k -o '%(title)s.%(ext)s' https://www.youtube.com/watch?v=cCEK6XrcUx0

Started playing it with the modified version of IINA and… Not only no spatial audio, but no audio at all.

Clicking on Inspector under the Window does not list an audio driver:
driver-missing

Checking the file with MediaInfo shows a 4 channel audio track is available:

MediaInfo output: 
low-batt@gag issue-3444$ mediainfo YouTube\ Spatial\ Audio\ Test\ Demo.webm 
General
Complete name                            : YouTube Spatial Audio Test Demo.webm
Format                                   : WebM
Format version                           : Version 4
File size                                : 9.06 MiB
Duration                                 : 1 min 3 s
Overall bit rate                         : 1 196 kb/s
Frame rate                               : 29.970 FPS
Writing application                      : Lavf61.1.100
Writing library                          : Lavf61.1.100

Video
ID                                       : 1
Format                                   : VP9
Format profile                           : 0
Count of views                           : 2
Codec ID                                 : V_VP9
Duration                                 : 1 min 3 s
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 29.970 (30000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Language                                 : English
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

Audio
ID                                       : 2
Format                                   : Opus
Codec ID                                 : A_OPUS
Duration                                 : 1 min 3 s
Channel(s)                               : 4 channels
Sampling rate                            : 48.0 kHz
Bit depth                                : 32 bits
Compression mode                         : Lossy
Delay relative to video                  : 7 ms
Language                                 : English
Default                                  : Yes
Forced                                   : No


low-batt@gag issue-3444$

Checking the mpv log files shows many of these messages:

[   2.858][e][ad] Converting libavcodec frame to mpv frame failed.

I believe this is mpv issue mpv-player/mpv#13735. I encountered the same issue with 360 Video Spatial Audio Test and Ambisonic Audio Test + 360° Video Visualization of Spatial Audio [Headphones On!].

The video THX Spatial Audio Demo by American Meme (Listen over headphones) sort of works. Using yt-dlp to list the available formats YouTube provides shows the issue:

Available formats: 
low-batt@gag issue-3444$ yt-dlp --list-formats https://www.youtube.com/watch?v=r-ZqYdw7f14
[youtube] Extracting URL: https://www.youtube.com/watch?v=r-ZqYdw7f14
[youtube] r-ZqYdw7f14: Downloading webpage
[youtube] r-ZqYdw7f14: Downloading ios player API JSON
[youtube] r-ZqYdw7f14: Downloading android player API JSON
WARNING: [youtube] Skipping player responses from android clients (got player responses for video "aQvGIIdgFDM" instead of "r-ZqYdw7f14")
[youtube] r-ZqYdw7f14: Downloading m3u8 information
[info] Available formats for r-ZqYdw7f14:
ID      EXT   RESOLUTION FPS CH │   FILESIZE   TBR PROTO │ VCODEC          VBR ACODEC      ABR ASR MORE INFO
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
sb2     mhtml 48x27        2    │                  mhtml │ images                                  storyboard
sb1     mhtml 80x45        1    │                  mhtml │ images                                  storyboard
sb0     mhtml 160x90       1    │                  mhtml │ images                                  storyboard
233     mp4   audio only        │                  m3u8  │ audio only          unknown             Default
234     mp4   audio only        │                  m3u8  │ audio only          unknown             Default
139-drc m4a   audio only      2 │  387.04KiB   49k https │ audio only          mp4a.40.5   49k 22k low, DRC, m4a_dash
139     m4a   audio only      2 │  387.04KiB   49k https │ audio only          mp4a.40.5   49k 22k low, m4a_dash
249     webm  audio only      2 │  363.01KiB   46k https │ audio only          opus        46k 48k low, webm_dash
250     webm  audio only      2 │  503.37KiB   64k https │ audio only          opus        64k 48k low, webm_dash
140-drc m4a   audio only      2 │    1.00MiB  130k https │ audio only          mp4a.40.2  130k 44k medium, DRC, m4a_dash
140     m4a   audio only      2 │    1.00MiB  130k https │ audio only          mp4a.40.2  130k 44k medium, m4a_dash
251     webm  audio only      2 │  991.59KiB  125k https │ audio only          opus       125k 48k medium, webm_dash
394     mp4   256x144     30    │  474.46KiB   60k https │ av01.0.00M.08   60k video only          144p, mp4_dash
269     mp4   256x144     30    │ ~  1.33MiB  172k m3u8  │ avc1.4D400C    172k video only
160     mp4   256x144     30    │  593.24KiB   75k https │ avc1.4D400C     75k video only          144p, mp4_dash
603     mp4   256x144     30    │ ~  1.20MiB  155k m3u8  │ vp09.00.11.08  155k video only
278     webm  256x144     30    │  573.17KiB   73k https │ vp09.00.11.08   73k video only          144p, webm_dash
395     mp4   426x240     30    │  860.00KiB  109k https │ av01.0.00M.08  109k video only          240p, mp4_dash
229     mp4   426x240     30    │ ~  2.42MiB  312k m3u8  │ avc1.4D4015    312k video only
133     mp4   426x240     30    │    1.24MiB  161k https │ avc1.4D4015    161k video only          240p, mp4_dash
604     mp4   426x240     30    │ ~  2.24MiB  289k m3u8  │ vp09.00.20.08  289k video only
242     webm  426x240     30    │    1.08MiB  140k https │ vp09.00.20.08  140k video only          240p, webm_dash
396     mp4   640x360     30    │    1.67MiB  216k https │ av01.0.01M.08  216k video only          360p, mp4_dash
230     mp4   640x360     30    │ ~  5.65MiB  730k m3u8  │ avc1.4D401E    730k video only
134     mp4   640x360     30    │    2.63MiB  341k https │ avc1.4D401E    341k video only          360p, mp4_dash
18      mp4   640x360     30  2 │ ≈  3.63MiB  470k https │ avc1.42001E         mp4a.40.2       44k 360p
605     mp4   640x360     30    │ ~  4.39MiB  567k m3u8  │ vp09.00.21.08  567k video only
243     webm  640x360     30    │    2.05MiB  266k https │ vp09.00.21.08  266k video only          360p, webm_dash
397     mp4   854x480     30    │    2.94MiB  381k https │ av01.0.04M.08  381k video only          480p, mp4_dash
231     mp4   854x480     30    │ ~  9.86MiB 1273k m3u8  │ avc1.4D401F   1273k video only
135     mp4   854x480     30    │    4.97MiB  644k https │ avc1.4D401F    644k video only          480p, mp4_dash
606     mp4   854x480     30    │ ~  7.25MiB  936k m3u8  │ vp09.00.30.08  936k video only
244     webm  854x480     30    │    3.71MiB  480k https │ vp09.00.30.08  480k video only          480p, webm_dash
22      mp4   1280x720    30  2 │ ≈ 11.15MiB 1444k https │ avc1.64001F         mp4a.40.2       44k 720p
398     mp4   1280x720    30    │    5.68MiB  736k https │ av01.0.05M.08  736k video only          720p, mp4_dash
232     mp4   1280x720    30    │ ~ 19.28MiB 2489k m3u8  │ avc1.64001F   2489k video only
136     mp4   1280x720    30    │   10.15MiB 1316k https │ avc1.64001F   1316k video only          720p, mp4_dash
609     mp4   1280x720    30    │ ~ 13.33MiB 1720k m3u8  │ vp09.00.31.08 1720k video only
247     webm  1280x720    30    │    7.29MiB  945k https │ vp09.00.31.08  945k video only          720p, webm_dash
399     mp4   1920x1080   30    │   10.25MiB 1329k https │ av01.0.08M.08 1329k video only          1080p, mp4_dash
270     mp4   1920x1080   30    │ ~ 36.99MiB 4774k m3u8  │ avc1.640028   4774k video only
137     mp4   1920x1080   30    │   20.30MiB 2631k https │ avc1.640028   2631k video only          1080p, mp4_dash
614     mp4   1920x1080   30    │ ~ 22.47MiB 2900k m3u8  │ vp09.00.40.08 2900k video only
248     webm  1920x1080   30    │   12.88MiB 1670k https │ vp09.00.40.08 1670k video only          1080p, webm_dash
low-batt@gag issue-3444$

Nothing available except two channel audio. This sort of works because macOS supports stereo up-mix:
stereo

Many other videos I found also only had two channel audio. The video Airpods Pro Spatial audio | spatial audio test | airpods surround test is another stereo only example.

With more searching today I found AirPods Pro 2, Airpods 3 & AirPods Max Spatial Audio Test #3 | Dolby ATMOS 7.1 Core Universe. From the title there should be 8 audio channels, yes? Listing the available formats shows a choice between stereo and 6 channels:

Available formats: 
low-batt@gag issue-3444$ yt-dlp --list-formats https://www.youtube.com/watch?v=RH1jIP8UL-Y
[youtube] Extracting URL: https://www.youtube.com/watch?v=RH1jIP8UL-Y
[youtube] RH1jIP8UL-Y: Downloading webpage
[youtube] RH1jIP8UL-Y: Downloading ios player API JSON
[youtube] RH1jIP8UL-Y: Downloading android player API JSON
WARNING: [youtube] Skipping player responses from android clients (got player responses for video "aQvGIIdgFDM" instead of "RH1jIP8UL-Y")
[youtube] RH1jIP8UL-Y: Downloading m3u8 information
[info] Available formats for RH1jIP8UL-Y:
ID      EXT   RESOLUTION FPS HDR CH │   FILESIZE    TBR PROTO │ VCODEC           VBR ACODEC      ABR ASR MORE INFO
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
sb3     mhtml 48x27        1        │                   mhtml │ images                                   storyboard
sb2     mhtml 80x45        1        │                   mhtml │ images                                   storyboard
sb1     mhtml 160x90       1        │                   mhtml │ images                                   storyboard
sb0     mhtml 320x180      1        │                   mhtml │ images                                   storyboard
233     mp4   audio only            │                   m3u8  │ audio only           unknown             [en] Default
234     mp4   audio only            │                   m3u8  │ audio only           unknown             [en] Default
139-drc m4a   audio only          2 │    1.09MiB    49k https │ audio only           mp4a.40.5   49k 22k [en] low, DRC, m4a_dash
249-drc webm  audio only          2 │    1.07MiB    48k https │ audio only           opus        48k 48k [en] low, DRC, webm_dash
250-drc webm  audio only          2 │    1.40MiB    62k https │ audio only           opus        62k 48k [en] low, DRC, webm_dash
139     m4a   audio only          2 │    1.09MiB    49k https │ audio only           mp4a.40.5   49k 22k [en] low, m4a_dash
249     webm  audio only          2 │    1.06MiB    47k https │ audio only           opus        47k 48k [en] low, webm_dash
250     webm  audio only          2 │    1.40MiB    62k https │ audio only           opus        62k 48k [en] low, webm_dash
256     m4a   audio only          6 │    4.37MiB   195k https │ audio only           mp4a.40.5  195k 24k [en] low, m4a_dash
140-drc m4a   audio only          2 │    2.90MiB   130k https │ audio only           mp4a.40.2  130k 44k [en] medium, DRC, m4a_dash
251-drc webm  audio only          2 │    2.73MiB   122k https │ audio only           opus       122k 48k [en] medium, DRC, webm_dash
140     m4a   audio only          2 │    2.90MiB   130k https │ audio only           mp4a.40.2  130k 44k [en] medium, m4a_dash
251     webm  audio only          2 │    2.74MiB   122k https │ audio only           opus       122k 48k [en] medium, webm_dash
380     m4a   audio only          6 │    8.61MiB   384k https │ audio only           ac-3       384k 48k [en] high, m4a_dash
328     m4a   audio only          6 │    8.61MiB   384k https │ audio only           ec-3       384k 48k [en] high, m4a_dash
258     m4a   audio only          6 │    8.69MiB   388k https │ audio only           mp4a.40.2  388k 48k [en] high, m4a_dash
602     mp4   256x144     12        │ ~  3.42MiB   152k m3u8  │ vp09.00.10.08   152k video only
269     mp4   256x144     24        │ ~  3.79MiB   169k m3u8  │ avc1.4D400C     169k video only
160     mp4   256x144     24        │    1.14MiB    51k https │ avc1.4D400C      51k video only          144p, mp4_dash
603     mp4   256x144     24        │ ~  6.08MiB   271k m3u8  │ vp09.00.11.08   271k video only
278     webm  256x144     24        │    1.27MiB    57k https │ vp09.00.11.08    57k video only          144p, webm_dash
631     mp4   256x144     24 10     │ ~  4.32MiB   193k m3u8  │ vp09.02.11.10   193k video only
330     webm  256x144     24 10     │    1.55MiB    69k https │ vp09.02.11.10    69k video only          144p HDR, webm_dash
229     mp4   426x240     24        │ ~  7.04MiB   314k m3u8  │ avc1.4D4015     314k video only
133     mp4   426x240     24        │    2.53MiB   113k https │ avc1.4D4015     113k video only          240p, mp4_dash
604     mp4   426x240     24        │ ~ 13.86MiB   619k m3u8  │ vp09.00.20.08   619k video only
242     webm  426x240     24        │    2.02MiB    90k https │ vp09.00.20.08    90k video only          240p, webm_dash
632     mp4   426x240     24 10     │ ~  6.51MiB   290k m3u8  │ vp09.02.21.10   290k video only
331     webm  426x240     24 10     │    2.70MiB   120k https │ vp09.02.21.10   120k video only          240p HDR, webm_dash
230     mp4   640x360     24        │ ~ 17.97MiB   802k m3u8  │ avc1.4D401E     802k video only
134     mp4   640x360     24        │    4.92MiB   219k https │ avc1.4D401E     219k video only          360p, mp4_dash
18      mp4   640x360     24      2 │ ≈  7.81MiB   348k https │ avc1.42001E          mp4a.40.2       44k [en] 360p
605     mp4   640x360     24        │ ~ 28.57MiB  1275k m3u8  │ vp09.00.21.08  1275k video only
243     webm  640x360     24        │    3.69MiB   165k https │ vp09.00.21.08   165k video only          360p, webm_dash
633     mp4   640x360     24 10     │ ~ 14.45MiB   645k m3u8  │ vp09.02.30.10   645k video only
332     webm  640x360     24 10     │    5.93MiB   265k https │ vp09.02.30.10   265k video only          360p HDR, webm_dash
231     mp4   854x480     24        │ ~ 30.39MiB  1356k m3u8  │ avc1.4D401E    1356k video only
135     mp4   854x480     24        │    9.94MiB   444k https │ avc1.4D401E     444k video only          480p, mp4_dash
606     mp4   854x480     24        │ ~ 47.57MiB  2122k m3u8  │ vp09.00.30.08  2122k video only
244     webm  854x480     24        │    6.47MiB   289k https │ vp09.00.30.08   289k video only          480p, webm_dash
634     mp4   854x480     24 10     │ ~ 24.47MiB  1092k m3u8  │ vp09.02.31.10  1092k video only
333     webm  854x480     24 10     │   11.19MiB   500k https │ vp09.02.31.10   500k video only          480p HDR, webm_dash
22      mp4   1280x720    24      2 │ ≈ 22.43MiB  1001k https │ avc1.64001F          mp4a.40.2       44k [en] 720p
232     mp4   1280x720    24        │ ~ 57.66MiB  2573k m3u8  │ avc1.4D401F    2573k video only
136     mp4   1280x720    24        │   19.54MiB   872k https │ avc1.4D401F     872k video only          720p, mp4_dash
609     mp4   1280x720    24        │ ~101.76MiB  4540k m3u8  │ vp09.00.31.08  4540k video only
247     webm  1280x720    24        │   13.28MiB   593k https │ vp09.00.31.08   593k video only          720p, webm_dash
635     mp4   1280x720    24 10     │ ~ 51.28MiB  2288k m3u8  │ vp09.02.40.10  2288k video only
334     webm  1280x720    24 10     │   26.61MiB  1188k https │ vp09.02.40.10  1188k video only          720p HDR, webm_dash
270     mp4   1920x1080   24        │ ~109.17MiB  4871k m3u8  │ avc1.640028    4871k video only
137     mp4   1920x1080   24        │   45.06MiB  2011k https │ avc1.640028    2011k video only          1080p, mp4_dash
614     mp4   1920x1080   24        │ ~143.56MiB  6406k m3u8  │ vp09.00.40.08  6406k video only
248     webm  1920x1080   24        │   22.21MiB   991k https │ vp09.00.40.08   991k video only          1080p, webm_dash
636     mp4   1920x1080   24 10     │ ~ 88.10MiB  3931k m3u8  │ vp09.02.41.10  3931k video only
335     webm  1920x1080   24 10     │   45.98MiB  2052k https │ vp09.02.41.10  2052k video only          1080p HDR, webm_dash
620     mp4   2560x1440   24        │ ~292.57MiB 13054k m3u8  │ vp09.00.50.08 13054k video only
271     webm  2560x1440   24        │   65.28MiB  2914k https │ vp09.00.50.08  2914k video only          1440p, webm_dash
639     mp4   2560x1440   24 10     │ ~247.28MiB 11033k m3u8  │ vp09.02.50.10 11033k video only
336     webm  2560x1440   24 10     │  142.59MiB  6364k https │ vp09.02.50.10  6364k video only          1440p HDR, webm_dash
625     mp4   3840x2160   24        │ ~573.65MiB 25597k m3u8  │ vp09.00.50.08 25597k video only
313     webm  3840x2160   24        │  126.76MiB  5658k https │ vp09.00.50.08  5658k video only          2160p, webm_dash
642     mp4   3840x2160   24 10     │ ~551.89MiB 24625k m3u8  │ vp09.02.51.10 24625k video only
337     webm  3840x2160   24 10     │  350.91MiB 15663k https │ vp09.02.51.10 15663k video only          2160p HDR, webm_dash
low-batt@gag issue-3444$

Checking with MediaInfo shows yt-dlp picked one of the 6 channel tracks:

MediaInfo output: 
low-batt@gag issue-3444$ mediainfo AirPods\ Pro\ 2\,\ Airpods\ 3\ \&\ AirPods\ Max\ Spatial\ Audio\ Test\ #3\ ｜\ Dolby\ ATMOS\ 7.1\ Core\ Universe.mkv 
General
Unique ID                                : 92842067832525757765328317720064109193 (0x45D8BB5AC46E671882E7226887D72A89)
Complete name                            : AirPods Pro 2, Airpods 3 & AirPods Max Spatial Audio Test #3 ｜ Dolby ATMOS 7.1 Core Universe.mkv
Format                                   : Matroska
Format version                           : Version 4
File size                                : 360 MiB
Duration                                 : 3 min 7 s
Overall bit rate                         : 16.0 Mb/s
Frame rate                               : 23.976 FPS
Writing application                      : Lavf61.1.100
Writing library                          : Lavf61.1.100
ErrorDetectionType                       : Per level 1

Video
ID                                       : 1
Format                                   : VP9
Format profile                           : 2
HDR format                               : SMPTE ST 2086, HDR10 compatible
Codec ID                                 : V_VP9
Duration                                 : 3 min 7 s
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 23.976 (24000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 10 bits
Language                                 : English
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.2020
Transfer characteristics                 : PQ
Matrix coefficients                      : BT.2020 non-constant
Mastering display color primaries        : Display P3
Mastering display luminance              : min: 0.0050 cd/m2, max: 1000 cd/m2

Audio
ID                                       : 2
Format                                   : AAC LC
Format/Info                              : Advanced Audio Codec Low Complexity
Codec ID                                 : A_AAC-2
Duration                                 : 3 min 7 s
Channel(s)                               : 6 channels
Channel layout                           : C L R Ls Rs LFE
Sampling rate                            : 48.0 kHz
Frame rate                               : 46.875 FPS (1024 SPF)
Compression mode                         : Lossy
Title                                    : ISO Media file produced by Google Inc.
Default                                  : Yes
Forced                                   : No
VENDOR_ID                                : [0][0][0][0]


low-batt@gag issue-3444$

A little concerning that IINA lists the channels as unknown6:
unknown6

Using the mpv IPC interface we can see that is coming from mpv track-list property:

mpv IPC output: 
low-batt@gag issue-3444$ echo '{ "command": ["get_property", "track-list"] }' | socat - /tmp/mpv-socket | jq
{
  "data": [
    {
      "id": 1,
      "type": "video",
      "src-id": 1,
      "lang": "eng",
      "image": false,
      "albumart": false,
      "default": true,
      "forced": false,
      "dependent": false,
      "visual-impaired": false,
      "hearing-impaired": false,
      "external": false,
      "selected": true,
      "main-selection": 0,
      "ff-index": 0,
      "decoder": "vp9",
      "decoder-desc": "Google VP9",
      "codec": "vp9",
      "codec-desc": "Google VP9",
      "codec-profile": "Profile 2",
      "demux-w": 3840,
      "demux-h": 2160,
      "demux-fps": 23.976024,
      "demux-par": 1.000000
    },
    {
      "id": 1,
      "type": "audio",
      "src-id": 2,
      "audio-channels": 6,
      "image": false,
      "albumart": false,
      "default": true,
      "forced": false,
      "dependent": false,
      "visual-impaired": false,
      "hearing-impaired": false,
      "external": false,
      "selected": true,
      "main-selection": 0,
      "ff-index": 1,
      "decoder": "aac",
      "decoder-desc": "AAC (Advanced Audio Coding)",
      "codec": "aac",
      "codec-desc": "AAC (Advanced Audio Coding)",
      "codec-profile": "LC",
      "demux-channel-count": 6,
      "demux-channels": "unknown6",
      "demux-samplerate": 48000
    }
  ],
  "request_id": 0,
  "error": "success"
}
low-batt@gag issue-3444$

The General tab reports 5.1 for channels:
general-tab

IINA obtains that information from the mpv audio-params property:

low-batt@gag issue-3444$ echo '{ "command": ["get_property", "audio-params"] }' | socat - /tmp/mpv-socket | jq
{
  "data": {
    "samplerate": 48000,
    "channel-count": 6,
    "channels": "5.1",
    "hr-channels": "5.1",
    "format": "floatp"
  },
  "request_id": 0,
  "error": "success"
}
low-batt@gag issue-3444$

The good news is that playing this YouTube video did enable spatial audio:
worked

Given that, the next test in the series would also be good, yes? Checking AirPods Pro 2, Airpods 3 & AirPods Max Spatial Audio Test #4 | Dolby ATMOS 7.1 Element:

Available formats: 
low-batt@gag issue-3444$ yt-dlp --list-formats https://www.youtube.com/watch?v=6N1xKSYUcak
[youtube] Extracting URL: https://www.youtube.com/watch?v=6N1xKSYUcak
[youtube] 6N1xKSYUcak: Downloading webpage
[youtube] 6N1xKSYUcak: Downloading ios player API JSON
[youtube] 6N1xKSYUcak: Downloading android player API JSON
WARNING: [youtube] Skipping player responses from android clients (got player responses for video "aQvGIIdgFDM" instead of "6N1xKSYUcak")
[youtube] 6N1xKSYUcak: Downloading m3u8 information
[info] Available formats for 6N1xKSYUcak:
ID      EXT   RESOLUTION FPS CH │   FILESIZE    TBR PROTO │ VCODEC           VBR ACODEC      ABR ASR MORE INFO
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
sb2     mhtml 48x27        1    │                   mhtml │ images                                   storyboard
sb1     mhtml 80x45        1    │                   mhtml │ images                                   storyboard
sb0     mhtml 160x90       1    │                   mhtml │ images                                   storyboard
233     mp4   audio only        │                   m3u8  │ audio only           unknown             Default
234     mp4   audio only        │                   m3u8  │ audio only           unknown             Default
139-drc m4a   audio only      2 │  666.55KiB    49k https │ audio only           mp4a.40.5   49k 22k low, DRC, m4a_dash
139     m4a   audio only      2 │  666.65KiB    49k https │ audio only           mp4a.40.5   49k 22k low, m4a_dash
249     webm  audio only      2 │  664.55KiB    49k https │ audio only           opus        49k 48k low, webm_dash
250     webm  audio only      2 │  877.22KiB    64k https │ audio only           opus        64k 48k low, webm_dash
140-drc m4a   audio only      2 │    1.72MiB   130k https │ audio only           mp4a.40.2  130k 44k medium, DRC, m4a_dash
140     m4a   audio only      2 │    1.72MiB   130k https │ audio only           mp4a.40.2  130k 44k medium, m4a_dash
251     webm  audio only      2 │    1.68MiB   126k https │ audio only           opus       126k 48k medium, webm_dash
602     mp4   256x144     15    │ ~  1.41MiB   105k m3u8  │ vp09.00.10.08   105k video only
269     mp4   256x144     30    │ ~  2.70MiB   202k m3u8  │ avc1.4D400C     202k video only
160     mp4   256x144     30    │    1.07MiB    81k https │ avc1.4D400C      81k video only          144p, mp4_dash
603     mp4   256x144     30    │ ~  2.24MiB   167k m3u8  │ vp09.00.11.08   167k video only
278     webm  256x144     30    │  902.60KiB    66k https │ vp09.00.11.08    66k video only          144p, webm_dash
229     mp4   426x240     30    │ ~  4.43MiB   332k m3u8  │ avc1.4D4015     332k video only
133     mp4   426x240     30    │    2.34MiB   176k https │ avc1.4D4015     176k video only          240p, mp4_dash
604     mp4   426x240     30    │ ~  4.04MiB   303k m3u8  │ vp09.00.20.08   303k video only
242     webm  426x240     30    │    1.54MiB   116k https │ vp09.00.20.08   116k video only          240p, webm_dash
230     mp4   640x360     30    │ ~  9.84MiB   737k m3u8  │ avc1.4D401E     737k video only
134     mp4   640x360     30    │    4.88MiB   367k https │ avc1.4D401E     367k video only          360p, mp4_dash
18      mp4   640x360     30  2 │ ≈  6.59MiB   495k https │ avc1.42001E          mp4a.40.2       44k 360p
605     mp4   640x360     30    │ ~  7.79MiB   583k m3u8  │ vp09.00.21.08   583k video only
243     webm  640x360     30    │    2.60MiB   196k https │ vp09.00.21.08   196k video only          360p, webm_dash
231     mp4   854x480     30    │ ~ 17.37MiB  1301k m3u8  │ avc1.4D401F    1301k video only
135     mp4   854x480     30    │    9.12MiB   686k https │ avc1.4D401F     686k video only          480p, mp4_dash
606     mp4   854x480     30    │ ~ 13.43MiB  1006k m3u8  │ vp09.00.30.08  1006k video only
244     webm  854x480     30    │    4.44MiB   334k https │ vp09.00.30.08   334k video only          480p, webm_dash
22      mp4   1280x720    30  2 │ ≈ 19.17MiB  1440k https │ avc1.64001F          mp4a.40.2       44k 720p
136     mp4   1280x720    30    │   17.45MiB  1312k https │ avc1.64001f    1312k video only          720p, mp4_dash
247     webm  1280x720    30    │    7.85MiB   590k https │ vp9             590k video only          720p, webm_dash
311     mp4   1280x720    60    │ ~ 53.43MiB  4002k m3u8  │ avc1.640020    4002k video only
298     mp4   1280x720    60    │   23.53MiB  1768k https │ avc1.640020    1768k video only          720p60, mp4_dash
612     mp4   1280x720    60    │ ~ 38.89MiB  2913k m3u8  │ vp09.00.40.08  2913k video only
302     webm  1280x720    60    │   11.76MiB   884k https │ vp09.00.40.08   884k video only          720p60, webm_dash
312     mp4   1920x1080   60    │ ~ 90.44MiB  6774k m3u8  │ avc1.64002A    6774k video only
299     mp4   1920x1080   60    │   40.66MiB  3056k https │ avc1.64002A    3056k video only          1080p60, mp4_dash
617     mp4   1920x1080   60    │ ~ 59.74MiB  4475k m3u8  │ vp09.00.41.08  4475k video only
303     webm  1920x1080   60    │   16.85MiB  1266k https │ vp09.00.41.08  1266k video only          1080p60, webm_dash
623     mp4   2560x1440   60    │ ~149.21MiB 11175k m3u8  │ vp09.00.50.08 11175k video only
308     webm  2560x1440   60    │   57.66MiB  4333k https │ vp09.00.50.08  4333k video only          1440p60, webm_dash
628     mp4   3840x2160   60    │ ~242.84MiB 18188k m3u8  │ vp09.00.51.08 18188k video only
315     webm  3840x2160   60    │  100.34MiB  7541k https │ vp09.00.51.08  7541k video only          2160p60, webm_dash
low-batt@gag issue-3444$

Stereo only.

I did find AirPods Pro 2 & Airpods Max Spatial Audio Test | Dolby Atmos which is also 6 channels.

Thus, you can't trust the title, you need to check the encoding and expect lots of the time to find the test file only has two channels.

@low-batt
Copy link
Contributor

low-batt commented May 4, 2024

On this question:

Is it possible to build locally with mpv v0.38.0 using the download_libs.sh script?

Usually download_libs.sh will provide the libraries used in the latest release of IINA. Normally these libraries are only rebuilt close to a release. Currently you will get the old version of mpv used with IINA 1.3.4.

The procedure used by IINA to build the libraries described in Building mpv manually can be painful to get working. Painful enough that I no longer do it that way for my own testing.

The symbolic linking mentioned by @Rozzemak in this comment above might be the easiest way, but the official mpv 0.38.0 release will have the restriction about opening multiple windows due to issue #4789.

For this testing I built mpv using mpv-build configured with these option files (.txt extension added to make GitHub happy):
ffmpeg_options.txt
mpv_options.txt

Note that these option files change the build to generate libmpv, not the mpv application.

Getting your Mac setup to build mpv and FFmpeg will require iterating through build failures and installing missing libraries using brew. Once you have gotten past the initial pain and successfully created a build then mpv-build is easy to use. Usually.

@ruihe774
Copy link

ruihe774 commented May 4, 2024 •
edited

I should have explained more but was tired from the search for a usable test video.

I am using this from Apple with stereo, 5.1 and 7.1.

@ShlomoCode
Copy link

ShlomoCode commented May 4, 2024 •
edited

@low-batt After installing several dependencies using brew (brew install autoconf automake libtool), the build goes without errors (but with warnings), but I don't find any .dylib files created similar to what was created with the download_libs script
Do you understand my mistake?
This is the full output
And this is the list of created files

Image

CleanShot 2024-05-05 at 02 14 36@2x

@low-batt
Copy link
Contributor

low-batt commented May 5, 2024

@ruihe774 Thank you very much for the link to the Apple examples. I had searched for Apple examples but ended up getting lost in Apple Music.

@ShlomoCode Did you strip the .txt off the end of the two configuration files I posted and use them in your mpv-build build?

The library should end up in a mpv/build subdirectory as seen here:

low-batt@gag mpv-build (master %=)$ ls -la mpv/build/libmpv.2.dylib
-rwxr-xr-x  1 low-batt  staff  30528048 Apr 19 19:46 mpv/build/libmpv.2.dylib
low-batt@gag mpv-build (master %=)$

The mpv_options file sets libmpv to true:

-Dbuildtype=debug
-Dlibmpv=true
-Db_asneeded=false
-Dc_link_args=-fuse-ld=/usr/bin/ld
-Dcplayer=false

Without that the mpv app will be built instead.

@ShlomoCode
Copy link

ShlomoCode commented May 5, 2024 •
edited

mpv/build/libmpv.2.dylib

Thanks! That's what I was missing. I just couldn't locate the file...
now I copied and replaced the files from mpv-build/mpv/build to iina/deps/lib and built, and I get Spatial Audio with iina! Wow this is wonderful 💫

@ShlomoCode
Copy link

ShlomoCode commented May 5, 2024 •
edited

Update: I changed in Build Settings > Architectures to "arm64" only and it solved the error for me.
@low-batt Running Xcode development (▶️ button) works and I can run Spatial Audio, but when I try to generate an .app file to put in my Applications (Product > Archive in Xcode), I get an "Undefined symbols" build error:
CleanShot 2024-05-05 at 04 33 03@2x

@low-batt
Copy link
Contributor

low-batt commented May 5, 2024

Glad to hear your build worked! And yes mpv-build does not generate a universal library like the libmpv provided by download_libs.sh. The mpv-build generated library is only usable for the architecture it was built under.

@low-batt
Copy link
Contributor

low-batt commented May 5, 2024

To get spatial audio to work you set ao on the Advanced tab of IINA's settings as I mentioned above, yes?

@ShlomoCode
Copy link

ShlomoCode commented May 5, 2024 •
edited

Yes, Or the change presented at #4909. Both worked for me (with Airpods Pro 2).

@dasti555
Copy link

dasti555 commented May 5, 2024

Heh, so when OTA for non-dev people? 😁

@low-batt low-batt mentioned this issue May 5, 2024
Closed
@low-batt
Copy link
Contributor

low-batt commented May 5, 2024

On this:

Heh, so when OTA for non-dev people?

See this comment above on what I think the release plans are. I'm a junior IINA developer and can't comment on when the releases might happen. I can only say that work is ongoing.

@Sponge-bink
Copy link

I was able to make IINA output Spacial Audio by temporarily replacing mpv that came with IINA with a version from Homebrew. Here are the steps for someone who needs them.

Backup the original mpv that came with IINA:

cp /Applications/IINA.app/Contents/Frameworks/libmpv.2.dylib /Applications/IINA.app/Contents/Frameworks/libmpv_back.2.dylib

Then delete the original mpv that came with IINA:

rm -f /Applications/IINA.app/Contents/Frameworks/libmpv.2.dylib

Install mpv if you haven't:

brew update && brew install mpv

Create a symbolic link to the Homebrew mpv (0.38.0 as of now) where the original mpv was:

ln -s /opt/homebrew/Cellar/mpv/0.38.0/lib/libmpv.2.dylib /Applications/IINA.app/Contents/Frameworks/libmpv.2.dylib

Then perform the action demonstrated by @low-batt. Now IINA should be capable of playing in Spacial Audio.

Screenshot 2024-05-06 at 3 42 12 PM

And if you want to undo it:

Remove the symbolic link you created:

rm -f /Applications/IINA.app/Contents/Frameworks/libmpv.2.dylib

Recover from the backup:

cp /Applications/IINA.app/Contents/Frameworks/libmpv_back.2.dylib /Applications/IINA.app/Contents/Frameworks/libmpv.2.dylib

You can leave the backup there or delete it:

rm -f /Applications/IINA.app/Contents/Frameworks/libmpv_back.2.dylib

At last, don't forget to undo the process of setting the ao option of mpv to avfoundation. Otherwise there will be no audio.

@ShlomoCode
Copy link

ShlomoCode commented May 7, 2024 •
edited

@low-batt I found a bug (seemingly) that only occurs when ao is set to avfoundation: when you drag a file into iina, it doesn't start playing by itself. The playback position must be changed and only then the video starts playing.
A detailed demo video is attached with:

  1. The latest iina code from develop (73c9608)
  2. Updated mpv dylb from mpn-build
CleanShot.2024-05-08.at.01.45.49-converted.mp4

@ShlomoCode
Copy link

In addition, when Spatialize Stereo is turned on/off, there is a skip of about a second in the audio. With QuickTime it doesn't happen:

Screen.Recording.2024-05-08.at.2.10.39-converted.2.mov

This was referenced May 8, 2024
Open
Open
@low-batt
Copy link
Contributor

low-batt commented May 8, 2024

I've created issues #4920 and #4921 for the problems. Let's continue the discussion in those issues.

@gingerbeardman
Copy link

Thanks @Sponge-bink

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@low-batt low-batt

Labels
progress: pr ready type: feature request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add support for Spatial Audio, #3444 iina/iina