When can we expect attachCamera & attachAudio support for visionOS?

[Zigonja]

Hello!

Currently attachCamera & attachAudio is not available in visionOS as it's blocked out by the macros in the code. Is it possible to just enable them or is there more to it as to why they're excluded?

[shogo4405]

I've heard that visionOS doesn't support camera or microphone input. At least, during the visionOS beta, there wasn't any API like https://developer.apple.com/documentation/avfoundation/avcapturesession available. That's why it's blocked out with macros.

Looking at it now, it seems to be enabled in the API documentation for vision OS 1.0+. However, it's unclear if it's actually usable.

[shogo4405]

If AVAudioEngine is available, I've prepared an interface to send AVAudioBuffer output from AVAudioEngine like this.

• https://github.com/shogo4405/HaishinKit.swift/blob/main/Examples/iOS/AudioCapture.swift
• https://shogo4405.github.io/HaishinKit.swift/Classes/IOStream.html#/s:10HaishinKit8IOStreamC6append_4whenySo13AVAudioBufferC_So0F4TimeCtF

[devon3000]

I've tested this approach on VisionOS for RTMP Streaming and it works with 1.9.7, but I can't figure out to get it to work in 2.0.x because we need to use MediaMixer, which has a dependency on AVCaptureAudioDataOutput.

It would be very helpful if there was some fallback version of MediaMixer.attachAudio in 2.0.x which would use AVAudioEngine when compiling for visionOS, so we could get microphone audio on Vision Pro in 2.0.x implementations.

[shogo4405]

There is a similar method available in MediaMixer, and I intend for it to be used as well.
https://github.com/shogo4405/HaishinKit.swift/blob/main/HaishinKit/Sources/Mixer/MediaMixer.swift#L274-L281

Please try it.

[devon3000]

Thank you for the pointer! This was exactly what I needed.

I have published an example project that uses HaishinKit and the Apple Enterprise API to live-stream the Vision Pro front camera to YouTube.
https://github.com/devon3000/AVPEnterpriseAPI-CameraStreaming

Thank you very much @shogo4405 for your excellent library, this would have been much more difficult without it.

[shogo4405]

@devon3000
That’s great! Please make sure to register and introduce it in a new Discussion. It might be helpful for others and could lead to collaboration!

https://github.com/shogo4405/HaishinKit.swift/discussions/new?category=show-and-tell