Merge video and audio stream not working

[nono313]

I am trying to merge an audio stream and a video stream on a Raspberry Pi. Here is my config:

streams:
  picam:
    - "exec:rpicam-vid -t 0 --inline -o -#ffmpeg=v4l2,format=h264,preset=ultrafast,tune=zerolatency,g=15,b:v=3500k,flags=nobuffer,fps_mode=cfr#rtsp=8554/picam"
  audio:
    - "exec:arecord -D plughw:2 -f S16_LE -r 48000 -c 1 -#ffmpeg=format=s16le,ar=48000,ac=1,acodec=aac#rtsp=8554/audio" 
  picam_audio:
    - "rtsp://localhost:8554/picam#audio=rtsp://localhost:8554/audio#ffmpeg=acodec=aac"

webrtc:
  listen: :1984
  candidates:
    - stun:stun.l.google.com:19302

rtsp:
  listen: :8554

The picam_audio stream only contains video stream and no audio. ffprobe gives me this:

ffprobe rtsp://<IP>:8554/audio -rtsp_transport tcp
ffprobe version n7.1 Copyright (c) 2007-2024 the FFmpeg developers
  built with gcc 14.2.1 (GCC) 20240910
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-frei0r --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libdav1d --enable-libdrm --enable-libdvdnav --enable-libdvdread --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgsm --enable-libharfbuzz --enable-libiec61883 --enable-libjack --enable-libjxl --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libplacebo --enable-libpulse --enable-librav1e --enable-librsvg --enable-librubberband --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpl --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-nvdec --enable-nvenc --enable-opencl --enable-opengl --enable-shared --enable-vapoursynth --enable-version3 --enable-vulkan
  libavutil      59. 39.100 / 59. 39.100
  libavcodec     61. 19.100 / 61. 19.100
  libavformat    61.  7.100 / 61.  7.100
  libavdevice    61.  3.100 / 61.  3.100
  libavfilter    10.  4.100 / 10.  4.100
  libswscale      8.  3.100 /  8.  3.100
  libswresample   5.  3.100 /  5.  3.100
  libpostproc    58.  3.100 / 58.  3.100
Input #0, rtsp, from 'rtsp://<IP>:8554/audio':
  Metadata:
    title           : go2rtc/1.9.8
  Duration: N/A, start: 0.000000, bitrate: 768 kb/s
  Stream #0:0: Audio: pcm_s16be, 48000 Hz, mono, s16, 768 kb/s

Am I missing something ?

[AlexxIT]

I have no idea where you got that config format from:

rtsp://localhost:8554/picam#audio=rtsp://localhost:8554/audio#ffmpeg=acodec=aac

There is nothing similar in the documentation at all.

[nono313]

Sorry for that. I have to admit as I did not found anything on how to merge audio and video streams in the doc, I tried to ask an LLM which obviously was not a good idea.

EDIT: I managed to get both audio and video in one stream with this:

streams:
  picam:
    - "exec:rpicam-vid -t 0 --inline -o -#ffmpeg=v4l2,format=h264,preset=ultrafast,tune=zerolatency,g=15,b:v=3500k,flags=nobuffer,fps_mode=cfr"
    - "exec:arecord -D plughw:2 -f S16_LE -r 48000 -c 1 -#ffmpeg=format=s16le,acodec=pcm_s16le,ab=128k,ar=48000,ac=1,rtsp_transport=tcp" 

webrtc:
  listen: :1984
  candidates:
    - stun:stun.l.google.com:19302

rtsp:
  listen: :8554

With the web UI it's great and I get almost no latency. Now I need to figure out how to get to to frigate with restream (and another instance of go2rtc) as there the audio is not present there.