RTSP streams stops when using olympe take_photo

[Rz-Rz]

I am currently working on a project using a Jetson AGX Orin with JetPack 6, a Skycontroller 4 Black, and an Anafi USA. The goal is to keep the RTSP stream running while I trigger a still photo from an Olympe Python script.

I am able to play the live stream with GStreamer without issues with the following prompt :

gst-launch-1.0 -v \
  rtspsrc location="rtsp://192.168.53.1/live" ! \
  rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! \
  autovideosink sync=false

But as soon as I take a photo using Olympe, the GStreamer pipeline stops with EOS and exits. I am using the following code:

import olympe
from olympe.messages.camera import set_camera_mode, set_photo_mode, take_photo, photo_progress

SC_IP = "192.168.53.1"
CAM_ID = 0

drone = olympe.Drone(SC_IP)
drone.connect()

# (also tried without these two calls once the drone is preconfigured)
drone(set_camera_mode(cam_id=CAM_ID, value="photo")).wait().success()
drone(set_photo_mode(cam_id=CAM_ID, mode="single",
                     format="rectilinear", file_format="jpeg")).wait().success()

waiter = drone(photo_progress(result="photo_saved", _policy="wait"))
assert drone(take_photo(cam_id=CAM_ID)).wait().success()
assert waiter.wait(_timeout=20).success()

And the console shows:

Got EOS from element pipeline0
Execution ended after 0:00:09.xxxxxx
Setting pipeline to NULL
Freeing pipeline

I have tried removing the set_camera_mode and set_photo_mode commands (since the drone is already pre-configured in photo mode), but the issue still persists.

Question:

Is it expected for the RTSP stream to stop after a still capture on Anafi USA?
If not, is there a recommended way to keep the live stream alive while taking photos?

Solution I am exploring :
Using pdraw backend to take a screenshot while demuxing the stream.

[Rz-Rz]

[UPDATE]
It turns out what I am seeing is basically the same behavior described here:

RTSP stream and photo/video capture in parallel seems to be not possible

Hello, Here is what happens with the video stream on ANAFI 4K, Thermal and USA products when camera commands are issued: The video stream is encoded in H.264 and payloaded in RTP/AVP over UDP. RTSP over TCP is user for session management. See the capture below where a stream is established, then a video to photo mode switch occurs, then 2 photos are taken. You can observe this with Wireshark for example. The RTSP session (“5251864a3d82f97e” in the example below) is never stopped once establi…

When I switch the drone to photo mode and trigger a take_photo with Olympe, the RTSP stream drops because the media pipeline is reconfigured on the drone side. From what I understand, the SSRC (and possibly some stream parameters) change when switching to photo mode, which effectively kills the existing RTSP session.

I also tried grabbing frames via pdraw instead of gstreamer (hoping it would be more tolerant to this change), but the result is the same: as soon as I switch to photo mode, the stream stops.

I came across the PhotoStreamingMode consideration in the docs:

In interrupt mode (default), the live stream is temporarily interrupted when a photo is taken.

In continuous mode, the drone tries to keep streaming while capturing photos.

Right now I am looking into that direction to see whether using / forcing a continuous PhotoStreamingMode could avoid the RTSP drop or at least make the transition smoother. If I manage to get a stable setup (RTSP + photo capture without the stream dying), I’ll share the details here.

I am also looking if rtspsrc in GStreamer could follow seamlessly SSRC changes throughout the rtsp lifecycle with mode changes on the drone via Olympe.