Access Video Stream via Olympe Anafi

I’m trying to get the video stream via olympe on an anafi drone with the following code example.

The following error messages are repeatedly issued when the code is executed:
Olympe streaming example output dir: /tmp/olympe_streaming_test_k7lclwjf
Takeoff if necessary…
2021-11-10 16:43:00,891 [ERROR] ulog - pdraw_sink_coded_video - VideoDecoder#3: coded video media format H264/AVCC not supported
2021-11-10 16:43:00,925 [ERROR] ulog - vmeta - vmeta_frame_proto_get_buffer:408: err=71(Protocol error)
2021-11-10 16:43:00,926 [ERROR] olympe.mp4mux - add_coded_frame - vmeta_frame_proto_get_buffer returned -71
2021-11-10 16:43:00,929 [ERROR] ulog - vmeta - vmeta_frame_proto_get_buffer:408: err=71(Protocol error)
2021-11-10 16:43:00,929 [ERROR] olympe.mp4mux - add_coded_frame - vmeta_frame_proto_get_buffer returned -71

I would be grateful for help.

@DHBW
Are you trying to run the streaming.py example unmodified? We had issues early on with streaming because we were trying to run a modified example where the drone didn’t take off (as we were in a lab setting). It appears that the drone has to actually take off and get to a hover state before the stream will start working. Once we tried the unmodified streaming.py example in the field, it worked just fine after takeoff and we were able to view the frames.

Yes, we (me and a fellow student) are trying to get the unmodified version of streaming.py running. This is only for testing purposes, our real goal is to realize an object recognition together with OpenCv. What we have done so far is that we first installed Ubuntu (20.04.3) and Python (3.8.10). Then we upgraded pip to version 21.3.1 and installed Olympe over it. A manual installation is currently not possible as listed in this post:

The path (/products/olympe/linux/env) is not available and therefore the shell is missing. The first thing was that we tried the Taking off - “Hello world” example:
https://developer.parrot.com/docs/olympe/userguide.html

This one worked. After that we wanted to get the camera feed via the unmodified streaming.py script. At first some Python errors occurred. These could then be solved by manually installing PySDL2, OpenGL and the dependencies given in the post mentioned above (Postinst missing in release 7.0.1) After that, the in our first Post mentioned errors occurred when running.

Hi @DHBW ,

We have had no trouble running the streaming.py out of the box. It sounds like perhaps your environment is not setup quite right and you may be missing some libraries. We are using docker to encapsulate all of the dependencies for Olympe and have a Dockerfile that contains the minimum set of commands to set up your environment. Even if you aren’t using docker, you should be able to just take the commands in the Dockerfile and run them on your bare metal system. I have pasted the core of the Dockerfile below so you can either use it directly with docker build or execute the commands individually. Once you set up the environment, I would try to the streaming.py example again.

FROM ubuntu:20.04
ENV DEBIAN_FRONTEND=noninteractive

# Install build and runtime dependencies
RUN apt-get update && apt-get install -y \
    build-essential \
    clinfo \
    curl \
    debconf-utils \
    git \
    libgl1-mesa-glx \
    lsb-release \
    python3 \
    python3-opencv \
    python3-pip \
    sudo \
    vim \
    wget \
 && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

RUN pip3 install --upgrade pip
RUN python3 -m pip install jupyter ipython jinja2 zmq opencv-python

RUN ln -s /usr/bin/python3 /usr/bin/python
#install Olympe
RUN useradd -ms /bin/bash ubuntu
RUN echo "ubuntu ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
USER ubuntu
RUN  python3 -m pip install --user parrot-olympe

First of all, thanks for the quick answers!
I ran the above commands and got the same errors as before.
At some point I got the idea that it was also due to the venv of Pycharm which I am currently using. After running the script from the command line, I did get a video stream(sigh), but the errors still occurred…
The video and JSON file are also created correctly, but the drone doesn’t take off!
After playing around with the script a bit and commenting out lines of code and re-running it, I found that the Persistently Repeated Error:

2021-11-11 20:07:58,083 [ERROR] olympe.mp4mux - add_coded_frame - vmeta_frame_proto_get_buffer returned -71
2021-11-11 20:07:58,103 [ERROR] ulog - vmeta - vmeta_frame_proto_get_buffer:408: err=71(Protocol error)

disappears when I only comment out the following code:

    self.drone.streaming.set_output_files(
        video=os.path.join(self.tempd, "streaming.mp4"),
        metadata=os.path.join(self.tempd, "streaming_metadata.json"),
    )

So it has to do something with outputting a h264 formatted Frame to an .mp4 file??

The one-time error:

2021-11-11 20:04:17,912 [ERROR] ulog - pdraw_sink_coded_video - VideoDecoder#3: coded video media format H264/AVCC not supported.

disappears if I comment out self.drone.streaming.start()

I would appreciate the help to fix the remaining bugs.

I see this error regularly, even though streaming works fine after that. We have ignored it thus far.

We are not relying on the h264 frames so we made the h264 callback just do a pass. Similarly, we didn’t care about saving output files (and creating an .mp4 at the end) so we removed most of that code and just focused on handling the frame in the yuv callback.

In our scripts, we issue the TakeOff() command prior to starting the streaming thread. In the streaming.py example, the thread is started prior to the fly() method which actually takes off. If you aren’t getting to the point where it takes off, it could be that some exception is thrown in the streaming thread code causing the script to exit prematurely.

This error message is coming from libpdraw and is “normal” in the context of Olympe. @Nicolas, aren’t we supposed to get rid of it at some point ? :slight_smile:

The video stream should work regardless of the state of the drone. I am not sure to understand what’s going on on your side.

Olympe 7.0.2 has been released on github and should fix this issue. I’ve updated the installation procedure and fixed the missing “-L” flag in the curl command. The online documentation is still lagging behind and should be up to date by next Monday.

If this error persists repeatedly, this is a real issue. If you don’t need to record the metadata file you should just remove the “metadata” parameter of the streaming.set_output_file() method.

@ndessart,

Prior to 7.0.0 we could not get a modified streaming.py to work while the drone was in a landed state in our lab (we were just trying to stream from the camera which the drone was on the ground). None of the frames would pop up in the cv2.imshow window and it stop abruptly. Once we tried the example in the field, it worked well, though the stream takes a few seconds to start, only once the drone had finished taking off. This is what led us to believe streaming only worked once the drone was in a particular state (i.e. after take off). We haven’t tried streaming in the lab with the drone stationary since we have upgraded to 7.0.0,.