Object Recognition on the Parrot Bebop 2

Hello all, I am planning on programming the Parrot Bebop 2 drone to use it’s the camera to do object recognition, and recognize things such as household objects and human faces, essentially Microsoft’s Azure’s computer vision, and possibly program a reaction. Alternatively, I could also use OpenCV.

I am wondering how I would go about implementing this if there’s any source code, tutorials, recommendations, suggested libraries, and tips I should be aware of.

I appreciate any help I can get. Thank you.

Hi, I’m making that. We get in touch by my Twitter Text me when you stay available.

Greetings from Ecuador!

I have done something similar to this on the Anafi drone. So I am quite sure it is possible on the Bebop. I did the following:

  1. Download Parrot Olympe, which allows you to connect to the parrot via Python scripts.
  2. Use this to get access to frames from the camera stream. Here is a snippet of the code I am using to do just that (note: this code was a modification of either the tutorials, documentation, or answers from this forum. I cant remember exactly which one. EDIT: it might have been this):
    def yuv_frame_cb(yuv_frame):
        """
        This function will be called by Olympe for each decoded YUV frame.
            :type yuv_frame: olympe.VideoFrame
        """
        # the VideoFrame.info() dictionary contains some useful information
        # such as the video resolution
        info = yuv_frame.info()
        height, width = info["yuv"]["height"], info["yuv"]["width"]

        # convert pdraw YUV flag to OpenCV YUV flag
        cv2_cvt_color_flag = {
            olympe.PDRAW_YUV_FORMAT_I420: cv2.COLOR_YUV2BGR_I420,
            olympe.PDRAW_YUV_FORMAT_NV12: cv2.COLOR_YUV2BGR_NV12,
        }[info["yuv"]["format"]]

        # yuv_frame.as_ndarray() is a 2D numpy array with the proper "shape"
        # i.e (3 * height / 2, width) because it's a YUV I420 or NV12 frame

        # Use OpenCV to convert the yuv frame to RGB
        cv2frame = cv2.cvtColor(yuv_frame.as_ndarray(), cv2_cvt_color_flag)

        # Use OpenCV to show this frame
        cv2.imshow("Olympe Streaming Example", cv2frame)
        cv2.waitKey(1)  # please OpenCV for 1 ms...

drone = olympe.Drone(MainDroneController.SIMULATED_IP)
drone.connection()
drone.set_streaming_callbacks(raw_cb=yuv_frame_cb)
drone.start_video_streaming()
drone.start_piloting()
...
  1. You now have access to the image as an OpenCV frame. You can then use a wide range of object recognition tools. One of my favorites is Yolo, which already has an implementation in OpenCV. Many tutorials exist on how to use this, for example, this tutorial.

The results of this would be something similar to the image below. Here the camera feed is from the drone near the origin. You can see it has identified the second drone, marked its bounding box and identified it as an “aeroplane”.

Avoidance

Hope this helps :slight_smile:

1 Like