Rtp video streaming from bebop2


#1

hey there,

I’m trying to capture the rtp video stream from my Bebop2 with no success.
I tried two paths:

  • I tried to use the Go implementation of the Bebop api. I couldn’t find the documentation of this element “VideoStreamMode(num)”, but I’m guessing from this link that I should have used option number"2"? any how this is the main block of code:
func main() {
	bebop := client.New()
	if err := bebop.Connect(); err != nil {
		fmt.Println(err)
		return
	}
	if err := bebop.VideoEnable(true); err != nil {
		fmt.Println(err)
		return
	}
	if err := bebop.VideoStreamMode(2); err != nil {
		fmt.Println(err)
		return
	}
    time.Sleep(99 * time.Second)
}

Indeed, I’m able to see on WireShark the rtp packets

I wrote this SDP file :

c=IN IP4 192.168.42.1
m=video 55004 RTP/AVP 96
a=rtpmap:96 H264/90000

But I can’t play it In Vlc. returns this error:
live555 error: no data received in 10s, aborting

  • I tried also to use the SDK directly, following this exemple. unfortunately it didn’t work either. it seems that i’m encountering some connection problem…
root@1ea5398b7536:/bebop/BebopStreamVLC# ./BebopDroneStartStream
[INF] BebopDroneStartStream | 12:35:24:288 | main:237 - -- Bebop Drone Start Stream --
[INF] BebopDroneStartStream | 12:35:24:288 | main:239 - -- Starting --
[INF] BebopDroneStartStream | 12:35:24:288 | ardiscoveryConnect:370 - - ARDiscovery Connection
[INF] BebopDroneStartStream | 12:35:24:305 | ARDISCOVERY_Connection_ReceiveJsonCallback:579 -     - ReceiveJson:{ "status": 0, "c2d_port": 54321, "arstream_fragment_siz
e": 65000, "arstream_fragment_maximum_number": 4, "arstream_max_ack_interval": -1, "c2d_update_port": 51, "c2d_user_port": 21, "arstream2_server_stream_port": 5004, "ar
stream2_server_control_port": 5005, "arstream2_max_packet_size": 1500, "arstream2_max_latency": 0, "arstream2_max_network_latency": 200, "arstream2_max_bitrate": 150000
0, "arstream2_parameter_sets": "TBD" }
[INF] BebopDroneStartStream | 12:35:24:305 | startNetwork:402 - - Start ARNetwork
[INF] BebopDroneStartStream | 12:35:24:305 | sendBeginStream:512 - - Send Streaming Begin
[ERR] ARNETWORK_Sender | 12:35:24:827 | ARNETWORK_Sender_ProcessBufferToSend:401 - [0xc3c150] Timeout waiting for ack in buffer 11
[ERR] ARNETWORK_Sender | 12:35:24:827 | ARNETWORK_Sender_ProcessBufferToSend:417 - [0xc3c150] Will retry sending data of buffer 11
[ERR] ARNETWORK_Sender | 12:35:25:327 | ARNETWORK_Sender_ProcessBufferToSend:401 - [0xc3c150] Timeout waiting for ack in buffer 11
[ERR] ARNETWORK_Sender | 12:35:25:327 | ARNETWORK_Sender_ProcessBufferToSend:417 - [0xc3c150] Will retry sending data of buffer 11
[ERR] ARNETWORK_Sender | 12:35:25:827 | ARNETWORK_Sender_ProcessBufferToSend:401 - [0xc3c150] Timeout waiting for ack in buffer 11
[ERR] ARNETWORK_Sender | 12:35:25:827 | ARNETWORK_Sender_ProcessBufferToSend:417 - [0xc3c150] Will retry sending data of buffer 11
[ERR] ARNETWORK_Sender | 12:35:26:327 | ARNETWORK_Sender_ProcessBufferToSend:401 - [0xc3c150] Timeout waiting for ack in buffer 11
[WNG] ARNETWORK_Sender | 12:35:26:327 | ARNETWORK_Sender_TimeOutCallback:651 - [0xc3c150] Did timeout sending command : project = 1 | class = 15 | command = 0 - 0
[INF] ARNETWORKAL_WifiNetwork | 12:35:29:310 | ARNETWORKAL_WifiNetwork_Receive:890 - [0xc17550] connection lost (too long time without reception)

Help would be much appreciated!


#2

I’m trying to do the same thing with NodeJS, but I’m unable to even get the stream started. Discovery and standard piloting commands are OK. But I’m unable to establish RTP connection. Can you, please, share your Go code for Connect() and VideoEnable() methods?


#3

no problem - it’s taken from this github repository.

look in the example, I only trimmed the ffmpeg part.

// +build example
//
// Do not build by default.

/*
This example will connect to the Bebop and stream its video to a webpage
via ffserver. This requires you to have both ffmpeg and ffserver installed
on your computer.
In order to run this example you will first need to start ffserver with:
$ ffserver -f ff.conf
then in a separate terminal run this program:
$ go run video.go
You will then be able to view the video feed by navigatin to
http://localhost:8090/bebop.mjpeg in a web browser. NOTE firefox works
best for viewing the video feed.
*/
package main

import (
“fmt”
“io/ioutil”
“os/exec”
“time”

“gobot.io/x/gobot/platforms/parrot/bebop/client”
)

func main() {
bebop := client.New()

if err := bebop.Connect(); err != nil {
fmt.Println(err)
return
}

if err := bebop.VideoEnable(true); err != nil {
fmt.Println(err)
return
}

if err := bebop.VideoStreamMode(0); err != nil {
fmt.Println(err)
return
}

ffmpeg := exec.Command(“ffmpeg”, “-i”, “pipe:0”, “http://localhost:8090/bebop.ffm”)

ffmpegErr, err := ffmpeg.StderrPipe()

if err != nil {
fmt.Println(err)
return
}

ffmpegIn, err := ffmpeg.StdinPipe()

if err != nil {
fmt.Println(err)
return
}

if err := ffmpeg.Start(); err != nil {
fmt.Println(err)
return
}

go func() {
for {
buf, err := ioutil.ReadAll(ffmpegErr)
if err != nil {
fmt.Println(err)
}
if len(buf) > 0 {
fmt.Println(string(buf))
}
}
}()

go func() {
for {
if _, err := ffmpegIn.Write(<-bebop.Video()); err != nil {
fmt.Println(err)
}
}
}()

time.Sleep(99 * time.Second)
}


#4

oh! i forgot - I tried to do it first with the node implementation, but it didn’t work for me either.
I suspect that it makes the drone transmit tcp stream of jpeg. for some reason they deprecated the command that chooses the streaming mode (you can still see that someone used it here). in the Go code it’s implemented like that:
bebop.VideoStreamMode(<the streaming mode code>)

Maybe someone can verify/elaborate.


#5

Thanks for your help. From your example, I figured that I did not have “arstream2_client_stream_port”: 55004, “arstream2_client_control_port”: 55005 included in the discovery json. It looks like these have to be sent for the drone to accept VideoEnable command.

Now I’m getting the stream and I was able to play it in VLC.

About your problem… I noticed that my Wireshark shows ICMP errors (port closed) from drone to my computer when there is nothing listening to 5004. When I start my own UDP listener on this port, or I run SDP file on VLC I start receiving packets. Your Wireshark screenshots shows a small sample of data, so I’m not sure if you have some service listening to 5004 or not?


#6

Hey!

did you use the Go code? or did you use Nodejs?
When i’m running the Go code, packets seem to arrive to port 55004. I get no errors (I attached the wireshark log). When I’m trying to consume them with VLC (using the SDP file described above) nothing seem to happen… [quote=“idowebiks, post:1, topic:5576”]
c=IN IP4 192.168.42.1
m=video 55004 RTP/AVP 96
a=rtpmap:96 H264/90000
[/quote]

What did you do to make it work?


#7

I used NodeJS. I’m working on my own wrapper so it has just a minimum of functionalities as a proof of concept. I’ve added ffmpeg to generate jpg image every second so you can try it as well.

I just uploaded it to GitHub https://github.com/dselmanovic/parrotterminal


#8

Hey @dselmanovic,

I tried to use your code, but i’m facing the same problem - I can see the packets arriving in wireShark, But when i’m trying to play the SDP file in VLC - nothing happens. Do you know maybe what am i doing wrong…?

(I used your functions: connect, then startvideo (saw the packets on wireshark), then played the SDP file in VLC)

Thanks


#9

WireShark seem to not recognizance the RTP protocol (just the udp layer) - could that induce the problem? Maybe my drone doesn’t acknowledge the VideoStreamMode() command?


#10

And this is the error i’m getting if i’m using your “save” command (after enabling the video) -
> $PT > save
> $PT > exec error: Error: Command failed: ffmpeg -protocol_whitelist file,udp,rtp -i bebop.sdp -f image2 -r 1 -updatefirst 1 outimage.jpg
> ffmpeg version N-84679-gd65b595 Copyright © 2000-2017 the FFmpeg developers
> built with gcc 6.3.0 (GCC)
> configuration: --enable-gpl --enable-version3 --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-nve
> nc --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray
> --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-lib
> mp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librtmp
> --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc
> --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libx
> vid --enable-libzimg --enable-lzma --enable-zlib
> libavutil 55. 51.100 / 55. 51.100
> libavcodec 57. 86.103 / 57. 86.103
> libavformat 57. 67.100 / 57. 67.100
> libavdevice 57. 3.101 / 57. 3.101
> libavfilter 6. 78.100 / 6. 78.100
> libswscale 4. 3.101 / 4. 3.101
> libswresample 2. 4.100 / 2. 4.100
> libpostproc 54. 2.100 / 54. 2.100
> [udp @ 0000000002572c60] ‘circular_buffer_size’ option was set but it is not supported on this build (pthread support is required)
> [udp @ 0000000002572c60] bind failed: Error number -10048 occurred
> bebop.sdp: Invalid data found when processing input


#11

I see just UDP as well on Wireshark, so that should not be issue. I had to add UDP as safe protocol for ffmpeg command, Bebop is transferring RTP over UDP.

check the version of ffmpeg on your computer, the message in your log looks like there is a build version issue. You can try to run ffmeg from terminal (instead of “save” command), as I just spawn the child process anyway. Just start the video before that.

As for VLC, I know it sounds weird, but have you tried to open the file in different ways? Open network, open file, double click the SDP file, etc. If I remember correctly I had a problem when I opened the file in some of the ways.

It looks like that you have a problem with decoding libraries on your computer, so I think the problrm might be there. If you have problem with ffmpeg and VLC at the same time, so I would look into versions of those apps.

What is the OS you are using? I’m on OSX


#12

Oh, and I just noticed that you get UDP bind error for ffmpeg. Which brings me back to my first assumption that some other process on your computer is already bind to 5004 UDP port.

You can try to test that by uncommenting streamServer.bind line in my code (just check which port it will try to bind) and should throw the error if the port is taken. Or you can check your open ports and services that are listening them, depending on the OS you are using.

let me know when you check this, I’d like to know what hapened. @idowebiks


#13

Hey, first of all thank you so much for your time,

I’m working on windows,
About the ports - I checked the occupied ports, and it seems alright

C:\Users\ido\projects\parrotterminal>netstat -a

Active Connections

  Proto  Local Address          Foreign Address        State
  TCP    0.0.0.0:80             ADMINRG-1MG0106:0      LISTENING
  TCP    0.0.0.0:135            ADMINRG-1MG0106:0      LISTENING
  TCP    0.0.0.0:445            ADMINRG-1MG0106:0      LISTENING
  TCP    0.0.0.0:2179           ADMINRG-1MG0106:0      LISTENING
  TCP    0.0.0.0:5357           ADMINRG-1MG0106:0      LISTENING
  TCP    0.0.0.0:7680           ADMINRG-1MG0106:0      LISTENING
  TCP    0.0.0.0:49664          ADMINRG-1MG0106:0      LISTENING
  TCP    0.0.0.0:49665          ADMINRG-1MG0106:0      LISTENING
  TCP    0.0.0.0:49666          ADMINRG-1MG0106:0      LISTENING
  TCP    0.0.0.0:49667          ADMINRG-1MG0106:0      LISTENING
  TCP    0.0.0.0:49668          ADMINRG-1MG0106:0      LISTENING
  TCP    0.0.0.0:49669          ADMINRG-1MG0106:0      LISTENING
  TCP    0.0.0.0:62640          ADMINRG-1MG0106:0      LISTENING
  TCP    10.0.75.1:139          ADMINRG-1MG0106:0      LISTENING
  TCP    10.1.0.35:55206        213.180.193.119:https  TIME_WAIT
  TCP    10.1.0.35:55212        213.180.193.119:https  TIME_WAIT
  TCP    10.1.0.35:55213        213.180.193.119:https  TIME_WAIT
  TCP    127.0.0.1:1234         ADMINRG-1MG0106:0      LISTENING
  TCP    127.0.0.1:1234         ADMINRG-1MG0106:64421  ESTABLISHED
  TCP    127.0.0.1:2375         ADMINRG-1MG0106:0      LISTENING
  TCP    127.0.0.1:5939         ADMINRG-1MG0106:0      LISTENING
  TCP    127.0.0.1:5939         ADMINRG-1MG0106:62647  ESTABLISHED
  TCP    127.0.0.1:6004         ADMINRG-1MG0106:0      LISTENING
  TCP    127.0.0.1:60398        ADMINRG-1MG0106:60399  ESTABLISHED
  TCP    127.0.0.1:60399        ADMINRG-1MG0106:60398  ESTABLISHED
  TCP    127.0.0.1:62647        ADMINRG-1MG0106:5939   ESTABLISHED
  TCP    127.0.0.1:62649        ADMINRG-1MG0106:62650  ESTABLISHED
  TCP    127.0.0.1:62650        ADMINRG-1MG0106:62649  ESTABLISHED
  TCP    127.0.0.1:64421        ADMINRG-1MG0106:1234   ESTABLISHED
  TCP    127.0.0.1:64448        ADMINRG-1MG0106:64449  ESTABLISHED
  TCP    127.0.0.1:64449        ADMINRG-1MG0106:64448  ESTABLISHED
  TCP    127.0.0.1:64456        ADMINRG-1MG0106:64457  ESTABLISHED
  TCP    127.0.0.1:64457        ADMINRG-1MG0106:64456  ESTABLISHED
  TCP    192.168.42.63:139      ADMINRG-1MG0106:0      LISTENING
  TCP    192.168.42.63:55222    111.221.29.164:https   SYN_SENT
  TCP    192.168.42.63:55223    111.221.29.90:https    SYN_SENT
  TCP    192.168.56.1:139       ADMINRG-1MG0106:0      LISTENING
  TCP    [::]:80                ADMINRG-1MG0106:0      LISTENING
  TCP    [::]:135               ADMINRG-1MG0106:0      LISTENING
  TCP    [::]:445               ADMINRG-1MG0106:0      LISTENING
  TCP    [::]:2179              ADMINRG-1MG0106:0      LISTENING
  TCP    [::]:5357              ADMINRG-1MG0106:0      LISTENING
  TCP    [::]:7680              ADMINRG-1MG0106:0      LISTENING
  TCP    [::]:49664             ADMINRG-1MG0106:0      LISTENING
  TCP    [::]:49665             ADMINRG-1MG0106:0      LISTENING
  TCP    [::]:49666             ADMINRG-1MG0106:0      LISTENING
  TCP    [::]:49667             ADMINRG-1MG0106:0      LISTENING
  TCP    [::]:49668             ADMINRG-1MG0106:0      LISTENING
  TCP    [::]:49669             ADMINRG-1MG0106:0      LISTENING
  TCP    [::]:62640             ADMINRG-1MG0106:0      LISTENING
  TCP    [::1]:64522            ADMINRG-1MG0106:0      LISTENING
  UDP    0.0.0.0:123            *:*
  UDP    0.0.0.0:3702           *:*
  UDP    0.0.0.0:3702           *:*
  UDP    0.0.0.0:5050           *:*
  UDP    0.0.0.0:5353           *:*
  UDP    0.0.0.0:5353           *:*
  UDP    0.0.0.0:5353           *:*
  UDP    0.0.0.0:5353           *:*
  UDP    0.0.0.0:5353           *:*
  UDP    0.0.0.0:5353           *:*
  UDP    0.0.0.0:5353           *:*
  UDP    0.0.0.0:5355           *:*
  UDP    0.0.0.0:51701          *:*
  UDP    0.0.0.0:52217          *:*
  UDP    0.0.0.0:63171          *:*
  UDP    10.0.75.1:137          *:*
  UDP    10.0.75.1:138          *:*
  UDP    10.0.75.1:1900         *:*
  UDP    10.0.75.1:2177         *:*
  UDP    10.0.75.1:5353         *:*
  UDP    10.0.75.1:59411        *:*
  UDP    127.0.0.1:1900         *:*
  UDP    127.0.0.1:52118        *:*
  UDP    127.0.0.1:53491        *:*
  UDP    127.0.0.1:53939        *:*
  UDP    127.0.0.1:54332        *:*
  UDP    127.0.0.1:55040        *:*
  UDP    127.0.0.1:57265        *:*
  UDP    127.0.0.1:57964        *:*
  UDP    127.0.0.1:59414        *:*
  UDP    127.0.0.1:59990        *:*
  UDP    127.0.0.1:61285        *:*
  UDP    127.0.0.1:63229        *:*
  UDP    127.0.0.1:63298        *:*
  UDP    127.0.0.1:64955        *:*
  UDP    127.0.0.1:65118        *:*
  UDP    192.168.3.59:5353      *:*
  UDP    192.168.42.63:137      *:*
  UDP    192.168.42.63:138      *:*
  UDP    192.168.42.63:1900     *:*
  UDP    192.168.42.63:2177     *:*
  UDP    192.168.42.63:59413    *:*
  UDP    192.168.56.1:137       *:*
  UDP    192.168.56.1:138       *:*
  UDP    192.168.56.1:1900      *:*
  UDP    192.168.56.1:2177      *:*
  UDP    192.168.56.1:5353      *:*
  UDP    192.168.56.1:59412     *:*
  UDP    [::]:123               *:*
  UDP    [::]:3702              *:*
  UDP    [::]:3702              *:*
  UDP    [::]:5353              *:*
  UDP    [::]:5353              *:*
  UDP    [::]:5353              *:*
  UDP    [::]:5353              *:*
  UDP    [::]:5355              *:*
  UDP    [::]:51702             *:*
  UDP    [::]:52218             *:*
  UDP    [::]:63171             *:*
  UDP    [::1]:1900             *:*
  UDP    [::1]:59410            *:*
  UDP    [fe80::1d2b:efd3:9cdd:c056%3]:5353  *:*
  UDP    [fe80::486c:1a1b:9054:60ee%2]:1900  *:*
  UDP    [fe80::486c:1a1b:9054:60ee%2]:2177  *:*
  UDP    [fe80::486c:1a1b:9054:60ee%2]:59408  *:*
  UDP    [fe80::48bf:286c:cf9e:5fba%15]:546  *:*
  UDP    [fe80::48bf:286c:cf9e:5fba%15]:1900  *:*
  UDP    [fe80::48bf:286c:cf9e:5fba%15]:2177  *:*
  UDP    [fe80::48bf:286c:cf9e:5fba%15]:59409  *:*
  UDP    [fe80::fd55:e26e:2e1c:3b01%6]:1900  *:*
  UDP    [fe80::fd55:e26e:2e1c:3b01%6]:2177  *:*
  UDP    [fe80::fd55:e26e:2e1c:3b01%6]:59407  *:*

I tried to uncomment the streamServer.bind(streamPort);. no error were thrown - the streamPort was 55004,
I tried to double click, open as file and open from network (as file) - didn’t work.

But I tried to run the ffmpeg command from the command line ffmpeg -protocol_whitelist file,udp,rtp -i bebop.sdp -f image2 -r 1 -updatefirst 1 outimage.jpg and that seem to work fine (I can see the picture changing periodically)


#14

Hey @dselmanovic I tried it also on different computer (linux based) - didn’t work either /:


#15

So ffmpeg works but VLC does not? I’m afraid that this is beyond my paygrade :smiley: But the only thing I can assume is that something is wrong with VLC.

Have you tried ffplay? @idowebiks


#16

@dselmanovic

  • FFmpeg works when converting to image - ffmpeg -protocol_whitelist file,udp,rtp -i bebop.sdp -f image2 -r 1 -updatefirst 1 outimage.jpg
  • FFplay - ffplay -protocol_whitelist file,udp,rtp -i bebop.sdp - throws Could not find codec parameters for stream 0 (Video: h264, none): unspecified size Consider increasing the value for the 'analyzeduration' and 'probesize' options
  • FFplay - ffplay -protocol_whitelist file,udp,rtp -f h264 -i bebop.sdp - throws Could not find codec parameters for stream 0 (Video: h264, none): unspecified size Consider increasing the value for the 'analyzeduration' and 'probesize' options
  • FFplay - ffplay -protocol_whitelist file,udp,rtp -f rtp -i bebop.sdp - throws [rtp @ 0000000002484fe0] Received too short packetsq= 0B f=0/0 Last message repeated 24 times [rtp @ 0000000002484fe0] Received too short packetsq= 0B f=0/0 Last message repeated 25 times [rtp @ 0000000002484fe0] Received too short packetsq= 0B f=0/0 Last message repeated 24 times [rtp @ 0000000002484fe0] Received too short packetsq= 0B f=0/0 L

I don’t really care about the VLC - but i’m concerned that it implies something is wrong with the transmission…


#17

I think that the picture decoding is the exception -
When playing it with ffplay It throws : "No start code is found - error splitting the input into NAL units".
Implying that the stream is corrupted (or at least can’t be played as video).

I reinstalled VLC. I’m really curious how come the video plays on your computer… Have you tried FFplay?


#18

Hi, were you able to solve this issue?
I’ve tried the Go library, experienced similar issues :frowning:


#19

Hi, where to get the golang lib?


#20

go get -d -u gobot.io/x/gobot/…
got it