BebopVideoView to Mat


#21

thread management… You don’t want to leave the cv processing thread running while the activity is not active.

resume starts the thread and pause tears it down


Construct cv::Mat from the frame Bebop send to iOS (ARCONTROLLER_Frame_t)
#22

I am working on an image processing app on bebop drone and I am facing the same problem, i.e., the taken image is black:( My template is parrot sample project At first, i changed BebobVideoView into TextureView. Then, when I try to get image from BebobActivity as

final Bitmap bit = mVideoView.getBitmap();
ImageView im = (ImageView) findViewById(R.id.imageView);
im.setImageBitmap(bit);

the image is black. So I tried to create class in original BebopActivity class, where the new class extends thread. The process of taking bitmap is then run in an own thread as is it seen in synman example but again the image is black:(

Can someone please post working BebopActivity where working getBitmap() is shown? Thank you


#23

I have used to take CVClassifierView and create own view of this class. In bebobActitivity I take the id and call resume. I the CVClassifierView, i try to paint the bitmap, but again, the output is black:(


#24

Solved:) It has needed to append "this.setSurfaceTextureListener(this); " into the bebopVideoView


#26

@hmattolou

hello.
I guess @mastemace comment would mean the code below.


[BebopActivity.java]

private void initIHM() {
      mVideoView = (BebopVideoView) findViewById(R.id.videoView);
      mVideoView.setSurfaceTextureListener(mVideoView);
 ...
}

[BebopVideoView.java]

public BebopVideoView(Context context) {
    this(context, null);
    ...
    setSurfaceTextureListener(this);
}

public BebopVideoView(Context context, AttributeSet attrs) {
    this(context, attrs, 0);
    ...
    setSurfaceTextureListener(this);
}

public BebopVideoView(Context context, AttributeSet attrs, int defStyleAttr) {
    super(context, attrs, defStyleAttr);
    ...
    setSurfaceTextureListener(this);
}

BebopVideoView is TextureView.


I think you can solve the problem by writing code like above.
Thanks.


#28

Hi synman!!!
I’m mingyu from south korea and student.
Actually i have a final project with BebopDrone for my graduate what is SelfiDrone.
I saw your codes BebopVideoView & CVClassifierView file. But i don’t understand about thread.
If you don’t mind, Could you send me your wholecode of project? (i want to know procedure)

My email address is ‘darkpmg@gmail.com’.

Thank you synman.
Regards


#29

Hi! mastemace!
I have some problem. If you are fine, could you send me BebopActivity.java and activity_bebop.xml?

my email is darkpmg@gmail.com

thank you


#31

hello .Can you show how to set the argument of sps pps buffers . Any help will be appreciated!


#32

Take a look at the ARSDK3 video stream events. You’ll find that the SDK provides them as part of setup.


#33

Thanks!synman. Could you send me the whole project that get Mat from the frame?I try seveal days .But the Bitamp I saved is a Black picture .I dont know how to deal with it …Any help will be appreciated! Thank you!


#34

I cannot give you my entire project as it is a retail application. If you have a specific question I am more than happy to help. The examples provided here, linked to over at stackoverflow work well with the parrot provided android sample.


#35

Thank you . The android sample provide H264VideoView.java.I modify it with your BebopVideoView.java you provided above. And BebopAcitivity.java in the sample In the function configureDecoder(ARControllerCodec
codec)call the function mVideoView.configureDecoder(codec) to init the view. But configureDecoder didn’t include in your BebopVideoView.
my BebopActivity.jave
package com.parrot.sdksample.activity;

import android.annotation.TargetApi;
import android.app.ProgressDialog;
import android.content.DialogInterface;
import android.content.Intent;
import android.graphics.Bitmap;
import android.graphics.BitmapFactory;
import android.graphics.ImageFormat;
import android.graphics.Rect;
import android.graphics.YuvImage;
import android.media.Image;
import android.media.MediaCodec;
import android.media.MediaExtractor;
import android.media.MediaFormat;
import android.os.Build;
import android.os.Bundle;
import android.os.Environment;
import android.support.v7.app.AppCompatActivity;
import android.util.Log;
import android.view.MotionEvent;
import android.view.View;
import android.widget.Button;
import android.widget.TextView;

import com.parrot.arsdk.arcommands.ARCOMMANDS_ARDRONE3_MEDIARECORDEVENT_PICTUREEVENTCHANGED_ERROR_ENUM;
import com.parrot.arsdk.arcommands.ARCOMMANDS_ARDRONE3_PILOTINGSTATE_FLYINGSTATECHANGED_STATE_ENUM;
import com.parrot.arsdk.arcontroller.ARCONTROLLER_DEVICE_STATE_ENUM;
import com.parrot.arsdk.arcontroller.ARControllerCodec;
import com.parrot.arsdk.arcontroller.ARFrame;
import com.parrot.arsdk.ardiscovery.ARDiscoveryDeviceService;
import com.parrot.sdksample.R;
import com.parrot.sdksample.drone.BebopDrone;
import com.parrot.sdksample.view.H264VideoView;

import org.opencv.android.BaseLoaderCallback;
import org.opencv.android.LoaderCallbackInterface;
import org.opencv.android.OpenCVLoader;
import org.opencv.android.Utils;
import org.opencv.core.Mat;
import org.opencv.imgcodecs.Imgcodecs;
import org.opencv.imgproc.Imgproc;

import java.io.ByteArrayOutputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.nio.ByteBuffer;

public class BebopActivity extends AppCompatActivity {
// Used to load the ‘native-lib’ library on application startup.
static {
System.loadLibrary(“bebop_video_decoder-lib”);
}
// public native String arframeToRGB();

private static final String TAG = "BebopActivity";
private BebopDrone mBebopDrone;

private ProgressDialog mConnectionProgressDialog;
private ProgressDialog mDownloadProgressDialog;

private H264VideoView mVideoView;

private TextView mBatteryLabel;
private Button mTakeOffLandBt;
private Button mDownloadBt;

private int mNbMaxDownload;
private int mCurrentDownloadIndex;
private  int pic_count=0;
public boolean b_openCV_load=false;
public boolean b_first_init=true;

@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_bebop);

    initIHM();

    Intent intent = getIntent();
    ARDiscoveryDeviceService service = intent.getParcelableExtra(DeviceListActivity.EXTRA_DEVICE_SERVICE);
    mBebopDrone = new BebopDrone(this, service);
    mBebopDrone.addListener(mBebopListener);

}

@Override
protected void onStart() {
    super.onStart();

    // show a loading view while the bebop drone is connecting
    if ((mBebopDrone != null) && !(ARCONTROLLER_DEVICE_STATE_ENUM.ARCONTROLLER_DEVICE_STATE_RUNNING.equals(mBebopDrone.getConnectionState())))
    {
        mConnectionProgressDialog = new ProgressDialog(this, R.style.AppCompatAlertDialogStyle);
        mConnectionProgressDialog.setIndeterminate(true);
        mConnectionProgressDialog.setMessage("Connecting ...");
        mConnectionProgressDialog.setCancelable(false);
        mConnectionProgressDialog.show();

        // if the connection to the Bebop fails, finish the activity
        if (!mBebopDrone.connect()) {
            finish();
        }
    }


}

@Override
public void onBackPressed() {
    if (mBebopDrone != null)
    {
        mConnectionProgressDialog = new ProgressDialog(this, R.style.AppCompatAlertDialogStyle);
        mConnectionProgressDialog.setIndeterminate(true);
        mConnectionProgressDialog.setMessage("Disconnecting ...");
        mConnectionProgressDialog.setCancelable(false);
        mConnectionProgressDialog.show();

        if (!mBebopDrone.disconnect()) {
            finish();
        }
    }
}

@Override
public void onDestroy()
{
    mBebopDrone.dispose();
    super.onDestroy();
}

private BaseLoaderCallback mLoaderCallback = new BaseLoaderCallback(this) {
    @Override
    public void onManagerConnected(int status) {
        switch (status) {
            case LoaderCallbackInterface.SUCCESS: {
                Log.i(TAG, "OpenCV loaded successfully");
                b_openCV_load=true;

// mOpenCvCameraView.enableView();
// mOpenCvCameraView.setOnTouchListener(ColorBlobDetectionActivity.this);
}
break;
default: {
super.onManagerConnected(status);
Log.i(TAG, “fail”);
}
break;
}
}
};

@Override
protected void onResume() {
    super.onResume();
   OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_3_4_0, this, mLoaderCallback);
    new Thread(new Runnable(){
        @Override
        public void run() {
            final Mat firstmat = new Mat();
            final Mat image = new Mat();
            while (true) {
                try {
                    Thread.sleep(1000);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
                // pic_count++;
                if (b_openCV_load == true && pic_count%50==0) {
                    if (pic_count >= 500)
                        pic_count = 0;


                    b_first_init=false;


                    final Bitmap mybitmap = mVideoView.getBitmap();
                    if (mybitmap != null) {

                        Utils.bitmapToMat(mVideoView.getBitmap(), image);
                        firstmat.assignTo(image);
                        Imgproc.cvtColor(image, image, Imgproc.COLOR_RGBA2GRAY);
                        // String pic_name=getApplicationContext().getFilesDir().getAbsolutePath()+"/kkkk" + pic_count + ".png";
                        String pic_name = getExternalFilesDir(Environment.DIRECTORY_MOVIES).toString() + "/kkkk" + pic_count + ".jpg";

                        if (Imgcodecs.imwrite(pic_name, image))
                            Log.d("onFrameReceived", "save " + pic_name + "  Success!");
                        else
                            Log.d("onFrameReceived", "save " + pic_name + "  Fail!!");
                    }
                }

            }
        }
    }).start();
}

    private void initIHM () {
        mVideoView = (H264VideoView) findViewById(R.id.videoView);

        findViewById(R.id.emergencyBt).setOnClickListener(new View.OnClickListener() {
            public void onClick(View v) {
                mBebopDrone.emergency();
            }
        });



            @Override
            public void onFrameReceived(ARFrame frame) {

                //mVideoView.displayFrame(frame);
              }

        };
    }
}

#36

I can get Bitmap saved ,but alway be a black picture. I change the Surface To TextView in H264VideoView.jave.

Mat image = new Mat();
Mat firstMat = new Mat();
Bitmap mybitmap = mVideoView.getBitmap();
if (mybitmap != null) {

                       Utils.bitmapToMat(mybitmap, firstMat);
                       firstMat.assignTo(image);

                       Imgproc.cvtColor(image,image,Imgproc.COLOR_RGBA2GRAY/*Imgproc.COLOR_YUV2BGR*/);

#37

Are you seeing a good video stream in the actual textureview?

You need to reformat your code above so that is is legible. I’ll try to look at it when I have time to decipher the wall of text you pasted :slight_smile: