Using JSVideoView sample code


#1

Hello, anyone out there who has made use of JSVideoView sample code included in the samples master. I’m here trying to get RGB Mat frame from the ARFrame. Is there anything helpful in the JSVideoView to fulfill my task(get RGB Mat frame from the ARFrame)

here’s the source code of JSVideoView

package com.parrot.sdksample.view;

import android.content.Context;
import android.graphics.Bitmap;
import android.graphics.BitmapFactory;
import android.os.Handler;
import android.util.AttributeSet;
import android.widget.ImageView;

import com.parrot.arsdk.arcontroller.ARFrame;

public class JSVideoView extends ImageView{

private static final String TAG = "JSVideoView";

private final Handler mHandler;

private Bitmap mBmp;

public JSVideoView(Context context) {
    super(context);
    // needed because setImageBitmap should be called on the main thread
    mHandler = new Handler(context.getMainLooper());
    customInit();
}

public JSVideoView(Context context, AttributeSet attrs) {
    super(context, attrs);
    // needed because setImageBitmap should be called on the main thread
    mHandler = new Handler(context.getMainLooper());
    customInit();
}

public JSVideoView(Context context, AttributeSet attrs, int defStyleAttr) {
    super(context, attrs, defStyleAttr);
    // needed because setImageBitmap should be called on the main thread
    mHandler = new Handler(context.getMainLooper());
    customInit();
}

private void customInit() {
    setScaleType(ScaleType.CENTER_CROP);
}

public void displayFrame(ARFrame frame) {
    byte[] data = frame.getByteData();
    synchronized (this) {
        mBmp = BitmapFactory.decodeByteArray(data, 0, data.length);
    }

    mHandler.post(new Runnable() {
        @Override
        public void run() {
            synchronized (this) {
                setImageBitmap(mBmp);
            }
        }
    });
}

}


#2

This is rather trivial…

import org.opencv.android.Utils;
import org.opencv.core.Mat;

public void displayFrame(ARFrame frame) {
    byte[] data = frame.getByteData();
    synchronized (this) {
        mBmp = BitmapFactory.decodeByteArray(data, 0, data.length);

        if (mBmp != null) {
                final Mat mat = new Mat();
                Utils.bitmapToMat(mBmp, mat);
                //
                // do whatever you want to do here or propagate
                // thru a runnable / etc
                //
        }
    }

    mHandler.post(new Runnable() {
        @Override
        public void run() {
            synchronized (this) {
                setImageBitmap(mBmp);
            }
        }
    });
}

#3

thank you so much for your response @synman :slight_smile:
I’m developing a face detection application using bebop video stream. I’m using a separate cpp file and a header file with JNI call to do my opencv work. And here inside the displayFrame method I need to pass the nativeObjectAdrress of each Mat frame to my cpp class which contains the opencv face detection function. Do you think that this approach is correct to perform face detection or is there any other alternative way to do that? Really appreciate your response


#4

If you’re going to keep it native then you may want to Marshall just the byte array to your native layer and do your hard work within it.

An alternative is the Java opencv library, which is what my example builds from.

You’ll find more details about my implementation around here somewhere… I did mine by modifying BebopVideoView to use a TextureView and built all my CV classifiers, filters, etc using the Java opencv library.


#5

Here are more details:


#6

thanks again @synman :slight_smile:
here in the BebopVideoView you have passed sps pps buffers as part of displayFrame method. but its not necessary right? We can still go with the default initialization as in the sample? Basically I’m developing the application with the pre compiled sdk and trying to go further by developing the sample since I’m not much familiar with this. can I continue my work with pre compiled sdk or do i need to compile my own sdk for this purpose?


#7

Precompiled should work fine


#8

hello synman . i have been following ur posts to get mat from the dataframe . but i cant get this to working .

in this solution when i right this

synchronized (this) {
mBmp = BitmapFactory.decodeByteArray(data, 0, data.length);
}

i always get this error
06-25 04:54:47.976 26451-26644/com.parrot.sdksample D/skia: — SkImageDecoder::Factory returned null

and when i tried to view mBmp its always null

can i know why is that …

thanks a lot


#9

Hi,

The decodeByteArray method used in JSVideoView can only be done on a byte array representing a compressed image. It is the case on the JumpingSumo where the video is mjpeg encoded (each frame is an I-Frame which is jpeg encoded).
However, on the Bebop, the video stream can’t be decoded that way because it is a H264 encoded stream. This is why you need to decode it with a video decoder. To do that, you can see BebopVideoView.