Opening a sink from a video stream

[PALequeu]

Hello everyone,

I am making a kotlin app in which i display the drone’s camera and all that works well. I’d like to now process this live video for object detection using ML Kit. I think i need to open a sink using stream.openSink() for that, but it needs a Stream.Sink.Config to work, and I was unable to find any further information on it.
What are thoses objects ? How do I use one ? I did not find anything on the documentation or on this forum.

Thanks for the help !

[PALequeu]

Some updates,

I was able to open a sink using RawVideoSink, and defined my callback like that.

@OptIn(UnstableApi::class)
class StreamCallBack(activity: MainActivity) : RawVideoSink.Callback {


    private var width : Int = 0
    private var height : Int = 0
    private var mainActivity = activity

    override fun onFrame(sink: RawVideoSink, frame: RawVideoSink.Frame) {
        super.onFrame(sink, frame)

        try {
            val rel = frame.released
            val myToast =
                Toast.makeText(mainActivity, rel.toString(), Toast.LENGTH_SHORT)
            myToast.show()
        } catch (e: IllegalStateException) {
            val myToast =
                Toast.makeText(mainActivity, e.toString(), Toast.LENGTH_SHORT)
            myToast.show()

        }


        //frame.release()
        //val myToast2 =
         //   Toast.makeText(mainActivity, frame.released.toString(), Toast.LENGTH_SHORT)
        //myToast2.show()

    }

    override fun onStart(sink: RawVideoSink, videoFormat: VideoFormat) {
        super.onStart(sink, videoFormat)
        width = videoFormat.resolution.width.toInt()
        height = videoFormat.resolution.height.toInt()
        val myToast =
            Toast.makeText(mainActivity, "$width $height", Toast.LENGTH_SHORT)
        myToast.show()
    }

    override fun onStop(sink: RawVideoSink) {
        super.onStop(sink)
    }
}

The onStart works well, however, the onFrame has a problem : it seems like the Frame is already released (the app shows a toast “true”). Is it a bug due to the API being instable ? or am I handling things wrong ?