yuv_frame.vmeta()[1] sometimes returns an empty object, while the yuv_frame itself is at least not None
Is that possible?
Any response?
Hi,
This is possible for the very first frames when the streaming session just started.
It definitely also happens during a connection:
This topic was automatically closed after 30 days. New replies are no longer allowed.
Hello,
Video metadata can sometimes be absent in a frame, as we can loose packets on a wireless connection.
So it should be handled in the code that the metadata object can sometimes be None even if a (partial) frame is available.
Ok, it’s not none, it’s an empty dict. But this is marginal. What does that mean to the accompanying video frame? Is this most likely garbage, should be discarded too?
We have error concealment on the frames which tries to output frames that are only partially received but can still be rendered and displayed. So the absence of metadata can also mean that the frame is only partially decoded and contains visual errors, which can make it unusable for computer vision algorithms for example.
To know that, see Olympe API Reference Documentation - 7.4
The frame info dictionary contains a “has_errors” value indicating that the frame may be visually broken. It also contains a “is_silent” value indicating that the frame is not intended to be displayed (for the first intra-refresh frames, for example).
Ah, thanks. Good to know. Is it an enhancement of 7.4 or was it already present in earlier versions?
Hi,
Is has been present at least since the 7.0 release.
Today I was going to test that has_errors flag on my simulator instance and found each frame tagged like so (in contrary to the frames coming from a real drone):
Simulator:
{'format': 'RAW', 'raw': {'frame': {'timestamp': 20881453667, 'timescale': 1000000, 'index': 625, 'format': 'YUV420/ABCD/LINEAR/8/PLANAR/HIGH/BE/8', 'info': {'full_range': False, 'color_primaries': 'BT709', 'transfer_function': 'BT709', 'matrix_coefs': 'BT709', 'width': 1280, 'height': 720, 'sar_width': 1, 'sar_height': 1}}}, 'has_errors': 1, 'is_silent': 0, 'is_sync': 0, 'is_ref': 1, 'ntp_timestamp': 20883071798, 'ntp_unskewed_timestamp': 20883071800, 'ntp_raw_timestamp': 20881453667, 'ntp_raw_unskewed_timestamp': 20881453669, 'play_timestamp': 20883070976, 'capture_timestamp': 449767433, 'local_timestamp': 21267790454}
{'format': 'RAW', 'raw': {'frame': {'timestamp': 20881486667, 'timescale': 1000000, 'index': 626, 'format': 'YUV420/ABCD/LINEAR/8/PLANAR/HIGH/BE/8', 'info': {'full_range': False, 'color_primaries': 'BT709', 'transfer_function': 'BT709', 'matrix_coefs': 'BT709', 'width': 1280, 'height': 720, 'sar_width': 1, 'sar_height': 1}}}, 'has_errors': 1, 'is_silent': 0, 'is_sync': 0, 'is_ref': 1, 'ntp_timestamp': 20880348411, 'ntp_unskewed_timestamp': 20880348443, 'ntp_raw_timestamp': 20881486667, 'ntp_raw_unskewed_timestamp': 20881486699, 'play_timestamp': 20880349184, 'capture_timestamp': 449800800, 'local_timestamp': 21265067067}
{'format': 'RAW', 'raw': {'frame': {'timestamp': 20881519667, 'timescale': 1000000, 'index': 627, 'format': 'YUV420/ABCD/LINEAR/8/PLANAR/HIGH/BE/8', 'info': {'full_range': False, 'color_primaries': 'BT709', 'transfer_function': 'BT709', 'matrix_coefs': 'BT709', 'width': 1280, 'height': 720, 'sar_width': 1, 'sar_height': 1}}}, 'has_errors': 1, 'is_silent': 0, 'is_sync': 0, 'is_ref': 1, 'ntp_timestamp': 20880381409, 'ntp_unskewed_timestamp': 20880381486, 'ntp_raw_timestamp': 20881519667, 'ntp_raw_unskewed_timestamp': 20881519744, 'play_timestamp': 20880381952, 'capture_timestamp': 449834167, 'local_timestamp': 21265100065}
{'format': 'RAW', 'raw': {'frame': {'timestamp': 20881553667, 'timescale': 1000000, 'index': 628, 'format': 'YUV420/ABCD/LINEAR/8/PLANAR/HIGH/BE/8', 'info': {'full_range': False, 'color_primaries': 'BT709', 'transfer_function': 'BT709', 'matrix_coefs': 'BT709', 'width': 1280, 'height': 720, 'sar_width': 1, 'sar_height': 1}}}, 'has_errors': 1, 'is_silent': 0, 'is_sync': 0, 'is_ref': 1, 'ntp_timestamp': 20880415407, 'ntp_unskewed_timestamp': 20880415528, 'ntp_raw_timestamp': 20881553667, 'ntp_raw_unskewed_timestamp': 20881553788, 'play_timestamp': 20880414720, 'capture_timestamp': 449867533, 'local_timestamp': 21265134063}
{'format': 'RAW', 'raw': {'frame': {'timestamp': 20881586667, 'timescale': 1000000, 'index': 629, 'format': 'YUV420/ABCD/LINEAR/8/PLANAR/HIGH/BE/8', 'info': {'full_range': False, 'color_primaries': 'BT709', 'transfer_function': 'BT709', 'matrix_coefs': 'BT709', 'width': 1280, 'height': 720, 'sar_width': 1, 'sar_height': 1}}}, 'has_errors': 1, 'is_silent': 0, 'is_sync': 0, 'is_ref': 1, 'ntp_timestamp': 20880719025, 'ntp_unskewed_timestamp': 20880719188, 'ntp_raw_timestamp': 20881586667, 'ntp_raw_unskewed_timestamp': 20881586830, 'play_timestamp': 20880719872, 'capture_timestamp': 449900900, 'local_timestamp': 21265437681}
{'format': 'RAW', 'raw': {'frame': {'timestamp': 20881619667, 'timescale': 1000000, 'index': 630, 'format': 'YUV420/ABCD/LINEAR/8/PLANAR/HIGH/BE/8', 'info': {'full_range': False, 'color_primaries': 'BT709', 'transfer_function': 'BT709', 'matrix_coefs': 'BT709', 'width': 1280, 'height': 720, 'sar_width': 1, 'sar_height': 1}}}, 'has_errors': 1, 'is_silent': 0, 'is_sync': 0, 'is_ref': 1, 'ntp_timestamp': 20880752023, 'ntp_unskewed_timestamp': 20880752205, 'ntp_raw_timestamp': 20881619667, 'ntp_raw_unskewed_timestamp': 20881619849, 'play_timestamp': 20880752640, 'capture_timestamp': 449934267, 'local_timestamp': 21265470679}
{'format': 'RAW', 'raw': {'frame': {'timestamp': 20881653667, 'timescale': 1000000, 'index': 631, 'format': 'YUV420/ABCD/LINEAR/8/PLANAR/HIGH/BE/8', 'info': {'full_range': False, 'color_primaries': 'BT709', 'transfer_function': 'BT709', 'matrix_coefs': 'BT709', 'width': 1280, 'height': 720, 'sar_width': 1, 'sar_height': 1}}}, 'has_errors': 1, 'is_silent': 0, 'is_sync': 0, 'is_ref': 1, 'ntp_timestamp': 20883485166, 'ntp_unskewed_timestamp': 20883485351, 'ntp_raw_timestamp': 20881653667, 'ntp_raw_unskewed_timestamp': 20881653852, 'play_timestamp': 20883484672, 'capture_timestamp': 449967633, 'local_timestamp': 21268231007}
{'format': 'RAW', 'raw': {'frame': {'timestamp': 20881686667, 'timescale': 1000000, 'index': 632, 'format': 'YUV420/ABCD/LINEAR/8/PLANAR/HIGH/BE/8', 'info': {'full_range': False, 'color_primaries': 'BT709', 'transfer_function': 'BT709', 'matrix_coefs': 'BT709', 'width': 1280, 'height': 720, 'sar_width': 1, 'sar_height': 1}}}, 'has_errors': 1, 'is_silent': 0, 'is_sync': 0, 'is_ref': 1, 'ntp_timestamp': 20883518169, 'ntp_unskewed_timestamp': 20883518356, 'ntp_raw_timestamp': 20881686667, 'ntp_raw_unskewed_timestamp': 20881686854, 'play_timestamp': 20883517440, 'capture_timestamp': 450000000, 'local_timestamp': 21268264010}
{'format': 'RAW', 'raw': {'frame': {'timestamp': 20881719667, 'timescale': 1000000, 'index': 633, 'format': 'YUV420/ABCD/LINEAR/8/PLANAR/HIGH/BE/8', 'info': {'full_range': False, 'color_primaries': 'BT709', 'transfer_function': 'BT709', 'matrix_coefs': 'BT709', 'width': 1280, 'height': 720, 'sar_width': 1, 'sar_height': 1}}}, 'has_errors': 1, 'is_silent': 0, 'is_sync': 0, 'is_ref': 1, 'ntp_timestamp': 20883551172, 'ntp_unskewed_timestamp': 20883551172, 'ntp_raw_timestamp': 20881719667, 'ntp_raw_unskewed_timestamp': 20881719667, 'play_timestamp': 20883550208, 'capture_timestamp': 450033367, 'local_timestamp': 21268297013}
{'format': 'RAW', 'raw': {'frame': {'timestamp': 20881753667, 'timescale': 1000000, 'index': 634, 'format': 'YUV420/ABCD/LINEAR/8/PLANAR/HIGH/BE/8', 'info': {'full_range': False, 'color_primaries': 'BT709', 'transfer_function': 'BT709', 'matrix_coefs': 'BT709', 'width': 1280, 'height': 720, 'sar_width': 1, 'sar_height': 1}}}, 'has_errors': 1, 'is_silent': 0, 'is_sync': 0, 'is_ref': 1, 'ntp_timestamp': 20881825764, 'ntp_unskewed_timestamp': 20881825764, 'ntp_raw_timestamp': 20881753667, 'ntp_raw_unskewed_timestamp': 20881753667, 'play_timestamp': 20881825792, 'capture_timestamp': 450066733, 'local_timestamp': 21266571605}
{'format': 'RAW', 'raw': {'frame': {'timestamp': 20881786667, 'timescale': 1000000, 'index': 635, 'format': 'YUV420/ABCD/LINEAR/8/PLANAR/HIGH/BE/8', 'info': {'full_range': False, 'color_primaries': 'BT709', 'transfer_function': 'BT709', 'matrix_coefs': 'BT709', 'width': 1280, 'height': 720, 'sar_width': 1, 'sar_height': 1}}}, 'has_errors': 1, 'is_silent': 0, 'is_sync': 0, 'is_ref': 1, 'ntp_timestamp': 20881858764, 'ntp_unskewed_timestamp': 20881858777, 'ntp_raw_timestamp': 20881786667, 'ntp_raw_unskewed_timestamp': 20881786680, 'play_timestamp': 20881858560, 'capture_timestamp': 450100100, 'local_timestamp': 21266604605}
{'format': 'RAW', 'raw': {'frame': {'timestamp': 20881819667, 'timescale': 1000000, 'index': 636, 'format': 'YUV420/ABCD/LINEAR/8/PLANAR/HIGH/BE/8', 'info': {'full_range': False, 'color_primaries': 'BT709', 'transfer_function': 'BT709', 'matrix_coefs': 'BT709', 'width': 1280, 'height': 720, 'sar_width': 1, 'sar_height': 1}}}, 'has_errors': 1, 'is_silent': 0, 'is_sync': 0, 'is_ref': 1, 'ntp_timestamp': 20881891765, 'ntp_unskewed_timestamp': 20881891816, 'ntp_raw_timestamp': 20881819667, 'ntp_raw_unskewed_timestamp': 20881819718, 'play_timestamp': 20881891328, 'capture_timestamp': 450133467, 'local_timestamp': 21266637606}
{'format': 'RAW', 'raw': {'frame': {'timestamp': 20881853667, 'timescale': 1000000, 'index': 637, 'format': 'YUV420/ABCD/LINEAR/8/PLANAR/HIGH/BE/8', 'info': {'full_range': False, 'color_primaries': 'BT709', 'transfer_function': 'BT709', 'matrix_coefs': 'BT709', 'width': 1280, 'height': 720, 'sar_width': 1, 'sar_height': 1}}}, 'has_errors': 1, 'is_silent': 0, 'is_sync': 0, 'is_ref': 1, 'ntp_timestamp': 20881662272, 'ntp_unskewed_timestamp': 20881662397, 'ntp_raw_timestamp': 20881853667, 'ntp_raw_unskewed_timestamp': 20881853792, 'play_timestamp': 20881661952, 'capture_timestamp': 450166833, 'local_timestamp': 21266408113}
{'format': 'RAW', 'raw': {'frame': {'timestamp': 20881886667, 'timescale': 1000000, 'index': 638, 'format': 'YUV420/ABCD/LINEAR/8/PLANAR/HIGH/BE/8', 'info': {'full_range': False, 'color_primaries': 'BT709', 'transfer_function': 'BT709', 'matrix_coefs': 'BT709', 'width': 1280, 'height': 720, 'sar_width': 1, 'sar_height': 1}}}, 'has_errors': 1, 'is_silent': 0, 'is_sync': 0, 'is_ref': 1, 'ntp_timestamp': 20881695271, 'ntp_unskewed_timestamp': 20881695450, 'ntp_raw_timestamp': 20881886667, 'ntp_raw_unskewed_timestamp': 20881886846, 'play_timestamp': 20881694720, 'capture_timestamp': 450200200, 'local_timestamp': 21266441112}
{'format': 'RAW', 'raw': {'frame': {'timestamp': 20881919667, 'timescale': 1000000, 'index': 639, 'format': 'YUV420/ABCD/LINEAR/8/PLANAR/HIGH/BE/8', 'info': {'full_range': False, 'color_primaries': 'BT709', 'transfer_function': 'BT709', 'matrix_coefs': 'BT709', 'width': 1280, 'height': 720, 'sar_width': 1, 'sar_height': 1}}}, 'has_errors': 1, 'is_silent': 0, 'is_sync': 0, 'is_ref': 1, 'ntp_timestamp': 20881728271, 'ntp_unskewed_timestamp': 20881728472, 'ntp_raw_timestamp': 20881919667, 'ntp_raw_unskewed_timestamp': 20881919868, 'play_timestamp': 20881727488, 'capture_timestamp': 450233567, 'local_timestamp': 21266474112}
Real drone:
{'format': 'RAW', 'raw': {'frame': {'timestamp': 9509500, 'timescale': 1000000, 'index': 282, 'format': 'YUV420/ABCD/LINEAR/8/PLANAR/HIGH/BE/8', 'info': {'full_range': False, 'color_primaries': 'BT709', 'transfer_function': 'BT709', 'matrix_coefs': 'BT709', 'width': 1280, 'height': 720, 'sar_width': 1, 'sar_height': 1}}}, 'has_errors': 0, 'is_silent': 0, 'is_sync': 0, 'is_ref': 1, 'ntp_timestamp': 9513185, 'ntp_unskewed_timestamp': 9505322, 'ntp_raw_timestamp': 9509500, 'ntp_raw_unskewed_timestamp': 9501637, 'play_timestamp': 9513185, 'capture_timestamp': 104227317, 'local_timestamp': 868622535}
{'format': 'RAW', 'raw': {'frame': {'timestamp': 9542867, 'timescale': 1000000, 'index': 283, 'format': 'YUV420/ABCD/LINEAR/8/PLANAR/HIGH/BE/8', 'info': {'full_range': False, 'color_primaries': 'BT709', 'transfer_function': 'BT709', 'matrix_coefs': 'BT709', 'width': 1280, 'height': 720, 'sar_width': 1, 'sar_height': 1}}}, 'has_errors': 0, 'is_silent': 0, 'is_sync': 0, 'is_ref': 0, 'ntp_timestamp': 9544863, 'ntp_unskewed_timestamp': 9536968, 'ntp_raw_timestamp': 9542867, 'ntp_raw_unskewed_timestamp': 9534972, 'play_timestamp': 9544863, 'capture_timestamp': 104260667, 'local_timestamp': 868654213}
{'format': 'RAW', 'raw': {'frame': {'timestamp': 9576233, 'timescale': 1000000, 'index': 284, 'format': 'YUV420/ABCD/LINEAR/8/PLANAR/HIGH/BE/8', 'info': {'full_range': False, 'color_primaries': 'BT709', 'transfer_function': 'BT709', 'matrix_coefs': 'BT709', 'width': 1280, 'height': 720, 'sar_width': 1, 'sar_height': 1}}}, 'has_errors': 0, 'is_silent': 0, 'is_sync': 0, 'is_ref': 0, 'ntp_timestamp': 9578229, 'ntp_unskewed_timestamp': 9570321, 'ntp_raw_timestamp': 9576233, 'ntp_raw_unskewed_timestamp': 9568325, 'play_timestamp': 9578229, 'capture_timestamp': 104294033, 'local_timestamp': 868687579}
{'format': 'RAW', 'raw': {'frame': {'timestamp': 9609600, 'timescale': 1000000, 'index': 285, 'format': 'YUV420/ABCD/LINEAR/8/PLANAR/HIGH/BE/8', 'info': {'full_range': False, 'color_primaries': 'BT709', 'transfer_function': 'BT709', 'matrix_coefs': 'BT709', 'width': 1280, 'height': 720, 'sar_width': 1, 'sar_height': 1}}}, 'has_errors': 0, 'is_silent': 0, 'is_sync': 0, 'is_ref': 1, 'ntp_timestamp': 9611595, 'ntp_unskewed_timestamp': 9603670, 'ntp_raw_timestamp': 9609600, 'ntp_raw_unskewed_timestamp': 9601675, 'play_timestamp': 9611595, 'capture_timestamp': 104327400, 'local_timestamp': 868720945}
{'format': 'RAW', 'raw': {'frame': {'timestamp': 9642967, 'timescale': 1000000, 'index': 286, 'format': 'YUV420/ABCD/LINEAR/8/PLANAR/HIGH/BE/8', 'info': {'full_range': False, 'color_primaries': 'BT709', 'transfer_function': 'BT709', 'matrix_coefs': 'BT709', 'width': 1280, 'height': 720, 'sar_width': 1, 'sar_height': 1}}}, 'has_errors': 0, 'is_silent': 0, 'is_sync': 0, 'is_ref': 0, 'ntp_timestamp': 9643174, 'ntp_unskewed_timestamp': 9635228, 'ntp_raw_timestamp': 9642967, 'ntp_raw_unskewed_timestamp': 9635021, 'play_timestamp': 9643174, 'capture_timestamp': 104360767, 'local_timestamp': 868752524}
{'format': 'RAW', 'raw': {'frame': {'timestamp': 9676333, 'timescale': 1000000, 'index': 287, 'format': 'YUV420/ABCD/LINEAR/8/PLANAR/HIGH/BE/8', 'info': {'full_range': False, 'color_primaries': 'BT709', 'transfer_function': 'BT709', 'matrix_coefs': 'BT709', 'width': 1280, 'height': 720, 'sar_width': 1, 'sar_height': 1}}}, 'has_errors': 0, 'is_silent': 0, 'is_sync': 0, 'is_ref': 0, 'ntp_timestamp': 9676540, 'ntp_unskewed_timestamp': 9668581, 'ntp_raw_timestamp': 9676333, 'ntp_raw_unskewed_timestamp': 9668374, 'play_timestamp': 9676540, 'capture_timestamp': 104394117, 'local_timestamp': 868785890}
{'format': 'RAW', 'raw': {'frame': {'timestamp': 9709700, 'timescale': 1000000, 'index': 288, 'format': 'YUV420/ABCD/LINEAR/8/PLANAR/HIGH/BE/8', 'info': {'full_range': False, 'color_primaries': 'BT709', 'transfer_function': 'BT709', 'matrix_coefs': 'BT709', 'width': 1280, 'height': 720, 'sar_width': 1, 'sar_height': 1}}}, 'has_errors': 0, 'is_silent': 0, 'is_sync': 0, 'is_ref': 1, 'ntp_timestamp': 9709907, 'ntp_unskewed_timestamp': 9701932, 'ntp_raw_timestamp': 9709700, 'ntp_raw_unskewed_timestamp': 9701725, 'play_timestamp': 9709907, 'capture_timestamp': 104427483, 'local_timestamp': 868819257}
{'format': 'RAW', 'raw': {'frame': {'timestamp': 9743067, 'timescale': 1000000, 'index': 289, 'format': 'YUV420/ABCD/LINEAR/8/PLANAR/HIGH/BE/8', 'info': {'full_range': False, 'color_primaries': 'BT709', 'transfer_function': 'BT709', 'matrix_coefs': 'BT709', 'width': 1280, 'height': 720, 'sar_width': 1, 'sar_height': 1}}}, 'has_errors': 0, 'is_silent': 0, 'is_sync': 0, 'is_ref': 0, 'ntp_timestamp': 9735377, 'ntp_unskewed_timestamp': 9727381, 'ntp_raw_timestamp': 9743067, 'ntp_raw_unskewed_timestamp': 9735071, 'play_timestamp': 9735377, 'capture_timestamp': 104460850, 'local_timestamp': 868844727}
{'format': 'RAW', 'raw': {'frame': {'timestamp': 9776433, 'timescale': 1000000, 'index': 290, 'format': 'YUV420/ABCD/LINEAR/8/PLANAR/HIGH/BE/8', 'info': {'full_range': False, 'color_primaries': 'BT709', 'transfer_function': 'BT709', 'matrix_coefs': 'BT709', 'width': 1280, 'height': 720, 'sar_width': 1, 'sar_height': 1}}}, 'has_errors': 0, 'is_silent': 0, 'is_sync': 0, 'is_ref': 0, 'ntp_timestamp': 9768746, 'ntp_unskewed_timestamp': 9760750, 'ntp_raw_timestamp': 9776433, 'ntp_raw_unskewed_timestamp': 9768437, 'play_timestamp': 9768746, 'capture_timestamp': 104494217, 'local_timestamp': 868878096}
Q: Is this flag supposed to be supported on simulator?