Sphinx 1.8 - Camera stops responding after a certain period every time

[danielizham]

Hello there Parrot Team,

I have been using Sphinx 1.8 to train an Anafi drone using RL and the experience has been great. However, there is a problem with the camera that always cuts the training short forcing me to break the training into many runs instead of one.

What is strange to me is that the camera stops working after the same number of time steps of training every run and that, even after this happens, the movement commands still work as usual.

I believe it is nothing to do with the system requirements because I’m using Ubuntu 18.04, RTX 3060 and plenty of RAM and the resources have never been completely used up. Also, the image on the simulated drone has always been deleted after processing and I have always checked that the drone’s storage (according to http://10.202.0.1/api/v1/media/medias) was empty.

Sometimes I receive this error message as the camera stops working but not always:

[Err] [ShmFrameServer.cc:118] Waited too long in horizontal_came

I’d appreciate if you can help me debug the problem. Thank you.

Regards,
Daniel Izham

[GCid]

I am experiencing the same problem (coincidentally, I am trying to do the same thing - RL).
I made a post a while back but received no answer.
Basically, I have the about same setup (same generation GPU, plenty of RAM and CPU) as you and delete the photos after processing each one. It surely has nothing to do with that, as the simulation speed is always above 0.9.
Hope this time we get some help from the developers.

Using the latest version of sphinx doesn’t help either. For some reason it doesn’t capture photos at all.

[danielizham]

Thank you @GCid for the extra information especially the fact the latest Sphinx is also showing the same problem. I hope it’s about a feature which has not been clearly documented that the Parrot Team can enlighten us about.

At the moment, I am trying instead to parse the data field coming out of one of the parrot-gz topics:

parrot-gz topic -e /gazebo/default/anafi4k/gimbal_2/horizontal_camera/image

hoping that it won’t affect the performance of the camera as using Olympe does. However, it has not been as straightforward as I thought because of the format of the data field.

[GCid]

Hello again!

A new olympe version was release some weeks ago (ver. 7.1) which according to the logs should have fixed our issue.
I have tested it out but it seems that the issue still persists, although this time there is no error logged from Sphinx.
Can you try it too and let me know?
In the meantime, I have contacted parrot developer support via e-mail.

I will also try the parrot-gz topic approach you are following.
Could you share the code you’re using to parse the image?
Or a high-level algorithm would be useful either way!

[danielizham]

Hi @GCid,

Thank you for the extra information. I did not notice that about the new version. I have actually moved on to applications that can do without the camera. But I will try it and get back to you.

With regards to the parrot-gz topic, shortly after posting my message, I stopped working on parsing it when I found out that the topic depends directly on the simulated camera as well. So after the camera fails, no data comes out of the topic.

If you want to try parsing the data, you can try these two techniques which I have found to be the most promising but they are done outside of the Python code: Convert Gazebo camera ImageStamped message to OpenCV Mat and Save camera images as jpeg files.

[GCid]

I see!
I have also moved away from using the camera, but still need it for the future.
At the time of my first reply I contacted parrot developer support via email and they let me know that someone was assign to the issue, so I hope it will get resolved at some point.

Thank you for the parrot-gz topic pointers!