Obstacle Avoidance with Sphinx

Hi,

I’m trying the obstacle avoidance of Anafi AI in sphinx in parrot-ue4-sphx-tests with obstacle_avoidance map.

Unfortunately, I cannot move drone by manual command, it is stuck in the same position after the takeoff.

Any ideas ?

Best,
Clément

What does the ulog say when your manual commands are executed?

Hi @DonMMK,

I have the getLastStereoFrame:226: videoFrame err=22(Invalid argument) error in ULOGS.
But no one was able to help me on this : getLastStereoFrame:226: videoFrame err=22(Invalid argument) - #15

It looks like I’m alone to have this error …

Best,
Clément

Here are the full ULOGS :
log.doc (952.7 KB)

Moreover, the displayed Trajectory has an offset from the drone’s position :

Can you provide us with more info:

  • The output of fdc show instances <instance_name>
  • sphinx logs
  • Your sphinx command line
  • Does sphinx-cli camera depth_camera show depth frames?

Thanks!

Hi @ocrave,

  • The output of fdc show instances tremulous_johanna :
name: tremulous_johanna
sha1: 99d3f0a4c8301d51a53140a62541306da96f8278
base_workspace: /var/cache/firmwared/mount_points//instances/99d3f0a4c8301d51a53140a62541306da96f8278
id: 0
pid: 295740
state: started
firmware_path: /usr/share/firmwared/firmwares//anafi2-pc.ext2.zip.84f64f7e-1fdc-4b00-9a00-ed688a2bb04c.firmware
root: /var/cache/firmwared/mount_points//instances/99d3f0a4c8301d51a53140a62541306da96f8278/union
fs_size: unlimited
inner_pts: /dev/pts/5
outer_pts: /dev/pts/4
time: Fri Jun 24 10:33:44 2022
virt_iface: eth1
stolen_interface: eth0:eth0:192.168.42.1/24
cmdline: LD_PRELOAD=librt.so:libc.so.6:libpthread.so.0:libstdc++.so.6 /sbin/boxinit ro.boot.console=/pts/5 ro.hardware=anafi2 ro.model=anafi2 ro.debuggable=1 ro.revision=pv1 ro.debug_ipaddr=10.202.0.1 wifid.bridge=0 ro.simulator.kernel.version=5.13.0-51-generic ro.simulator.kernel.build=#58~20.04.1-Ubuntu_SMP_Tue_Jun_14_11:29:12_UTC_2022 ro.simulator.host.arch=x86_64 ro.simulator.host.user=cllebi ro.simulator.host.ip0=172.17.0.1 ro.simulator.host.mac0=02:42:ff:6f:8a:7b ro.simulator.host.ip1=192.168.0.179 ro.simulator.host.mac1=34:48:ed:84:14:7e ro.simulator.gpu.hardware=UHD_Graphics_630_(Mobile) ro.simulator.gpu.driver=driver=i915_latency=0 ro.simulator.gpu.nvidia.info=NVIDIA_UNIX_x86_64_Kernel_Module__510.73.05__Sat_May__7_05:30:26_UTC_2022 ro.simulator.host.name=pp257l.ad.phedre.fr ro.simulator.os.version=Ubuntu_20.04.4_LTS ro.simulator.sphinx.version=2.8.2 ro.with-front-cam=1 ro.gimbald=1 ro.depth.hil=0 ro.tracking.hil=0 ro.storage.sphinx_pattern=__undefined__
debug_ipaddr: 10.202.0.1
mountbind:
wan_access: 0
remote_ctrl_ip:
hil_active: 0
int_storage_size: 500MB
ext_storage_size: 10G
owner: pp257l.ad.phedre.fr-293293
  • Does sphinx-cli camera depth_camera show depth frames ? YES

  • My Sphinx Command Lines :

sudo systemctl start firmwared.service
sphinx "/opt/parrot-sphinx/usr/share/sphinx/drones/anafi_ai.drone"::firmware=`pwd`/anafi2-pc.ext2.zip::p
ose="SUV" & parrot-ue4-sphx-tests -ams-config-file=demo.yaml -ams-path=DefaultPath,Suv
  • My Sphinx logs :
Parrot-Sphinx simulator version 2.8.2

Connecting to firmwared version: 2.8.2
starting Telemetryd to capture true data...
Gazebo multi-robot simulator, version 11.2.0
Copyright (C) 2012 Open Source Robotics Foundation.
Released under the Apache 2 License.
http://gazebosim.org

[Msg] Waiting for master.
[Msg] Connected to gazebo master @ http://127.0.0.1:11345
[Msg] Publicized address: 127.0.0.1
[Msg] created parameter server on http:8383
[Msg] Waiting for a UE4 application to start...
[Msg] Waiting for a UE4 application to start...
[Msg] Waiting for a UE4 application to start...
[Msg] Waiting for a UE4 application to start...
[Msg] Waiting for a UE4 application to start...
[Msg] Waiting for a UE4 application to start...
[Msg] Waiting for a UE4 application to start...
[Msg] Waiting for a UE4 application to start...
[Msg] Waiting for a UE4 application to start...
[Msg] Waiting for a UE4 application to start...
[Msg] World params for 'default':
[Msg] 	spawn_points:
[Msg] 		- name: suv
[Msg] 		  pose: 3.69923 0.399228 1.77178 0.000556 -0.000556 2e-06
[Msg] 		- name: default
[Msg] 		  pose: 0 0 0.2 0 -0 0
[Msg] connected to firmwared
[Msg] Preparation of firmware /home/cllebi/Documents/NEXTER/nexter-poc-drone/sphinx/anafi2-pc.ext2.zip
[Msg] firmware /usr/share/firmwared/firmwares//anafi2-pc.ext2.zip.84f64f7e-1fdc-4b00-9a00-ed688a2bb04c.firmware supported hardwares: 
[Msg] 	anafi2
[Msg] All firmwares prepared.
[Msg] Selected params to build .sdf file for anafi_ai:
	param drone_iface = eth0
	param drone_ip = 192.168.42.1/24
	param hil_ip = auto
	param invisible_body = 0
	param photo_high_res = 0
	param sdcard_serial = __undefined__
	param virt_iface = eth1
	param with_depth_hil = 0
	param with_external_cams = 1
	param with_front_cam = 1
	param with_front_stcam = 1
	param with_gimbal = 1
	param with_tracking_hil = 0
	param xmlns = 
[Msg] Drone anafi_ai[99d3f0a4c8301d51a53140a62541306da96f8278] loaded
[Msg] Using Wifi interface 'eth0'
[Msg] Web dashboard is accessible at http://localhost:9002
[Msg] Instance tremulous_johanna[99d3f0a4c8301d51a53140a62541306da96f8278] restarted
[Msg] All drones instantiated

I hope it can help to find the issue …

In the output of fdc we can see that your driver is the intel one:

ro.simulator.gpu.hardware=UHD_Graphics_630_(Mobile) ro.simulator.gpu.driver=driver=i915_latency=0 ro.simulator.gpu.nvidia.info=NVIDIA_UNIX_x86_64_Kernel_Module__510.73.05__Sat_May__7_05:30:26_UTC_2022

I am wondering if you need to run this command to select the nvidia driver:

prime-select nvidia

Otherwise, I don’t see any issue.

Hi @ocrave,

Indeed it seems that sphinx isn’t using the right gpu hardware. Unfortunately, even with sudo prime-select nvidia, the fdc are the same :

ro.simulator.gpu.hardware=UHD_Graphics_630_(Mobile) ro.simulator.gpu.driver=driver=i915_latency=0 ro.simulator.gpu.nvidia.info=NVIDIA_UNIX_x86_64_Kernel_Module__510.73.05__Sat_May__7_05:30:26_UTC_2022

Is there a way to force Sphinx to use my Nvidia GPU ?

I’ve never encounter this issue with all other big software like Blender, Carla, etc. Hope there is a solution …

Best,
Clément

I think that the driver displayed by fdc is not the one used by Unreal. To make sure that Unreal detects your Nvidia GPU, you can start any Unreal app with this command line option:

-LogCmds=\"global log, logchecksubobjects none\"'

It should display the list of GPUs and the one it selects. If Unreal selects your Nvidia GPU, you don’t to worry about the output of fdc.

Hi @ocrave,

Here is the full log :
log.doc (224.5 KB)

It seems to use my GPU :

[2022.06.27-08.07.15:736][  0]LogVulkanRHI: Display: Device 0: Intel(R) UHD Graphics 630 (CFL GT2)
[2022.06.27-08.07.15:736][  0]LogVulkanRHI: Display: Device 1: llvmpipe (LLVM 12.0.0, 256 bits)
[2022.06.27-08.07.15:737][  0]LogVulkanRHI: Display: Device 2: Quadro T1000
[2022.06.27-08.07.15:738][  0]LogVulkanRHI: Display: Using Device 2: Geometry 1 Tessellation 1 BufferAtomic64 1

Also, my nvidia-smi show that gzserver and UnrealApp are using my Quatro T1000 :

nvidia-smi
Mon Jun 27 10:12:09 2022       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.73.05    Driver Version: 510.73.05    CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Quadro T1000        Off  | 00000000:01:00.0  On |                  N/A |
| N/A   51C    P0    17W /  N/A |   2627MiB /  4096MiB |     31%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      2138      G   /usr/lib/xorg/Xorg                 65MiB |
|    0   N/A  N/A      3509      G   /usr/lib/xorg/Xorg                390MiB |
|    0   N/A  N/A      4243      G   /usr/bin/gnome-shell               70MiB |
|    0   N/A  N/A      6912      G   ...RendererForSitePerProcess      129MiB |
|    0   N/A  N/A      7977      G   /usr/lib/firefox/firefox          185MiB |
|    0   N/A  N/A     44362      G   evolution                          35MiB |
|    0   N/A  N/A     71109    C+G   .../Binaries/Linux/UnrealApp     1733MiB |
|    0   N/A  N/A     71537      G   gzserver                            6MiB |
+-----------------------------------------------------------------------------+

Do you have other idea to fix this 01-01 01:00:46.481 E guidance (guidance-376) : getLastStereoFrame:226: videoFrame err=22(Invalid argument) error ?

Best,
Clément

OK, so there is no issue with your GPU since Unreal selects the right one.

I am unable to reproduce your getLastStereoFrame issue. I tried with several firmware versions. It is an important issue because it means that guidance fails to read depth frames.

Do you always have this issue? You can try this command to restart guidance:

adb connect 10.202.0.1:9050 ; adb shell prestart guidance

HI @ocrave,

Yes always have this issue on this computer.
The problem still occurs after running the command you sent.

Best,
Clément