Parsing "tlm-data-logger"

Hey there!

I am willing to use the data logger CLI tool a.k.a. tlm-data-logger to provide feedback to ML models. Despite I know there are parsing tools available for logs recorded with the --datalog option at Sphinx, I know no script with the same functionality for tlm-data-logger.

Do you use or know any? Do you mind sharing it? If not, what would be the easiest way to parse its output in a nearly-real-time fashion? I’d run it like tlm-data-logger ... |

Thanks in advance!



You probably want to look at the telemetry library itself:

You need a workspace to build this library. You can use the ARSDK workspace if you already have it installed:
The ARSDK workspace does not contain the telemetry library but you can clone it inside the packages directory along with its dependencies (I believe libshdata is required).

Then, inside your workspace, you can build tlm-data-logger:

./ -p arsdk-native -A tlm-data-logger

And execute it while sphinx is running:

. out/arsdk-native/staging/
tlm-data-logger -r 0 inet:

Now, you can have a look at the source code in telemetry/tools/gndctrl_datalogger.cpp. Basically, GndCtrlApp::sample is called each time a new sample is received. The parameters sectionId and varId can help you identify the variables you are interested in.

If you plan on parsing your data in python, you may prefer to reproduce the code contained in this file directly in your application:

It works the same way as tlm-data-logger.

It’d be nice to see some comments in the code, it’s quite daunting to get into such a file when even the filename isn’t understandable.

Anyway, so far I’ve been trying to run tlm-data-logger using a pipe

tlm-data-logger inet: | rostopic pub sphinx/raw std_msgs/String

also like

tlm-data-logger inet: |

Neither way I got the data piped from the tlm tool. I think this may be because the first process piped doesn’t end, so the next one isn’t executed.

I am currently logging all the data generated in a file, then reading it from a Python script

tlm-data-logger inet: > telemetry_log.txt

However, this way is quite inefficient. With this knowledge, am I able to reproduce some of your code or toolings in a such a way that the telemetry datalogs can be directly inputted in a Python script (without using intermediate files) ?

Thanks in advance!

Also, this script has heavy dependencies on pomp Py module, which seems to be somewhat related to Parrot’s libpomp. I tried installing both and running the script, but I’m still getting pomp-related errors. Is there any documentation available?

Thank you