Merge with PX4 upstream?

[mrpollo]

Hey @jonas-eschmann, this project is exciting, and it might benefit from being ported over to PX4 upstream, where it makes sense, so it doesn't go stale over time. It reduces your maintenance burden. I'm happy to help you with the porting and introduce you to other maintainers who can assist as well.

[jonas-eschmann]

Hey @mrpollo,

Thank you for reaching out! Yes, I believe that would make sense! Here are some concrete steps that I think are necessary before that makes sense:

1. Split out the "core" of RLtools: Currently, we organize RLtools as a kind of monorepo where the top level (including the RLtools source) is lean, and we have submodules to track everything else. Some of these submodules are quite heavy (like test data, PX4 itself, etc.), so I don't think you would want that amount of recursive dependencies in your tree. It has been a goal of mine to spin out the header-only core (+ maybe some target source and tests but 0 submodules) and include it as a submodule in the main repo. Then this minimal core (~8MB including full commit history IIRC) would be the only dependency required to run the RAPTOR policy.

2. Better integration into the PX4 controller stack. Right now, we are just bypassing all of it. We created an additional topic actuator_motors_mux and swapped the input of FunctionMotors.hpp to the new topic. This is the only change to PX4 proper that is required. Everything else happens in an external/OOT module right now. This topic is served by a slim multiplexer module that either selects the outputs of the RL policy or of the PX4 control stack. This way, we can easily switch the controllers, even in mid-air. Last time I checked, there seemed to be some cross-dependencies between the control stack and other safety supervisors, etc., so some guidance on how to best integrate it might be helpful such that we end up in a maintainable state.

3. RLtools' philosophy heavily emphasizes compile-time configuration, while PX4 is very run-time configurable. The main point of contact is the IMU_GYRO_RATEMAX here. Currently, the "gear-ratio" is set at compile time (and there are some dmesg warnings if the actual rate deviates too much). Furthermore, optimally, the inference control frequency should be a multiple of the training control frequency (which is 100Hz), so the 250Hz setting is not matching greatly. Practically, I don't expect the latter to be a big issue because the RAPTOR policy is quite robust against these kinds of deviations.

So the major points would be 1) and 2), 3) should be quite straightforward. I would be very interested in your feedback on point 2) in particular. The RAPTOR policy works well with position and velocity set points as inputs and outputs direct motor commands. So I would assume that sets the boundaries/location in the stack quite clearly, but I would be very interested in your take!

Thank you for reaching out again!

[AlexKlimaj]

It sounds like you can use the neural network module as a template for upstreaming.

https://docs.px4.io/main/en/advanced/neural_networks.html

[Pedro-Roque]

I think that item 2) should be somewhat "easy" to integrate into the control allocator module, since that's where actuator_motors are generated. I do understand that muxing after it is simpler, as the module is quite convoluted, but we might be able to work this out. I think we should actually take advantage of these works (NN and RAPTOR) to motivate some of the control allocator upgrades. Something to bring up at the next call.

[Pedro-Roque]

@jonas-eschmann not sure if you talked to @Jaeyoung-Lim about this, I think it could be a good push for upstream

[Jaeyoung-Lim]

@jonas-eschmann Go go go!

[jonas-eschmann]

@Pedro-Roque @Jaeyoung-Lim working on it here: jonas-eschmann/PX4-Autopilot. Should hopefully be ready for a PR soon