force exerted on the feet

[CoderWangcai]

Thank you very much for open-sourcing such excellent work.
Many robots do not have foot force data. May I ask if it can still work?

[ylenianistico]

Thank you very much for open-sourcing such excellent work. Many robots do not have foot force data. May I ask if it can still work?

Hi @CoderWangcai! Thanks for opening the first issue!

Unfortunately, contact detection is necessary for proprioceptive state estimation, so the current code version would not work without it. But it is in my short-term plans to add exteroception as soon as possible, and in that case, it will still work without contact detection.

In the meantime, can I ask you which kind of robot you are working with? Just to understand if I can give you some "workaround" to have the current version of MUSE working with your robot.

[CoderWangcai]

Thank you very much for open-sourcing such excellent work. Many robots do not have foot force data. May I ask if it can still work?

Hi @CoderWangcai! Thanks for opening the first issue!

Unfortunately, contact detection is necessary for proprioceptive state estimation, so the current code version would not work without it. But it is in my short-term plans to add exteroception as soon as possible, and in that case, it will still work without contact detection.

In the meantime, can I ask you which kind of robot you are working with? Just to understand if I can give you some "workaround" to have the current version of MUSE working with your robot.

Thank you very much for your prompt reply! It’s reassuring to see how quickly you address questions, and I’m also excited to hear about the exteroception features you plan to add.

In the meantime, I’m working with a custom-built quadruped robot that doesn’t have foot-end sensors like those on commercial models (e.g., from Unitree). I’m wondering if there’s any workaround or temporary solution that would let me run the current version of MUSE without contact detection. If you have any suggestions or related resources, I would greatly appreciate it.

Thanks again for your time, and please keep me posted on any updates!

[ylenianistico]

@CoderWangcai you could compute the ground reaction forces from the joint torques. The procedure is actually explained in the paper!

I will provide a complete code implementation soon (it's already done, I just need to clean up the code a bit). I'll keep you posted

[lnotspotl]

@ylenianistico Will the code incorporating exteroceptive information be also included in the next release? Can we expect that any time soon. Would love to give that a go.

Regardless, thank you for open-sourcing muse. It's a great piece of software. 🤗

Also, I'd like to ask a question regarding an implementation detail. During the first iteration, the sensor fusion plugin stores the world to base rotation matrix which is then used throughout the rest of time on line 154. Is this correct? Will this not pose an issue when the robot's orientation changes?

[ylenianistico]

@ylenianistico Will the code incorporating exteroceptive information be also included in the next release? Can we expect that any time soon. Would love to give that a go.

Regardless, thank you for open-sourcing muse. It's a great piece of software. 🤗

Also, I'd like to ask a question regarding an implementation detail. During the first iteration, the sensor fusion plugin stores the world to base rotation matrix which is then used throughout the rest of time on line 154. Is this correct? Will this not pose an issue when the robot's orientation changes?

Hi @lnotspotl , thanks a lot for your kind words and for taking the time to dive into the details!

1. Yes, the code incorporating exteroceptive information will be included in the next release, I’m aiming to have it out by the end of July (hopefully :') )

2. You're right about the implementation detail you pointed out. Thanks for catching an error! Fortunately, in practice, the orientation drift shouldn't impact too heavily, since the vertical direction tends to remain relatively stable. That said, it should be corrected. The correct expression should be: u = w_R_b * f_b + gravity;
   I'll make sure to fix this soon in the codebase. Really appreciate the feedback!

[lnotspotl]

Awesome. Thanks for the prompt response.

One more question I'd like to ask. Is there any reason to believe muse should not work on other robot embodiments? In my view, deploying this framework on G1, for instance, should be as easy as slightly modifying the code to reflect the fact that G1 only has two legs instead of four. Am I missing something does that align with your expertise?

If this is the case, maybe muse could be generalized to work on any legged machine (with the assumption of a single point contact, not really sure how valid it is for different robots). I'd be willing to contribute the changes to muse in that case. Let me hear your thoughts on this first though.

[ylenianistico]

Awesome. Thanks for the prompt response.

One more question I'd like to ask. Is there any reason to believe muse should not work on other robot embodiments? In my view, deploying this framework on G1, for instance, should be as easy as slightly modifying the code to reflect the fact that G1 only has two legs instead of four. Am I missing something does that align with your expertise?

If this is the case, maybe muse could be generalized to work on any legged machine (with the assumption of a single point contact, not really sure how valid it is for different robots). I'd be willing to contribute the changes to muse in that case. Let me hear your thoughts on this first though.

In theory, it should work on a humanoid as well! The attitude estimation module shouldn't require any major changes, just adjustments like the IMU-to-base frame rotations in the config file. The leg odometry and contact estimation code would need some modifications, since it's currently written for a robot with four legs, but the underlying logic remains the same for a humanoid as well.
If you're interested in trying muse on G1, I'd be absolutely thrilled and happy to help you get started 🥹

[lnotspotl]

Certainly! I'd love to start integrating muse for G1 to tbai next week, so I'll definitely keep you updated. In fact, we already have integrated muse into tbai, it works nicely for Anymal D and Go2. Cannot wait to see how muse performs once the exteroceptive estimation support is out.

[ylenianistico]

Very happy to hear that! If you have modified the code and you want, you can very well open a pull request...I will be happy to merge it @lnotspotl

[lnotspotl]

Hey @ylenianistico,

things got a little different priorities so I did not get around to actually testing the state estimator on the actual robots of ours, though it's in our backlog and I'll get to this soon.

[ylenianistico]

Hey @ylenianistico,

things got a little different priorities so I did not get around to actually testing the state estimator on the actual robots of ours, though it's in our backlog and I'll get to this soon.

I understand the feeling 🥲 I'm also late with the new release

[chenxiaocongAI]

Hello, @ylenianistico may I ask if the code for calculating end effector force through joint torque has not been open-source yet? Can you give me an example to calculate the force myself, because I want to estimate whether the robot is in contact with the ground without a force sensor. In addition, I did not see the skid detection module function in the code.
I would like to know how to get the desired speed and position in the calculation of the slip process.

[ylenianistico]

Hi @chenxiaocongAI, sorry, I didn't have time to open-source the code yet. If you want to know how to compute the force yourself, then you can read MUSE paper, or this one.

Regarding the slip detector, the desired speed and position are given by the specific controller you are using on your robot. That's why I didn't include it in this repo

[Abban-Fahim]

Hi @ylenianistico! Is there any update on fusing exteroceptive sensing into this repo. And out of curiosity, what modalities are planned to be supported (like depth maps, point clouds, range data etc.)?

Though it has been said on this thread numerous times before, thank you for open-sourcing this software :)