@rwn17
First of all, thank you very much for sharing this excellent work!
I noticed that the official repository currently only releases the inference code. It was also mentioned in Issue #7 that the training code is planned for release this June. In order to train models on our own data based on the Fin3R method—rather than just loading pre-trained LoRA weights for inference—I would like to kindly request the release of the complete training code.
Based on my analysis of the existing codebase, I believe the complete training code should include the following key modules:
-
Teacher Model Abstraction
The paper mentions using monocular teacher models (e.g., MiDaS, DPT, etc.) for knowledge distillation, but the current code does not show a unified interface for interacting with teacher models. I hope the released code can provide a clear teacher model abstraction layer to facilitate the replacement of different teacher models.
-
Teacher Target Dataset/Cache
Distillation training requires pre-computing or generating teacher model outputs (e.g., depth maps, normal maps, etc.) on the fly. For large-scale training, an efficient pre-computation and caching mechanism is crucial. I look forward to seeing the implementation of this aspect.
-
Distillation Loss
This is the core of Fin3R. I hope the released code can fully implement the various distillation losses described in the paper (e.g., L1/L2 loss, perceptual loss, gradient loss, etc.) and allow users to flexibly combine them.
-
LoRA Training Initialization & Adapter Checkpoint Management
The existing code demonstrates how to load LoRA weights (model.apply_lora()), but lacks the code for training these LoRA weights. I hope the release includes complete LoRA initialization, training loops, and adapter checkpoint saving/loading logic.
-
Encoder/Decoder Freezing Strategy
One of the core strategies of Fin3R is to "freeze the decoder and only fine-tune the encoder". I hope the training code clearly demonstrates how to implement this parameter grouping and freezing logic.
-
Dataset Recipe
The paper mentions sampling 20,000 images from SA-1B, 1,000 from Hypersim, and 1,000 from TartanAir per epoch. I hope the release provides the specific preprocessing scripts and configurations for these datasets.
Thank you again for your open-source contribution. I look forward to the release of the training code!
Fei
@rwn17
First of all, thank you very much for sharing this excellent work!
I noticed that the official repository currently only releases the inference code. It was also mentioned in Issue #7 that the training code is planned for release this June. In order to train models on our own data based on the Fin3R method—rather than just loading pre-trained LoRA weights for inference—I would like to kindly request the release of the complete training code.
Based on my analysis of the existing codebase, I believe the complete training code should include the following key modules:
Teacher Model Abstraction
The paper mentions using monocular teacher models (e.g., MiDaS, DPT, etc.) for knowledge distillation, but the current code does not show a unified interface for interacting with teacher models. I hope the released code can provide a clear teacher model abstraction layer to facilitate the replacement of different teacher models.
Teacher Target Dataset/Cache
Distillation training requires pre-computing or generating teacher model outputs (e.g., depth maps, normal maps, etc.) on the fly. For large-scale training, an efficient pre-computation and caching mechanism is crucial. I look forward to seeing the implementation of this aspect.
Distillation Loss
This is the core of Fin3R. I hope the released code can fully implement the various distillation losses described in the paper (e.g., L1/L2 loss, perceptual loss, gradient loss, etc.) and allow users to flexibly combine them.
LoRA Training Initialization & Adapter Checkpoint Management
The existing code demonstrates how to load LoRA weights (model.apply_lora()), but lacks the code for training these LoRA weights. I hope the release includes complete LoRA initialization, training loops, and adapter checkpoint saving/loading logic.
Encoder/Decoder Freezing Strategy
One of the core strategies of Fin3R is to "freeze the decoder and only fine-tune the encoder". I hope the training code clearly demonstrates how to implement this parameter grouping and freezing logic.
Dataset Recipe
The paper mentions sampling 20,000 images from SA-1B, 1,000 from Hypersim, and 1,000 from TartanAir per epoch. I hope the release provides the specific preprocessing scripts and configurations for these datasets.
Thank you again for your open-source contribution. I look forward to the release of the training code!
Fei