Thank you for sharing this excellent work.
The paper mentions that APPO does not require SFT or CoT. However, reinforcement learning methods may encounter issues when there is no cold start, such as sparse reward signals during early deployment and inefficient gradient estimation. Does APPO also suffer from these issues? Have you tried performing SFT before APPO training, or using a method similar to the Hybrid Online-Offline Training adopted in EasyVideoR1? Would performance be higher if APPO training were conducted after SFT?
Additionally, could you please provide the evaluation code?
Also, it seems that the project does not include a requirements.txt. Could you please provide one?
Looking forward to your reply.
Thank you for sharing this excellent work.
The paper mentions that APPO does not require SFT or CoT. However, reinforcement learning methods may encounter issues when there is no cold start, such as sparse reward signals during early deployment and inefficient gradient estimation. Does APPO also suffer from these issues? Have you tried performing SFT before APPO training, or using a method similar to the Hybrid Online-Offline Training adopted in EasyVideoR1? Would performance be higher if APPO training were conducted after SFT?
Additionally, could you please provide the evaluation code?
Also, it seems that the project does not include a requirements.txt. Could you please provide one?
Looking forward to your reply.