Clarification on baseline model settings for NTP vs. teacherless (Seq2Seq config error)

[YaraLiao]

Dear authors,

I’m reproducing the baselines from Roll the dice & look before you leap and I’m unsure which model setting to use for the NTP and teacherless (multi-token) setups.

When I run your sibling-discovery/teacherless training script, I hit:

ValueError: You must specify a Seq2Seq config OR encoder_type, encoder_name, and decoder_name OR encoder_type and encoder_decoder_name

This comes from simpletransformers.seq2seq.Seq2SeqModel initialization.Could you please clarify the intended backbone/configuration for each baseline?

Thank you for your time and for the excellent benchmark.

[ChenWu98]

Hi! Can you try to install the simpletransformers under this repo (via pip install -e .) instead of the official, latest version? I believe this issue will be resolved after that.

[YaraLiao]

Thank you for your reply! It works for me.
Btw, I would like to ask if you have re-trained the SEDD model? Or did you perform a fine-tuning instead?

[Nazzyyj]

Hi @YizhenLiao ,
Did you reproduce the gemma2b experiments of ntp and teacherless? The results I got from ntp are relatively low, and looks weird. Can you achieve the result in the paper under the authors' setting? Looking foward to your reply ~