Questions on reproducing the paper (datasets, multi-node, compute, logs)

[JihwanEom]

Hi! Thank you for releasing such a clean and inspiring piece of work.
I read the paper carefully and really enjoyed it — the minimalist generative VLP direction feels genuinely refreshing, and as someone who also liked SuperClass, GenLIP has quickly become one of my favorite VLP papers. I'd love to reproduce the reported results faithfully and build on top of them, so I have a few questions if you'd be willing to clarify.

1. Stage-wise dataset configuration.
The training scripts reference Recap-DataComp-1B for Stage 1, and Infinity-MM (stage1) + BLIP3o-Pretrain-Long-Caption for Stage 2, plus two optional datasets (CapRL-2M, PLM-Image-Auto caption parts). The paper only mentions the first three. Are the reported numbers reproducible using just the paper-mentioned datasets? And could you share the role of the optional ones — were they used in additional experiments / ablations not covered in the paper, or are they suggested for a stronger variant?

2. Multi-node training setup.
The current script launches with torchrun on a single node, but I assume the actual runs were multi-node. Could you share the multi-node launch configuration (number of nodes, etc.) you used, so we can match the original setup as closely as possible?

3. Codebase parity.
Just to confirm — is the current public codebase the same one used to train the final reported models? i.e. if we follow it with the matching config, should we expect to land near the paper's numbers?

4. Training / eval logs, if possible.
I realize this is a bigger ask, but if any portion of the training or evaluation logs could be shared, it would be enormously helpful for sanity-checking our reproduction — loss curves and intermediate eval numbers alone would already be plenty.

Thanks again for the open release! Really looking forward to building on this work, and happy to contribute back anything useful that comes out of the reproduction effort.

[YanFangCS]

Hi, thanks for your interest in our work.

1. Stage-wise dataset configuration

For the reported results, using the paper-mentioned datasets is sufficient: Recap-DataComp-1B for Stage 1, and Infinity-MM (stage1 subset) + BLIP3o-Pretrain-Long-Caption for Stage 2.

The additional datasets, CapRL-2M and PLM-Image-Auto caption parts, are optional alternatives we explored for Stage 2. In our experiments, they can replace BLIP3o-Pretrain-Long-Caption and get similar results but needs more time for data processing. We leave them here just for recommendation, but not needed for Stage2.

2. Multi-node training

For the reported runs, we launched training with torchrun in a multi-node setup, e.g., NNODES=4 on our internal cluster. The key environment variables are the standard distributed-training ones, such as MASTER_ADDR, MASTER_PORT, NNODES, NODE_RANK, and NPROC_PER_NODE:

torchrun \
  --nnodes=${NNODES} \
  --nproc_per_node=${NPROC_PER_NODE} \
  --node_rank=${NODE_RANK}  \
  --master_addr=${MASTER_ADDR} \
  --master_port=${MASTER_PORT} \
...

In addition, we provide two multi-node training script templates, jobs/train_multinode.sh and jobs/train_slurm_multinode.sh, for easier adaptation.

3. Codebase parity

Yes. The current public codebase is a cleaned-up version of the codebase we used to train the released GenLIP models, with only minor documentation and usability updates. With the matching configurations and datasets, it should reproduce results close to those reported in the paper.

We also expect that access to more high-quality caption data could further improve the model, although this has not been exhaustively explored.

4. Training / evaluation logs

Yes. We are currently organizing the training and evaluation logs and plan to release several representative logs publicly once they are ready (we aim to finish by next Monday). We will update this issue accordingly.

Please feel free to let us know if you encounter any other issues during reproduction.

[Gary-code]

Hi, thanks for your interest in our work.

1. Stage-wise dataset configuration

For the reported results, using the paper-mentioned datasets is sufficient: Recap-DataComp-1B for Stage 1, and Infinity-MM (stage1 subset) + BLIP3o-Pretrain-Long-Caption for Stage 2.

The additional datasets, CapRL-2M and PLM-Image-Auto caption parts, are optional alternatives we explored for Stage 2. In our experiments, they can replace BLIP3o-Pretrain-Long-Caption and get similar results but needs more time for data processing. We leave them here just for recommendation, but not needed for Stage2.

2. Multi-node training

For the reported runs, we launched training with torchrun in a multi-node setup, e.g., NNODES=4 on our internal cluster. The key environment variables are the standard distributed-training ones, such as MASTER_ADDR, MASTER_PORT, NNODES, NODE_RANK, and NPROC_PER_NODE:

torchrun
--nnodes=${NNODES}
--nproc_per_node=${NPROC_PER_NODE}
--node_rank=${NODE_RANK}
--master_addr=${MASTER_ADDR}
--master_port=${MASTER_PORT}
...
In addition, we provide two multi-node training script templates, jobs/train_multinode.sh and jobs/train_slurm_multinode.sh, for easier adaptation.

3. Codebase parity

Yes. The current public codebase is a cleaned-up version of the codebase we used to train the released GenLIP models, with only minor documentation and usability updates. With the matching configurations and datasets, it should reproduce results close to those reported in the paper.

We also expect that access to more high-quality caption data could further improve the model, although this has not been exhaustively explored.

4. Training / evaluation logs

Yes. We are currently organizing the training and evaluation logs and plan to release several representative logs publicly once they are ready (we aim to finish by next Monday). We will update this issue accordingly.

Please feel free to let us know if you encounter any other issues during reproduction.

Could you provide the training logs for stages 1 and 2?