ADE20K linear performance drop when increasing samples seen (2B → 8B)

[tommiekerssies]

Hi,

Amazing work!

Do you have an explanation for why the ADE20K linear probing performance drops when increasing the number of samples seen from 2B to 4B to 8B?

Thanks!

[dfan]

Thank you for reading and reaching out! I think the network becomes more "semantic" as it trains for longer on MC-2B. If this is the case, then performance might drop on benchmarks that need more granular visual information. We follow DINO's evaluation protocol for these vision benchmarks, which take features from a fixed set of layers (either last layer, last 4 layers, or evenly spaced 4 layers). I suspect that extracting features from earlier in the network might improve the ADE20K performance, but I haven't had time to try this