Skip to content

Mix real code-switching corpora into training, not just evaluation #7

Description

@Awshesh12

All our training data is synthetic. Real code-switched speech is used only for evaluation. The paper's central finding is that this costs a lot of generalisation — the obvious next move is to close the gap from the training side.

Candidate sources

  • CS-FLEURS xtts/train — 2097 JA–EN rows already available via data/load_csfleurs.py, currently unused. Note these are TTS, so they help with switch-point modelling but not acoustic realism.
  • CS-FLEURS read — real voices, but it is our test set. Would require carving out a disjoint train split and re-freezing the benchmark, which invalidates comparison with the current paper. Do not do this casually.
  • SEAME — Mandarin–English, wrong pair, but usable for a cross-lingual transfer experiment
  • Japanese monolingual real speech (JSUT, Common Voice ja) mixed with English (LibriSpeech, Common Voice en) — cheap, real acoustics, and the composition ablation in Table 6 already shows monolingual data punches above its weight

Experiment to run

A curve: hold total training volume fixed and vary the synthetic:real ratio (100:0, 75:25, 50:50, 25:75). How much real data is needed before real-speech ScriptAcc saturates? That result would be a genuine contribution on its own and directly extends §8.

Watch out

Any use of CS-FLEURS in training breaks the clean held-out property the current benchmark depends on. Document clearly whichever way it goes.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    dataCorpora, augmentation, dataset constructiontrainingFine-tuning recipes, ablations, hyperparameters

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions