Durable mod metadata cache + ghost mods (allowGhostMods)

[tomjn]

Summary

SPADS persists full map metadata durably (mapInfoCache.dat for start
positions/dimensions/options, mapHashes.dat for version-keyed hashes), but
mod metadata (hash, options, sides) only ever lived in the in-memory
%cachedMods and was lost on restart. As a result there was no "ghost mod"
equivalent to ghost maps: a dedicated relay host always required the game
archive locally, even though it never simulates.

This PR mirrors the existing map machinery for mods and adds a allowGhostMods
setting so a dedicated host can host a configured game whose archive is absent,
using previously-cached metadata.

Changes

• Durable mod cache (SpadsConf.pm): modHashes.dat (version-keyed
  checksums, dumped in dumpDynamicData alongside mapHashes.dat) and
  modInfoCache.dat (Storable: options + sides). New accessors
  getCachedModInfo / cacheModInfo / getModHash / saveModHash.
• Persist on load (loadArchivesPostActions): a mod's hash and info are
  written to the durable cache the first time its archive is loaded.
• Cache-aware accessors: getModHash / getModOptions / getModSides
  now fall back to the durable cache, mirroring getMapHash / getMapOptions.
• allowGhostMods setting (preset setting, mirrors allowGhostMaps) plus
  canUseGhostMod() and ghost-mod resolution in updateTargetMod() and on
  initial load.

Design notes

The data splits cleanly by version-sensitivity: the checksum is
engine-version-sensitive (kept keyed by springMajorVersion, like map hashes),
while options and sides are intrinsic to the archive (version-independent).

canUseGhostMod() requires both a version-matching cached hash and
cached info (options + sides) before a ghost mod is used, so a battle is never
opened with incomplete mod metadata.

Existing configs need allowGhostMods added to spads.conf (handled by the
normal config update flow, as with any new setting). Generated setting docs
(doc/*.html) are not updated in this PR.

Testing status (please read)

This is compile-checked only, not runtime-tested. The repo has no automated
tests, and I was not able to run SPADS in my environment (no Spring/unitsync
install), so I verified each change with perl -c only. The open-battle path in
particular should be exercised on a real dedicated host before this is relied
on. Feedback on correctness and on anything I've missed is very welcome.

Scope is the "complete the cache" half of the work; an optional external
metadata import/export/fetch layer is intentionally left out of this PR.

[Yaribz]

Thanks for this PR. However, can you explain what motivated implementing mod metadata cache in SPADS ? Did you notice a bothersome delay due to mod archive scan or something like that ?

Maps metadata are cached in SPADS because maps are changed quite often between games, and it would take too much time to pre-load all maps metadata at startup.

On the opposite, only the metadata of the currently selected mod are loaded by SPADS at startup. Changing the currently hosted mod requires closing the battleroom anyway (lobby protocol limitation), so it's not problematic to spend a couple seconds to load the new mod metadata when this happens.
Furthermore, often when the currently hosted mod is changed it is because a new version was released, so the mod metadata wouldn't be cached anyway...

Also, the fact that the map hashes are broadcasted to everyone connected to the lobby server makes the ghost map functionality very effective when used in conjunction with the automatic map hashes learning functionality (see autoLearnMaps). This is not possible with mods because mod hashes aren't broadcasted, they are only sent to the clients once they joined a battle lobby.

If the mod metadata cache purpose is only to serve as a base for the "ghost mod" functionality with external metadata provider, see my answer in the dedicated PR.

[tomjn]

@Yaribz this was quite a while ago, me and Scary le poo were discussing the problem of needing to keep archives on disk and storage, and bandwidth issues, especially when hosting multiple SPADs adding multiple overheads.

The main issue being, that sure there's a single game, but if you're constantly creating and destroying SPADS instances, all of them need that game and to generate that cache, meaning what was a handful of MB now becomes up to a GB. That's a non-issue if you have a VPS and a single SPADS instance that runs for a long time.

[Yaribz]

if you're constantly creating and destroying SPADS instances, all of them need that game and to generate that cache, meaning what was a handful of MB now becomes up to a GB

The idea is that you don't need multiple SPADS installations to have several SPADS instances running in parallel, they can all share the same binaries and the same archives datadir. The recommended way to run massive parallel SPADS instances on a single host is to use the ClusterManager plugin, which takes care of automatic scaling and various optimizations related to parallel hosting.