Data Models

Data Models

🌐 English · Deutsch

Source registry (providers.py)

Every streaming source is registered as a Provider dataclass: URL patterns (series/season/episode) plus the corresponding model classes.

Source	series_cls	season_cls	episode_cls
AniWorld	AniworldSeries	AniworldSeason	AniworldEpisode
SerienStream (serienstream.to)	SerienstreamSeries	SerienstreamSeason	SerienstreamEpisode
FilmPalast	—	—	FilmPalastEpisode (movies have no season structure)
MegaKino (series)	MegakinoSeries	MegakinoSeason	MegakinoEpisode (one post = one season; synthetic …?episode=N URLs)
MegaKino (movie)	—	—	MegakinoMovie
hanime (18+)	HanimeSeries	HanimeSeason	HanimeEpisode (franchise = series; synthetic …?ep=N URLs; browser-resolved AES-128 HLS)

resolve_provider(url) normalises the URL (e.g. /serie/stream/<slug> → /serie/<slug>, trailing slash) and returns the matching provider — or raises ValueError("Unsupported URL").

from mediaforge.providers import resolve_provider

prov = resolve_provider("https://aniworld.to/anime/stream/one-piece")
series = prov.series_cls(url="https://aniworld.to/anime/stream/one-piece")

Series

Lazy-loading properties (HTML is fetched on first access and cached):

title, title_cleaned, description, genres, release_year, poster_url, directors, actors, producer, country, age_rating, rating, imdb (IMDB ID), mal_id (AniWorld only), has_movies, seasons, season_count.

Note on poster URLs: some serienstream.to series use a hash ID instead of the series slug in the poster path — the extraction is generalised accordingly.

Season

AniworldSeason(url, series=None) — properties include season_number, episodes, are_movies (AniWorld "movies" season, skipped e.g. by AutoSync).

Episode

Core class for downloads:

ep = prov.episode_cls(
    url="https://aniworld.to/anime/stream/x/staffel-1/episode-1",
    selected_language="German Dub",
    selected_provider="VOE",
    selected_path="/path/optional",   # optional
)
ep.download(cancel_event=event)        # blocking; aborts when the event is set

• provider_data (property): mapping (Audio, Subtitles) → {host → embed URL} — the basis for language checks (e.g. in AutoSync) and host selection. A module's episode class may use a plain string as the key instead ("German Dub" → {host → embed URL}); /api/providers takes such a key as the label it already is.
• download(): invoke extractor → direct link → yt-dlp/FFmpeg → target file according to MEDIAFORGE_NAMING_TEMPLATE; sets language tags (deu/eng/jpn) on audio/subtitle tracks. The finished file path is available in _episode_path afterwards.
• Progress is published globally via models/common/common.get_ffmpeg_progress() (phase download = yt-dlp, ffmpeg = muxing) — the queue UI and /api/v1/status read from it.

Language system (config.py)

Site language keys are mapped to semantic enums:

Key	Label (aniworld)	Audio	Subtitles
1	German Dub	GERMAN	—
2	English Sub	JAPANESE	ENGLISH
3	German Sub	JAPANESE	GERMAN
4	English Dub	ENGLISH	—

serienstream.to uses the same keys with its own labels (1 = German Dub, 2 = English Dub, 3 = English Dub with German Sub). Helper maps: LANG_KEY_MAP, LANG_LABELS, LANG_CODE_MAP (ISO 639-2 for FFmpeg tags) and their inverses.

URL patterns

Defined as compiled regexes in config.py (MEDIAFORGE_SERIES_PATTERN, …_SEASON_…, …_EPISODE_…, analogous SERIENSTREAM_*, plus FILMPALAST_EPISODE_PATTERN in providers.py). AniWorld episode URLs also cover movies (/filme/film-N).

Audio / Subtitles enums

The Audio / Subtitles enums this page relies on are not one shared pair. models/s_to/episode.py defines its own (Audio.GERMAN / ENGLISH / JAPANESE, Subtitles.NONE / GERMAN), intentionally separate from the ones in config.py that AniWorld uses — s.to offers an "English audio + German subtitles" combination AniWorld's LANG_KEY_MAP does not model the same way.

Because the two enum classes are distinct types, models/common/common.py's download() tells the two apart via hasattr(self, "_normalize_language"), not isinstance.

episode_number vs file_episode_number

A real contract, and one that bites: episode_number is what the page shows and what filters use; file_episode_number is what gets matched against files on disk.

It exists because of AniWorld's absolute episode numbering (option aniworld_absolute_episodes): an episode titled … [Episode 062] stays in the season AniWorld lists it in but is written as S02E062 instead of S02E002.

• episode_number — the season-relative number from the URL. Page display, filters, everything user-facing.
• absolute_episode_number — parsed from the [Episode NNN] title marker, or None. AniWorld-only.
• file_episode_number — the number folder path and file name are built from. Equals episode_number unless absolute numbering is on and this episode actually carries a marker. Movies (/filme) are never renumbered.
• file_number_candidates — every (season, episode) pair a file for this episode may carry, current scheme first. Used for "do I already have this?" so flipping the setting does not make an existing library look empty and re-download the whole show under a second scheme.

All three are defined in models/aniworld_to/episode.py. No other provider has them — s.to, MegaKino, FilmPalast and hanime episodes only have episode_number, so every call site must use getattr(ep, "file_episode_number", ep.episode_number) and getattr(ep, "file_number_candidates", ((season, ep.episode_number),)). Real consumers today: the path/file-name building in models/aniworld_to/episode.py, web/autosync_worker.py's skip check, and the OpenSubtitles metadata builder in models/common/common.py.

models/common/

Site-agnostic pieces every episode model shares. models/common/__init__.py re-exports ProviderData, check_downloaded, clean_title, loose_title_key, titles_match, and the shared download() / watch() / syncplay() episode actions from common.py.

loose_title_key() / titles_match() — is this folder this title?

Two helpers in common.py for the one question every "already downloaded" badge asks: does this folder on disk belong to this card?

• loose_title_key(title) reduces a title to its letters and digits, lower-cased — no spaces, no punctuation, no dashes.
• titles_match(folder_name, title) compares the two loose keys in both directions: the folder key may start with the title key, or the title key may start with the folder key. The reverse direction additionally requires the folder key to be at least 10 characters, so a Naruto folder cannot claim Naruto Shippuden.

Four call sites used to do folder.name.lower().startswith(provider_title) by hand, which is why a tick showed up for one provider and not for another: a series folder is named after whichever provider downloaded it first, so every other provider's spelling of the same title missed. static/app.js carries the JS twin (_looseTitleKey + _looseFolderHolds) so the badges in the browser agree with the backend.

Out of scope on purpose: titles that share no prefix at all. Attack on Titan and Shingeki no Kyojin are the same show and will not match here — that needs an id, not string surgery. Which is exactly what web/library_aliases.py supplies (below); titles_match() stays a pure string function and the two are consulted side by side.

web/library_aliases.py — the names a folder answers to

Where titles_match() runs out, the connection is looked up instead of guessed. Every library folder is resolved against TMDB once, in the background, and the show's localized title, original-language title and TMDB alternative_titles are stored in the library_aliases table (see Database).

Function	Purpose
alias_index()	{loose_title_key: folder} over everything resolved so far — one SQL read, built for the callers that ask about many titles at once.
folder_holds_title(folder_name, title, index=None)	The alias-aware counterpart to titles_match(): does this folder answer to this title?
folder_for_title(title, index=None)	Which folder (if any) holds this title.
resolve_pending(limit=RESOLVE_BATCH)	One batch of unresolved folders; returns how many it wrote.
start_alias_resolver()	Starts the background loop.

Constants: RESOLVE_INTERVAL 900 s, RESOLVE_BATCH 25, RETRY_MISS_AFTER 30 days.

Four decisions worth keeping:

• Persisted, not cached. tmdb_cache is keyed by the asking title and expires after 24 h — right for "what is this card's rating", useless for a fact about a folder.
• Never resolved inside a request. Doing this in GET /api/downloaded-folders would turn a first home-page load on a large library into a multi-minute wait. The route serves what is already resolved; the string match answers for the rest, exactly as before.
• Confident matches only. A fuzzy TMDB hit contributes no names. A wrong alias produces a false "already downloaded", which silently suppresses a wanted download — strictly worse than the miss it would fix.
• No API key, no work. Without CineInfo/TMDB the module does nothing at all and reports nothing.

ProviderData

The class behind the provider_data mapping documented above. It wraps dict[(Audio, Subtitles)][provider_name] -> url and behaves like a dictionary: get((audio, subs)) returns {} for an unknown key, __getitem__ raises. __str__ renders a readable "German audio + English subtitles / - VOE -> …" block, which is what the CLI prints.

String keys are allowed. /api/providers understands both key shapes: an (Audio, Subtitles) tuple is translated into a label through the language maps, a plain string key is taken as the label it already is. A module is free to skip the enums entirely and write {"German Dub": {"VOE": url}} — which is what .examples/thirdparties/README.md and example_content_source/source.py always documented. Before that, a string key was silently dropped and the module's series modal opened with empty Language and Hoster dropdowns. Built-in tuple keys are unchanged.

dupecheck.py — duplicate handling

Two opt-in checks that run when the episode is already on disk. Both default to off.

• dl_quality_upgrade — enumerate the source's formats (yt-dlp, metadata only, nothing downloaded) and compare against the file on disk. Deliberately through yt-dlp and not ffprobe: the resolved stream URL is usually an HLS master playlist, and ffprobe would report the variant that playlist defaults to while the download itself takes bestvideo+bestaudio/best. Only a strictly higher video height (HEIGHT_TOLERANCE 32 px) or a meaningfully higher bitrate at the same height (BITRATE_MARGIN 0.20) counts as better; anything unknown counts as not better. is_better_quality(), explain().
• dl_audio_track_merge — the episode exists in another language and the current job is a different one. With language separation on (or {language} in the naming template) that file lives under a different folder and name, so the mux path never saw it and wrote a near-duplicate. find_existing_variant() closes the gap by resolving the same episode across sibling language folders, matching on an S01E01 / 1x01 token rather than on the full name.

The subtitle chain

_gather_subtitles() in models/common/common.py runs three stages, cheapest first, so the three download branches (full / audio-only / video-only) cannot drift apart:

1. yt-dlp sidecars (subtitles.py) — every subtitle rendition the stream itself offers, written next to the video and muxed into the .mkv as tagged soft-sub tracks. Setting dl_subtitles, default on; capped at MAX_SUBTITLE_TRACKS (12). writeautomaticsub stays off on purpose — ASR captions are not the source's subtitles.
2. The hoster's player config (fetch_hoster_subtitles()) — out-of-band tracks an extractor reports that the stream does not carry.
3. OpenSubtitles.com (opensubtitles.py) — external, off by default (opensubtitles_enabled plus credentials), and only for languages still missing after 1 and 2. moviehash first (size + first/last 64 KiB — identifies the exact release, so the timing fits), title + season_number/episode_number second. The daily quota is cached per process, and the /download response's URL is checked against _ALLOWED_DOWNLOAD_HOSTS over https only, so a spoofed response cannot turn into an SSRF.

Nothing in this chain may raise: a missing subtitle is a cosmetic loss, the video is the deliverable. Language codes are normalised to ISO 639-2/B (normalize_lang(), so a ger track and a deu track are not two languages).

Stage 3 is extensible. subtitle_sources.py (core, no mediaforge.web import) offers register_subtitle_source(item_id, source_id, label, fetch) / unregister_subtitle_source(item_id); fetch(video_path, have_langs, meta) -> [Path, …] must never re-fetch a language already in have_langs and returns sidecars named <video stem>.<lang>.<ext>. opensubtitles is a reserved source_id. Registrations are keyed by item_id, so web/thirdparties/registry.py's unregister_module() removes them when a module is disabled.

models/direct_link/ — the Direct Link path

A direct link has no series/season/provider/dub-sub structure to reconcile: the user pastes a raw stream URL (typically an .m3u8 master playlist, but anything yt-dlp supports works), picks one of the probed quality variants and gives the job a filename. So this path bypasses resolve_provider() entirely — web/queue_worker.py branches on item.get("provider") == "Direct" and constructs the class directly.

episode.py — DirectLinkEpisode

DirectLinkEpisode(url, title, selected_path=None, format_id=None, source_provider=None). It exists purely to satisfy the small interface the queue worker's dispatch loop expects from an "episode" (.download(cancel_event=…), ._episode_path, title_cleaned), so the retry/watchdog/history machinery is reused unchanged. Output is always <folder>/<title_cleaned>.mkv; format_id defaults to bestvideo+bestaudio/best.

download() always re-runs the full discovery rather than reusing what was resolved at probe time — many embed hosts hand out short-lived, signed CDN URLs that expire while the job sits in the queue. The first attempt may use the 10-minute resolve cache; a retry forces a fresh resolution.

resolver.py

The "which hosts can we resolve ourselves" layer. The host list is derived at runtime, not hard-coded: every name in models/megakino_to/scraper.py's _HOSTER_DOMAINS is kept if and only if extractors/provider/ has a real (non-stub) extractor for it — _extractor_implemented() probes with an empty URL and drops anything raising NotImplementedError. fast_providers() returns them in the user's configured provider order (Settings → Provider order).

FAST_PROVIDERS is an instance of _FastProviderSet, a live view over fast_providers() rather than the hard-coded set it used to be — existing name in FAST_PROVIDERS checks keep working while picking up newly enabled providers without a restart.

resolve_stream_for_provider(name, url, timeout=12) calls that host's extractor and returns (stream_url, headers), headers seeded from PROVIDER_HEADERS_D plus DIRECT_LINK_USER_AGENT (a desktop Chrome UA; some CDNs reject yt-dlp's default outright).

probe.py

discover_and_resolve(url, timeout=12, use_cache=True) is the whole pipeline, used identically by the probe step and by the download-time re-resolution. In order, retrying across all candidates at each step (a link that merely looks like a supported host is often a dead mirror):

1. url itself, if it is already a known embed-host link (detect_fast_provider()).
2. find_candidate_urls_in_page() — a static HTML scan for embedded hoster links (iframe/href, plus URLs inside inline <script> JSON after un-escaping \/), sorted by the user's provider order. It cannot find a link a site only reveals via a click-driven AJAX call.
3. browser_sniff.sniff_media_url() — last resort.

It always returns (provider_name_or_None, stream_url, headers); on total failure (None, url, {"User-Agent": …}), so the caller can still hand the URL to yt-dlp's generic extraction.

SSRF gate. Direct Link fetches, yt-dlp-probes and even headless-renders whatever URL a logged-in user pastes, so assert_safe_url() refuses non-http(s) schemes and private/loopback/metadata addresses, raising UnsafeUrlError (a ValueError subclass). It reuses web/stream_proxy.py's is_safe_url() — the same check the HLS proxy applies — imported lazily to keep the model layer free of an import-time dependency on the web package. The gate is applied to the pasted URL, to every redirect hop (redirects are followed manually, up to MAX_PAGE_REDIRECTS, so each hop is checked before it is fetched), to every in-page candidate, and to the stream URL the host finally hands back.

probe_direct_link_formats(url) is the read-only "what's in here" step behind the format-picker modal: it returns {"title", "provider", "formats"}, where formats[0] is always the safe bestvideo+bestaudio/best default and the rest are one entry per resolution (highest bitrate wins), with +bestaudio appended for video-only HLS renditions.

browser_sniff.py

Site-agnostic fallback: render the page in a headless browser (patchright; absent → returns None), watch its network traffic and return the first .m3u8/.mp4 request that is not obvious ad noise. It nudges playback with a couple of centre clicks, since many players only issue the real media request after a click (autoplay blocked, or a "click to play" overlay). Returned headers carry a Referer of the original page. Only tried last — launching a browser costs seconds.

Books / eBooks (web/books/)

Books get an indexing pass of their own and are stored under a separate books key in the library cache, alongside titles. That split is the point: nothing in the video pipeline — scanning, stats, calendar, auto-sync, upscaling — can be reached by a book entry. Because set_library_cache() replaces the whole row, a video-only partial update has to carry books / books_version over explicitly, or the shelf silently empties until the next full scan.

The video scanner in routes/library.py is positional (a directory under the library root is a title; a file counts only if its name carries an SxxExx marker). Neither assumption survives a Calibre layout, where the top-level folder is the author and the book sits one level down in Title (id)/. So the book scan is extension-driven instead: every file under the base, at any depth, whose extension is a book format is a candidate. Where it sits only informs the metadata; it never decides whether it counts.

• scanner.py — scan_books(base). Walks the tree, builds candidates, merges them, and emits the book entries the API returns. It never opens an EPUB/MOBI/PDF; it reads the filesystem plus metadata.opf and Calibre's metadata.db where present. Entries carry key, title, sort_title, authors, series/series_index, language, isbn, published, publisher, rating, description, tags, cover_path, formats, total_size, added_at, media_kind: "book". The scan result is versioned via BOOKS_FORMAT_VERSION.
• identity.py — pure string logic (no filesystem, no DB) deciding which files are the same book. This is the part that has to be right: too eager and two books collapse into one card, too timid and the same novel appears five times — which is not hypothetical, because Calibre stores one record per format, so EPUB + MOBI + AZW3 + PDF is four folders with four metadata.opf files, plus the loose copy in the library root. normalize(), clean_title(), split_filename() (stem → title/author), split_series(), group_key(title, author) and merge_groups(candidates).
• opf.py — parse_opf() for Calibre's metadata.opf sidecar. Best metadata available without opening the book, and it carries the untruncated title that the filename does not. Size-capped at 512 KB against XML bombs.
• calibre_db.py — load_catalogue() / lookup() against metadata.db, opened strictly read-only (file:…?mode=ro&immutable=1, so no -wal/-shm sidecars and no lock Calibre would trip over). Every failure is soft. Authoritative for series/tags/rating/description, but never overrides a title or author already resolved from the file itself.
• covers.py — where the cover picture hides per format: the EPUB package document's cover (EPUB 3 properties="cover-image" and EPUB 2 <meta name="cover">), MOBI/AZW3/AZW via the converted EPUB, a sidecar cover.jpg, and nothing for PDF. Caching and downscaling live in web/covercache.py, shared with the comic shelf.
• convert.py — no browser renders Mobipocket, so the reader asks for an EPUB and the server makes one via the mobi package (GPL-3.0). Cached by (path, mtime, size) under the config directory; nothing is ever written next to the user's book, which is what makes a read-only library mount work.

How a book identity differs from a video title. A video title is identified positionally (folder under the root) and keyed by season/episode numbers. A book is identified by normalize(title)|normalize(author) — path-independent by design, so the same novel loose in the root and inside Author/Title (1234)/ merges into one entry. The formats become a formats list on that one entry (each with path, size, readable, drm) instead of separate items, and author-less entries get their own namespace so they can be attached later rather than force-merged with a same-titled book by a different author.

Extending the registry from a module

Third-party modules can add their own content source at runtime via register_provider(item_id, provider) / unregister_provider(item_id) in providers.py — the same Provider dataclass documented at the top of this page. Registrations are keyed by the module's item_id so they are cleaned up on uninstall.

Details, the full registry list and the rules a module has to follow are in the Module API — not repeated here.