v0.3: Sefaria importer#14
Merged
Merged
Conversation
Saved Sefaria v3 API response shapes for William Davidson (Hebrew with niqqud), Vilna Edition (plain), and Kaufmann Manuscript (plene). Three witnesses, three mishnayot, intentional Davidson/Vilna/Kaufmann differences to exercise EXACT, NIQQUD_STRIPPED, and PLENE_DEFECTIVE tiers. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Single GET against a Sefaria API URL, JSON-parsed and returned as dict. Module-level _http_get is the patch target for tests; _fetch_json wraps it with UTF-8 decoding and json.loads. No external HTTP dependency. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Fetches one reference from the Sefaria v3 API (mockable via _fetch_json), selects the requested versionTitle, concatenates segment strings with spaces, and runs the result through tracealign.tokenize with the caller-supplied seq_label.
Returns one token list per segment. Each segment gets a derived
seq_label of the form {base}:{n} so token IDs stay unique across
segments and across witnesses.
Sefaria text fields can contain <b>/<i>/<sup>. _strip_html removes all tags; <sup>...</sup> footnote markers are dropped entirely so footnote digits never pollute the token stream.
dict[version_title, list[list[Token]]] for one reference across multiple versions. Each value is a list of token lists, one per segment.
Single hasattr test pinning load/load_segments/load_versions. No
top-level alias — the importer follows the existing
tracealign.io.{result,escriptorium,tei} import pattern.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
docs/usage.md gains a Sefaria subsection showing load_segments and load_versions; README adds a one-line mention so the importer surfaces in Highlights.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #11. Foundation for #12.
Adds `tracealign.io.sefaria` with three public functions, exercised by offline-fixture tests for Davidson, Vilna, and Kaufmann editions of Mishna Avot 1.
Public surface (`tracealign.io.sefaria`)
Implementation notes
Stats
Test plan