Skip to content

Add caching for document embedding and Weaviate persistence in RAG notebook - #3

Open
denis-samatov wants to merge 1 commit into
AnkunHuang:mainfrom
denis-samatov:pr-rag-caching-persistence
Open

Add caching for document embedding and Weaviate persistence in RAG notebook#3
denis-samatov wants to merge 1 commit into
AnkunHuang:mainfrom
denis-samatov:pr-rag-caching-persistence

Conversation

@denis-samatov

Copy link
Copy Markdown

Summary

  • Avoids re-embedding and re-loading the document set on every run by persisting the Weaviate vectorstore and reusing the local data file if already present
  • Along the way, fixes the source-data URL, which pointed at a GitHub blob-view HTML page (.../blob/master/...) instead of the raw file content (raw.githubusercontent.com/...)

Note: this diff also reformats the notebook's JSON (pretty-printed instead of minified), a side effect of editing it programmatically.

Test plan

  • Run the notebook twice; confirm the second run skips re-downloading/re-embedding and reuses the persisted vectorstore

…tebook

Avoids re-embedding and re-loading the document set on every run.

Note: this diff also reformats the notebook's JSON (pretty-printed
instead of minified), a side effect of editing it programmatically.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant