Langsmith tracing - #179
Conversation
| def _build_agent_trace_context( | ||
| *, | ||
| endpoint: str, | ||
| session_id: UUID | None, | ||
| thread_id: UUID, | ||
| body: models.AgentContext, | ||
| ) -> dict[str, str | int | list[int] | list[str] | None]: | ||
| env = settings.ENV | ||
| return { | ||
| "endpoint": endpoint, | ||
| "feature": "chat_agent", | ||
| "environment": env, | ||
| "session_id": str(session_id) if session_id else None, | ||
| "thread_id": str(thread_id), | ||
| "query_length": len(body.query or ""), | ||
| "sdg_filter": body.sdg_filter, | ||
| "corpora": list(body.corpora) if body.corpora else None, | ||
| } | ||
|
|
There was a problem hiding this comment.
Idk if there is risk of that or is it's a problem but if endpoint, thread_id are None or if body is wrongly structured (without corpora) several variables can return "None" or crash the function
| session_id: UUID | None, | ||
| thread_id: UUID, | ||
| body: models.AgentContext, | ||
| ) -> dict[str, str | int | list[int] | list[str] | None]: |
There was a problem hiding this comment.
🍞 Long return type hint, a dataclass or custom type could improve readability
| # LANGSMITH / LANGCHAIN TRACING | ||
| LANGSMITH_TRACING_ENABLED: bool = False | ||
| LANGSMITH_PROJECT: Optional[str] = None | ||
| LANGSMITH_ENDPOINT: Optional[str] = None | ||
| LANGSMITH_API_KEY: Optional[str] = None | ||
|
|
There was a problem hiding this comment.
For later refacto : https://typing.python.org/en/latest/reference/best_practices.html#shorthand-syntax
Even if "Optionnal" or "Union" are still valid, it's more pythonic to use " | None" or "X | Y"
Anyway, in this specific PR it's logical to stick to the syntax used in the rest of the file
| trace_context: dict[str, Any] = { | ||
| "component": "chat_non_agent", | ||
| "operation": operation, | ||
| "environment": settings.ENV, | ||
| "model": getattr(self.chat_client, "model", None), | ||
| } | ||
| trace_context.update(extra) |
There was a problem hiding this comment.
Same comment than for "build_agent_trace_context" the usage of a specific class or at least custom type could improve readability and the maintainability
| settings = get_settings() | ||
| trace_context: dict[str, Any] = { | ||
| "component": "chat_non_agent", | ||
| "operation": operation, |
| messages=messages, | ||
| trace_context=self._build_non_agent_trace_context( | ||
| "run_llm_with_json_parsing", | ||
| has_fallback_formatter=bool(fallback_formatter), |
There was a problem hiding this comment.
| has_fallback_formatter=bool(fallback_formatter), | |
| has_fallback_formatter=fallback_formatter is not None, |
It's more readable and more pythonic :)
| await self.client.close() | ||
|
|
||
| @log_time_and_error | ||
| @traceable(run_type="llm", name="Completion (non-agent)") |
There was a problem hiding this comment.
If an enumeration is implemented for "abst_chat" and hte values of method maybe it can be used here too for the name ?
| if inspect.isawaitable(response): | ||
| return await response |
| ) | ||
| super().__init__("UniversityTeacherAgent", model, system_prompt) | ||
| super().__init__( | ||
| "UniversityTeacherAgent", |
There was a problem hiding this comment.
| "UniversityTeacherAgent", | |
| self.__name__, |
idk if it's work but it could be better than rewrite the name of the class
| chat_model, | ||
| lang, | ||
| trace_tags=[*base_tags, "agent:university_teacher"], | ||
| trace_metadata={**base_metadata, "agent": "UniversityTeacherAgent"}, |
There was a problem hiding this comment.
If the name of the agent is always pass in this function can't we put the logic direclty into the class ?
2 participants
This pull request introduces LangSmith tracing for LangChain-based LLM calls, enabling improved observability and debugging of both agent and non-agent chat operations. The integration is configurable via environment variables, and trace context metadata is now propagated throughout the chat and LLM proxy layers. The most important changes are summarized below.
LangSmith/Tracing Integration:
langsmithas a dependency and new LangSmith-related environment variables to.env.exampleand the settings model (pyproject.toml,.env.example,src/app/core/config.py). [1] [2] [3]configure_langsmith_tracingto set up LangSmith tracing on app startup (src/app/core/langsmith.py,src/app/core/lifespan.py). [1] [2] [3]Trace Context Propagation:
trace_contextmetadata for agent and non-agent chat operations, including endpoint, session, thread, and query details (src/app/api/api_v1/endpoints/chat.py,src/app/api/api_v1/endpoints/chat_utils.py). [1] [2] [3] [4] [5] [6] [7] [8] [9]Non-Agent and Agent Chat Improvements:
_build_non_agent_trace_contexthelper and ensured all chat operations (e.g., formatting, language detection, rephrasing, chat messages) pass trace context to LLM calls (src/app/shared/infra/abst_chat.py). [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]LLM Proxy Enhancements:
LLMProxyto accept and logtrace_contextin all completion methods, and decorated key methods with@traceablefor LangSmith compatibility (src/app/shared/infra/llm_proxy.py). [1] [2] [3]These changes collectively enable end-to-end tracing and improved debugging for both agent and non-agent LLM-powered chat features.