LangChain Tool-Call & Tool-Output Cost Tracing with Arize Phoenix

​ I’m implementing cost and token observability for a Python/LangChain application using Arize Phoenix + OpenTelemetry. The main goal is to clearly distinguish between these four things: - LLM tool-call generation → input/output tokens + cost - Tool execution → tool name, arguments, latency, output, output size/tokens, provider cost (if available) - Tool output consumption → tokens added to the next LLM request + corresponding input cost - Final LLM response → output tokens + cost For example: LLM #1 ├─ input: 156 tokens ├─ output/tool-call: 17 tokens └─ cost: $0.0000896 ↓ Tool ├─ output: 850 tokens (estimated) ├─ latency: 3.49s └─ provider cost: if available ↓ LLM #2 ├─ input: 1478 tokens ├─ output: 101 tokens └─ cost: $0.0007528 The 850 tool-output tokens must NOT be treated as LLM output tokens. They should be measured separately, while the actual 1478 tokens sent to LLM #2 should come from the provider's usage data. Implementation requirements I’m planning to use a custom LangChain "BaseCallbackHandler" with: on_llm_start / on_llm_end on_chat_model_start / on_chat_model_end on_tool_start / on_tool_end Each tool should have a unique "tool.call.id" so the trace can correlate: LLM → Tool → Tool Output → Next LLM Phoenix should expose attributes such as: llm.model llm.token_count.prompt llm.token_count.completion llm.cost.input llm.cost.output llm.cost.total tool.name tool.call.id tool.arguments tool.output tool.output.token_count tool.output.size_bytes tool.execution.duration_ms tool.cost I also need: - Centralized, configurable model pricing - Exact vs estimated tool-output token counts - "unavailable" status when token usage/pricing isn't provided - Configurable masking of sensitive tool arguments/outputs - Tracing failures must never break the actual agent/tool execution - Parent/child span correlation in Phoenix The key requirement: never collapse tool-call tokens, tool-output tokens, LLM input tokens, and tool-provider costs into a single metric. Has anyone implemented something similar with LangChain + Arize Phoenix? I’d especially appreciate examples or recommendations for the best way to correlate the tool span with both the LLM that generated the call and the subsequent LLM that consumed the tool output.