Skip to content

📡 Say in the log whether usage is forwarded to Valohai LLM - #45

Open
petterir wants to merge 7 commits into
mainfrom
feat/41-log-valohai-llm-forwarding
Open

petterir wants to merge 7 commits into
mainfrom
feat/41-log-valohai-llm-forwarding

Conversation

@petterir

Copy link
Copy Markdown
Collaborator

Fixes #41.

At the default log level you can now tell whether usage is being forwarded to Valohai LLM, how much was sent, and why nothing is sent when calls carry no X-VH- header.

# no API key
INFO conduit_proxy: Valohai LLM forwarding off; set VALOHAI_LLM_API_KEY or [valohai-llm] api_key to turn it on

# key set, 6 tagged calls
INFO conduit_proxy: forwarding usage to Valohai LLM at https://llm.valohai.com
INFO conduit_proxy::sync: sent 6 usage records to Valohai LLM

# key set, untagged calls
INFO conduit_proxy::sync: 3 recorded calls had no X-VH- headers and stay local; add an X-VH-Tag-<key> header (e.g. X-VH-Tag-Context) to forward them

Commits

  1. Say at startup whether usage is forwarded. One info line in either case. It never includes the key.
  2. Pull one flush out of the poster loop. No behaviour change; flush_unsent returns how many records the endpoint accepted, which makes the flush testable.
  3. Log how many usage records each flush sent. One info line per flush that sent something, with the total across batches. The per-POST debug line stays.
  4. Say when recorded calls stay local. Counts calls recorded without X-VH- headers and says so once.

For the reviewer

  • The stay-local note is counted in memory for this run, not queried from the database, so old records in conduit.db don't trigger it. It prints at info the first time untagged calls are seen and at debug after that, so a steady stream of untagged calls doesn't add a line every 5 seconds. It only prints when forwarding is on.
  • The count is "recorded calls with no X-VH- header". A tagged call with no usage data, such as a stream without include_usage, is also never sent and is not counted.
  • The hint differs from the issue's wording. "or use valohai-llm" is dropped because nothing by that name exists in the repo besides the config section. The header names come from examples/python/*_tag_monitoring.py.
  • FakeTransitStorage is added to the proxy crate's testutil, for tests only, so the flush and the counter can be tested without a database.
  • Not changed here: if mark_sent fails after a successful POST, the flush loop lists and posts the same records again. That predates this PR, but the new sent count would include the repeats.

Testing

cargo fmt --all --check, cargo clippy --workspace --tests --locked and cargo test --workspace --locked pass on each commit. The binary was also run against a stand-in provider and ingest endpoint with dummy keys, which produced the lines above (with the stand-in's URL in place of https://llm.valohai.com).

🤖 Generated with Claude Code

Petteri Raatikainen and others added 4 commits November 19, 2026 12:30
At the default log level, Conduit now prints one line at startup that
tells whether usage leaves the machine. Before, a run with forwarding on
looked the same as one without an API key.

    INFO conduit_proxy: forwarding usage to Valohai LLM at https://llm.valohai.com
    INFO conduit_proxy: Valohai LLM forwarding off; set VALOHAI_LLM_API_KEY or [valohai-llm] api_key to turn it on

The line names the URL and never the API key.

Refs #41

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The body of each 5-second tick is now its own function that returns how
many records the endpoint accepted. Nothing changes in what is sent or
logged; this lets the next commit log a per-flush count, and lets tests
drive a flush against a local endpoint and an in-memory storage.

Refs #41

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each 5-second flush that sends something now logs one info line with
the total across all of its batches:

    INFO conduit_proxy::sync: sent 6 usage records to Valohai LLM

Before, sends were logged at debug only, so at the default level there
was no way to tell from the log that forwarding worked. Flushes that
send nothing stay quiet, and the per-POST debug line is kept.

Refs #41

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conduit only forwards calls that carry an X-VH- header, and calls
without one were not mentioned at any level. With forwarding on, the
first flush that finds such calls from this run now logs one info line
with a hint on how to tag them:

    INFO conduit_proxy::sync: 3 recorded calls had no X-VH- headers and stay local; add an X-VH-Tag-<key> header (e.g. X-VH-Tag-Context) to forward them

Later counts go to debug, so a steady stream of untagged calls does not
add an info line every five seconds. The count is kept in memory for
this run only, so records from earlier runs never trigger it. With
forwarding off, the startup line already says nothing is sent.

Fixes #41

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The default-level URL log can expose credentials or tokens, and the new activity messages can be missing or inaccurate.

4 open issues
What changed in this PR

Adds default-level logging so users can tell whether Conduit forwards usage to Valohai LLM, addressing #41.

Changes:

  • Logs forwarding status at startup and accepted record totals after each nonempty flush.
  • Adds a one-time INFO hint for untagged calls kept local.
  • Extracts flush logic and adds database-free tests for batching, status messages, and counters.
File Description
crates/​proxy/​src/​sync.rs Adds forwarding messages, flush totals, local-only tracking, and tests.
crates/​proxy/​src/​lib.rs Wires startup logging and counters; adds shared test storage.
crates/​proxy/​src/​inspect/​mod.rs Counts untagged identity reports and tests counter behavior.

Review effort: Balanced

Comment thread crates/proxy/src/sync.rs Resolved
Comment thread crates/proxy/src/inspect/mod.rs Resolved
Comment thread crates/proxy/src/inspect/mod.rs Resolved
Comment thread crates/proxy/src/sync.rs Resolved
Petteri Raatikainen and others added 3 commits November 19, 2026 14:59
The Valohai LLM url is free-form config, so it can carry credentials as
userinfo (`https://user:secret@host`) or in the query string, and the
startup line printed it as written at the default log level.

The line now shows only scheme, host, port and path. A url that doesn't
parse to a host is shown as `<invalid url>` instead of being echoed.

Refs #41

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The stay-local note counted an identity as soon as its report arrived. If
storing the batch then failed, say on a full disk, the log still said the
calls were recorded, and could spend the one info-level note on calls
that were lost.

The count is now taken per batch and added only after the batch is
stored.

Refs #41

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A flush kept posting until no unsent records were left. With records
arriving as fast as they are sent it would not return, so neither the
sent count nor the stay-local note was logged while batches went out
fine.

A flush now also ends once it has run for the flush interval. Its tick is
already due by then, so the next flush starts right away and the backlog
keeps draining at the same rate.

Refs #41

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Say in the log whether forwarding to Valohai LLM is on, and what it sent or kept local

2 participants