fix(qwen): report token usage from Qwen's own chat log

agents/entire-agent-qwen/AGENT.md

@@ -39,6 +39,7 @@ Session filename portability: Windows reserved device basenames (case-insensitiv
 | `extract-prompts` | `UserPromptSubmit` | Reads sidecar prompt records |
 | `extract-summary` | `Stop`/failure | Reads final assistant/error message |
 | `compact-transcript` | Sidecar JSONL | Emits base64 Entire Transcript Format JSONL |
+| `calculate-tokens` | Native chat-log usage | Sums the `AssistantUsage` sidecar records after `--offset`; see Token Usage |
 
 ## Sidecar Record Shape
 
@@ -57,18 +58,52 @@ always runs the generic compactor over the raw transcript.
 The projection is derived data — `readSidecarRecords` ignores `type` and
 `message` — so a sidecar written before it existed still reads.
 
+Records carry `v: 2` since the adapter started copying token usage; `v: 1`
+records come from an older adapter. The two differ only in that the token
+usage copy starts at the first `v: 2` record, or at the prompt of the turn it
+lands in (see Token Usage).
+
 | event | projected as | content |
 |---|---|---|
 | `UserPromptSubmit` | `user` | text block with the prompt |
 | `PreToolUse` | `assistant` | `tool_use` (`type`/`id`/`name`/`input`, the only fields `stripAssistantContent` keeps) |
 | `PostToolUse` / `PostToolUseFailure` | `user` | `tool_result` with snake_case `tool_use_id`, so cli inlines the output into the preceding `tool_use` |
 | `Stop` / `StopFailure` | `assistant` | text block with the final message or error details |
+| `AssistantUsage` (adapter-written, see Token Usage) | not projected | dropped from the compact transcript |
 | everything else | not projected | dropped from the compact transcript |
 
 `message` is stored as raw JSON so a record round-trips whichever shape it
 carries. Typed as a string, a projected line would fail to unmarshal and take
 the whole sidecar read with it.
 
+## Token Usage
+
+Read from Qwen Code 0.26.9 source (`QwenLM/qwen-code` 11c83aa) and checked
+with a live run of the 0.26.9 npm package against a local OpenAI-compatible
+mock that streams usage: `ChatRecordingService.recordAssistantTurn` stores the
+provider's usage, normalized to Gemini field names (`promptTokenCount`,
+`candidatesTokenCount`, `cachedContentTokenCount`, `thoughtsTokenCount`,
+`totalTokenCount`), as `usageMetadata` on each assistant record of the native
+chat log. The OpenAI-compatible pipeline requests it with
+`stream_options.include_usage`; the Anthropic and Responses generators convert
+their usage to the same shape.
+
+| Step | Behavior |
+|---|---|
+| Copy | At `PreToolUse`, `Stop`, `StopFailure`, `SubagentStop` and `SessionEnd`, each native `assistant` record with `usageMetadata` not yet copied becomes an `AssistantUsage` sidecar record holding its usage, `uuid` (`native_uuid`) and the byte offset of its line (`native_offset`) |
+| Wait | At `Stop`, until the log holds the answer to the turn's last prompt, the copy polls for up to 1 s: Qwen queues the turn's final assistant record before it fires `Stop`, but the write can still be in flight. The answer is the log's last record other than `system` records, when that is an assistant record that calls no tool and is no older than the sidecar's last `UserPromptSubmit` after the last copy (Qwen also fires `UserPromptSubmit`, with an empty prompt, before the call that follows a tool round). A line still being written is waited for too. A log replaced (Qwen rewrites it by renaming a new file over it) or cut shorter during the wait is read again from the start |
+| Resume | A copy finds the last `AssistantUsage` record for the log by reading the sidecar backwards from its end, and starts after that record's line while the line still holds the same record. Otherwise it rereads the log (Qwen can rewrite it, e.g. when prompts are deleted) and copies its calls again; the count keeps one copy of each |
+| Skip | Lines still being written (no newline yet), records `/branch` copied from a parent session (`forkedFrom`), subagent records (`isSidechain`), and records older than the sidecar's first `v: 2` record: the history of a resumed session whose sidecar was removed from the temp directory, or of a session an older adapter started. When the older adapter's records leave a turn open (a typed prompt, not the empty one before a tool round's next call, with no end of turn or session after it), that prompt is the cutoff instead, so an upgrade in the middle of a turn still counts that turn, whichever hook runs first after it |
+| Count | `calculate-tokens --offset <line>` sums `AssistantUsage` records from that sidecar line, counting each `native_uuid` once. Input is `promptTokenCount` minus `cachedContentTokenCount`, cache reads are `cachedContentTokenCount`, and output counts reasoning once, decided by `totalTokenCount` as in Qwen's own `outputTokenCount` |
+
+Qwen writes its log through a queue that no hook waits for, so an API call can
+land after the hook that follows it (seen once in six live runs, at
+`PreToolUse`; that call was copied at `Stop`). `Stop` waits for the turn's
+final call as above; a call that lands later still is copied at the next hook
+and counted with that turn or checkpoint. Calls outside the session's chat log
+(subagents, the auto-memory extractor and other background helpers) are not
+counted, and Qwen keeps no separate cache-write count.
+
 ## Lifecycle Mapping
 
 | Qwen Hook | Entire Hook Verb | Entire Event |

agents/entire-agent-qwen/README.md

@@ -10,7 +10,30 @@ External agent binary that teaches the Entire CLI how to work with Qwen Code.
 | transcript_analyzer | Yes, via Entire sidecar JSONL |
 | compact_transcript | Yes, emits Entire compact transcript JSONL |
 | transcript_preparer | No |
-| token_calculator | No |
+| token_calculator | Yes, from the per-call usage Qwen Code writes to its own chat log, copied into the sidecar at hooks |
+
+## Token Usage
+
+Qwen Code's hook payloads carry no token counts, but its chat log
+(`~/.qwen/projects/<project>/chats/<session>.jsonl`, the hooks'
+`transcript_path`) stores the provider's `usageMetadata` on every assistant
+record. At `PreToolUse`, `Stop`, `StopFailure`, `SubagentStop` and
+`SessionEnd` the adapter copies the usage of each new assistant record into the
+sidecar as an `AssistantUsage` line, and `calculate-tokens` sums those lines
+after `--offset`. The sidecar stays the only transcript, so a stored checkpoint
+still reports its tokens on a machine without the chat log.
+
+Qwen writes its log through a queue and fires `Stop` without waiting for it,
+so a turn's last API call can still be on its way when the `Stop` hook reads
+the log. Until the log holds the model's answer to the turn's last prompt,
+`Stop` waits up to a second for it. A call that arrives later still is copied
+at the next hook and counted with the next turn or checkpoint; totals are
+unaffected. A session that an older adapter started is counted from
+the first hook after the upgrade, or from the prompt of a turn still under way
+then. Calls Qwen does not record in the session's
+chat log, such as subagents and background helpers like its auto-memory
+extractor, are not counted. Cache writes are not reported separately: for Anthropic models Qwen
+folds them into the prompt count, so they appear as input tokens.
 
 ## Installation
 

agents/entire-agent-qwen/cmd/entire-agent-qwen/main.go

@@ -60,6 +60,8 @@ func main() {
 		err = protocol.HandleExtractPrompts(os.Args[2:], os.Stdout, agent)
 	case "extract-summary":
 		err = protocol.HandleExtractSummary(os.Args[2:], os.Stdout, agent)
+	case "calculate-tokens":
+		err = protocol.HandleCalculateTokens(os.Args[2:], os.Stdin, os.Stdout, agent)
 	default:
 		fatalf("unknown subcommand: %s", os.Args[1])
 	}

agents/entire-agent-qwen/internal/protocol/protocol.go

@@ -60,6 +60,10 @@ type transcriptAnalyzer interface {
 	ExtractSummary(sessionRef string) (string, bool, error)
 }
 
+type tokenCalculator interface {
+	CalculateTokens(data []byte, offset int) (TokenUsageResponse, error)
+}
+
 func WriteJSON(w io.Writer, v any) error {
 	enc := json.NewEncoder(w)
 	enc.SetEscapeHTML(false)
@@ -321,6 +325,24 @@ func HandleExtractSummary(args []string, stdout io.Writer, analyzer transcriptAn
 	return WriteJSON(stdout, ExtractSummaryResponse{Summary: summary, HasSummary: hasSummary})
 }
 
+func HandleCalculateTokens(args []string, stdin io.Reader, stdout io.Writer, calculator tokenCalculator) error {
+	fs := flag.NewFlagSet("calculate-tokens", flag.ContinueOnError)
+	fs.SetOutput(io.Discard)
+	offset := fs.Int("offset", 0, "offset")
+	if err := fs.Parse(args); err != nil {
+		return err
+	}
+	data, err := io.ReadAll(stdin)
+	if err != nil {
+		return err
+	}
+	usage, err := calculator.CalculateTokens(data, *offset)
+	if err != nil {
+		return err
+	}
+	return WriteJSON(stdout, usage)
+}
+
 func DefaultSessionDir(repoPath string) string {
 	return filepath.Join(repoPath, ".entire", "tmp")
 }

agents/entire-agent-qwen/internal/protocol/types.go

@@ -77,6 +77,14 @@ type ExtractSummaryResponse struct {
 	HasSummary bool   `json:"has_summary"`
 }
 
+type TokenUsageResponse struct {
+	InputTokens         int `json:"input_tokens"`
+	CacheCreationTokens int `json:"cache_creation_tokens"`
+	CacheReadTokens     int `json:"cache_read_tokens"`
+	OutputTokens        int `json:"output_tokens"`
+	APICallCount        int `json:"api_call_count"`
+}
+
 type CompactTranscriptResponse struct {
 	Transcript string                       `json:"transcript"`
 	Assets     []CompactTranscriptAssetJSON `json:"assets,omitempty"`

agents/entire-agent-qwen/internal/qwen/agent.go

@@ -47,6 +47,7 @@ func (a *Agent) Info() protocol.InfoResponse {
 		Capabilities: protocol.DeclaredCapabilities{
 			Hooks:              true,
 			TranscriptAnalyzer: true,
+			TokenCalculator:    true,
 			CompactTranscript:  true,
 			UsesTerminal:       true,
 		},

agents/entire-agent-qwen/internal/qwen/hooks.go

@@ -21,6 +21,12 @@ func (a *Agent) ParseHook(hookName string, input []byte) (*protocol.EventJSON, e
 	raw.HookEventName = qwenEventName(hookName, raw.HookEventName)
 	normalizeRawHook(&raw, hookName)
 
+	if syncsUsage(hookName) {
+		// Copy token usage from Qwen's own chat log first, so it precedes
+		// this hook's record; see tokens.go. Best effort: the hook itself
+		// must not fail because the native log could not be read.
+		_ = a.syncNativeUsage(hookName, raw)
+	}
 	if err := a.appendSidecar(raw); err != nil {
 		return nil, err
 	}
@@ -189,7 +195,10 @@ func normalizeRawHook(raw *qwenHookInputRaw, hookName string) {
 		raw.SessionID = stubSessionID
 	}
 	if strings.TrimSpace(raw.Timestamp) == "" {
-		raw.Timestamp = time.Now().UTC().Format(time.RFC3339)
+		// Sub-second, like Qwen's own timestamps: the Stop wait tells a
+		// turn's answer from the previous turn's by comparing record times
+		// with the prompt's (tokens.go).
+		raw.Timestamp = time.Now().UTC().Format(time.RFC3339Nano)
 	}
 	if strings.TrimSpace(raw.Prompt) == "" {
 		raw.Prompt = raw.UserPrompt