Skip to content

[2093] feat: Add instrumentation support for openai decision model api's - #438

Open
muhammedrazalak wants to merge 1 commit into
developfrom
feat/decision-api-instrumentation
Open

muhammedrazalak wants to merge 1 commit into
developfrom
feat/decision-api-instrumentation

Conversation

@muhammedrazalak

Copy link
Copy Markdown
Collaborator

Summary

Adds tracing for OpenAI's new Decisions API (client.decisions.create, sync and async), added in the openai 3.x SDK. Each call now produces an openai.decision span with the input, the questions asked, the typed answers and token usage.

Older openai versions without the Decisions API keep working: the instrumentor skips decisions with a DEBUG log, so nothing changes for them.

Changes

netra/instrumentation/libraries/openai/__init__.py

  • Wraps openai.resources.decisions → Decisions.create / AsyncDecisions.create in a new _instrument_decisions() helper.
  • If the module is missing on older openai versions, it logs at DEBUG and skips, so there's no error log.
  • _uninstrument() unwraps both classes. It passes the owning class path (openai.resources.decisions.Decisions, "create"), because unwrap(module, "Class.method") silently does nothing.

netra/instrumentation/libraries/openai/wrappers.py

  • New decisions_wrapper / adecisions_wrapper. The API has no streaming mode, so each call is a single non-streaming span.
  • Span openai.decision, kind CLIENT, llm.request.type = "decision".
  • Records the same timing attributes as the other wrappers (llm.response.duration, time to first token), sets status OK or ERROR (and re-raises), and honours instrumentation suppression.

netra/instrumentation/libraries/openai/utils.py

  • Request (operation_type == "decision"):
    • input:
      • A string becomes one user prompt (gen_ai.prompt.0).
      • A list is recorded item by item:
        • messages keep their role, with content parts stored as JSON
        • function_call / custom_tool_call → assistant entries holding {"name", "arguments"}
        • function_call_output / custom_tool_call_output → tool entries
        • tool items also get tool_call_id
    • questions: stored as a JSON array in gen_ai.decision.questions.
    • Only lists and tuples are read. The SDK accepts any iterable, and reading a generator here would empty the request before it's sent.
    • Capture is best-effort: a serialisation failure is logged and never fails the user's call.
  • Response: answers (predicate / choice / score / refusal) are stored in gen_ai.decision.answers and as the span's assistant completion (gen_ai.completion.0), so they show up in the UI like any other model output.
    • Usage (prompt / completion / cache read / cache write / reasoning / total tokens) and the response model go through the existing handlers with no changes.

tests/test_openai_instrumentation.py

  • Updated the wrap/unwrap call counts (6 → 8 wraps, 8 → 10 unwraps).
  • 19 new tests covering:
    • wrapping and unwrapping of the decision methods, and the quiet skip on older openai versions
    • string and mixed list input, question capture, generators left unread, unserialisable questions
    • answers and usage on the response
    • sync and async wrappers, the error path, and suppression
    • an end-to-end test through the real openai Decisions resource with an in-memory exporter (skipped on openai versions without the API)

New span attributes

Attribute Value
llm.request.type decision
gen_ai.decision.questions JSON array of the questions as sent
gen_ai.decision.answers JSON array of the answers, in question order
gen_ai.prompt.* / gen_ai.completion.0.* input items / answers
gen_ai.usage.* token usage

Backend / dashboard: spans carry a new request type (decision) and model (gpt-6-luna). Pricing and views may need to recognise them.

Testing

  • tests/test_openai_instrumentation.py: 69 passed on openai 3.50.0; 68 passed, 1 skipped on openai 2.45.0.
  • Full suite: no new failures. The same pre-existing failures and errors (environment-related) show up with and without this change.
  • black, isort and flake8 are clean on the changed files; mypy reports no new errors.
  • Manual check: the openai_decision_api sample in netra-ai-sample-apps runs against the live API and emits an openai.decision span under its workflow span.

Notes

  • Out of scope: the existing _uninstrument() calls for chat, embeddings and responses use unwrap("openai.resources.chat.completions", "Completions.create"), which never removes those wrappers. That should be a follow-up fix.

@muhammedrazalak
muhammedrazalak force-pushed the feat/decision-api-instrumentation branch from 75046d8 to f39aaf5 Compare November 13, 2026 08:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant