Skip to content

feat(outputschema): let an agent ask OpenAI for strict structured output - #3286

Draft
danielle-mustillo wants to merge 1 commit into
kagent-dev:mainfrom
danielle-mustillo:danielle-mustillo/openai-strict-output
Draft

danielle-mustillo wants to merge 1 commit into
kagent-dev:mainfrom
danielle-mustillo:danielle-mustillo/openai-strict-output

Conversation

@danielle-mustillo

Copy link
Copy Markdown
Contributor

What changes for users

An AgentTemplate can ask OpenAI and Azure OpenAI to follow its output schema while they generate (strict mode), instead of kagent only validating the answer afterward. It is opt-in per agent.

kind: AgentTemplate
spec:
  outputSchemaStrict: true
  outputSchema:
    type: object
    additionalProperties: false
    required: [due, code]
    properties:
      due: {type: string}
      code: {type: string}
AgentTemplate Result
no outputSchemaStrict (default) unchanged. The request has no strict key
outputSchemaStrict: true, compatible schema strict: true on Chat Completions response_format.json_schema and on Responses text.format
outputSchemaStrict: true, no schema kubectl apply refused: outputSchemaStrict requires outputSchema or outputSchemaFrom
outputSchemaStrict: true, a property missing from required Agent Compatible=False: the root object must list property "code" in required
outputSchemaStrict: true, an object without additionalProperties: false (root, nested, items, anyOf, $defs) Agent Compatible=False, naming the path
outputSchemaStrict: true, anyOf at the root Agent Compatible=False
any provider other than OpenAI / Azure OpenAI field ignored

Resolves solo-io/enterprise-kagent#403. That issue asks for a per-agent switch because strict accepts fewer schemas than outputSchema does, so turning it on for every agent would break schemas that work today.

How

  • API. AgentTemplateSpec.outputSchemaStrict (bool), with a CEL rule that it needs a schema. It goes into the compiled config as ADKOutputConfig.strict (proto field 3).
  • Compile check (go/adk/pkg/outputschema/strict.go, called from the kagent Harness compiler in adkconfig/runtime.go). It checks the rules OpenAI documents for strict mode: every object sets additionalProperties: false and lists every property in required, and the root does not use anyOf. OpenAI rejects a strict request that breaks them, so without this check the Agent would be Ready and every request would fail.
  • Runtime. OpenAIModel.StrictOutputSchema is set from the agent's output config. Both request builders add strict: true when it is set. Azure uses the same model type.

Testing

  • make -C go lint: 0 issues.
  • Unit tests: CheckStrict table (7 cases), compiler accept/reject, request body has strict: true on both APIs. The existing tests that require strict to be omitted by default still pass.
  • go test ./adk/... ./core/internal/... ./api/... on macOS: all pass except packages that need a local Docker for Postgres (service/session, service/sandbox). Those are left to CI.

Kind cluster, Azure OpenAI gpt-4.1-mini, Chat Completions

Check Result
apply strict without schema refused by CRD rule
strict with an optional property Compatible=False UnsupportedConfiguration, Ready=False
12-level nested schema, strict off request accepted, the model answered (it did not match the schema)
same schema, strict on provider 400. Direct call shows Azure's reason 12 levels of nesting exceeds limit of 10, which Azure returns only in strict mode. This proves the flag reaches the provider
flat 6-string schema, 5 fresh sessions each off 4/5 completed, on 4/5 completed. Both failures were root agent output is not valid JSON

The last row is not caused by this change. Ten direct streaming calls to the same deployment with the same prompt and strict: true returned valid JSON ten times. The failing kagent tasks reported about twice the output tokens of passing ones. The cause is inside kagent's request or response path and is still open.

Strict's value shows on schemas with format and pattern (#3284). An earlier prototype with both changes went from 0/9 to 5/5 on the same Azure deployment. This PR does not depend on #3284.

Open questions

  • Field shape. A bool is the smallest change. Kubernetes API conventions prefer an enum when more modes may follow.
  • Provider limits. The compile check covers OpenAI's structural rules, not its size limits (10 nesting levels, 5000 properties, 1000 enum values). A strict schema past those passes compilation and fails at the provider. The provider error is also reduced to OpenAI API error: 400 Bad Request in the Task message.

Add AgentTemplate outputSchemaStrict. When set, the Go ADK sends strict: true
with the output schema on the Chat Completions and Responses APIs. The kagent
Harness compiler rejects a strict schema that OpenAI would refuse: an object
without additionalProperties false, a property missing from required, or an
anyOf root.

Signed-off-by: Danielle Mustillo <danielle.mustillo@solo.io>
@github-actions github-actions Bot added the enhancement New feature or request label Oct 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant