diff --git a/spec/router-openapi.yaml b/spec/router-openapi.yaml index 445b63b..382b4ae 100644 --- a/spec/router-openapi.yaml +++ b/spec/router-openapi.yaml @@ -198,7 +198,7 @@ paths: /v2/models/{provider}/{model}/estimate: post: summary: Price a model call before running it, without running it. - description: 'Returns what Comfy will charge for running this body on this model, in US dollars, priced from the same rate card the charge is billed against. The body is validated against the model''s input schema first, as a default run (no `strict_mode`, no `reject_unknown_fields`) validates it, so a body that fails the model''s schema is the same `422` here as on the run; nothing is dispatched to a provider and nothing is charged. The answer is not a price lock: it is priced at the rates in force at `pricing_as_of`, and a run is charged at the rates in force when it is rated. While the feature is rolling out the route answers `403` with `error_type: not_enabled`.' + description: 'Returns a pre-flight estimate, in US dollars, of what running this body on this model will cost, priced from the same rate card the run is billed against (a model billed the cost its vendor reports is estimated from the vendor''s published price list instead). It is an estimate, not the final charge: the settled charge in your workspace''s usage and billing records is what you pay. The body is validated against the model''s input schema first, as a default run (no `strict_mode`, no `reject_unknown_fields`) validates it, so a body that fails the model''s schema is the same `422` here as on the run; nothing is dispatched to a provider and nothing is charged. The answer is not a price lock: it is priced at the rates in force at `pricing_as_of`, and a run is charged at the rates in force when it is rated. While the feature is rolling out the route answers `403` with `error_type: not_enabled`.' operationId: estimateRouterModelCost tags: - Comfy Router @@ -344,6 +344,8 @@ paths: $ref: '#/components/responses/RouterIdempotencyConflict' '422': $ref: '#/components/responses/RouterModelValidationError' + '429': + $ref: '#/components/responses/RouterQueueSubmitTooManyRequests' '503': $ref: '#/components/responses/RouterRequestUnavailable' /v2/models/{provider}/{model}/requests/{request_id}: @@ -403,8 +405,6 @@ paths: $ref: '#/components/responses/RouterRequestError' '504': $ref: '#/components/responses/RouterRequestError' - '422': - $ref: '#/components/responses/RouterModelValidationError' default: $ref: '#/components/responses/RouterRequestError' /v2/models/{provider}/{model}/requests/{request_id}/cancel: @@ -474,6 +474,12 @@ paths: $ref: '#/components/headers/RouterRequestIdHeader' Retry-After: $ref: '#/components/headers/RouterQueuePollAfterHeader' + X-Comfy-Estimated-Duration-Seconds: + $ref: '#/components/headers/RouterEstimatedDurationHeader' + X-Comfy-Estimated-Duration-P90-Seconds: + $ref: '#/components/headers/RouterEstimatedDurationP90Header' + X-Comfy-Estimate-Source: + $ref: '#/components/headers/RouterEstimateSourceHeader' content: application/json: schema: @@ -498,7 +504,7 @@ components: example: unknown RouterCostEstimate: type: object - description: 'What Comfy will charge for this call, priced before dispatch from the same rate card the charge is billed against, in US dollars. `source` says how much to trust the number: `exact` carries `amount`; `estimated` carries `min_amount` and `max_amount` and no `amount`; `unknown` carries neither and a `reason`. Never read an absent `amount` as free. Not a price lock: see `pricing_as_of`.' + description: 'A pre-flight estimate of what this call will cost, in US dollars, priced before dispatch from the same rate card the run is billed against (or, for a model billed the cost its vendor reports, estimated from the vendor''s published price list). It is not the final charge: the settled charge in your workspace''s usage and billing records is what you pay. `X-Comfy-Credits-Used`, reported after the run, is a rounded post-hoc figure, not a receipt. `source` says how much to trust the number: `exact` carries `amount`; `estimated` carries `min_amount` and `max_amount` and no `amount`; `unknown` carries neither and a `reason`. Never read an absent `amount` as free. Not a price lock: see `pricing_as_of`.' required: - source - currency @@ -508,19 +514,19 @@ components: properties: source: type: string - description: '`exact` | `estimated` | `unknown`; read any other value as `unknown`.' + description: '`exact` | `estimated` | `unknown`; read any other value as `unknown`. `exact` and `estimated` both label an estimate; neither is the final charge. `unknown` carries no figure.' currency: type: string description: Always `USD`. amount: type: string - description: Dollars as a decimal string, e.g. "0.04", with up to 8 decimals and no exponent. A chargeable sub-cent price is never rendered as "0". Present only when `source` is `exact`. + description: 'The estimated cost in dollars as a decimal string, e.g. "0.04", with up to 8 decimals and no exponent. A chargeable sub-cent price is never rendered as "0". Present only when `source` is `exact`. Not the final charge: the settled charge in your workspace''s usage and billing records is.' min_amount: type: string - description: The lower bound in dollars as a decimal string. Present only when `source` is `estimated`. + description: 'The low end of the estimate in dollars as a decimal string. It is an estimate, not a guaranteed floor: a call can be billed less, for example when the provider returns less than the request asked for (such as no images), when the estimate is a typical price learned from past calls, or when the price does not rise steadily with the amount requested. Present only when `source` is `estimated`.' max_amount: type: string - description: The upper bound in dollars as a decimal string. Present only when `source` is `estimated`. + description: 'The high end of the estimate in dollars as a decimal string. It is an estimate, not a guaranteed cap: a call can be billed more, for example when the provider returns more than was asked (such as more images than requested), when the estimate is learned from past calls, or when the price does not rise steadily with the amount requested. Present only when `source` is `estimated`.' amount_cents: type: number format: double @@ -536,7 +542,7 @@ components: credits: type: number format: double - description: '`amount` in Comfy credits using the platform conversion ($1 = 211 credits), rounded to 2 decimals - the figure `X-Comfy-Credits-Used` will carry for an `exact` quote.' + description: '`amount` in Comfy credits using the platform conversion ($1 = 211 credits), rounded to 2 decimals - the figure `X-Comfy-Credits-Used` is expected to carry for an `exact` quote. That header is a rounded post-hoc figure, not a receipt; the settled charge in your workspace''s usage and billing records is final.' provider: type: string description: The provider leg this quote is for. On a queued submit it is the leg that will run. @@ -545,7 +551,7 @@ components: pricing_as_of: type: string format: date-time - description: 'When the rate card this quote was priced from was fetched. The run is charged at the rates in force when it is rated, not at this instant. Whenever no rate card was read (`reason: pricing_unavailable`, `byok`, or a `not_quotable` call that cannot be priced from any card), this is when the quote was answered.' + description: 'When the rate card this quote was priced from was fetched. The estimate is not a price lock: the run is charged at the rates in force when it is rated, not at this instant. An `estimated` quote taken from the vendor''s published price list (a model billed the cost the vendor reports) carries the date that list was read instead. Whenever no rate card was read (`reason: pricing_unavailable`, `byok`, or a `not_quotable` call that cannot be priced from any card), this is when the quote was answered.' reason: type: string description: 'Present only when `source` is `unknown`: `not_quotable` (Router cannot price this call before it runs - its price depends on what the provider reports back, or the body does not bound a billed quantity), `unpriced_model` (the rate card does not price this request), `byok` (queued submit only: the run uses your own provider key, so Comfy bills nothing for it), or `pricing_unavailable` (queued submit only: the rate card could not be read when the request was admitted; this is transient, and the run is still charged as usual). Read any other value as `not_quotable`.' @@ -679,6 +685,10 @@ components: $ref: '#/components/schemas/RouterModelSegment' billing: $ref: '#/components/schemas/RouterModelBilling' + schema_etag: + type: string + description: 'The current strong `ETag` of this model''s `GET /v2/models/{provider}/{model}/openapi.json` document. Send it as `If-None-Match` on that read to get a `304` while the schema is unchanged, or compare it with the value seen last time to learn which models'' schemas changed without fetching every document. Absent whenever that read would not answer `200` for this caller at this moment: the schema service is not available, the model''s document cannot be rendered, or the model stopped being available to this caller while the response was being built. An absent tag is not a schema change; re-read the model''s document to learn its current tag.' + example: '"9f2c1e0b7a4d3c6e8f5a2b1d0c9e8f7a6b5c4d3e2f1a0b9c8d7e6f5a4b3c2d1e"' required: - id - provider @@ -733,7 +743,7 @@ components: example: bfl RouterQueueCancelResponse: type: object - description: The answer to a cancellation ask on the two statuses that describe a request this route resolved - the `202` and the `400`. One body shape across both rather than a success envelope plus an error envelope, because both are the same statement - what cancelling found - and a client that has to parse a different type per status code gains nothing from the split. + description: The answer to a cancellation ask on the two statuses that describe a request this route resolved - the `202` and the `409`. One body shape across both rather than a success envelope plus an error envelope, because both are the same statement - what cancelling found - and a client that has to parse a different type per status code gains nothing from the split. properties: request_id: $ref: '#/components/schemas/RouterQueueRequestId' @@ -806,7 +816,7 @@ components: estimate: allOf: - $ref: '#/components/schemas/RouterCostEstimate' - description: 'The pre-flight cost estimate for this request, present on a fresh admission when the cost estimate is enabled for the caller. It quotes the provider leg this request was pinned to (the queue has no fallback loop, so `fallback_provider` never changes the leg that runs). When no figure can be given it is still present, with `source: unknown`; the submission never fails because of the estimate. Omitted on an idempotent replay (`Idempotent-Replayed: true`), because the original quote is not stored. It is not a price lock: the run is charged at the rates in force when it is rated.' + description: 'The pre-flight cost estimate for this request, present on a fresh admission when the cost estimate is enabled for the caller. It quotes the provider leg this request was pinned to (the queue has no fallback loop, so `fallback_provider` never changes the leg that runs). When no figure can be given it is still present, with `source: unknown`; the submission never fails because of the estimate. Omitted on an idempotent replay (`Idempotent-Replayed: true`), because the original quote is not stored. It is an estimate, not the final charge, and not a price lock: the run is charged at the rates in force when it is rated, and the settled charge in your workspace''s usage and billing records is final.' required: - request_id - status @@ -982,6 +992,17 @@ components: application/json: schema: $ref: '#/components/schemas/RouterErrorResponse' + RouterQueueSubmitTooManyRequests: + description: 'Inspect `X-Comfy-Error-Type`: `queue_backlog_full` means you already hold as many unfinished queued requests as you are allowed, so submit again once one of them finishes; `rate_limited` means wait for the allowance window. Nothing was queued or charged, and the `Idempotency-Key` is not consumed. A submit at your in-flight limit is queued rather than refused, so `concurrency_limit_exceeded` is never answered here.' + headers: + X-Comfy-Error-Type: + $ref: '#/components/headers/RouterErrorTypeHeader' + X-Comfy-Request-Id: + $ref: '#/components/headers/RouterRequestIdHeader' + content: + application/json: + schema: + $ref: '#/components/schemas/RouterErrorResponse' RouterRequestError: description: A Router request-level failure - the request never reached the model, or failed for a reason the model itself did not report. The body is `RouterErrorResponse` and the bucket is repeated on `X-Comfy-Error-Type`. headers: @@ -1030,14 +1051,14 @@ components: name: fallback_provider in: query required: false - description: 'Controls whether Router retries this call against another of the model''s registered providers when the first attempt fails for a reason attributable to Router''s own side or to the specific provider tried - never for a reason attributable to the request itself (an unretried failure is refused exactly as it always was). Omitted, or any value other than `false`, turns fallback on (the default) and Router scans the model''s registered alternates in a fixed order and retries once against the first eligible one. `false` turns fallback off: a failure is refused, never retried. A successful fallback response carries the `X-Comfy-Router-Fallback-Provider` header, naming the provider that served it; a fallback attempt that itself also fails does not carry the header, and no case retries a generation that may already have been submitted to a provider. Fallback also turns itself off, regardless of this parameter, on a request that resolved a bring-your-own-key credential, on a request whose body selects a multipart operation, and under `strict_mode=true` - each binds the call to one specific provider and cannot be faithfully replayed against another: the alternate leg would dispatch on Comfy''s own key rather than on the caller''s credential, no alternate translator is guaranteed to carry the multipart media field, and a strict body is already shaped for one provider rather than for the native contract. Each of those is inert, not refused - no retry is attempted and the first attempt''s own failure reaches the caller unchanged. Only an explicit `model_provider` is refused, and see that parameter for the `409`s it answers with; note that fallback''s multipart disable is unconditional, while `model_provider`''s multipart `409` applies only to a leg whose translator cannot serve the operation.' + description: 'Controls whether Router retries this call against another of the model''s registered providers when the first attempt fails for a reason attributable to Router''s own side or to the specific provider tried - never for a reason attributable to the request itself (an unretried failure is refused exactly as it always was). Omitted, or any value other than `false`, turns fallback on (the default) and Router scans the model''s registered alternates in a fixed order and retries once against the first eligible one. `false` turns fallback off: a failure is refused, never retried. A successful fallback response carries the `X-Comfy-Router-Fallback-Provider` header, naming the provider that served it; a fallback attempt that itself also fails does not carry the header, and no case retries a generation that may already have been submitted to a provider. Fallback also turns itself off, regardless of this parameter, on a request that resolved a bring-your-own-key credential, on a request whose body selects a multipart operation, and under `strict_mode=true` - each binds the call to one specific provider and cannot be faithfully replayed against another: the alternate leg would dispatch on Comfy''s own key rather than on the caller''s credential, no alternate translator is guaranteed to carry the multipart media field, and a strict body is already shaped for one provider rather than for the native contract. Each of those is inert, not refused - no retry is attempted and the first attempt''s own failure reaches the caller unchanged. Only an explicit `model_provider` is refused, and see that parameter for the `409`s it answers with; note that fallback''s multipart disable is unconditional, while `model_provider`''s multipart `409` applies only to a leg whose translator cannot serve the operation. A workspace configured for direct provider calls only never runs this retry, and refuses a non-default `model_provider` with `403` unless the request replays an already accepted `Idempotency-Key`.' schema: type: string ModelProvider: name: model_provider in: query required: false - description: 'Selects an alternate provider to serve this model, instead of its current default. Omitting it runs native dispatch on the model''s own default provider; `default`, `comfy`, and `comfyui` are aliases for that same native behavior and name no override, because Comfy Router is never itself a serving backend. The alternate providers Router can retarget a model onto are `fal`, `wavespeed`, `runware`, and `higgsfield`; which of them a given model supports is reported by `GET /v2/models/{provider}/{model}`. When an alternate provider is selected, the native request body is translated into that provider''s real schema unless `strict_mode=true`; see `strict_mode` and `fallback_provider`. Its refusals are checked in a fixed order, and an earlier one answers whether or not a later one would. A value that is not a registered provider at all is refused `400` with `error_type: invalid_input`. Then a request whose body selects a multipart operation (an edit - for example gpt-image''s `image` field) is refused `409`, also with `error_type: invalid_input`, whose `detail` begins "this request''s `image` selects the edit operation" - but only when the named provider''s translator for this model cannot itself serve that operation; a leg whose translator does carry the media is not refused and proceeds normally, so this is a per-leg refusal rather than a blanket one. Then a request that has already resolved a bring-your-own-key credential for the provider in the path is refused `409`, also with `error_type: invalid_input`, whose `detail` begins `this request resolved a BYOK credential` - see the `409` on this route, where that `detail` is the only thing separating this case, and the multipart case above, from the `Idempotency-Key` one. That BYOK check runs whether or not the named provider has a leg for this model. Then the named provider''s own gate refuses `403` with `error_type: not_enabled` when that provider is not turned on for you, and `503` with `error_type: service_unavailable` when the gate cannot be evaluated - a flag-evaluation failure, or a missing or nil gate entry. Only past all of those is a real provider that does not serve this model refused `400` with `error_type: invalid_input`, the same answer as a value that is not a registered provider at all - both mean the `model_provider` value cannot serve this model, and that `400`''s `detail` is human-readable and not a contract, so read `GET /v2/models/{provider}/{model}` to learn which providers a model does support rather than parsing it; past that `400`, this workspace''s partner-provider policy for the named vendor is evaluated too and can refuse `403` or `503` of its own. Neither this parameter nor `fallback_provider` is available on a BYOK request, and the two are unavailable in different ways: the credential was resolved for the provider named in the path while every alternate leg dispatches on Comfy''s own key, so an explicit `model_provider` is refused with that `409`, and `fallback_provider` is inert rather than refused - no retry against an alternate provider is attempted and the first attempt''s own failure is what the caller receives. The same split applies to a multipart body: only an explicit `model_provider` reaches the `409` above, and only for a leg whose translator cannot serve the operation, while the automatic on-failure retry `fallback_provider` controls is simply skipped (inert, not refused) for any multipart body. Router defines no `provider_not_available` or `validation_error` `error_type`: these conditions fold onto `invalid_input`. The `error_type` set can still grow, so treat any value you do not recognize as `internal_error` rather than switching exhaustively.' + description: 'Selects an alternate provider to serve this model, instead of its current default. Omitting it runs native dispatch on the model''s own default provider; `default`, `comfy`, and `comfyui` are aliases for that same native behavior and name no override, because Comfy Router is never itself a serving backend. The alternate providers Router can retarget a model onto are `fal`, `wavespeed`, `runware`, and `higgsfield`; which of them a given model supports is reported by `GET /v2/models/{provider}/{model}`. When an alternate provider is selected, the native request body is translated into that provider''s real schema unless `strict_mode=true`; see `strict_mode` and `fallback_provider`. Its refusals are checked in a fixed order, and an earlier one answers whether or not a later one would. A value that is not a registered provider at all is refused `400` with `error_type: invalid_input`. Then a workspace configured for direct provider calls only refuses any non-default `model_provider` with `403` and `error_type: forbidden`, and Router never runs its `fallback_provider` retry for it; a resubmit carrying the `Idempotency-Key` of a request already accepted is answered as a replay of that request rather than refused. Then a request whose body selects a multipart operation (an edit - for example gpt-image''s `image` field) is refused `409`, also with `error_type: invalid_input`, whose `detail` begins "this request''s `image` selects the edit operation" - but only when the named provider''s translator for this model cannot itself serve that operation; a leg whose translator does carry the media is not refused and proceeds normally, so this is a per-leg refusal rather than a blanket one. Then a request that has already resolved a bring-your-own-key credential for the provider in the path is refused `409`, also with `error_type: invalid_input`, whose `detail` begins `this request resolved a BYOK credential` - see the `409` on this route, where that `detail` is the only thing separating this case, and the multipart case above, from the `Idempotency-Key` one. That BYOK check runs whether or not the named provider has a leg for this model. Then the named provider''s own gate refuses `403` with `error_type: not_enabled` when that provider is not turned on for you, and `503` with `error_type: service_unavailable` when the gate cannot be evaluated - a flag-evaluation failure, or a missing or nil gate entry. Only past all of those is a real provider that does not serve this model refused `400` with `error_type: invalid_input`, the same answer as a value that is not a registered provider at all - both mean the `model_provider` value cannot serve this model, and that `400`''s `detail` is human-readable and not a contract, so read `GET /v2/models/{provider}/{model}` to learn which providers a model does support rather than parsing it; past that `400`, this workspace''s partner-provider policy for the named vendor is evaluated too and can refuse `403` or `503` of its own. Neither this parameter nor `fallback_provider` is available on a BYOK request, and the two are unavailable in different ways: the credential was resolved for the provider named in the path while every alternate leg dispatches on Comfy''s own key, so an explicit `model_provider` is refused with that `409`, and `fallback_provider` is inert rather than refused - no retry against an alternate provider is attempted and the first attempt''s own failure is what the caller receives. The same split applies to a multipart body: only an explicit `model_provider` reaches the `409` above, and only for a leg whose translator cannot serve the operation, while the automatic on-failure retry `fallback_provider` controls is simply skipped (inert, not refused) for any multipart body. Router defines no `provider_not_available` or `validation_error` `error_type`: these conditions fold onto `invalid_input`. The `error_type` set can still grow, so treat any value you do not recognize as `internal_error` rather than switching exhaustively.' schema: type: string RejectUnknownFields: @@ -1188,6 +1209,26 @@ components: required: true schema: $ref: '#/components/schemas/RouterErrorType' + RouterEstimateSourceHeader: + description: 'Sent only on the status read while `status` is `IN_PROGRESS`, beside `X-Comfy-Estimated-Duration-Seconds`: where that generation-duration estimate came from (not the cost estimate''s `source`), `historical` (measured on recent runs) or `default` (a per-media-kind fallback, used while too few recent runs are recorded).' + required: false + schema: + type: string + example: historical + RouterEstimatedDurationHeader: + description: 'Sent only on the status read while `status` is `IN_PROGRESS`: Router''s estimate, in whole seconds, of the generation''s whole run once the provider has accepted it (not the time remaining, and never time spent `IN_QUEUE`). It is the median of recent runs at the closest level of model, endpoint and settings that has enough history, or a per-media default when none has. The same figure is repeated on every read. It is never sent on the collect `202`, and it can first appear on a later `IN_PROGRESS` read than the first one, once the provider has accepted the job; absent when no estimate can be given.' + required: false + schema: + type: integer + minimum: 1 + example: 30 + RouterEstimatedDurationP90Header: + description: 'Sent only on the status read while `status` is `IN_PROGRESS`, beside `X-Comfy-Estimated-Duration-Seconds`: the 90th percentile, in whole seconds, of that generation-duration estimate.' + required: false + schema: + type: integer + minimum: 1 + example: 90 RouterFallbackProviderHeader: description: 'Present, naming the provider, only when `fallback_provider` actually retried this call against a second provider and that retry succeeded - the provider that ultimately served the call, never one that was attempted and also failed. Absent when the primary attempt itself succeeded, and absent on an error response. It is stored with the `Idempotency-Key` record and replayed unchanged on a same-key retry (alongside `Idempotent-Replayed: true`), so a retry names the same provider the original call did rather than reading as an unsubstituted run. In the one case where the stored headers were too large to keep in full, the record is marked non-replayable and the retry is refused with `409` rather than served without this header, so a same-key retry never reads as an unsubstituted run either way. See `fallback_provider` for the retry policy this discloses.' required: false @@ -1214,7 +1255,7 @@ components: type: string example: openai RouterQueuePollAfterHeader: - description: Seconds to wait before polling this queued request again. It is Router's own estimate of when asking again is worth the round trip, and it moves with how far the request has actually got - a request at the back of the queue is told to wait longer than one already running. + description: 'Seconds to wait before polling this queued request again. It is Router''s own estimate of when asking again is worth the round trip: while the request waits it grows with its place in the queue, while it runs it grows with how long it has been running, and it stops growing at 10.' required: false schema: type: integer