Skip to content

fix: reject invalid timezone offsets and arrays in JSON schema validation - #680

Open
OsamaAnsar wants to merge 1 commit into
withcatai:masterfrom
OsamaAnsar:fix/gbnf-validator-offset-and-array
Open

OsamaAnsar wants to merge 1 commit into
withcatai:masterfrom
OsamaAnsar:fix/gbnf-validator-offset-and-array

Conversation

@OsamaAnsar

Copy link
Copy Markdown

Description of change

Two cases where validateObjectAgainstGbnfSchema (used by LlamaJsonSchemaGrammar.parse and function-call param parsing) accepts values that the grammar for the same schema can never produce:

  1. timeRegex / datetimeRegex had an optional hour in the timezone offset ([+-]([01][0-9]|2[0-3])?:[0-5][0-9]), so 12:00:00+:30 and 2024-01-01T10:00:00-:00 were valid. The grammar in GbnfFormatString.ts requires the hour (("+" | "-") ([01] [0-9] | "2" [0-3]) ":" [0-5] [0-9]). The ? is removed.
  2. validateObject checked typeof object !== "object" || object === null, so an array passed for an object schema (typeof [] === "object"). An object schema's grammar starts with {, so an array is never valid. Arrays are now rejected, with the error message saying "array" instead of "object" (and "null" for null).

Before / after (real output from validateObjectAgainstGbnfSchema, run with npx tsx; "throws" is LlamaJsonSchemaValidationError):

                                            before    after
time "12:00:00+:30"                         true      throws: Expected a valid time string but got "12:00:00+:30"
date-time "2024-01-01T10:00:00-:00"         true      throws: Expected a valid date-time string but got ...
time "12:00:00+05:30" / "12:00:00Z"         true      true
[] vs {type: "object", properties: {}}      true      throws: Expected an object but got "array"
{} vs {type: "object", properties: {}}      true      true
[] vs oneOf [object, null]                  true      throws: Expected one of 2 schemas but got []

Tests: new standalone test/standalone/utils/validateObjectAgainstGbnfSchema.test.ts (does not need the native addon or a model); it also asserts that the grammar text for the schema requires the offset hour and that an object grammar starts with {. With only the source fix reverted 5 of the 10 tests fail; with the fix all 10 pass. tsc --noEmit and eslint on src/utils/gbnfJson and the new test pass.

Not tested: the native addon could not be built in my environment, so LlamaGrammar.test.ts and other suites that need it, and the model-dependent tests, were not run; the grammar was checked only as text, not compiled by llama.cpp.

Pull-Request Checklist

  • Code is up-to-date with the master branch
  • npm run format to apply eslint formatting
  • npm run test passes with this change - the new tests, tsc --noEmit and eslint pass; tests that need the native addon / models were not run
  • This pull request links relevant issues as Fixes #0000 - N/A, no existing issue found
  • There are new or updated unit tests validating the change
  • Documentation has been updated to reflect this change - N/A, no API or usage change
  • The new commits and pull request title follow conventions explained in pull request guidelines (PRs that do not follow this convention will not be merged)

…tion

The JSON schema validator accepted `time` and `date-time` strings whose timezone offset has no
hour (for example `12:00:00+:30`), because the hour group in the regexes was optional, while the
grammar built for the schema requires it. It also accepted an array for an `object` schema, since
`typeof [] === "object"`.

Make the offset hour mandatory and reject arrays when an object is expected.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant