Repository navigation
fix: reject invalid timezone offsets and arrays in JSON schema validation - #680
Open
OsamaAnsar wants to merge 1 commit into
Open
OsamaAnsar wants to merge 1 commit into
OsamaAnsar wants to merge 1 commit into
Conversation
…tion The JSON schema validator accepted `time` and `date-time` strings whose timezone offset has no hour (for example `12:00:00+:30`), because the hour group in the regexes was optional, while the grammar built for the schema requires it. It also accepted an array for an `object` schema, since `typeof [] === "object"`. Make the offset hour mandatory and reject arrays when an object is expected.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of change
Two cases where
validateObjectAgainstGbnfSchema(used byLlamaJsonSchemaGrammar.parseand function-call param parsing) accepts values that the grammar for the same schema can never produce:timeRegex/datetimeRegexhad an optional hour in the timezone offset ([+-]([01][0-9]|2[0-3])?:[0-5][0-9]), so12:00:00+:30and2024-01-01T10:00:00-:00were valid. The grammar inGbnfFormatString.tsrequires the hour (("+" | "-") ([01] [0-9] | "2" [0-3]) ":" [0-5] [0-9]). The?is removed.validateObjectcheckedtypeof object !== "object" || object === null, so an array passed for anobjectschema (typeof [] === "object"). An object schema's grammar starts with{, so an array is never valid. Arrays are now rejected, with the error message saying"array"instead of"object"(and"null"for null).Before / after (real output from
validateObjectAgainstGbnfSchema, run withnpx tsx; "throws" isLlamaJsonSchemaValidationError):Tests: new standalone
test/standalone/utils/validateObjectAgainstGbnfSchema.test.ts(does not need the native addon or a model); it also asserts that the grammar text for the schema requires the offset hour and that an object grammar starts with{. With only the source fix reverted 5 of the 10 tests fail; with the fix all 10 pass.tsc --noEmitand eslint onsrc/utils/gbnfJsonand the new test pass.Not tested: the native addon could not be built in my environment, so
LlamaGrammar.test.tsand other suites that need it, and the model-dependent tests, were not run; the grammar was checked only as text, not compiled by llama.cpp.Pull-Request Checklist
masterbranchnpm run formatto apply eslint formattingnpm run testpasses with this change - the new tests,tsc --noEmitand eslint pass; tests that need the native addon / models were not runFixes #0000- N/A, no existing issue found