Repository navigation
feat(core): ryzenai-corelib 0.103.0; a JSON Schema pattern matches anywhere in the string - #1047
Conversation
…esting_npu_arch Moves the core backend to ryzenai-corelib 0.70.0 (corelib main at 39c30eab). What changed in corelib 0.61 to 0.70 that reaches FastFlowLM: - 0.70, NPU query: ryzenai_corelib_npu_usable and the testing hook ryzenai_corelib_testing_npu_arch are gone. ryzenai_corelib_get_npu (none / aiem / aie2p) and ryzenai_corelib_npu_name replace them, so corelib_api.hpp no longer includes corelib/testing.h. - The runtime's readiness check now asks for aiem. The old npu_usable was false on AIE2P, whose kernels corelib runs only behind an xclbin FastFlowLM never gives it; get_npu now answers aie2p there, so checking for "any NPU" would have offered the core backend on Strix. - NpuArch names the enum with npu_name, "" for none, as before, so dev04 / aie2p platform detection is unchanged. - 0.61 / 0.68, model structs: model_metadata gains max_draft_tokens and audio_token_id; session_options gains draft_tokens. FastFlowLM zero-initialises both at their full sizeof, so no draft is asked for. - 0.62 / 0.67, matmul: the 4-bit layout is mladf_v4 (was mladf) beside AIE2P's mladf_v2, and matmul_bf16_kernel_info gains format. FastFlowLM names neither; the fake corelib fills format. - 0.68: ssmlp_bf16_weights_desc gains ungated; unused here. - Model libraries and their names are unchanged. The weights loader's switch from bad_argument to unsupported for a wrong tensor type does not matter here: FastFlowLM does not branch on corelib's status codes. The fake corelib answers get_npu from one enum and npu_name as corelib spells it; the tests refuse none and aie2p, and read "" for none. On an AIE4 (Medusa) test machine, qwen3-core 0.6b/1.7b/4b/8b, phi4-mini-it-core:4b, qwen3.5-core 4b/9b and llama3.2-core:1b each load, pass the readiness check through get_npu, report no fallback and answer correctly, at decode rates within a few percent of corelib 0.59's. Co-authored-by: Cursor <cursoragent@cursor.com>
…keeps a one-character class's length bounds
JSON Schema finds a string pattern anywhere in the string; xgrammar
matches the regex against the whole string. A field written as
{"type": "string", "minLength": 1, "maxLength": 128, "pattern": "\\S"}
("not blank") therefore allowed exactly one character, and xgrammar also
drops minLength/maxLength whenever a pattern is present (it logs "pattern
combined with minLength/maxLength is not supported").
Found running an agent benchmark with qwen3.6-moe-core:35b-a3b: every
free-text argument of a forced chart-generation tool call came back as one
character ("caption":"7","alt_text":"A","labels":["P","C"]). The caller
rejected it, the grammar left the model no other call to make, and it
repeated until the tool-iteration limit.
normalize_schema() now:
- pads the side(s) a string pattern leaves unanchored with ".*" ("\S"
becomes ".*(\S).*"). A pattern anchored at both ends is left as it is,
so "^\d{4}-\d{2}-\d{2}$" keeps its meaning.
- lets a lone character class that only has to occur somewhere ("\S",
"\d", "[a-z]", ...) lead, and turns the schema's length bounds into the
repetition after it: "\S" with maxLength 128 becomes "\S.{0,127}", so
the bounds xgrammar would drop are enforced.
- keeps minLength/maxLength and drops the pattern instead when that
repetition would be longer than 128. xgrammar compiles a longer regex
repetition into a rule of its own and cannot build that rule inside a
nested object ("cannot build the decoding grammar: Check failed:
per_rule_fsm_hashes[...].has_value()"), so the request would fail with
HTTP 400; a string past its maxLength is the likelier mistake.
response_format schemas now go through normalize_schema() as tool
parameters already did, so they get the same reading.
New live cases in src/test/openai_api_test: test_tool_calls
function_unanchored_pattern and function_nested_long_bounded_pattern, and
test_response_format schema_unanchored_pattern. The two unanchored cases
fail on main ('P' for "Paris"); the nested one fails with the HTTP 400
above if the bounds are folded at any length. On an AIE4 (Medusa) test
machine with qwen3.6-moe-core:35b-a3b:
test_tool_calls.py 18/18, test_response_format.py 12/12.
Co-authored-by: Cursor <cursoragent@cursor.com>
…npu_name Moves the core backend to ryzenai-corelib 0.87.0 (corelib main at 5bf19507). What changed in corelib 0.71 to 0.87 that reaches FastFlowLM: - 0.73, NPU query: ryzenai_corelib_get_npu, ryzenai_corelib_npu_name and the ryzenai_corelib_npu enum are gone. ryzenai_corelib_has_feature and ryzenai_corelib_feature_name replace them, over ryzenai_corelib_feature (npu, npu_aiem, npu_aie2p, gpu, npu_direct). - The runtime's readiness check asks has_feature(npu_aiem). AIE2P is still refused, since FastFlowLM never gives corelib an xclbin, and the error names npu_aie2p or none. - NpuArch passes corelib's spelling through (npu_aiem, npu_aie2p, ""), and platform_from_corelib_arch maps npu_aiem to dev04. FastFlowLM still keeps no table of devices. - Everything else FastFlowLM resolves is unchanged: every other entry point in FLM_CORELIB_FUNCTIONS keeps its signature, and model_metadata and session_options keep their layout. 0.87's single PDI list per stream, NPU direct, the .amd.gguf exports (0.80, 0.83) and the new ops do not touch FastFlowLM's calls. - 0.83: corelib's 4-bit GGUF types are renamed (AMD_Q4_G128, AMD_Q4_G32, new AMD_Q4_G64). Files published under the old type codes (ggml 128, 129) are still read as before, so the catalog's AMD files need no change. - Model libraries: ryzenai_corelib_model_d3 is new. The core build copies ryzenai_corelib_model_*.dll by glob, so it lands beside flm.exe without a change; no -core tag uses it yet. - Not taken here: corelib now lets Llama 3.2-1B/3B and 3.1-8B read a Q4_G128 GGUF in place of their Q8_0. Q8_0 still loads, so the catalog's Llama entries stay on Q8_0; switching them is a separate change. The fake corelib answers has_feature from one NPU feature, or none, and feature_name as corelib spells it. The tests refuse none and npu_aie2p, each named in the error, and read "" for none. The Linux setup guide's version check now prints 0, 87, 0. On an AIE4 (Medusa) test machine, with the core flm.exe statically linked against corelib's default build (no --npu-direct): every -core tag except Llama 3B/8B loads, passes the readiness check through has_feature(npu_aiem), reports no fallback and answers correctly, at decode rates matching 0.70's. Gemma 4 E2B/E4B, Qwen3.8 and Qwen3.6 recall a code from the middle of 4k and 12k prompts (Qwen3.6 also at 24k). An flm.exe linked against a corelib built with --npu-direct engages NPU direct for Qwen3.6 and gives the same greedy output, with prefill about 10-15% faster from 12k tokens. The core build stays on corelib's default build: --npu-direct needs CHESS in WSL, so turning it on is a build-only follow-up. Co-authored-by: Cursor <cursoragent@cursor.com>
…e DFlash entries hold no licence Two checks added on main left two Windows-only core suites failing on main itself. Only tests change here. - test_phi4_frontend, test_llama_frontend: ROCm#874 added a check to AutoModel::_sample that refuses logits shorter than the sampler's in_features, which is vocab_size from config.json. Each test's FakeEngine returned buffer<bf16>(1) from forward and prefill, so every test that samples failed with "Backend returned fewer logits than the frontend vocabulary". They now return kVocabSize logits, as the core backend does: 200064 for Phi-4 and 128256 for Llama. config.json takes vocab_size from the same constant. - test_model_downloader: ROCm#950 dropped LICENSE.OpenMDW-1.1 and LICENSE from the files of nemotron3.5-bolt-dflash:e4b and qwen3.8-dflash2:27b, but the test still expected them. It now expects config.json and dflash_weight.q4nx, as the catalog lists. The check itself is unchanged. The real core backend returns logits as wide as corelib's vocab_size, which equals config.json's for the shipped -core tags, so it does not trip there. Co-authored-by: Cursor <cursoragent@cursor.com>
Moves the core backend to ryzenai-corelib 0.103.0 (corelib main at 49a8ed3c). What changed in corelib 0.88 to 0.103 that reaches FastFlowLM: - Every entry point in FLM_CORELIB_FUNCTIONS keeps its signature, and model_metadata keeps its layout, so the only code change is the required version in corelib_api.hpp. - model_session_options: lora is gone (0.97: a session runs LoRA kernels when its model has an adapter set), and low_memory (0.96) and kv_cache_bits (0.98) are appended. FastFlowLM value-initialises the struct and sets only max_sequence_length, so the defaults hold: no LoRA, the embedding table mapped, a BF16 KV cache. - Additive only, as far as FastFlowLM's calls go: uint8 tensors, stream_set_low_memory, PQ2_0 weights, head_gate_bf16, hadamard_bf16, and NPU direct on AIE2P. FastFlowLM still refuses AIE2P. - Model libraries: qwen2, lfm2, lfm3, hyv3, glide, minicpm_ocr and minicpmv47 are new. The core build copies ryzenai_corelib_model_*.dll by glob, so they land beside flm.exe without a change; no -core tag uses them yet. - Not taken here: the 8-bit KV cache (kv_cache_bits 8, AIE4 NPU direct only) and low_memory sessions. Both are opt-in and would be separate changes. The docs' version checks and the required-version notes move to 0.103. On an AIE4 (Medusa) test machine, qwen3-core 0.6b/1.7b/4b/8b, phi4-mini-it-core:4b, qwen3.5-core 4b/9b and llama3.2-core:1b each load, report no fallback and answer correctly, at decode rates matching 0.87's. Co-authored-by: Cursor <cursoragent@cursor.com>
zaneni6
left a comment
There was a problem hiding this comment.
Reviewed the grammar change and the CI history. Summary:
CI: the XRT failure is deterministic and comes from the new test, not flakiness. api/test_tool_calls.function_pattern_keeps_length_bounds [gemma4-it:e2b] fails on Test Linux XRT portable and Test Windows XRT in every attempt (run 43657612538 attempts 1–4, run 43985532394 attempt 1). The model copies the whole 113-char note into a field with maxLength: 20. Gemma 4 is the only model in the XRT suite that is shown tools (qwen3:0.6b skips them), and the HRX suites don't run test_tool_calls, so this never showed up in the PR's own testing (qwen3.6-moe-core / qwen3.5-oga-hipep both go through xgrammar).
Root cause: GEMMA4 (and LFM2 pythonic) tool calls don't use xgrammar's JSON Schema converter. tool_tags() → gemma4_arguments_grammar() → property_value() / value_rule() maps every type: string to the hand-written unbounded rule string ::= "<|\"|>" ([^<] | "<" [^|])* "<|\"|>" (chat_grammar.cpp ~L337, ~L445), ignoring pattern, minLength and maxLength, on main as well as here. So for Gemma 4, function_unanchored_pattern and function_nested_long_bounded_pattern also pass only because nothing is constrained. schema_unanchored_pattern does go through xgrammar via response_format, so that pass is real. Options: skip/relax the bounds test for the GEMMA4 / LFM2 formats, or emit a bounded repeat (([^<] | "<" [^|]){min,max}) in that string rule when the schema has length bounds.
The other failures look like infrastructure, not this PR:
- First-round Windows HRX on xconucstrhl102: ERT state 5 /
hrx_buffer_allocate failed. HRX doesn't run the tool tests, and it passed on re-run, though on a different runner. - Attempt 2 Windows XRT on xconucstrhl101:
flm.exeexit 0xC0000139, a DLL entry point not found. - Latest attempt 1 Windows XRT on xsjnuckrck01: the runner lost communication.
XRT / HRX AIE2 impact:
- Build: none. The corelib changes are under
FLM_ENABLE_CORE, andnpu_platform.hpponly renames the label the core backend matches (AIEM→npu_aiem). - Runtime: only through
chat_grammar.cpp, which is shared CPU code. It changes tool calls andresponse_formatwith stringpatternon every model. Nothing touches the NPU runtime.
The grammar fix itself is correct for the xgrammar paths (Hermes JSON, qwen_xml, response_format). I checked the helpers against the repo's xgrammar in a scratch harness. The both-anchored pass-through, odd/even \ before $, []] / [^]], and the 128 cutoff (it matches kLargeRepeatThreshold, no off-by-one) all behave as intended. Some edge cases are in the inline comments.
Reviewed with the help of Claude Code.
| if (core.empty()) { | ||
| return ".*"; | ||
| } | ||
| return (starts ? "^(" : ".*(") + core + (ends ? ")$" : ").*"); |
There was a problem hiding this comment.
Edge cases of the .* padding, found in a scratch harness against the bundled xgrammar:
- Non-ASCII pattern (most serious): xgrammar compiles it through
RegexToEBNFwithoutjson_string(json_schema_converter.cc ~L2271), where.matches",\and newline. So"é"becomes.*(é).*, and the grammar accepts{"city": "café", "x": "y"}, an extra key injected into the JSON. Before this PR, that pattern only allowed one character. - qwen_xml: the top-level patterned string is built with
force_cfg_expansion=trueand no excludes, so a padded value can contain</parameter>. Found in the code, not reproduced. (?i)abcnow fails: it becomes.*((?i)abc).*, which xgrammar refuses to build, so the request now gets a 400. It used to work.- Anchor inside a group:
(?:^abc)or(a$)isn't detected, so the pattern gets padded. xgrammar ignores mid-regex anchors, soxabcis accepted. - Top-level alternation with one anchor:
^a|bbecomes^(a|b).*, which is stricter than ECMA (it was also wrong before).
The non-ASCII one probably needs a guard, e.g. skip padding or pad with [^"\\]* when the pattern isn't printable ASCII.
There was a problem hiding this comment.
Thanks, all confirmed against the bundled xgrammar and addressed in 8268970:
- Non-ASCII: the padding is now
[^"\\\x00-\x1f]*(what a JSON string holds unescaped) instead of.*, and the folded repetition after a lone class uses the same class. On the previous code the newtest_string_patterncheck reproduces the injection ((é|x)tookx", "k": "y); it is rejected now. (?i)abc: a pattern opening with an inline flag is left as sent. In the unit harness the padded form did not just fail to build, it crashed the process.- Anchor inside a group: a
^or$anywhere but at an edge (outside classes, escapes and groups) leaves the pattern as sent, so(?:^abc)rejectsxabcagain. - Top-level alternation: each alternative is padded on its own, so
^a|bnow means "starts with a, or holds b". - qwen_xml: left as is. A patterned top-level value would need the model to write
</parameter>inside it, and the parser ends the value there as it does for any string; I'd rather not special-case it without a repro.
Each case has a check in src/test/chat_tools.
| } | ||
| const int64_t low = std::max<int64_t>(min_length.get<int64_t>(), 1) - 1; | ||
| if (max_length.is_number_integer() && max_length.get<int64_t>() - 1 < low) { | ||
| return whole_string_pattern(pattern); // unsatisfiable, which xgrammar reports |
There was a problem hiding this comment.
Minor: for minLength: 0, maxLength: 0 this branch keeps the bounds, but xgrammar ignores bounds once a pattern is present and only errors when min > max. So it doesn't report the schema as unsatisfiable; it generates e.g. "x". The comment is misleading.
There was a problem hiding this comment.
Right, the comment was wrong. Reworded in 8268970: unsatisfiable bounds fall through to the pattern alone, since xgrammar ignores bounds beside a pattern.
| return std::nullopt; | ||
| } | ||
| const std::string high = max_length.is_null() ? std::string() : std::to_string(max_length.get<int64_t>() - 1); | ||
| schema.erase("minLength"); |
There was a problem hiding this comment.
Suggest keeping minLength / maxLength here rather than erasing them. xgrammar ignores them once a pattern is present, so the xgrammar paths behave the same. But the hand-written GEMMA4 / LFM2 argument grammars (property_value / value_rule) would then still see the bounds if they learn to apply them. That is the fix the failing function_pattern_keeps_length_bounds [gemma4-it:e2b] needs.
Also worth a comment: the folded form forces the class to the front, so " Paris" or \d + bounds with "Room 5" can no longer be produced. That's narrower than JSON Schema but always valid.
There was a problem hiding this comment.
Addressed differently, via @tawei-amd's 3d4a048: the Gemma4/LFM2 grammars now read tool_def.declared, the schema as sent, so they see minLength/maxLength without normalize_schema keeping them (which would also bring back xgrammar's pattern combined with minLength/maxLength warning). The narrowing is now noted in the comment on string_pattern (8268970).
| json response_schema(const chat_constraint_request& request) { | ||
| if (!request.json_schema.is_null()) { | ||
| return request.json_schema; | ||
| return normalize_schema(request.json_schema); |
There was a problem hiding this comment.
Note that this widens the behaviour change: every response_format schema now gets the pattern rewrite, including the edge cases above. Users will also see that an unanchored "\d{5}" changes from exactly five digits to contains five digits. That's spec-correct, but worth a line in the release notes.
normalize_schema also recurses into const / enum / default / examples / not values. Rewriting a pattern inside not changes its meaning. Low risk, but cheap to exclude those keys.
There was a problem hiding this comment.
8268970 stops normalize_schema from entering const, enum, default, examples and not; maps keyed by name (properties, patternProperties, $defs, definitions, dependentSchemas) still recurse, so a property called default is still a schema. Both have checks in test_chat_tools. I've added the release-note line about unanchored patterns (\d{5} now means contains five digits) to the PR description.
| body = api.body(user(f"Save this note word for word: {NOTE}"), tools=[tool], | ||
| tool_choice=function_choice("save_note")) | ||
| text = json.loads(expect_calls(api.ok(body), [tool])[0]["function"]["arguments"])["text"] | ||
| expect(1 < len(text) <= 20, f"text {text!r} is outside 2..20 characters") |
There was a problem hiding this comment.
This fails deterministically on both XRT jobs for gemma4-it:e2b, because the GEMMA4 tool-argument grammar never applies pattern / minLength / maxLength (see the review summary). It needs a skip for the GEMMA4 / LFM2 formats, or a grammar fix there.
Separately, 1 < len(text) disagrees with the schema's minLength: 1: a valid one-character answer would fail. The same goes for the len(...) > 1 checks in function_unanchored_pattern and schema_unanchored_pattern. Those are trying to catch the old one-character bug, so maybe check against the expected word (e.g. contains "Paris") instead.
|
The two NPU test failures here (Test Linux XRT portable, Test Windows XRT) both come from one case: Root cause: Gemma4 (and LFM2) tool arguments aren't constrained through xgrammar's JSON-schema path. FLM writes their EBNF itself ( Fix: branch
Tested locally (Linux, Strix NPU):
@chizamd could you cherry-pick it onto Or if you'd rather I push it to your branch (maintainer edits are on), just say so. 🤖 Generated with Claude Code |
…xLength
The argument grammars FastFlowLM writes itself (Gemma4, LFM2) gave every
string property the unbounded string rule, so a tool parameter with
maxLength 20 took a whole sentence: test_tool_calls
function_pattern_keeps_length_bounds failed on gemma4-it:e2b, Linux and
Windows. normalize_schema had also folded the bounds into a pattern that
only xgrammar's JSON schema reads.
These grammars now read the schema as sent and spell bounds up to
XGRAMMAR_REGEX_REPEAT_LIMIT: Gemma4 through counted gstr_<low>_<high>
rules (a "<" step takes the next character too, so lengths stay exact
and "<|" still cannot occur), LFM2 through a {low,high} repetition where
an escape is one character. Array items keep their bounds too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… flags and inner anchors as sent
Review follow-ups to the pattern rewrite:
- The padding is now a class of what a JSON string holds unescaped, not
".*". xgrammar keeps '"', '\' and control characters out of a pattern's
matches only when the pattern is printable ASCII; any other pattern
compiles without that guard, where ".*" could close the string and add
keys ("(é|x)" took x", "k": "y). The folded repetition after a lone
class uses the same class.
- A pattern opening with an inline flag such as "(?i)" is left as sent:
padded, it no longer builds.
- A pattern with an anchor inside a group, such as "(?:^abc)", is left as
sent: xgrammar ignores an anchor in the middle, so padding would have
let "xabc" through.
- Each top-level alternative is padded on its own, so "^a|b" reads as
"starts with a, or holds b" rather than "^(a|b).*".
- normalize_schema no longer enters const, enum, default or examples,
which are values, nor "not", whose meaning a rewrite would change. A
property named like one of them is still a schema.
- The comment on unsatisfiable bounds now says what happens: xgrammar
ignores the bounds beside a pattern. The folded form's narrowing is
noted.
The live tests check for the expected word instead of a length above
one, which a valid one-character answer would have failed.
New checks in test_chat_tools cover each case through the grammar engine;
on the previous code "^a|b" and the non-ASCII injection fail, and the
inline-flag and inner-anchor patterns crash the harness.
Co-authored-by: Cursor <cursoragent@cursor.com>
|
@tawei-amd thanks, cherry-picked as 3d4a048 (authorship kept). I read through it and it looks right to me; one note for the record: a bounded Gemma4 string can no longer hold 8268970 on top addresses @zaneni6's inline comments (padding that cannot end the string, inline flags, inner anchors, alternation, literal keywords, test assertions); the replies are on each thread. |
Follow-up to #927. Moves the
corebackend from ryzenai-corelib 0.60 to 0.103.0 (through 0.70 and 0.87), fixes how a JSON Schema stringpatternis turned into a grammar, and fixes two core test suites that #874 and #950 left failing onmain. Seven commits onmainatfbf2228d(the sixth by @tawei-amd); they merge cleanly withmainate450e495.Release note: tool-call arguments and
response_formatstrings now read a JSON Schemapatternas the spec does, matching anywhere in the string unless anchored: an unanchored"\d{5}"now means contains five digits, not is five digits. Anchor with^...$to match the whole string.Commits
39c30eab).npu_usableand thetesting_npu_archhook go;get_npu(none/aiem/aie2p) andnpu_namereplace them. The readiness check asks foraiemspecifically:get_npuanswersaie2pon Strix, where corelib needs an xclbin FastFlowLM never gives it, so "any NPU" would have offered the core backend there. The newmodel_metadata/session_optionsfields (draft tokens, audio token) are zero-initialised, so no draft is asked for.patternanywhere in the string, xgrammar matches it against the whole string, so{"type": "string", "minLength": 1, "pattern": "\\S"}("not blank") allowed exactly one character. Seen withqwen3.6-moe-core:35b-a3bin an agent benchmark: every free-text argument of a forced tool call came back as one character, and the model looped until the tool-iteration limit.normalize_schema()now pads an unanchored side with.*, folds a lone character class'sminLength/maxLengthinto the regex (xgrammar otherwise drops them), and keeps the length bounds and drops the pattern when that repetition would pass 128, which xgrammar cannot build inside a nested object (HTTP 400).response_formatschemas now go throughnormalize_schema()too. Not core-specific: this applies to every model's tool calls andresponse_format.5bf19507).get_npu,npu_nameand the NPU enum go;has_feature(npu_aiem)is the readiness check andfeature_namethe spellingNpuArchpasses through (npu_aiemmaps todev04). Every other entry point and struct FastFlowLM resolves is unchanged. corelib's renamed 4-bit GGUF types still read the old type codes, so no catalog file changes; the newryzenai_corelib_model_d3library is copied by the existing glob and no tag uses it yet. The Linux setup guide's version check now prints 0, 87, 0.test_phi4_frontendandtest_llama_frontendstubbed one logit, which Add stable model extension support #874's vocabulary check now refuses; they returnvocab_sizelogits as the core backend does.test_model_downloaderstill expected the licence files model list: drop license and repo-doc files from model downloads #950 dropped from the two DFlash entries. Tests only.49a8ed3c). Every entry point FastFlowLM resolves keeps its signature andmodel_metadataits layout, so the only code change is the required version. Inmodel_session_options,lorais gone andlow_memory/kv_cache_bitsare appended; FastFlowLM value-initialises it and sets onlymax_sequence_length, so the defaults hold (no LoRA, a BF16 KV cache). The new model libraries (qwen2,lfm2,lfm3,hyv3,glide,minicpm_ocr,minicpmv47) are copied by the existing glob; no-coretag uses them yet. The 8-bit KV cache andlow_memorysessions are opt-in and not taken here.function_pattern_keeps_length_bounds [gemma4-it:e2b]failed on both XRT jobs. They now read the schema as sent (tool_def.declared) and spell bounds up to 128: counted rules for Gemma4, a{low,high}repetition for LFM2.[^"\\\x00-\x1f]*rather than.*(xgrammar drops its JSON string guard for a non-ASCII pattern, where.*could close the string and add keys); a pattern opening with(?i)-style flags or holding an anchor inside a group is left as sent; each top-level alternative is padded on its own (^a|b);normalize_schemano longer entersconst/enum/default/examples/not. The live tests check for the expected word rather than a length above one.Only the core backend (
FLM_ENABLE_CORE), the chat grammar and tests change; no FastFlowLM (aie2p) engine or catalog entry is touched.Testing
Built on Windows with the fake corelib: against the 0.103.0 headers at the head of this branch,
core_model4/4,gemma4_core3/3,llama_core1/1,model_backend1/1,model_list_platform1/1,phi4_core5/5; against the 0.87.0 headers at commit 4, the same pluschat_tools1/1; at the head,chat_toolspasses with the new pattern and string-length checks, which on the code before commits 6 and 7 fail or crash the harness (test_real_corelibandtest_gemma4_tokenizerskip without an NPU and a GGUF).On an AIE4 (Medusa) test machine, core
flm.exelinked against corelib's default build,flm serveper tag withRYZENAI_CORELIB_REPORT_FALLBACKS=1, multi-turn capped at 2048 tokens. At 0.103.0 and at 0.87.0 every tag loads, passeshas_feature(npu_aiem), reports no fallback, and answers 2+2, a one-sentence prompt and a two-turn recall. Decode tok/s (the 2+2 answer / the one-sentence answer):qwen3-core:0.6b(Q8_0)qwen3-core:1.7bqwen3-core:4bqwen3-core:8bphi4-mini-it-core:4bqwen3.5-core:4bqwen3.5-core:9bllama3.2-core:1b(Q8_0)At 0.103 the first Phi-4-mini run answered 2+2 with "2" (at its default temperature 0.8); two reruns answered 4, and the table shows those. At 0.87 Qwen3-0.6B's second turn mentioned the number without saying it was the user's favourite. Phi-4-mini's 2+2 rate is over a 2-token answer.
At 0.87, Gemma 4, Qwen3.8 and Qwen3.6, pulled by this
flm.exe, recall a code placed in the middle of a long prompt (not rerun at 0.103: their GGUFs were no longer on the test machine):gemma4-it-core:e2bgemma4-it-core:e4bqwen3.8-core:27bqwen3.6-moe-core:35b-a3bThe chat fix was run live on the same machine with
qwen3.6-moe-core:35b-a3b:test_tool_calls.py18/18 andtest_response_format.py12/12, including the three new cases (function_unanchored_pattern,function_nested_long_bounded_pattern,schema_unanchored_pattern); the two unanchored cases fail onmain.Not run: Gemma 4, Qwen3.8 and Qwen3.6 at 0.103, Llama 3.2-3B / 3.1-8B on hardware, and the AIE2P refusal on a Strix machine.
NPU direct, not turned on
At 0.87, a core
flm.exelinked against a corelib built with--npu-directruns Qwen3.6's attention NPU direct from 12k keys, with the same greedy output and prefill about 10% faster at 12k and 15% at 24k. The core build stays on corelib's default build, since--npu-directneeds CHESS in WSL; turning it on later is a build-only change.