Skip to content

fix(ci): install evaluation dependencies through SDK extras - #711

Merged
jariy17 merged 1 commit into
mainfrom
fix/evaluation-dependency-constraints
Oct 23, 2026
Merged

jariy17 merged 1 commit into
mainfrom
fix/evaluation-dependency-constraints

Conversation

@jariy17

@jariy17 jariy17 commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

Summary

The evaluation job in run 39875066084 failed during pytest plugin loading, before running tests. Its unconstrained dependency install selected DeepEval 0.21.28 with LangChain 1.4.7; that DeepEval version imports the removed langchain.schema module.

Install the local SDK with its deepeval, autoevals, and ragas extras so CI uses the supported adapter constraints already declared in pyproject.toml. Resolve the SDK, adapters, and test tools in one pip invocation. Other matrix jobs default to the local SDK without extras and retain their existing additional dependencies.

Validation

  • Fresh Python 3.10 environment: pip installation succeeds, pip check passes, pytest plugin loading succeeds, and all three SDK adapters import successfully. Resolved DeepEval 4.3.1 with LangChain 1.4.7.
  • Verified the default editable install for jobs without evaluation extras.
  • Repository lockfile environment: 3,656 tests passed, 10 skipped, 4 xpassed; coverage 91.18% (90% required).
  • All pre-commit checks and Bandit passed.
  • Fresh dependency environment: 160 adapter tests passed and two unknown-scope error-code assertions failed. Both failures also reproduce on the unchanged base commit with the same dependencies (test_02_unrecognized_scope and test_15_unrecognized_scope_deepeval); no adapter or test source is changed here.

Live AWS/LLM integration execution remains for CI.

@jariy17
jariy17 requested a review from a team October 23, 2026 17:15
@jariy17
jariy17 merged commit 4bb326e into main Oct 23, 2026
17 of 18 checks passed
@github-actions github-actions Bot added the size/xs PR size: XS label Oct 23, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Oct 23, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/xs PR size: XS

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants