[mikkelke]

Fixes #2599.

The problem

PythonDeps.refresh_dep_graph reads and ast.parses every Python file under the apps directory. Dependencies.update calls it unconditionally, and check_app_updates calls update on every utility-loop iteration — so every pass re-parses the entire app tree even when nothing was touched.

Measured on a live install (79 app files, 2.76 MB, AppDaemon 4.5.13, Python 3.12), replaying the exact code path in-container:

py files considered : 79        total source bytes : 2,757,075
walk (iterdir+stat) :    17.5 ms
FileCheck stat pass :     0.9 ms
get_dependency_graph:  2235.3 ms  (failed=0)
  - pure read       :    13.5 ms
  - pure ast.parse  :  2191.8 ms

98% of the cost is ast.parse. Reading the files is 13 ms — this is CPU, not I/O.

It compounds: once the pass exceeds max_utility_skew, utility_loop skips its own sleep, so the loop goes hot. Measured from inside the container (482 sequential requests over 15 s):

slow (>200ms): 6, sum 12.91 s   max 2.240 s   median 0.0017 s
blocked fraction: 0.86

86% of the event loop blocked in ~2.2 s chunks every ~2.4 s, container pinned at 102% CPU. Because app-side sync API calls go through run_coroutine_threadsafe onto that same loop, app callbacks arrive up to ~2.4 s late — visible in app logs as stalls mid-callback.

The fix

FileCheck.update has already computed the new/modified/deleted sets by the time refresh_dep_graph is called, so it already knows whether anything needs re-parsing. This gates the refresh on there_were_changes via a needs_refresh hook, overridden only in PythonDeps.

AppDeps deliberately keeps refreshing unconditionally: its refresh_dep_graph walks the already-parsed app config rather than the filesystem, so it is cheap, and it is also called explicitly elsewhere when app configs change.

Files that previously failed to parse are still retried — a fixed file has a new mtime and therefore lands in FileCheck.modified. There is a test for exactly that.

Tests

Adds tests/unit/test_dependency_manager.py (7 tests): graph built on init; unchanged files do not rebuild; new/modified/deleted each do rebuild; a fixed bad file is retried; AppDeps still refreshes unconditionally. Three of them fail on dev and pass with the change. Full unit suite: 209 passed.

Note

This is a targeted fix, not the watchdog/inotify rework mentioned in #1897 — it just stops the redundant work. Raising utility_delay is a workaround but costs reload latency; this keeps detection at the normal interval.

[rmounce]

This comment was authored by Codex, an AI coding assistant.

I reviewed and tested PR head 13cc612. It contains only the new tests—the needs_refresh implementation described in the PR body appears to be missing. As checked out, the dependency-manager tests report 3 failed, 4 passed, including an AttributeError for the missing hook.

I added the described implementation locally: Dependencies.needs_refresh defaults to True, PythonDeps overrides it with files.there_were_changes, and Dependencies.update() gates the graph refresh on that property. With this change, 211 unit tests pass, including two additional checks for dependency-graph changes and unconditional AppDeps refresh.

With the account owner’s authorization, I also tested the guard in their live AppDaemon 4.5.13 / Python 3.12.12 instance:

Window	Process CPU	Main-thread CPU
Before, 30 s	14.47%	13.53%
Guard active, 30 s	4.90%	3.97%
Confirmed rollback, 30 s	14.03%	12.96%

CPU percentages are relative to one logical CPU. Workload varied, but rolling back reproduced the original load. A separate quiet 30-second sampling profile with the guard active showed no AST parsing samples. Live file addition, import modification, and deletion all triggered dependency updates, while polling continued approximately once per second.

Could you add the implementation to the branch? These results support the approach described here.