Repository navigation
Conversation
rp_debugger called cpuinfo.get_cpu_info() at import. On x86, py-cpuinfo measures clock speed in a child process around a fixed sleep(1), and the worker imports rp_debugger on every start, so every cold start paid about 1.17s for a value used only with --rp_debugger. - Probe lazily in a cached _get_processor(), called from get_debugger_output() - Import cpuinfo inside the function; its import also spawns a subprocess - Keep rp_debugger.PROCESSOR working via module __getattr__ Real GPU worker (RTX 4090): handler loaded -> first worker-start fitness check went from 1459/1469ms to 536ms. Linear: SLS-814
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
Comment on lines
+50
to
+51
| "import sys; import runpod.serverless.utils.rp_debugger; " | ||
| "print('cpuinfo' in sys.modules)", |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Comment on lines
+50
to
+51
| "import sys; import runpod.serverless.utils.rp_debugger; " | ||
| "print('cpuinfo' in sys.modules)", |
There was a problem hiding this comment.
🟢 Approval recommended
The focused changes preserve debugger behavior, include relevant regression coverage, and have no unresolved findings.
No open issues 🎉
What changed in this PR
Reduces serverless worker cold-start overhead by deferring CPU detection until debugger information is requested.
Changes:
- Lazily imports
cpuinfoand caches the processor lookup. - Preserves access to
rp_debugger.PROCESSORthrough a lazy module attribute. - Adds regression tests for deferred loading, caching, processor reporting, and fallback behavior.
| File | Description |
|---|---|
| tests/test_serverless/test_utils/test_debugger.py | Tests lazy CPU detection and compatibility behavior. |
| runpod/serverless/utils/rp_debugger.py | Defers CPU detection and caches its result. |
Review effort: Balanced
Member
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
rp_debugger.pyrancpuinfo.get_cpu_info()at import. The worker importsrp_debuggeron every start (worker→rp_local→rp_job→rp_debugger). On x86, py-cpuinfo measures clock speed in a child process around a fixedsleep(1)(CPUID.get_raw_hz), so every cold start paid about 1.2s for a value only used with--rp_debugger._get_processor(), called fromget_debugger_output()cpuinfois imported inside that function; importing it also spawns a subprocess (platform.architecture()runsfile)rp_debugger.PROCESSORstill works for existing callers, resolved lazily through a module__getattr__With
--rp_debuggerenabled, the probe moves to the first job and runs once per process.Impact
Measured on a real RTX 4090 serverless worker (AMD EPYC 7B13, 128 CPUs):
cpuinfo.get_cpu_info()"cpuinfo" in sys.moduleswasFalseinside the handler on this branch. On macOS arm64 the probe takes about 40ms because py-cpuinfo skips the x86 CPUID path, so local testing does not show the problem.Test plan
get_cpu_info(fails on main)cpuinfo, checked in a fresh interpreter (fails on main)PROCESSORattributemake quality-check: 1484 passed, 91.67% coverageLinear: SLS-814