Repository navigation
Release 0.1.1 - #3
Merged
Merged
Conversation
LFC excluded every gene whose two arm means differed by less than thres_diff before the BH correction. That filter selects on the effect estimate: the pairs it removes are the ones with p-values near 1, so BH over the remaining pairs was anti-conservative. On a Perturb-seq negative control (fake perturbations drawn from controls) it removed about 10% of the expressed pairs; keeping them flattened the null p-value histogram and lowered the null SD of the statistics from 1.09 to 1.04. thres_diff is now only the floor before the logarithm. The expression threshold (thres_min / min_counts) is unchanged: it looks at expression level, not at the effect. Docstrings and the LFC guide drop version-history prose and references to an analysis that is not in the repository; the versionchanged directives keep the history. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
3_run_batch.py no longer asserts version 0.1.0 and writes the nuisance file that the propensity refit reads. The README lists the refit script and which results file the notebook reads. A local analysis folder that .gitignore already excluded was still tracked; it is removed from the repository. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
comp_size_factor's median of ratios ran over every gene. In sparse single-cell data most nonzero counts are 1 or 2, so those ratios caught about a third of each cell's sequencing depth (SD of log size factors 0.12 against 0.35 for log total counts on Replogle controls). The leftover depth sits in highly expressed genes, which scale fully with depth, and when a group of cells differs from the controls in average depth by chance, all of them shift together. On a Replogle negative control (30 fake perturbations cut from the controls, full pipeline) each fake arm's shift tracked its leftover depth (correlation 0.98), the null SD for genes above 5 counts per cell was 1.32 and 114 pairs were called; Wilcoxon on the same genes called 2. Swapping in total-count offsets removed the shift, while cross-fitting and a Poisson outcome model changed nothing, which rules out the in-sample fit and dispersion. The median of ratios now uses genes with mean count >= min_mean (2), or the 100 highest-expressed genes if fewer pass. That keeps its resistance to composition changes, which total counts lack and which the GCATE deconfounding tests depend on, while tracking depth: 14 false discoveries on the same null, SD 1.08 for highly expressed genes, and no change on the mouse-brain negative control. min_mean=0 restores the old factors; method='libsize' uses total counts. Cells without counts now get a size factor of zero instead of NaN. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Guides are assigned at random, so the propensity model only has to absorb chance imbalance. Latent factors estimated from expression can instead track a perturbation's own effect; the model then partly separates that arm from the controls and the weights pile onto the treated cells that look least affected. select_propensity_factors, next to tune_penalty_factor and with the same interface, starts from every covariate. For each treatment whose treated ESS falls below 0.5, overlap below 0.3 or AUC above 0.9, it drops the covariate most imbalanced between that treatment and the controls, one at a time, until ESS and overlap recover; the intercept always stays. It returns a drop_by_treatment mapping for refit_propensity_scores and a per-arm report. The Replogle and Perturb-seq tutorials leave log library size out of the propensity model and choose the latent factors per arm with it: knocking down transcription machinery shrinks total RNA, so library size separated those arms from the controls almost completely. 4_refit_propensity.py applies the rule to every Replogle batch (29 of 200 arms adjusted, one or two factors dropped each); 4_cache_propensity_batch.py, which only served the penalty sweep, is removed and the refit script takes its number. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fit_gcate took its offset from kwargs_glm['size_factor'], which was only set for offset=True. A supplied offset array was therefore replaced by ones everywhere except the dispersion estimate. The array is now used, and it is checked for one finite value per row. offset=False still means no offset. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
usevar='unequal' is the by-arm Welch variance again, with Welch-Satterthwaite df. 'pooled' stays the default. In small case-control designs Welch is the more conservative choice: on 85 SEA-AD donors, permuted labels gave 0-1 false discoveries with it, against up to 3,654 under 'pooled'. The gcate docs note that with few units the factors can stay nearly orthogonal to treatment, while under 'pooled' they still shrink the standard errors. The SEA-AD tutorial README no longer lists results/sea_ad_lfc.csv as a notebook input. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Only results/replogle-r.csv, the JIC table on raw counts, is kept. The older table computed on log-normalised values is removed, with its mention in 2_estimate_r.py and the README; .gitignore no longer exempts it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 50 GB nuisance file only feeds the propensity walk-through, which the notebook skips when the file is absent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The R tutorial was last rendered with 0.0.10. It still kept library size in the propensity model and tuned a penalty on it. It now follows the Python notebook: library size is left out, factors are selected per arm with select_propensity_factors, then LFC runs with the selected scores. It is re-knit with 0.1.1 and gives 8,161 hits, as the Python notebook does. - The R discovery count no longer includes rows whose padj is NA. - The unreferenced perturbseq_files/ figure is removed. - LFC shows its per-treatment progress bar only with verbose=True, so the tutorial outputs no longer carry hundreds of progress lines. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Library size can confound (capture depth, cell quality) or reflect a treatment effect (total RNA), and the data cannot tell which. The LFC docs now say this in one place, with a rule of thumb: keep library size when every arm stays supported with it, and leave it out when arms separate. tune_penalty_factor is the middle ground. The Perturb-seq, Replogle and SEA-AD tutorials use the same wording and point to the docs, instead of calling library size a pure treatment effect. Only markdown changed; the results are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With latent factors in the outcome model, crispyx 0.1.5's structured GLM fitter was about 2.5 times slower than 0.1.6's, for the same estimates: 159 s against 62 s for LFC on 5,000 cells, 3,000 genes and 10 treatments (mouse-brain subsample). The full test suite passes with 0.1.6. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An Intel (x86_64) Python on an Apple Silicon Mac runs translated by Rosetta 2, which makes causarray's numerical code several times slower. On a 5,000-cell subsample LFC took 159 s translated against 32-57 s natively, and the test suite 270 s against 64 s. pip installs into whatever Python it is given, so packaging cannot prevent this. Instead, `import causarray` now issues one RuntimeWarning that explains how to create a native environment, and the README says how to check for one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The causarray and causarray-r environments must share an architecture, since reticulate cannot load an arm64 Python into an Intel R. The README now says to create both natively on Apple Silicon. environment-r.yaml lists dplyr and ggplot2, which the R tutorial loads directly. The R tutorial is re-knit with native R 4.4 and Python. Results are unchanged (8,161 hits; U9 and U7 dropped for Satb2). Seurat now loads quietly, and the newer pandoc changes heading style only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
L45Glut-py.ipynb is a causarray 0.1.1 tutorial on one cell type of a mouse-brain CRISPR screen (L4-5 IT Glut, 58 perturbations). It covers the per-cell-type pipeline, the propensity and calibration checks, the biological readouts and a comparison with Wilcoxon, and ends with the loop for all cell types. It is rendered in the docs as an orphan page, not listed in the index. pipeline.py, which the notebook imports, and the folder README are committed with it. The data, the results, the archived investigation notebook and its scripts stay local. .gitignore now ignores prepared SEA-AD data other than the committed excitatory-neuron pseudo-bulk, such as the subclass pseudo-bulks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- comp_size_factor: a cell with no counts in the well-expressed genes fell
back to ratios against all-gene geometric means, which sit near one for
sparse genes, so an almost-empty cell got a size factor near the median
(0.91 for a cell with 0.6% of the median library). It now gets its total
count, rescaled by the other cells' median ratio of size factor to total.
method='scale' returns one factor per row instead of one per column. The
unreachable old implementation and two unused helpers are removed.
- select_propensity_factors: a triggered treatment that already meets the
target keeps every covariate; on_infeasible ('best', 'all', 'none')
decides what an arm that never reaches the target keeps, instead of
always dropping every candidate; with mask, covariates are ranked on the
eligible cells that the support metrics score. The tutorial selections
are unchanged: every adjusted arm failed the target before and met it
after.
- _platform: look up sysctlbyname in the running process instead of a
hard-coded library path.
- Docstrings drop numbers tied to particular analyses (SEA-AD permutations,
Replogle dispersion, Perturb-seq timings, the Rosetta benchmark).
- The Welch test states the 2x ratio for equal arms and widens its bound.
- lfc.rst points up to the library-size section; the mouse-brain README
no longer lists local-only files; the L45Glut notebook's text now
matches its output for Ppp2r1a and Ap2s1 and no longer says the null
check shows the extra hits are real.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The three notebooks are re-executed with the 0.1.1 changes: no mean- difference filter before BH, size factors from well-expressed genes, and propensity factors chosen per arm. - SEA-AD uses usevar='unequal' and says it is the more conservative choice in a sample this small (2,683 genes). The GCATE cache is refitted. - Perturb-seq leaves library size out of the propensity model and selects factors per arm; only Satb2 is adjusted (U9 and U7 dropped, treated ESS 0.50 -> 0.68), and its section shows what the selection changed in one paragraph instead of hand-picked filtering variants. 8,161 hits. - Replogle reads the per-arm refit (145,340 discoveries; SUPT5H 1,591 hits, its down-regulated genes enriched for translation and ribosome biogenesis). Section 5 walks through the batch with SUPT5H and SUPT6H and skips the walk-through when the local nuisance file is absent. The fallback to the legacy rank table is removed, and a heading that skipped from level 2 to level 4 is fixed. - All three describe library size in the propensity model as a judgement call and point to the LFC docs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
If your organization's extra usage balance is empty, an organization admin can add extra usage credits at claude.ai/admin-settings/usage. If its monthly spend limit was reached, an admin can raise it on the same page. If neither applies, contact Anthropic support.
Once extra usage is available, comment @claude review on this pull request, or push a new commit, to trigger a review.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.