Repository navigation
CHS: freeze states after each person's last observed measurement - #104
Open
hmgaudecker wants to merge 1 commit into
Open
hmgaudecker wants to merge 1 commit into
hmgaudecker wants to merge 1 commit into
Conversation
Persons with no later observed measurement (death, attrition) were predicted through every remaining period. In a translog their predicted covariance explodes (about 1e19 at ages 90+ in health-cognition), the QR of the missing-measurement update gets a zero R diagonal, and the floored QR derivative overflows; the where-mask then turns 0 * inf into a NaN gradient while the log-likelihood stays finite. - The predict step runs only for persons with an observed measurement in a later update. All others keep their last filtered states and covariance; their predict step runs on placeholder inputs (zero states, identity covariance) and is discarded. - kalman_update builds the QR for missing measurements from an identity covariance; the result is discarded as before. - debug_loglike reports filtered states as NaN for the periods after a person's last observation. The log-likelihood is unchanged: the frozen updates contribute exactly zero before and after. Tests pin loglikeobs and gradients from the unchanged code and check the gradient against finite differences where it overflowed before. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y3CuwvNUhYTSpriKYSwqmq
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## asymmetric-translog #104 +/- ##
=======================================================
+ Coverage 96.05% 96.07% +0.02%
=======================================================
Files 147 148 +1
Lines 14998 15089 +91
=======================================================
+ Hits 14406 14497 +91
Misses 592 592 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #102 (asymmetric-translog).
Problem
Persons with no later observed measurement (death, attrition) are predicted through every remaining period. In a translog, their predicted covariance explodes (about 1e19 at ages 90+ in health-cognition). The QR of the missing-measurement update then has a zero R diagonal, the floored QR derivative overflows, and the where-mask turns 0 · inf into a NaN gradient. The log-likelihood stays finite.
Change
predict_if_observed_later(inchs/likelihood.py, used by both scans): the predict step runs only for persons with an observed measurement in a later update. All others keep their last filtered states and covariance. Their predict step runs on placeholder inputs (zero states, identity covariance) and is discarded (double-where).kalman_update: the QR for missing measurements is built from an identity covariance. The result is discarded as before.debug_loglike: filtered states are NaN for the periods after a person's last observation.The log-likelihood does not change, because the frozen updates contribute exactly zero, before and after the change.
Tests (
tests/test_frozen_after_last_observation.py)loglikeobsand the gradients at three random parameter vectors are equal to the values recorded on the unchanged code (9863a78).Full suite: 870 passed.
https://claude.ai/code/session_01Y3CuwvNUhYTSpriKYSwqmq