Repository navigation
[BUG] cl.concat(...,axis=2, ignore_index=True) does not follow origin grain #1489
Description
Are you on the latest chainladder version?
- Yes, this bug occurs on the latest version.
Describe the bug in words
When concatenating 2 triangles along the origin index, while setting ignore_index=True, the oldest origin period is set to 1970-01-01T00:00:00.000000000 (Unix epoch) and incremented by 1 nanosecond per period.
This ignores the triangle's original grain (Y, in the example below).
How can the bug be reproduced?
import chainladder as cl
raa = cl.load_sample("raa")
o1 = raa.iloc[:, :, :5, :]
o2 = raa.iloc[:, :, 5:, :]
res2 = cl.concat([o1, o2], axis=2, ignore_index=True)
print(res2)
12 24 36 48 60 72 84 96 108 120
1970 5012.0 8269.0 10907.0 11805.0 13539.0 16181.0 18009.0 18608.0 18662.0 18834.0
1970 106.0 4285.0 5396.0 10666.0 13782.0 15599.0 15496.0 16169.0 16704.0 NaN
1970 3410.0 8992.0 13873.0 16141.0 18735.0 22214.0 22863.0 23466.0 NaN NaN
1970 5655.0 11555.0 15766.0 21266.0 23425.0 26083.0 27067.0 NaN NaN NaN
1970 1092.0 9565.0 15836.0 22169.0 25955.0 26180.0 NaN NaN NaN NaN
1970 1513.0 6445.0 11702.0 12935.0 15852.0 NaN NaN NaN NaN NaN
1970 557.0 4020.0 10946.0 12314.0 NaN NaN NaN NaN NaN NaN
1970 1351.0 6947.0 13112.0 NaN NaN NaN NaN NaN NaN NaN
1970 3133.0 5395.0 NaN NaN NaN NaN NaN NaN NaN NaN
1970 2063.0 NaN NaN NaN NaN NaN NaN NaN NaN NaNprint(res2.origin_grain)
YWhat is the expected behavior?
Some decisions need to be made, but the goal should be to match the data with the grain.
- Is the Unix epoch OK as the default starting date?
- Should the origin grain be changed to match nanoseconds?
- Or should the default increment be faithful to the original grain?
If we decide to make the increment match the original grain:
12 24 36 48 60 72 84 96 108 120
1970 5012.0 8269.0 10907.0 11805.0 13539.0 16181.0 18009.0 18608.0 18662.0 18834.0
1971 106.0 4285.0 5396.0 10666.0 13782.0 15599.0 15496.0 16169.0 16704.0 NaN
1972 3410.0 8992.0 13873.0 16141.0 18735.0 22214.0 22863.0 23466.0 NaN NaN
1973 5655.0 11555.0 15766.0 21266.0 23425.0 26083.0 27067.0 NaN NaN NaN
1974 1092.0 9565.0 15836.0 22169.0 25955.0 26180.0 NaN NaN NaN NaN
1975 1513.0 6445.0 11702.0 12935.0 15852.0 NaN NaN NaN NaN NaN
1976 557.0 4020.0 10946.0 12314.0 NaN NaN NaN NaN NaN NaN
1977 1351.0 6947.0 13112.0 NaN NaN NaN NaN NaN NaN NaN
1978 3133.0 5395.0 NaN NaN NaN NaN NaN NaN NaN NaN
1979 2063.0 NaN NaN NaN NaN NaN NaN NaN NaN NaNWould you be willing to contribute this ticket?
- Yes, absolutely!
- Yes, but I would like some help.
- No.
Activity
henrydingliu commentedon Oct 16, 2026
can we first align that we should follow pandas convention wherever it makes sense but also don't blindly force pandas convention everywhere?
if we can be aligned on the above, then i don't think ignore_index=True should be allowed on axes 2 and 3. your example here is fairly benign, but a gigantic can of worms opens up if
o2 = [:, :, 4:, :]instead (i.e. res2 is now a non-triangular polygon that's likely to bug out every estimator we throw it into).genedan commentedon Oct 16, 2026
Agree. For this example, I think we'd have to take the 2nd route since origin is unique to chainladder. Pandas will discard the dtype and reset the axis to be integers
0, 1, ...I think unless we keep origin as a PeriodIndex, a bunch of actuarial calculations will break.Agree. The design decisions that we'd have to make won't be obvious until somebody proposes a use case. I think we can close this out by adding an exception. Looping in @casact/chainladder-maintainers if anyone has last words.
I wonder if there is a more serious bug that would happen if a user just overrides the origin/development indexes with whatever values they want. With grain being the same, I would expect things to break.
henrydingliu commentedon Oct 16, 2026
one use case i can think of: when the claims data cut over (e.g. migrating from one claims system to another) in the middle of an origin period. say we are doing OYDY analysis, and the migration happens on 7/1. and say the new claims system outputs data in a different schema. it is reasonable to want to keep the triangle generation pipeline of the old system separate from a triangle generation pipeline for the new system, and only combine the resulting triangles together. however, for this use case, we do still have
__add__that works perfectly.we should also update the AssertError that ignore_index=False emits. It's not helpful today
priyam0k commentedon Oct 19, 2026
Because
__add__already handles combining amounts across identical origin and development coordinates, havingcl.concatsynthesize arbitrary origin or development indices with ignore_index=True provides little practical value and breaks temporal grain invariants. We ran into a similar boundary in #1455 for Styler concatenation: actuarial axes require rigid temporal alignment, and treating them like generic pandas axes quickly creates invalid geometry.Disallowing ignore_index=True on axes 2 and 3 cleanly resolves both this and #1490. Replacing the bare assert statements across
concat(such as duplicate checks and non-concat axis alignment checks) with descriptive ValueErrors would also make failure modes much clearer. Open to other ideas.