Skip to content

Method summary #852

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
henrydingliu wants to merge 17 commits into
base: main
Choose a base branch
from
Open

Method summary #852

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
acedad0
Adding summary to methods
henrydingliu May 26, 2026
12072f3
bugbot fix
henrydingliu May 26, 2026
e90bbd4
bugbot fix
henrydingliu May 26, 2026
4c81050
Update sandbox_workbook_blank.ipynb
henrydingliu May 26, 2026
633a13c
Update plot_mack.ipynb
henrydingliu May 26, 2026
a63d9f5
Update stochastic-tutorial.ipynb
henrydingliu May 26, 2026
3c93489
Update sandbox_workbook_filled.ipynb
henrydingliu May 26, 2026
19b6d31
Update mack.py
henrydingliu May 26, 2026
6863d05
Update expectedloss.py
henrydingliu May 26, 2026
3d3ee60
Update capecod.py
henrydingliu May 26, 2026
01be8fb
Update bornferg.py
henrydingliu May 26, 2026
8aa4db4
Update chainladder.py
henrydingliu May 26, 2026
5333439
Update benktander.py
henrydingliu May 26, 2026
ca5f61e
Update base.py
henrydingliu May 26, 2026
d13b0fa
Update test_mack.py
henrydingliu May 26, 2026
5d3c97d
Update test_predict.py
henrydingliu May 26, 2026
1c3db5c
Update test_predict.py
henrydingliu May 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions chainladder/methods/base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,38 @@ def latest_diagonal(self):
else:
return self.X_.sum('development')

@property
def summary_(self):
return self._get_summary(self.X_,self.ultimate_)

def _get_summary(self,X,ult):
#columns for melt
ids = X.key_labels + ['origin','development','valuation']
#create dataframe for amount
amount = X.incr_to_cum().latest_diagonal.to_frame(implicit_axis=True,keepdims=True).reset_index().melt(id_vars=ids,var_name = 'column',value_name='latest')
amount_index = X.key_labels + ['origin','column']
amount = amount[amount_index + ['development','latest']]
amount.set_index(amount_index,inplace=True)
#create dataframe for ultimate
ultimate = ult.to_frame(implicit_axis=True,keepdims=True).reset_index().melt(id_vars=ids,var_name = 'column',value_name='ultimate')
#columns for melt for dev factors
dev_ids = X.ldf_.key_labels + ['origin','development','valuation']
#create dataframe for ldf
ldf = X.ldf_.to_frame(implicit_axis=True,keepdims=True).reset_index().melt(id_vars=dev_ids,var_name = 'column',value_name='ldf')
dev_factor_index = X.ldf_.key_labels + ['development','column']
ldf = ldf[dev_factor_index + ['ldf']]
ldf.set_index(dev_factor_index,inplace=True)
Comment on lines +85 to +88
Copy link
Copy Markdown
Collaborator

@kennethshsu kennethshsu May 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the LDFs useful? I think the CDFs are fine.

Copy link
Copy Markdown
Collaborator Author

@henrydingliu henrydingliu May 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doesn't have to be useful. we are trying to save the user extra work. manipulating ldf separately is not trivial.

Copy link
Copy Markdown
Collaborator

@kennethshsu kennethshsu May 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When would the user ever need LDFs in the summary?

Copy link
Copy Markdown
Collaborator Author

@henrydingliu henrydingliu May 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

easy example, if they want to calculate the expected emergence over the next calendar period

#create dataframe for cdf
cdf = X.cdf_.to_frame(implicit_axis=True,keepdims=True).reset_index().melt(id_vars=dev_ids,var_name = 'column',value_name='cdf')
cdf = cdf[dev_factor_index + ['cdf']]
cdf.set_index(dev_factor_index,inplace=True)
#assemble full summary. start from ultimate, as some methods (e.g. BF) return an ultimate without any actual amount
output = ultimate.drop(columns=['development','valuation'])
Copy link
Copy Markdown
Collaborator

@kennethshsu kennethshsu May 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would also add the ibnr_ in there.

Copy link
Copy Markdown
Collaborator Author

@henrydingliu henrydingliu May 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this actually came up a couple of weeks earlier. bugbot saw my friedland code and made the comment that ult - paid isn't IBNR. i think that's a valid point. ibnr_ isn't guaranteed to be IBNR.

Copy link
Copy Markdown
Collaborator

@kennethshsu kennethshsu May 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh ya that's another thing that bugs me, a better name would be unmerged or something like that. But this will be deprecate and move to a new name.

Copy link
Copy Markdown
Collaborator Author

@henrydingliu henrydingliu May 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, unlike ldf, "ibnr" can just be df["ultimate"] - df["latest"], whenever the user wants.

output = output.join(amount,on=amount_index,how='left')
output = output.join(ldf,on=dev_factor_index,how='left')
output = output.join(cdf,on=dev_factor_index,how='left')
return output[X.key_labels + ['column','origin','development','latest','ldf','cdf','ultimate']]

def fit(self, X, y=None, sample_weight=None):
"""Applies the chainladder technique to triangle **X**

Expand Down
6 changes: 6 additions & 0 deletions chainladder/methods/benktander.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,9 @@ class Benktander(MethodBase):
The ultimate losses per the method
ibnr_: Triangle
The IBNR per the method
summary_: Pandas Dataframe
a summary exhibit that contains latest, ldf, cdf, and ultimate


Examples
--------
Expand Down Expand Up @@ -243,9 +246,12 @@ def predict(self, X, sample_weight=None):
2012 15914.716737
2013 17193.715555
"""
if sample_weight is None:
raise ValueError("sample_weight is required.")
X_new = super().predict(X, sample_weight)
X_new.expectation_ = self._get_benktander_aprioris(X, sample_weight)
X_new.ultimate_ = self._get_ultimate(X_new, X_new.expectation_)
X_new.summary_ = self._get_summary(X_new,X_new.ultimate_)
X_new.n_iters = self.n_iters
X_new.apriori = self.apriori
return X_new
Expand Down
2 changes: 2 additions & 0 deletions chainladder/methods/bornferg.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ class BornhuetterFerguson(Benktander):
The ultimate losses per the method
ibnr_: Triangle
The IBNR per the method
summary_: Pandas Dataframe
a summary exhibit that contains latest, ldf, cdf, and ultimate

Examples
--------
Expand Down
2 changes: 2 additions & 0 deletions chainladder/methods/capecod.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ class CapeCod(Benktander):
The trended apriori vector developed by the Cape Cod Method
detrended_apriori_:
The detrended apriori vector developed by the Cape Cod Method
summary_: Pandas Dataframe
a summary exhibit that contains latest, ldf, cdf, and ultimate
Examples
--------
Expand Down
3 changes: 3 additions & 0 deletions chainladder/methods/chainladder.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ class Chainladder(MethodBase):
full_triangle_:
The ultimates back-filled to each development period in **X** retaining
the known data
summary_: Pandas Dataframe
a summary exhibit that contains latest, ldf, cdf, and ultimate
Examples
--------
Expand Down Expand Up @@ -188,6 +190,7 @@ def predict(self, X, sample_weight=None):
"""
X_new = super().predict(X, sample_weight)
X_new.ultimate_ = self._get_ultimate(X_new, sample_weight)
X_new.summary_ = self._get_summary(X_new,X_new.ultimate_)
return X_new

def _get_ultimate(self, X, sample_weight=None):
Expand Down
2 changes: 2 additions & 0 deletions chainladder/methods/expectedloss.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@ class ExpectedLoss(Benktander):
The ultimate losses per the method
ibnr_: Triangle
The IBNR per the method
summary_: Pandas Dataframe
a summary exhibit that contains latest, ldf, cdf, and ultimate

Examples
--------
Expand Down
14 changes: 8 additions & 6 deletions chainladder/methods/mack.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@ class MackChainladder(Chainladder):
full_triangle_:
The ultimates back-filled to each development period in **X** retaining
the known data
summary_:
summary of the model
mack_summary_:
summary of the mack model
full_std_err_:
The full standard error
total_process_risk_:
Expand All @@ -39,7 +39,9 @@ class MackChainladder(Chainladder):
The total prediction error by origin period
total_mack_std_err_:
The total prediction error across all origin periods

summary_: Pandas Dataframe
a summary exhibit that contains latest, ldf, cdf, and ultimate

Examples
--------
Fit the Mack chainladder method and inspect the headline summary table,
Expand All @@ -54,7 +56,7 @@ class MackChainladder(Chainladder):

tr = cl.load_sample('ukmotor')
model = cl.MackChainladder().fit(tr)
print(model.summary_)
print(model.mack_summary_)

.. testoutput:

Expand Down Expand Up @@ -401,7 +403,7 @@ def _get_total_mack_std_err_(self, obj):
return pd.DataFrame(out, index=obj.index, columns=obj.columns)

@property
def summary_(self):
def mack_summary_(self):
"""
Headline Mack summary table by origin: latest diagonal, IBNR,
ultimate, and Mack standard error.
Expand All @@ -423,7 +425,7 @@ def summary_(self):

tr = cl.load_sample('ukmotor')
model = cl.MackChainladder().fit(tr)
print(model.summary_)
print(model.mack_summary_)

.. testoutput::

Expand Down
6 changes: 3 additions & 3 deletions chainladder/methods/tests/test_mack.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@ def test_mack_to_triangle():
cl.Development().fit_transform(cl.load_sample("ABC"))
)
)
.summary_
.mack_summary_
== cl.MackChainladder()
.fit(cl.Development().fit_transform(cl.load_sample("ABC")))
.summary_
.mack_summary_
)


Expand All @@ -20,4 +20,4 @@ def test_mack_malformed():
b = a.iloc[:, :, :-1]
x = cl.MackChainladder().fit(a)
y = cl.MackChainladder().fit(b)
assert x.process_risk_.iloc[:,:,:-1] == y.process_risk_
assert x.process_risk_.iloc[:,:,:-1] == y.process_risk_
55 changes: 29 additions & 26 deletions chainladder/methods/tests/test_predict.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import chainladder as cl
import pandas as pd
import numpy as np
import pytest

raa = cl.load_sample("RAA")
Expand All @@ -8,37 +9,39 @@
apriori = cl_ult * 0 + (float(cl_ult.sum()) / 10) # Mean Chainladder Ultimate
apriori_1989 = apriori[apriori.origin < "1990"]

@pytest.mark.parametrize(
"estimators",
[
cl.CapeCod,
cl.BornhuetterFerguson,
cl.ExpectedLoss,
cl.Benktander,
cl.Chainladder
],
)
def test_predict_and_weights(estimators,atol):
est = estimators().fit(raa_1989, sample_weight=apriori_1989)
pred = est.predict(raa, sample_weight=apriori)
assert pred
assert np.all(abs(est.summary_['latest'].values - raa_1989.latest_diagonal.values.reshape(-1)) < atol)
assert np.all(abs(pred.summary_['latest'].values - raa.latest_diagonal.values.reshape(-1)) < atol)
assert np.all(abs(est.summary_['ultimate'].values - est.ultimate_.values.reshape(-1)) < atol)
assert np.all(abs(pred.summary_['ultimate'].values - pred.ultimate_.values.reshape(-1)) < atol)
#Test validation of sample_weight requirement. Should raise a value error if no weight is supplied.
if estimators != cl.Chainladder:
with pytest.raises(ValueError):
estimators().fit(raa_1989)
with pytest.raises(ValueError):
estimators().fit(raa_1989, sample_weight=apriori_1989).predict(raa)
else:
assert np.all(abs(est.summary_['ultimate'].values / est.summary_['latest'].values - est.summary_['cdf'].values) < atol)
assert np.all(abs(pred.summary_['ultimate'].values / pred.summary_['latest'].values - pred.summary_['cdf'].values) < atol)

def test_cc_predict():
cc = cl.CapeCod().fit(raa_1989, sample_weight=apriori_1989)
assert cc.predict(raa, sample_weight=apriori)

def test_bf_predict():
bf = cl.BornhuetterFerguson().fit(raa_1989, sample_weight=apriori_1989)
assert bf.predict(raa, sample_weight=apriori)

def test_el_predict():
bf = cl.ExpectedLoss().fit(raa_1989, sample_weight=apriori_1989)
assert bf.predict(raa, sample_weight=apriori)

def test_mack_predict():
mack = cl.MackChainladder().fit(raa_1989)
assert mack.predict(raa_1989)

def test_capecod_fit_weight():
"""
Test validation of sample_weight requirement. Should raise a value error if no weight is supplied.
"""
with pytest.raises(ValueError):
cl.CapeCod().fit(raa_1989)

def test_capecod_predict_weight():
"""
Test validation of sample_weight requirement. Should raise a value error if no weight is supplied.
"""
with pytest.raises(ValueError):
cc = cl.CapeCod().fit(raa_1989, sample_weight=apriori_1989)
cc.predict(raa)

def test_bs_random_state_predict(clrd):
tri = clrd.groupby("LOB").sum().loc["wkcomp", ["CumPaidLoss", "EarnedPremNet"]]
Expand Down Expand Up @@ -186,4 +189,4 @@ def test_odd_shaped_triangle():
)
ult1 = cl.Chainladder().fit(cl.Development(average="volume").fit_transform(tr.grain("OYDQ"))).ultimate_.sum()
ult2 = cl.Chainladder().fit(cl.Development(average="volume").fit_transform(tr)).ultimate_.grain("OYDQ").sum()
assert abs(ult1 - ult2) < 1e-5
assert abs(ult1 - ult2) < 1e-5
2 changes: 1 addition & 1 deletion docs/gallery/plot_mack.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@
"dev = cl.Development(average='volume')\n",
"mack.fit(dev.fit_transform(data))\n",
"\n",
"plot_data = mack.summary_.to_frame(origin_as_datetime=False)\n",
"plot_data = mack.mack_summary_.to_frame(origin_as_datetime=False)\n",
"plot_data"
]
},
Expand Down
16 changes: 8 additions & 8 deletions docs/getting_started/online_sandbox/sandbox_workbook_blank.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -697,7 +697,7 @@
"id": "bdb08c81-5921-4c41-ad63-96168ffd48b7",
"metadata": {},
"source": [
"MackChainladder also has a `summary_` attribute."
"MackChainladder also has a `mack_summary_` attribute."
]
},
{
Expand All @@ -707,7 +707,7 @@
"metadata": {},
"outputs": [],
"source": [
"__fill_in_code__.summary_"
"__fill_in_code__.mack_summary_"
]
},
{
Expand All @@ -726,15 +726,15 @@
"outputs": [],
"source": [
"plt.bar(\n",
" mcl_mod.summary_.to_frame().index.year,\n",
" mcl_mod.summary_.to_frame()[__fill_in_code__],\n",
" mcl_mod.mack_summary_.to_frame().index.year,\n",
" mcl_mod.mack_summary_.to_frame()[__fill_in_code__],\n",
" label=\"Reported\",\n",
")\n",
"plt.bar(\n",
" mcl_mod.summary_.to_frame().index.year,\n",
" mcl_mod.summary_.to_frame()[__fill_in_code__],\n",
" bottom=mcl_mod.summary_.to_frame()[__fill_in_code__],\n",
" yerr=mcl_mod.summary_.to_frame()[__fill_in_code__],\n",
" mcl_mod.mack_summary_.to_frame().index.year,\n",
" mcl_mod.mack_summary_.to_frame()[__fill_in_code__],\n",
" bottom=mcl_mod.mack_summary_.to_frame()[__fill_in_code__],\n",
" yerr=mcl_mod.mack_summary_.to_frame()[__fill_in_code__],\n",
" label=\"IBNR\",\n",
")\n",
"plt.legend(loc=\"upper left\")"
Expand Down
4 changes: 2 additions & 2 deletions docs/getting_started/online_sandbox/sandbox_workbook_filled.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4666,7 +4666,7 @@
"id": "bdb08c81-5921-4c41-ad63-96168ffd48b7",
"metadata": {},
"source": [
"MackChainladder also has a `summary_` attribute."
"MackChainladder also has a `mack_summary_` attribute."
]
},
{
Expand Down Expand Up @@ -4790,7 +4790,7 @@
}
],
"source": [
"mcl_mod.summary_"
"mcl_mod.mack_summary_"
]
},
{
Expand Down
16 changes: 8 additions & 8 deletions docs/getting_started/tutorials/stochastic-tutorial.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2008,7 +2008,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"With the `summary_` method, we can more easily look at the result of the `MackChainladder` model."
"With the `mack_summary_` method, we can more easily look at the result of the `MackChainladder` model."
]
},
{
Expand All @@ -2020,7 +2020,7 @@
}
},
"source": [
"mack.summary_"
"mack.mack_summary_"
],
"outputs": [
{
Expand Down Expand Up @@ -2148,15 +2148,15 @@
},
"source": [
"plt.bar(\n",
" mack.summary_.to_frame(origin_as_datetime=True).index.year,\n",
" mack.summary_.to_frame(origin_as_datetime=True)[\"Latest\"],\n",
" mack.mack_summary_.to_frame(origin_as_datetime=True).index.year,\n",
" mack.mack_summary_.to_frame(origin_as_datetime=True)[\"Latest\"],\n",
" label=\"Paid\",\n",
")\n",
"plt.bar(\n",
" mack.summary_.to_frame(origin_as_datetime=True).index.year,\n",
" mack.summary_.to_frame(origin_as_datetime=True)[\"IBNR\"],\n",
" bottom=mack.summary_.to_frame(origin_as_datetime=True)[\"Latest\"],\n",
" yerr=mack.summary_.to_frame(origin_as_datetime=True)[\"Mack Std Err\"],\n",
" mack.mack_summary_.to_frame(origin_as_datetime=True).index.year,\n",
" mack.mack_summary_.to_frame(origin_as_datetime=True)[\"IBNR\"],\n",
" bottom=mack.mack_summary_.to_frame(origin_as_datetime=True)[\"Latest\"],\n",
" yerr=mack.mack_summary_.to_frame(origin_as_datetime=True)[\"Mack Std Err\"],\n",
" label=\"Reserves\",\n",
")\n",
"plt.legend(loc=\"upper left\")\n",
Expand Down
Loading