Friedland Chapter 6 and half of Chapter 7

[henrydingliu]

Summary of Changes

first batch of deliverables for friedland reconstruction

Related GitHub Issue(s)

#588

Additional Context for Reviewers

please see #798 for discussion

• I passed tests locally for both code (uv run pytest) and documentation changes (uv run jb build docs --builder=custom --custom-builder=doctest)

---

Note

Low Risk
Documentation-only changes with executable doctests against existing friedland_* sample data; no library behavior changes beyond re-enabling Development API docs.

Overview
Adds a Friedland Recreation section to the Jupyter Book: landing page (friedland/index.rst), Chapter 6 (triangle diagnostics with doctests for premium/on-level ratios, claim and count triangles), and Chapter 7 (development/chain-ladder exhibits through Exhibit II; Exhibit III is marked WIP).

Chapter 7 documents Development, DevelopmentConstant, TailConstant, Chainladder, and helper flows for development/unpaid summaries aligned with Friedland tables. docs/_toc.yml wires chapters 6–7 only; chapter_8.ipynb is added as in-progress work (not in the TOC).

Minor doc fixes: chainladder.Development API page re-enables autoclass; CODEOWNERS newline; pyproject.toml docs extra list formatting.

^{Reviewed by Cursor Bugbot for commit ace9aac. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Unfinished notebook committed with rendered widget outputs

Low Severity

The chapter_8.ipynb file is committed but not included in _toc.yml, and the PR scope is "Chapter 6 and half of Chapter 7." The notebook contains incomplete work (cells with null execution counts alongside cells with counts of 3 and 20), and cell 4 includes hundreds of lines of rendered sklearn widget HTML/CSS/JavaScript output. This adds significant repository bloat from auto-generated content that doesn't belong in version control.

 

^{Reviewed by Cursor Bugbot for commit 006f47f. Configure here.}

[genedan]

I see that one downside of the *.rst migration is that the tables are still in code-block format rather than the HTML output you get with Jupyter notebooks. So there's an inconsistency in the formatting compared to the rest of the docs.

There seem to be ways around this according to claude, which lets you print the DataFrame without having to copy the HTML into the source code, although I'm not sure how close the tables will look compared to Jupyter until we try it:

  ---
  2. Custom Sphinx directive
  
  Write a small extension that executes Python and inserts the .to_html() output as a raw HTML node:

  # docs/_ext/df_table.py
  from docutils import nodes
  from docutils.parsers.rst import Directive
  import importlib

  class DataFrameTable(Directive):
      has_content = True

      def run(self):
          code = "\n".join(self.content)
          local = {}
          exec(code, {}, local)
          df = local["df"]
          html = df.to_html(classes="dataframe", border=0)
          return [nodes.raw("", html, format="html")]

  def setup(app):
      app.add_directive("df-table", DataFrameTable)

  Then in RST:
  .. df-table::

     import chainladder as cl
     df = cl.load_sample("raa").to_frame()

  ---
  3. sphinx-exec-directive (third-party)

  A pre-built version of option 2 — adds an .. exec-code:: directive that captures rich output including HTML.

  ---

I'd rather go with option 2, it's not much code and it will give us greater control and better dependency management than option 3. Option 1 (not shown) was sticking with Jupyter, which is something I can support if I'm outvoted on this.

It's not going to influence the approval, just an observation and a follow-up issue on how we'll reconcile the formatting across the docs.

[henrydingliu]

what about keeping the rst intact but nbconvert them all into notebooks (then html) during doc build?

[genedan]

It's worth a shot, I'd say whatever the path of least resistance is, but if formatting turns out weird, we can try lower-level techniques like the custom extension to work it out.

[kennethshsu]

Do you guys want to merge this into main? I know it's not quite complete, and I'm sorry I sneaked in chapter 8, which is nowhere near complete, but we can bring merge this and just continue to make improvements as things move along.

[henrydingliu]

i'm working on the rst to ipynb conversion build commands to address gene's concern over visual conformity

[henrydingliu]

I see that one downside of the *.rst migration is that the tables are still in code-block format rather than the HTML output you get with Jupyter notebooks. So there's an inconsistency in the formatting compared to the rest of the docs.

There seem to be ways around this according to claude, which lets you print the DataFrame without having to copy the HTML into the source code, although I'm not sure how close the tables will look compared to Jupyter until we try it:

  ---
  2. Custom Sphinx directive
  
  Write a small extension that executes Python and inserts the .to_html() output as a raw HTML node:

  # docs/_ext/df_table.py
  from docutils import nodes
  from docutils.parsers.rst import Directive
  import importlib

  class DataFrameTable(Directive):
      has_content = True

      def run(self):
          code = "\n".join(self.content)
          local = {}
          exec(code, {}, local)
          df = local["df"]
          html = df.to_html(classes="dataframe", border=0)
          return [nodes.raw("", html, format="html")]

  def setup(app):
      app.add_directive("df-table", DataFrameTable)

  Then in RST:
  .. df-table::

     import chainladder as cl
     df = cl.load_sample("raa").to_frame()

  ---
  3. sphinx-exec-directive (third-party)

  A pre-built version of option 2 — adds an .. exec-code:: directive that captures rich output including HTML.

  ---

I'd rather go with option 2, it's not much code and it will give us greater control and better dependency management than option 3. Option 1 (not shown) was sticking with Jupyter, which is something I can support if I'm outvoted on this.

It's not going to influence the approval, just an observation and a follow-up issue on how we'll reconcile the formatting across the docs.

@genedan I got a couple things cooking, but nothing is working great at the moment. i do have a more macro question for you. have you actually taken a look at any of the docstring examples born out of #704 in the doc? for example, bottom of here. these are not in html. these are all in basic text formatting. so never mind the display discrepancy on friedland. do you have a solution to this display discrepancy with API rst?

[genedan]

I've been so accustomed to seeing API reference docs in terminal format from other projects (like pandas) that I hadn't thought about it.

We should try to answer the question, how do we expect the users to follow along? If they're using Jupyter, then we'd want the output to be in HTML, but if they're using a terminal or getting their output in stdout, then the monospace shell formatting would make more sense.

But, what would be the merit of keeping the output format consistent throughout the entire docs? IIRC, the seminars conducted had people use Jupyter, and I think it would make sense to have the quickstart/onboarding tutorials be in Jupyter. But in-depth docs and API reference may be geared towards people writing programs, so maybe shell output would make more sense for those sections. In any case, most books I read spend at least a paragraph or two explaining the output formatting (such as remind users not to type in prompts if they are included in examples). As long as we're explicit about that kind of thing, I think we're okay.

From our discussions on the Friedland reconciliation I take it that getting the figures to match would be of higher priority than the output format. So I think if you think getting the formatting consistent across the docs is out of scope or a hinderance in getting Chapters 7+ done I'm ok with not going with HTML or saving it for another issue.

@kennethshsu what are your thoughts on the formatting?

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.99%. Comparing base (e76d85d) to head (ace9aac).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #837      +/-   ##
==========================================
+ Coverage   86.94%   86.99%   +0.05%     
==========================================
  Files          86       86              
  Lines        4994     5068      +74     
  Branches      644      652       +8     
==========================================
+ Hits         4342     4409      +67     
- Misses        462      466       +4     
- Partials      190      193       +3     

Flag	Coverage Δ
unittests	86.99% <ø> (+0.05%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[henrydingliu]

I like how tables look in jupyter notebooks. i just didn't want you to be surprised when we are almost done with api docstrings and they are all in standard text. after some trial and error, i couldn't figure out a way to make the same block of code in .rst to both act as a test AND output html. and the html rendering in rst doesn't look nearly as nice, though that's probably due to lack of css customization on my end.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit ace9aac. Configure here.}

[cursor]

Duplicated helper functions differ only in CDF logic

Low Severity

rounded_development_summary duplicates nearly all of development_summary — the shared setup logic for claims, ages, index alignment, and paid claims is identical. Only the CDF computation and ultimate calculation differ. Since both functions are reused across multiple exhibits in this chapter, consolidating them (e.g., by parameterizing the CDF/ultimate logic) would reduce maintenance burden if the shared setup logic ever needs updating.

Additional Locations (1)

• docs/friedland/chapter_7.rst#L475-L492

 

^{Reviewed by Cursor Bugbot for commit ace9aac. Configure here.}

[henrydingliu]

the 'best' solution is currently up in experimental. The actual doctest is hidden. then i use a combination of a non-executing code-block and a raw html block to essentially fake the html looking table. it's only a little tedious. but the html table doesn't look that great.