map_cleanup: phantom orientations — picker scores height, not prominence

docs/tuning/2026-08-11-orientation-picking.md

@@ -0,0 +1,136 @@
+# Orientation picking — the phantom wall direction, 2026-08-11
+
+**Verdict: the declutter pass picks wall orientations by their height above the
+angular energy's broadband floor (`peak_rel_threshold` 0.15 of the residual
+maximum), not by their height above zero (0.45 of the raw maximum).** The
+phantom direction that forced a hand-tuned threshold on the 2026-08-02 flat map
+is gone, and the shipped defaults now reproduce that hand-tuned output exactly.
+This doc is the measurement behind the change. Data:
+`2026-08-11-orientation-picking/`.
+
+## The defect
+
+`_pick_directions` kept any local maximum of the angular energy reaching
+`peak_rel_threshold` (0.45) of the global maximum, after suppressing peaks
+within 12° of a stronger one.
+
+Clutter, speckle and ragged wall edges put energy at *every* orientation, so a
+real map's angular energy is a modest set of wall peaks riding on a broad
+pedestal — **measured at 0.49 of the global maximum** on the 2026-08-02 flat
+map. A threshold expressed as a fraction of the raw maximum therefore spends
+more than half its range below the pedestal, where nothing is a direction and
+everything clears the gate.
+
+What survives that gate is whatever the 12° suppression radius happens to
+leave: on that map, a direction at **73.8°**, which is the shoulder of the real
+86.8° wall family 13.0° away — just outside the radius. It reaches 0.50 of the
+raw maximum against the weakest real family's 0.64, a ratio of 1.3, and no
+threshold placed between two such numbers is a measurement. The operator's
+workaround was `--peak-rel 0.55`, threaded between them by hand.
+
+It is not one map's curiosity. Of the **13 genuine occupancy maps** on this
+box, **6 carried such a shoulder**, every one of them 12.0–13.5° from a
+stronger peak.
+
+![before and after](2026-08-11-orientation-picking/energy-panel-before-after.png)
+
+## What ROSE itself does, and why this departs from it
+
+Worth stating plainly, because this module cites ROSE: **the shipped
+relative-height rule was never ROSE's rule**, and neither is the one replacing
+it. ROSE (Luperto et al., [arXiv:2004.08794](https://arxiv.org/abs/2004.08794),
+§3.1) selects directions by **topographic prominence**, thresholded at 50% of
+the curve's peak-to-trough range — literally, in the reference implementation
+([aislabunimi/ROSE2](https://github.com/aislabunimi/ROSE2),
+`src/rose_v1_repo/fft_structure_extraction.py`):
+
+```python
+find_peaks(pol_h, prominence=(np.max(pol_h) - np.min(pol_h)) * 0.5)
+```
+
+Run over our corpus (`rose-rule.txt`), that criterion is *more* conservative
+than either of ours: on all 13 maps it returns exactly the two strongest,
+near-orthogonal directions, and never a phantom. It also drops the real
+off-axis families along with them — 11.8° and 110.2° on the flat map — which is
+the two-direction result the operator rejected at `--peak-rel 0.65` for visibly
+eroding walls.
+
+That is a difference in the maps, not a mistake in the paper. ROSE scores large,
+overwhelmingly rectilinear building floor plans, where two directions is the
+right answer and a third is usually clutter; and its output is an abstracted
+floor plan. A small flat mapped by a 2-D lidar has genuine off-axis wall
+families, and this pass keeps *observed pixels* gated to real walls, so a
+dropped family costs real wall rather than detail.
+
+## Two fixes that do not work
+
+**Literal topographic prominence**, at a threshold loose enough to keep the four
+families, cannot exclude the phantom. The real off-axis family at 11.8° scores
+0.033 of the maximum against the phantom's 0.028 — that family sits on the tail
+of the dominant one and is a shoulder too, in exactly the same sense. So
+prominence has no setting that expresses what is wanted here: the paper's
+threshold gives two directions, and any threshold that admits the third and
+fourth admits the phantom with them. Thresholding it that low is worse still,
+because prominence rewards isolation and starts promoting lone bumps in the
+noise floor that carry no structural energy at all (47.2° on the tuning map;
+43.2° and 137.2° on the replay maps).
+
+**A wider suppression radius** cannot separate them either. The phantom is
+13.0° from its parent; the real off-axis family is 14.5° from its own. Any
+radius that suppresses the one suppresses the other.
+
+## The fix
+
+Subtract the broadband floor before thresholding — the curve smoothed over
+`FLOOR_HALFWIDTH_DEG` (45°), i.e. what is left once anything with a wall's
+angular sharpness is averaged away. This module already computed that residual
+for `angular_stats`; the declutter picker was simply not using it.
+
+On the same two peaks, the residual reads **0.012 for the phantom and 0.296 for
+the weakest real family** — 1.3× apart before, 24× apart after.
+
+The threshold moves with the curve: 0.15 of the residual maximum, the same
+number `angular_stats` already used on the same kind of curve. **Both halves
+have to travel together.** 0.45 on the residual would drop real families; 0.15
+on the raw curve returns four directions on a clean synthetic rectilinear map,
+two of them flank samples. A test pins that combination.
+
+The threshold's window is measured, and is not wide:
+
+| bound | map | direction | residual |
+| --- | --- | --- | --- |
+| from below | `sim/mote_world` | 137.2° phantom, admitted at 0.10 | 0.109 |
+| from above | `tuning/input_map` | 16.2°, a real family the shipped rule kept | 0.189 |
+
+## What changed, over every map on the box
+
+`corpus.txt` — 13 genuine ROS occupancy PNGs (the tuning input, the three sim
+world maps, and eight `map_raw_notraj.png` outputs of the bag-replay harness;
+`bag_replay_results/*/*/map.png` is a rendered *figure*, not a map, and is not
+in the corpus).
+
+* **7 maps unchanged**, including the whole tuning set and all three sim worlds.
+* **6 maps lost exactly one direction each**, and every one of them was a
+  shoulder 12.0–13.0° from a stronger peak, with residual 0.012–0.105 against a
+  weakest-kept of 0.204–0.285.
+* **No map gained a direction.**
+
+## Does it reproduce the hand-tuned map?
+
+Yes, byte for byte. The 2026-08-02 flat map cleaned with the new defaults is
+**identical** to the `--peak-rel 0.55` output the operator hand-threaded and
+shipped (0 differing cells of 43 259), and differs from the old default's output
+by 664 cells. `flat-map-output.txt`.
+
+Room segmentation, which picks directions through the same function to find the
+map's dominant rotation, is unchanged on the sim ladder — 30/33 hospital, 10/10
+office, 1/1 mote, zero merges, and the same again with the map turned +17° and
+−31°.
+
+## What this does not fix
+
+The picker still returns *directions*, not families, and `max_directions` (5) is
+still a cap rather than a decision: a building genuinely using six wall
+directions loses one, silently. Nothing here estimates how many directions a map
+*should* have — `angular_stats`'s frame table is the diagnostic that reads the
+answer back, and it is a diagnostic, not a gate.

docs/tuning/2026-08-11-orientation-picking/corpus.txt

@@ -0,0 +1,33 @@
+# Direction sets over every genuine ROS occupancy PNG on the dev box, 2026-08-11.
+# "before" reproduces the shipped rule (relative height on the raw angular
+# energy, 0.45); "after" is the patched pipeline (relative height on the
+# floor-subtracted residual, 0.15). Angles are spectral (wall normals), as the
+# declutter CLI prints them.
+#
+# Corpus: scratchpad_results/map_cleanup/{input_map,warehouse/cleaned}.png,
+# mote_simulation/sim_home/sites/*/floors/ground/maps/*/map.png, and
+# bag_replay_results/*/*/map_raw_notraj.png.
+
+map                                            before  after  directions
+--------------------------------------------------------------------------------------------------------------
+tuning/input_map                                    4      4  UNCHANGED
+tuning/warehouse                                    1      1  UNCHANGED
+sim/hospital_world                                  2      2  UNCHANGED
+sim/mote_world                                      2      2  UNCHANGED
+sim/office_world                                    2      2  UNCHANGED
+replay/20260802T142558/r2-chain10                   5      4  dropped 73.75
+      dropped  73.75: raw h/max 0.501  residual 0.012  raw prominence/max 0.028  gap to stronger peak 13.0 deg  (weakest kept residual 0.285)
+replay/20260802T162552/r2-chain10                   5      5  UNCHANGED
+replay/20260802T162659/r2-chain10                   5      4  dropped 170.25
+      dropped 170.25: raw h/max 0.472  residual 0.070  raw prominence/max 0.014  gap to stronger peak 12.0 deg  (weakest kept residual 0.204)
+replay/20260802T162758/r2-chain10                   5      4  dropped 96.75
+      dropped  96.75: raw h/max 0.538  residual 0.105  raw prominence/max 0.000  gap to stronger peak 12.0 deg  (weakest kept residual 0.235)
+replay/20260802T163028/r2-chain10                   5      4  dropped 100.75
+      dropped 100.75: raw h/max 0.526  residual 0.028  raw prominence/max 0.000  gap to stronger peak 12.0 deg  (weakest kept residual 0.270)
+replay/20260802T163104/r2-chain10                   5      4  dropped 98.25
+      dropped  98.25: raw h/max 0.525  residual 0.104  raw prominence/max 0.000  gap to stronger peak 12.0 deg  (weakest kept residual 0.259)
+replay/20260802T163141/r2-chain10                   4      4  UNCHANGED
+replay/20260802T163259/r2-chain10-finang            5      4  dropped 80.25
+      dropped  80.25: raw h/max 0.533  residual 0.065  raw prominence/max 0.000  gap to stronger peak 12.0 deg  (weakest kept residual 0.279)
+--------------------------------------------------------------------------------------------------------------
+7 maps unchanged, 6 changed

docs/tuning/2026-08-11-orientation-picking/flat-map-output.txt

@@ -0,0 +1,10 @@
+# The 2026-08-02 flat map through the new defaults, against the outputs cleaned
+# on 2026-08-02. `cleaned` is the old default (--peak-rel 0.45); `cleaned-rel*`
+# are the hand-threaded runs. 0.55 is the one the operator shipped.
+
+new default directions: [11.75, 86.75, 110.25, 177.25]
+occupied: 4474 -> 3245
+  vs cleaned          differing cells:    664 (1.535% of the grid)
+  vs cleaned-rel0.5   differing cells:    664 (1.535% of the grid)
+  vs cleaned-rel0.55  differing cells:      0 (0.000% of the grid)   IDENTICAL
+  vs cleaned-rel0.65  differing cells:   1321 (3.054% of the grid)

docs/tuning/2026-08-11-orientation-picking/rose-rule.txt

@@ -0,0 +1,28 @@
+# ROSE's own peak criterion over our corpus, 2026-08-11, against this PR's.
+#
+# ROSE (arXiv:2004.08794 §3.1) selects dominant directions by topographic
+# prominence at 50% of the curve's peak-to-trough range; the reference
+# implementation (aislabunimi/ROSE2, src/rose_v1_repo/fft_structure_extraction.py)
+# is find_peaks(pol_h, prominence=(max(pol_h) - min(pol_h)) * 0.5). Applied here
+# to the same angular energy curve our declutter pass measures, with scipy's
+# prominence definition transcribed (no scipy in the robot env).
+#
+# It never admits a phantom, and it never returns more than the two strongest
+# near-orthogonal directions -- including on the maps whose real off-axis
+# families this pass has to keep. Angles are spectral (wall normals).
+
+map                                          ROSE (prom >= 0.5*range)                      this PR
+----------------------------------------------------------------------------------------------------
+tuning/input_map                                           88.2 179.8        16.2 88.2 114.8 179.8
+tuning/warehouse                                                179.8                        179.8
+sim/hospital_world                                           0.2 90.2                     0.2 90.2
+sim/mote_world                                               0.2 90.2                     0.2 90.2
+sim/office_world                                           90.2 179.2                   90.2 179.2
+replay/20260802T142558/r2-chain10                          86.8 177.2        11.8 86.8 110.2 177.2
+replay/20260802T162552/r2-chain10                          80.2 174.8    7.2 80.2 92.2 104.8 174.8
+replay/20260802T162659/r2-chain10                            2.2 90.8          2.2 14.8 90.8 114.8
+replay/20260802T162758/r2-chain10                          84.8 178.2        10.8 84.8 109.2 178.2
+replay/20260802T163028/r2-chain10                          88.2 179.8        13.2 88.2 112.8 179.8
+replay/20260802T163104/r2-chain10                          86.2 178.2        11.2 86.2 110.8 178.2
+replay/20260802T163141/r2-chain10                            2.2 92.8          2.2 14.8 92.8 114.8
+replay/20260802T163259/r2-chain10-finang                     0.2 92.2          0.2 14.2 92.2 113.8

docs/tuning/2026-08-11-orientation-picking/threshold-sweep.txt

@@ -0,0 +1,139 @@
+# Threshold sweep behind peak_rel_threshold = 0.15, 2026-08-11. Same corpus as
+# corpus.txt. Schemes, all with the shipped 12 deg suppression radius and a cap
+# of 5 directions:
+#   raw @t  : threshold t * max on the raw angular energy (what shipped)
+#   res @t  : threshold t * max on the floor-subtracted residual (what ships now)
+#   prom@t  : threshold t * max on topographic prominence of the raw curve
+# prom@ is the rejected candidate: note the directions it invents (47.2 on the
+# tuning map, 43.2 / 137.2 on the replay maps) and the real families it drops.
+
+
+== tuning/input_map
+  raw @0.45 (ships):  16.25  88.25 114.75 179.75
+  res @0.10       :  16.25  88.25 114.75 179.75
+  res @0.15       :  16.25  88.25 114.75 179.75
+  res @0.20       :  88.25 114.75 179.75
+  res @0.25       :  88.25 114.75 179.75
+  prom@0.03       :  16.25  47.25  88.25 114.75 179.75
+  prom@0.05       :  16.25  47.25  88.25 114.75 179.75
+  prom@0.08       :  16.25  47.25  88.25 114.75 179.75
+
+== tuning/warehouse
+  raw @0.45 (ships): 179.75
+  res @0.10       : 179.75
+  res @0.15       : 179.75
+  res @0.20       : 179.75
+  res @0.25       : 179.75
+  prom@0.03       : 179.75
+  prom@0.05       : 179.75
+  prom@0.08       : 179.75
+
+== sim/hospital_world
+  raw @0.45 (ships):   0.25  90.25
+  res @0.10       :   0.25  90.25
+  res @0.15       :   0.25  90.25
+  res @0.20       :   0.25  90.25
+  res @0.25       :   0.25  90.25
+  prom@0.03       :   0.25  90.25
+  prom@0.05       :   0.25  90.25
+  prom@0.08       :   0.25  90.25
+
+== sim/mote_world
+  raw @0.45 (ships):   0.25  90.25
+  res @0.10       :   0.25  90.25 137.25
+  res @0.15       :   0.25  90.25
+  res @0.20       :   0.25  90.25
+  res @0.25       :   0.25  90.25
+  prom@0.03       :   0.25  47.25  69.75  90.25 137.25
+  prom@0.05       :   0.25  47.25  69.75  90.25 137.25
+  prom@0.08       :   0.25  47.25  90.25 137.25
+
+== sim/office_world
+  raw @0.45 (ships):  90.25 179.25
+  res @0.10       :  90.25 179.25
+  res @0.15       :  90.25 179.25
+  res @0.20       :  90.25 179.25
+  res @0.25       :  90.25 179.25
+  prom@0.03       :  90.25 179.25
+  prom@0.05       :  90.25 179.25
+  prom@0.08       :  90.25 179.25
+
+== replay/20260802T142558/r2-chain10
+  raw @0.45 (ships):  11.75  73.75  86.75 110.25 177.25
+  res @0.10       :  11.75  86.75 110.25 177.25
+  res @0.15       :  11.75  86.75 110.25 177.25
+  res @0.20       :  11.75  86.75 110.25 177.25
+  res @0.25       :  11.75  86.75 110.25 177.25
+  prom@0.03       :  47.25  86.75 110.25 137.25 177.25
+  prom@0.05       :  47.25  86.75 110.25 137.25 177.25
+  prom@0.08       :  47.25  86.75 110.25 137.25 177.25
+
+== replay/20260802T162552/r2-chain10
+  raw @0.45 (ships):   7.25  80.25  92.25 104.75 174.75
+  res @0.10       :   7.25  80.25  92.25 104.75 174.75
+  res @0.15       :   7.25  80.25  92.25 104.75 174.75
+  res @0.20       :   7.25  80.25  92.25 104.75 174.75
+  res @0.25       :   7.25  80.25  92.25 104.75 174.75
+  prom@0.03       :   7.25  80.25  92.25 104.75 174.75
+  prom@0.05       :   7.25  80.25  92.25 104.75 174.75
+  prom@0.08       :  80.25  92.25 104.75 174.75
+
+== replay/20260802T162659/r2-chain10
+  raw @0.45 (ships):   2.25  14.75  90.75 114.75 170.25
+  res @0.10       :   2.25  14.75  90.75 114.75
+  res @0.15       :   2.25  14.75  90.75 114.75
+  res @0.20       :   2.25  14.75  90.75 114.75
+  res @0.25       :   2.25  90.75 114.75
+  prom@0.03       :   2.25  14.75  73.75  90.75 114.75
+  prom@0.05       :   2.25  14.75  73.75  90.75 114.75
+  prom@0.08       :   2.25  14.75  90.75 114.75
+
+== replay/20260802T162758/r2-chain10
+  raw @0.45 (ships):  10.75  84.75  96.75 109.25 178.25
+  res @0.10       :  10.75  84.75  96.75 109.25 178.25
+  res @0.15       :  10.75  84.75 109.25 178.25
+  res @0.20       :  10.75  84.75 109.25 178.25
+  res @0.25       :  84.75 109.25 178.25
+  prom@0.03       :  40.75  65.25  84.75 109.25 178.25
+  prom@0.05       :  40.75  65.25  84.75 109.25 178.25
+  prom@0.08       :  84.75 109.25 178.25
+
+== replay/20260802T163028/r2-chain10
+  raw @0.45 (ships):  13.25  88.25 100.75 112.75 179.75
+  res @0.10       :  13.25  88.25 112.75 179.75
+  res @0.15       :  13.25  88.25 112.75 179.75
+  res @0.20       :  13.25  88.25 112.75 179.75
+  res @0.25       :  13.25  88.25 112.75 179.75
+  prom@0.03       :  13.25  88.25 112.75 151.25 179.75
+  prom@0.05       :  13.25  88.25 112.75 151.25 179.75
+  prom@0.08       :  13.25  88.25 112.75 179.75
+
+== replay/20260802T163104/r2-chain10
+  raw @0.45 (ships):  11.25  86.25  98.25 110.75 178.25
+  res @0.10       :  11.25  86.25  98.25 110.75 178.25
+  res @0.15       :  11.25  86.25 110.75 178.25
+  res @0.20       :  11.25  86.25 110.75 178.25
+  res @0.25       :  11.25  86.25 110.75 178.25
+  prom@0.03       :  48.75  86.25 110.75 139.25 178.25
+  prom@0.05       :  48.75  86.25 110.75 139.25 178.25
+  prom@0.08       :  86.25 110.75 178.25
+
+== replay/20260802T163141/r2-chain10
+  raw @0.45 (ships):   2.25  14.75  92.75 114.75
+  res @0.10       :   2.25  14.75  92.75 114.75
+  res @0.15       :   2.25  14.75  92.75 114.75
+  res @0.20       :   2.25  92.75 114.75
+  res @0.25       :   2.25  92.75
+  prom@0.03       :   2.25  14.75  73.75  92.75 114.75
+  prom@0.05       :   2.25  14.75  73.75  92.75 114.75
+  prom@0.08       :   2.25  92.75 114.75
+
+== replay/20260802T163259/r2-chain10-finang
+  raw @0.45 (ships):   0.25  14.25  80.25  92.25 113.75
+  res @0.10       :   0.25  14.25  92.25 113.75
+  res @0.15       :   0.25  14.25  92.25 113.75
+  res @0.20       :   0.25  14.25  92.25 113.75
+  res @0.25       :   0.25  14.25  92.25 113.75
+  prom@0.03       :   0.25  14.25  92.25 113.75 143.25
+  prom@0.05       :   0.25  14.25  92.25 113.75 143.25
+  prom@0.08       :   0.25  14.25  92.25 113.75

mkdocs.yml

@@ -156,3 +156,4 @@ nav:
       - Slip detection: tuning/2026-07-28-slip-detection.md
       - Camera layer decay: tuning/2026-07-29-camera-layer-decay.md
       - Monitor-node CPU: tuning/2026-08-11-monitor-cpu.md
+      - Orientation picking: tuning/2026-08-11-orientation-picking.md