Classical single-image reflection separation via gradient-domain optimization.
Separates a photo taken through glass into two layers:
- T (transmission): the scene behind the glass
- R (reflection): the reflected scene
I = T + R
minimize E(T) = λT * ||wT ⊙ ∇T||² + λR * ||wR ⊙ ∇R||² + λdata * ||T - I||²
Weights wT, wR are updated each outer IRLS step to enforce gradient sparsity.
Edge exclusion penalizes gradients appearing in both T and R at the same location.
A data fidelity term anchors T to the input, preventing degenerate solutions.
Solved per-channel via conjugate gradient (matrix-free, O(N) per step).
cargo build --releaseBinary: target/release/reflection-removal
Requires Rust edition 2024. Dependencies: rayon, clap, image, anyhow.
reflection-removal [OPTIONS] <INPUT>
Arguments:
<INPUT> Input image (PNG or JPEG)
Options:
-o, --output <PATH> Transmission output [default: <stem>_T.png]
-r, --reflection <PATH> Reflection layer output (optional)
-v, --verbose Print per-iteration diagnostics
-h, --help Print help
| Flag | Default | Meaning |
|---|---|---|
--lambda-t |
0.5 | T gradient sparsity weight. Keep low - T is the dominant scene layer with many gradients. |
--lambda-r |
2.0 | R gradient sparsity weight. Keep high - R must earn its gradients. |
--lambda-excl |
1.0 | Edge exclusion penalty. Suppresses T gradients where R has edges, and vice versa. |
--lambda-smooth |
0.3 | R smoothness prior. Inflates R's gradient cost, pushing R toward zero except at strong edges. |
--lambda-data |
0.5 | Data fidelity ` |
--init-blur-radius |
4 | Box-blur radius for initialization. T_init = blur(I), R_init = I - T_init. Gives R non-zero starting gradients. 0 to disable. |
-n, --iterations |
8 | Outer IRLS steps. More iterations = finer separation but slower. |
--inner-iters |
200 | Conjugate gradient steps per outer iteration. |
--epsilon |
0.05 | IRLS denominator floor. Caps max weight at lambda_r / epsilon. Lower = sharper sparsity but worse CG conditioning. |
Most impactful - try first:
| Scenario | Suggestion |
|---|---|
| Reflection edges overlap scene edges | Increase --lambda-excl to 2-5 |
| Reflection is blurry / diffuse | Increase --lambda-smooth to 0.5-2.0 |
| Result looks identical to input | Decrease --lambda-data to 0.1-0.3 |
| Output still contaminated after many iterations | Increase -n to 12-20 |
| T is over-smoothed (scene details lost) | Increase --lambda-t to 1.0-2.0 |
Aggressive settings for stubborn cases:
reflection-removal photo.jpg \
--lambda-excl 4 \
--lambda-smooth 1.5 \
--lambda-data 0.1 \
-n 15 \
-o result_T.png -r result_R.png -vEach outer iteration fixes per-pixel weights and solves a linear system for T:
(Dᵀ WT D + Dᵀ WR D + λdata I) T = (Dᵀ WR D + λdata I) I
where D is the forward-difference gradient operator and Dᵀ is its true adjoint (backward divergence with boundary zeros). The system is solved via conjugate gradient - matrix-free, computing Ax as div(W * grad(x)) + λdata * x in O(N) per step.
wT[i] = λT / (|∇T[i]| + ε) <- L1 sparsity, favors piecewise-constant T
wR[i] = λR / (|∇R[i]| + ε) <- L1 sparsity on R
The parameter epsilon is the key conditioning parameter: it caps the maximum weight at lambda / epsilon, preventing the CG system from becoming ill-conditioned.
After base IRLS weights:
wT[i] *= exp(-λexcl * |∇R[i]|) <- suppress T gradient where R is strong
wR[i] *= exp(-λexcl * |∇T[i]|) <- suppress R gradient where T is strong
To avoid the degenerate T=I fixed point, T is initialized as a box-blurred version of I and R as the high-frequency residual:
T_init = box_blur(I, radius)
R_init = I - T_init
This gives R non-zero starting gradients so IRLS weights begin in a balanced regime.
RGB channels are processed independently and in parallel (rayon). Input is converted from sRGB to linear before optimization; output is converted back to sRGB.
Single-image reflection removal is an ill-posed problem. For every observed pixel you want two values but have one. Every algorithm adds assumptions; when those assumptions are violated, quality degrades.
- Reflection is clearly blurrier than the scene (depth-of-field difference)
- Reflection is weaker than the scene (low-reflectance glass)
- Scene has dominant structure (large-scale edges, silhouettes) with reflection as texture
- Sharp reflection, sharp scene - gradient statistics are indistinguishable; the optimizer has no evidence for layer assignment
- Strong reflection - the T-dominant prior breaks down; the scene might actually be the weaker layer
- Large flat regions - gradient-domain methods only separate texture; color cast from reflection in flat areas is invisible to the optimizer
The classical single-image objective always has T = I, R = 0 as a valid solution (scene = everything, reflection = nothing). The sparsity and smoothness priors push toward a better solution when there is genuine statistical contrast between the layers - but without that contrast, no parameter tuning overcomes the ambiguity.
When you need reliable separation:
| Situation | Recommended approach |
|---|---|
| You can take multiple photos | Multi-image motion-based separation (different viewpoints give different parallax per layer) |
| Controlled capture setup | Polarizing filter at multiple angles |
| Focus difference between layers | Focus-bracketing separation |
| Training data available | Neural single-image reflection removal (e.g., ERRNet, IBCLN) |
reflection-removal/
├── Cargo.toml # workspace
├── crates/
│ ├── core/ # reflection-removal-core (lib)
│ │ └── src/
│ │ ├── lib.rs # public API
│ │ ├── image.rs # ImageF32, gradient ops, sRGB conversion, box blur
│ │ ├── weights.rs # IRLS weights, exclusion, smoothness
│ │ ├── solver.rs # conjugate gradient
│ │ └── optimize.rs # IRLS outer loop
│ └── cli/ # reflection-removal (bin)
│ └── src/
│ └── main.rs # CLI, image I/O
cargo testTests cover: gradient operator adjoints, constant-image gradients, sRGB roundtrip, CG convergence on a known PD system.
MIT