added examples/fitting/fitting_galsimfitter.py & examples/metacal/metacal_psf_response.py

[s-Sayan]

New files

• examples/fitting/fitting_galsimfitter.py — Demonstrates fitting with GalsimFitter, analogous to the existing fitter examples in the same directory.
• examples/metacal/metacal_psf_response.py — Demonstrates metacalibration using GalsimFitter with psf='dilate', including PSF response corrections. Follows the structure of examples/metacal/metacal.py.

Notes

Both examples are heavily inspired by the discussion in issue #255.

[esheldon]

Thanks for the examples. Do you think a unit test could also be made for these?

[s-Sayan]

Hi Erin, yes — these are unit tests (similar to existing examples in your directory: fitting_galsimfitter.py mirrors examples/fitting/fitting.py, and metacal_psf_response.py mirrors examples/metacal/metacal.py). They use round objects, a Moffat PSF (with slight PSF shears), and low-noise settings. Running the code should produce the following outputs:

fitting_galsimfitter.py

S/N: 805.3598771396064
true flux: 100   meas flux:  99.9136 +/- 0.427817  (99.7% conf)
true hlr:  0.5   meas hlr:   0.501831 +/- 0.0037977 (99.7% conf)
true g1:   0.05  meas g1:    0.0456032 +/- 0.00703185 (99.7% conf)
true g2:  -0.02  meas g2:   -0.022934 +/- 0.00702158 (99.7% conf)

metacal_psf_response.py (takes a bit of time — runs over 5 input shears: −0.02, −0.01, 0.0, 0.01, 0.02, then fits a line to obtain m and c)

S/N: 50976.7
R11: 0.999596
R11_psf: 2.65024e-05
m: 0.000421002 +/- 0.000118555 (99.7% conf)
c: 1.21787e-07 +/- 1.66119e-06 (99.7% conf)

I've added explanations and expected outputs to the docstrings at the top of each file. Is there any other test in particular you'd like me to add?

[esheldon]

I meant real unit tests that assert correctness, as in ngmix/tests/

but never mind that for now. I'll see if these run for me when I get a chance.

[s-Sayan]

Ah I see, thanks! I didn't know about pytest — that's pretty cool!

I can add something like that for testing the GalsimFitter in MetacalBootstrapper and the PSF response correction, and assert correctness for the additive c (something similar to ngmix/tests/test_metacal_accuracy.py). I guess I would only test this for psf='dilate' — is there any specific situation you would suggest testing? Like any weird PSF maybe?

I also see you already have a unit test for the GalsimFitter: ngmix/tests/test_ml_fitting_galsim.py::test_ml_fitting_galsim.

[s-Sayan]

Okay I added a unit test at ngmix/tests/test_metacal_galsim_psf_response.py. To make the test a bit more challenging, I used galsim.OpticalPSF to create a jittery PSF (using SuperBIT instrumentation details) and shear the PSF a bit, since I cannot use a .psf file, I thought this is the best I can do, here one such example of what that PSF looks like:

I check the additive bias c1 (using g1 component only) for different PSF ellipticity combinations (psf_g1 = -0.02, +0.02; psf_g2 = -0.02, +0.02) and assert correctness via cerr < 1e-5 and |c| < cerr (consistent with zero). Let me know if you'd like a different assertion condition or additional scenarios to be tested. The values I get for those four combinations are:

psf_g1	psf_g2	c1	3*cerr (99.7%)
−0.02	−0.02	6.24×10⁻⁷	7.63×10⁻⁶
+0.02	−0.02	4.09×10⁻⁷	7.63×10⁻⁶
−0.02	+0.02	7.37×10⁻⁷	7.62×10⁻⁶
+0.02	+0.02	5.41×10⁻⁷	7.61×10⁻⁶

P.S. I have also modified examples/metacal/metacal_psf_response.py to use the SuperBIT optical PSF with shear rather than Moffat PSF, and updated the expected output in the docstring. For both the unit test and the example, shear estimation uses only sheear_true = ±0.02 to estimate m and c, with ntrial=100 to keep things a bit faster.

[esheldon]

I don't think you would always expect c < cerr, so this unit test can fail (it did for me)

[esheldon]

These would be good checks I think

abs(m) < 0.001
abs(c) < 1.0e-5

I found setting the galaxy size to something smaller also helped (I used 128).

Also I ran tests with psf="azgauss" it it looks good too.

[s-Sayan]

Thanks! I have now updated the assertion conditions like that, in the same code I have also included the psf=azgauss settings to be tested with the superbit optical psf.

I found setting the galaxy size to something smaller also helped (I used 128).

I think I understand this but just to make sure — I use an Exponential profile with hlr=0.5 arcsec (with pixel scale 0.141). You mean setting nx, ny = 128 while doing:

​ im = obj.drawImage(nx=npix, ny=npix, scale=scale).array ​

Previously I was letting GalSim decide npix, and that was actually causing higher m and c, hence the failure for azgauss.

TL;DR: Is it a rule of thumb that psf_npix shouldn't be much higher than npix of the object? Or is it specific to the psf='azgauss' settings?

[s-Sayan]

Hey Erin, I have a quick question on how are we tracking if the psf='azgauss', is better? is it just looking at m and c. If so, then the difference between the psf='azgauss' and psf='gauss' is not so much (with the superbit psf test above), I am getting the c values small for both the settings.

1. psf='azgauss':

m: 0.00042347 +/- 0.000426856 (99.7% conf)
c: -1.71905e-06 +/- 8.53713e-06 (99.7% conf)

2. psf='gauss':

m: 0.000431744 +/- 0.00042775 (99.7% conf)
c: -1.82156e-06 +/- 8.555e-06 (99.7% conf)

I have also tried crancking up the noise, the m and c values are higher, but they are comparable for the both settings. Maybe I should try some other test to see the improvement over gauss?

[esheldon]

azgauss is better when the PSF image is noisy