TPHOT-Fast

TPHOT-Fast is an unofficial, performance-oriented fork of T-PHOT v2.x for prior-based, PSF-matched deconfusion photometry. It keeps the T-PHOT fitting model and file formats while removing the dominant repeated I/O and matrix assembly work in the cells-on-objects (COO) fitter. This version is fully developed by GPT-5.6 sol model.

This repository is the Python 3 port of the original Python 2-oriented T-PHOT codebase. The pipeline and installer have been migrated to Python 3 syntax and runtime APIs and are validated with Python 3.11. Python 2 is not supported by TPHOT-Fast.

On a validated 200 x 200 PACS-green tile with 630 templates, the optimized fitter reduced wall time from 243.41 s to 13.16 s (18.50x). The output catalog and cell file were byte-for-byte identical to the baseline build.

This is an experimental fork, not an official release from the original T-PHOT authors. Validate it on representative data before production use.

What changed

• Template FITS cache: each mod-ID.fits thumbnail is loaded once per fitter process instead of being reopened inside source-pair loops.
• Source-pair matrix cache: safe weighted template dot products are reused across COO cells.
• Partial covariance: when a COO cell writes only its central source, only the required column of the inverse matrix is obtained by back-substitution.
• Python 3 edition: the original Python 2-oriented pipeline has been ported to Python 3, including print, exception, input, dictionary, iterator, string-processing and timing API changes.
• Portable build: CFITSIO and FFTW are discovered from the active Conda environment or a user-supplied dependency prefix.

The matrix-pair optimization has guarded eligibility checks. If the requested fit is not mathematically equivalent to the cached operation, TPHOT-Fast uses the original computation. See docs/OPTIMIZATIONS.md.

Quick start with Conda

git clone https://github.com/AstroZzz3z/TPHOT-Fast.git
cd TPHOT-Fast

conda env create -f environment.yml
conda activate tphot-fast
python install.py

export TPHOT_HOME="$PWD"
export PATH="$TPHOT_HOME/bin:$PATH"
export PYTHONPATH="$TPHOT_HOME/lib${PYTHONPATH:+:$PYTHONPATH}"

Confirm that the optimized copy is selected:

command -v tphot
command -v tphot_fitter

Run T-PHOT with the same parameter-file interface as upstream:

tphot /path/to/pass1.param
tphot /path/to/pass2.param

The bundled upstream test can be run from its working directory:

cd test
tphot test.param

Install with existing system libraries

Python 3 is required (Python 3.11 is the validated version), together with NumPy, SciPy, Astropy, Matplotlib, CFITSIO, FFTW3, a C compiler and a C++ compiler. If CFITSIO and FFTW share a prefix:

export DEPS_PREFIX=/path/to/dependencies
python install.py

Without DEPS_PREFIX, the build uses CONDA_PREFIX when a Conda environment is active and /usr/local otherwise.

Runtime controls

All fast paths are enabled by default. Each can be disabled independently for compatibility testing:

export TPHOT_FAST_CACHE=0          # disable template FITS cache
export TPHOT_FAST_MATRIX_CACHE=0   # disable source-pair matrix cache
export TPHOT_FAST_PARTIAL_COVAR=0  # compute the complete covariance matrix

Unset the variables or set them to 1 to restore the optimized defaults.

For many independent tiles on a large CPU server, use process-level tile parallelism and avoid nested BLAS oversubscription:

export OMP_NUM_THREADS=1
export OPENBLAS_NUM_THREADS=1
export MKL_NUM_THREADS=1

Start with fewer workers than physical cores, measure memory and throughput, then increase the number of tile workers. Each fitter process owns its own template and matrix caches; all cache memory is released when that process exits.

Validation and limitations

• The published benchmark used COO mode, cellmask=true, no fitted background, zero template threshold, LU solving and one process.
• The catalog and cell outputs matched the baseline byte-for-byte.
• The optimization is currently CPU-only. It does not provide CUDA kernels or a multithreaded LAPACK replacement.
• A tile containing many sources is not automatically a large linear solve; the relevant size is the Number of objects in each fitting cell.
• Highly crowded cells can make the source-pair cache grow quadratically. See docs/OPTIMIZATIONS.md.

Full benchmark details and hashes are in docs/BENCHMARK.md.

Original T-PHOT source and citation

This fork is based on the public T-PHOT repository by Emiliano Merlin:

• Upstream source: https://gitlab.com/EMerlin/t-phot-v2.0
• Baseline commit: d8ed4678f6c19547353653b969f64fac89ddce78 (v2.1, 2019-10-01)
• Original installation guide: README
• Detailed provenance: docs/ORIGIN.md

If T-PHOT contributes to scientific work, cite the original papers:

• Merlin et al. 2015, T-PHOT: A new code for PSF-matched, prior-based, multiwavelength extragalactic deconfusion photometry, https://arxiv.org/abs/1505.02516
• Merlin et al. 2016, T-PHOT version 2.0: improved algorithms for background subtraction, local convolution, kernel registration, and new options, https://arxiv.org/abs/1609.00146

Please state separately that the analysis used the unofficial TPHOT-Fast fork when these optimizations are enabled.

License

The upstream project is distributed under the GNU General Public License v3. This fork retains that license and the original copyright notices. See COPYING. The original authors are not responsible for the changes in this fork.

---

中文快速说明

TPHOT-Fast 是原始 T-PHOT 的 Python 3 移植和性能优化版，不支持 Python 2。它保留原始 T-PHOT 的参数文件和科学模型，主要优化了 buildlinearsystem 中重复打开模板 FITS、重复计算源对矩阵元和不必要的完整 协方差回代。推荐使用独立 Conda 环境安装：

conda env create -f environment.yml
conda activate tphot-fast
python install.py
export PATH="$PWD/bin:$PATH"
export PYTHONPATH="$PWD/lib${PYTHONPATH:+:$PYTHONPATH}"

之后仍然使用原来的命令：tphot pass1.param。原始 T-PHOT 环境不需要修改。