Skip to content
View aadityat23's full-sized avatar

Block or report aadityat23

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
aadityat23/README.md
Typing SVG

LinkedIn Email GitHub Profile Views



Current Work

Research engineering at the intersection of LLM evaluation, AI systems infrastructure, and applied ML. Primary focus: building evaluation tooling that makes numerical reasoning failures in financial LLMs measurable, reproducible, and systematically addressable.

Three active research and engineering tracks — FinVerify Terminal (evaluation infrastructure), DARPAN (climate AI for ISRO BAH 2026), and SATSentinel-X extensions. ARR submission #2935 currently under review.


Research & Publications

[Under Review] Structured Numerical Hallucinations in Financial LLMs ACL Rolling Review · May 2026 · Submission #2935 · Track: FinNLP @ EMNLP

Introduces a Deterministic Verification Layer (DVL) — a post-inference verification architecture that parses financial reasoning chains from model outputs, independently executes arithmetic, and flags outputs where model-reported values diverge from verified computation. DVL is model-agnostic and requires no fine-tuning or weight modification.

Controlled evaluation on FinQA demonstrates execution accuracy of 42.61% under DVL intervention, compared to 1.0% under standard inference — a finding that holds across model variants. Cross-domain evaluation on DROP establishes that the failure mode is finance-specific: general numerical reasoning in LLMs does not exhibit the same rate of arithmetic inconsistency, confirming that the problem is domain-structural rather than a general capability gap.

Benchmark  : FinQA (primary) · DROP (cross-domain control)
Method     : Deterministic Verification Layer (DVL), post-inference
Infra      : FinVerifyBench — open evaluation framework
Key result : Finance-specific arithmetic inconsistency confirmed

[Published] SATSentinel-X: AI-Assisted Spacecraft Telemetry Intelligence IEEE Conference · ASTRA 2026 Oral Presentation · IIST Thiruvananthapuram

Ensemble anomaly detection for spacecraft telemetry streams. Three-stage fusion — statistical baseline (Z-Score), classical ML (Isolation Forest), deep learning (LSTM Autoencoder) — with SHAP explainability for sensor-level attribution. Deployed via FastAPI with Kafka-based streaming ingestion.

F1 Score  : 0.767      AUC-ROC  : 0.943
Ensemble  : Isolation Forest · LSTM Autoencoder · Z-Score
Explain   : SHAP feature attribution
Serving   : FastAPI backend · Kafka streaming pipeline

[Published × 2] IEEE Conference Papers Applied ML systems: breast cancer survival prediction framework; computer vision applications.


Research Engineering — FinVerify

FinVerify Terminal

Verification infrastructure for numerical reasoning in financial LLMs

Research Problem

Financial LLMs frequently produce outputs that are arithmetically inconsistent with the evidence in the input context — numbers are hallucinated, multi-step arithmetic fails silently, and standard evaluation metrics (perplexity, exact-match) do not surface these failures. The problem is structural: financial reasoning requires chained arithmetic over retrieved values, and nothing in standard LLM training enforces arithmetic closure.

Contribution: Deterministic Verification Layer (DVL)

The DVL is a post-inference interception architecture. It parses model-generated reasoning steps, extracts operands and operations, executes arithmetic independently, and compares results against the model's stated answer. Outputs failing arithmetic verification are flagged before reaching the user. The architecture is model-agnostic — it operates on the output layer of any LLM without access to weights, logits, or training procedures.

Research Engineering Work

The system required substantial infrastructure engineering beyond the core verification logic:

  • Designed the full evaluation pipeline: context retrieval → reasoning extraction → arithmetic execution → verification → metric scoring
  • Built FinVerifyBench as the reproducible evaluation substrate (see below)
  • Implemented adversarial answerability validation, surfacing approximately 4.5% false positives in existing benchmark labels
  • Repaired systematic data pipeline defects: TAT-QA silent key-mismatch causing empty table propagation; FinQA context truncation dropping operands; synthetic samples with referenced values absent from input context
  • Identified and eliminated oracle leakage in the DVL evaluation pipeline — a systematic bias that inflated reported metrics in earlier experimental runs
  • Designed six new evaluation metrics capturing verification coverage, execution precision, operand recall, and arithmetic chain consistency
  • Built full audit toolchain for benchmark provenance tracking and result reproducibility

Technical Impact

The primary contribution is not the accuracy number — it is demonstrating that structured post-inference verification is a tractable intervention for a class of failures that model scaling alone does not address. The evaluation infrastructure (FinVerifyBench) is designed to be reusable by the research community for studying numerical reasoning reliability in financial LLMs.

Production stack: Next.js · FastAPI · Supabase · Mistral-7B · HuggingFace Spaces · Python


FinVerifyBench

Research infrastructure for financial LLM evaluation

The evaluation framework open-sourced alongside the ARR submission. FinVerifyBench is not a leaderboard — it is an auditable evaluation substrate designed for researchers studying numerical reasoning reliability in financial language models.

Components

Module Description
Validators Schema-level and semantic validators for financial QA inputs; detects malformed contexts, ambiguous referents, and truncation artifacts
Execution metrics Arithmetic execution correctness, operand coverage, chain-of-thought consistency, verification coverage rate
Benchmark auditing Automated detection of label inconsistencies, context leakage, annotation artifacts, and answerability false positives
Adversarial validation Systematic label-level audit; identified ~4.5% false positives in existing answerability annotations
Reproducibility tooling Deterministic evaluation runs with full provenance logging and seed-locked execution
Audit pipeline End-to-end traceability from input context → model output → verification result → final metric

The framework currently covers FinQA, TAT-QA, and DROP. Versioned as v3/v4 following progressive benchmark hardening cycles.

Python FastAPI HuggingFace Spaces FinQA TAT-QA DROP


Selected Systems

DARPAN — Dynamic AI Risk Prediction and Analysis Network

AI-powered Digital Twin of India's Climate · ISRO BAH 2026, PS5

Integrates IMDAA reanalysis data, INSAT-3D satellite observations, and MOSDAC geospatial products into a unified climate forecasting pipeline. Implements ensemble data assimilation (Optimal Interpolation), XGBoost-based extreme weather classification with HSS (Heidke Skill Score) validation, and uncertainty-quantified multi-horizon forecasting.

Python XGBoost Xarray NetCDF MOSDAC API Streamlit


SATSentinel-X

Spacecraft telemetry anomaly detection

The full system behind the ASTRA 2026 paper. Multi-stage ensemble designed under the constraint that spacecraft telemetry anomalies are rare, labeled data is scarce, and false negatives carry mission-critical cost. SHAP explainability layer maps detections to sensor-level attributions for operator review.

Python scikit-learn PyTorch SHAP Kafka FastAPI


NeuroGenesis

Evolutionary intelligence and artificial life research

CTRNN (Continuous-Time Recurrent Neural Network) agents evolving under selective pressure. Core experiments: whether the Baldwin Effect — the capacity for learned behaviors to redirect evolutionary search — manifests measurably in simulated populations with constrained plasticity. Sits at the intersection of evolutionary computation and cognitive modeling.

Python NumPy CTRNN Evolutionary Algorithms


AlphaForgeX

Quantitative analytics platform

Bloomberg-style Streamlit dashboard: RandomForest alpha signal generation, systematic factor backtesting, WorldQuant BRAIN IQC alpha development. Engineering exposure to systematic trading infrastructure.

Python Streamlit Pandas scikit-learn WorldQuant BRAIN


Research Contributions

  • Designed and implemented FinVerifyBench — an open evaluation framework for financial LLM numerical reasoning, with validators, execution metrics, audit tooling, and adversarial label validation
  • Built the Deterministic Verification Layer (DVL) — a post-inference verification architecture establishing that arithmetic consistency in financial LLM outputs can be enforced without modifying model weights
  • Identified systematic data pipeline defects across FinQA, TAT-QA, and synthetic benchmark sources; repaired evaluation infrastructure to eliminate oracle leakage and label-level false positives
  • Developed reproducible verification pipeline with full provenance logging, deterministic execution, and cross-benchmark generalization testing
  • Built spacecraft telemetry intelligence system (SATSentinel-X) — ensemble anomaly detection with multi-method fusion and SHAP-based attribution; IEEE-published and presented at ASTRA 2026
  • Developed climate AI workflows integrating IMDAA reanalysis, INSAT-3D satellite observations, and MOSDAC geospatial data for India's climate digital twin (DARPAN)
  • Conducted cross-domain evaluation of financial LLM failure modes using DROP as a control benchmark, establishing finance-specificity of arithmetic inconsistency

Systems Built

System Summary
FinVerify Terminal Post-inference verification infrastructure for financial LLMs; research question: can arithmetic consistency be enforced deterministically at inference time; engineering challenge: eliminating oracle leakage and label inconsistencies across heterogeneous benchmark sources
FinVerifyBench Auditable evaluation substrate for financial LLM numerical reasoning; research question: what constitutes a valid, reproducible evaluation of arithmetic consistency; engineering challenge: adversarial validation of answerability labels and cross-source defect repair
SATSentinel-X Ensemble telemetry anomaly detection for spacecraft systems; research question: how to maximize recall when anomalies are rare and labeled data scarce; engineering challenge: fusing heterogeneous detection signals with explainable attribution
DARPAN AI climate digital twin integrating satellite and reanalysis data streams; research question: how to assimilate multi-source observational data into coherent forecasting under uncertainty; engineering challenge: HSS-validated extreme event classification with proper uncertainty quantification
NeuroGenesis Evolutionary CTRNN agent simulation; research question: does the Baldwin Effect measurably accelerate fitness under constrained plasticity; engineering challenge: stable CTRNN dynamics under evolutionary perturbation
AlphaForgeX Systematic quant analytics platform; research question: can ML-generated alpha signals generalize across factor regimes; engineering challenge: reproducible backtesting pipeline with configurable factor definitions

Research Interests

LLM Evaluation & Reliability   ─── numerical reasoning, hallucination taxonomy,
                                    verification-augmented inference, benchmark engineering

AI for Earth Systems            ─── climate forecasting, ensemble data assimilation,
                                    satellite observation integration, geospatial ML

Anomaly Detection               ─── telemetry intelligence, time-series classification,
                                    rare-event detection, SHAP explainability

Evolutionary Computation        ─── CTRNN dynamics, Baldwin Effect,
                                    phenotypic plasticity, emergent behavior

Data Infrastructure             ─── evaluation pipeline engineering, streaming architectures,
                                    benchmark design, audit tooling

Tech Architecture

Domain Stack
Research & ML Python PyTorch scikit-learn XGBoost SHAP
LLM & Eval HuggingFace Mistral · LangChain · FinQA · DROP · FinVerifyBench
Data Engineering Airflow Kafka PostgreSQL Docker
Backend & Infra FastAPI Next.js Supabase Vercel
Climate & Geospatial Xarray · NetCDF · MOSDAC · IMDAA · INSAT-3D
Core Git Linux SQL · NumPy · Pandas · JavaScript

Selected Highlights

ACL Rolling Review · May 2026 Submission #2935 · FinNLP @ EMNLP · Under Review
ASTRA 2026 · Oral Presentation Spacecraft telemetry intelligence · IIST Thiruvananthapuram
IEEE Publications × 3 2 published · 1 under review · Applied ML and AI systems
FinVerifyBench Open evaluation infrastructure for financial LLM numerical reasoning
SATSentinel-X Ensemble anomaly detection system · F1 0.767 · AUC-ROC 0.943
ISRO BAH 2026 · PS5 AI-powered climate digital twin of India (DARPAN)
IIT Bombay Eureka! Startup competition finalist
IITM BS Data Science Dual-programme · Universal College of Engineering, Mumbai

Research Timeline

2026
 │
 ├─ [Jun]  HACKATHON   DARPAN → ISRO BAH 2026 (PS5) · Climate Digital Twin
 │                     Integrates IMDAA · INSAT-3D · MOSDAC · XGBoost · HSS validation
 │
 ├─ [May]  RESEARCH    ACL Rolling Review · Submission #2935
 │                     "Structured Numerical Hallucinations in Financial LLMs"
 │                     Target: FinNLP @ EMNLP
 │
 ├─ [Apr]  OPEN SOURCE FinVerifyBench v4 released
 │                     Adversarial label validation · 6 new eval metrics · audit pipeline
 │
 └─ [Mar]  PRESENTATION ASTRA 2026 · IIST Thiruvananthapuram
                        SATSentinel-X oral presentation before researchers and scientists

2025
 │
 ├─ [Dec]  PUBLICATION  SATSentinel-X → IEEE Conference
 │                      F1 0.767 · AUC 0.943 · Kafka + FastAPI deployment
 │
 ├─ [Oct]  ENGINEERING  FinVerify Terminal production build
 │                      Next.js · FastAPI · Supabase · Mistral-7B · HuggingFace Spaces
 │
 ├─ [Aug]  COMPETITION  IIT Bombay Eureka! · Startup Competition Finalist
 │
 ├─ [Jun]  PUBLICATION  IEEE Conference Paper #2 · Applied ML systems
 │
 ├─ [Mar]  PUBLICATION  IEEE Conference Paper #1 · Applied ML systems
 │
 └─ [Jan]  RESEARCH     FinVerify initiated · DVL architecture design begins

2024
 │
 ├─ [Sep]  EDUCATION    IITM BS Data Science · dual programme enrollment
 │
 └─ [Jun]  LEADERSHIP   GDG Joint Design Head · Technical community building

Building Now

class ActiveWork:
    finverify   = "FinVerify Terminal — production hardening, benchmark v4 finalization"
    research    = "ARR #2935 — review cycle in progress"
    hackathon   = "DARPAN @ ISRO BAH 2026 — July 1 idea submission deadline"
    horizon     = "Financial Constraint Graph · Claim Provenance Engine"

GitHub Statistics


Leadership

Role Organization
Oral Presenter ASTRA 2026, IIST Thiruvananthapuram
Startup Finalist IIT Bombay Eureka! Competition
Joint Design Head Google Developer Groups (GDG)
Member E-Cell, IIT Bombay

Research collaborations · Engineering discussions · MS Research — ETH Zürich · TU Munich · TU Delft

Working on LLM evaluation, AI for Earth systems, or data infrastructure? Reach out.

LinkedIn Email

Pinned Loading

  1. ALPHAFORGEX ALPHAFORGEX Public

    Python 1

  2. ClimateTwinIndia ClimateTwinIndia Public

    Jupyter Notebook 1

  3. finverify-llm finverify-llm Public

    Jupyter Notebook 3 3

  4. FinverifyBench FinverifyBench Public

    Python 1

  5. Neurogenesis_V1 Neurogenesis_V1 Public

    Python 1

  6. SATSentinel-X SATSentinel-X Public

    Python 1