A professional portfolio of five completed encoder-decoder projects spanning structured generation, vision-language captioning, automatic speech recognition, data-to-text generation, and grammatical error correction.
Each project is developed as an end-to-end case study with reproducible source code, task-appropriate evaluation, GPU-aware training, automated validation, public model publishing, transparent deployment decisions, and a publicly accessible application.
Portfolio status: 5 completed projects · 5 published models · 5 live applications
Repository owner: Anmol Tripathi
Deployment portfolio: 3 Hugging Face Static Spaces · 2 Vercel applications · 5 Hugging Face model repositories
I am a Quality Data Scientist building advanced Machine Learning and Applied AI systems that connect rigorous analysis with production-minded modeling, reproducible experimentation, responsible deployment, and usable applications.
This portfolio demonstrates practical experience across:
- Natural Language Processing;
- Computer Vision and vision-language generation;
- Automatic Speech Recognition;
- sequence-to-sequence modeling;
- structured generation;
- controlled text generation;
- model evaluation and ablation;
- GPU optimization;
- ONNX conversion and quantization;
- browser/static and full-stack deployment;
- GitHub Actions and reproducible engineering;
- responsible AI communication.
The repository demonstrates how encoder-decoder architectures can be adapted to different modalities and output structures rather than being treated as one generic text-generation technique.
Across the five projects, the portfolio covers:
- text → structured SQL generation;
- image → natural-language caption generation;
- audio → text transcription;
- tables/KPIs → executive narrative generation;
- noisy text → corrected text generation.
Each project generally contains:
- a clearly defined analytical or business problem;
- reproducible data preparation and validation;
- a task-appropriate encoder-decoder architecture;
- baseline or cross-model comparison where meaningful;
- training, validation, and held-out evaluation;
- task-specific evaluation metrics;
- modular source code;
- saved metadata, reports, charts, and deployment assets;
- automated tests or GitHub Actions validation;
- a public application;
- a separately published model on Hugging Face;
- transparent limitations and responsible-use guidance.
| No. | Project | Encoder-Decoder Problem | Primary Deployment | Live Application | Published Model |
|---|---|---|---|---|---|
| 1 | Schema-Aware Text-to-SQL | Natural language + database schema → validated SQLite | Vercel | Live App | Model |
| 2 | Image Caption Generation | Image → multiple natural-language captions | Hugging Face Static | Live App | Model |
| 3 | Streaming Speech Recognition with Whisper | Audio / microphone input → transcription | Hugging Face Static | Live App | Model |
| 4 | Data-to-Text Executive Report Generator | KPI tables / CSV / JSON → controlled executive narrative | Vercel | Live App | Model |
| 5 | Grammar Error Correction with FLAN-T5 | Noisy English text → corrected text + tracked edits | Hugging Face Static | Live App | Model |
| Coverage Area | Demonstrated Through |
|---|---|
| Structured text generation | Project 01 — schema-aware Text-to-SQL |
| Vision-language generation | Project 02 — image captioning |
| Audio sequence modeling | Project 03 — Whisper speech recognition |
| Controlled business-language generation | Project 04 — executive report generation |
| Text-to-text correction | Project 05 — grammatical error correction |
| Fine-tuning and model adaptation | All five projects |
| Baseline and model comparison | Projects 01–05 |
| GPU-aware training | RTX 5090 workflows across the portfolio |
| Quantization / ONNX analysis | Projects with browser or deployment optimization experiments |
| Static ML deployment | Projects 02, 03, and 05 |
| Full-stack deployment | Projects 01 and 04 |
| Model publishing | All five projects |
| Automated validation | Project-specific GitHub Actions workflows |
| Responsible AI communication | All five projects |
Project 01 — Schema-Aware Text-to-SQL converts a natural-language business question and relational schema into SQLite, validates it before execution, rejects unsafe operations, and exposes schema-linking and execution evidence.
Key themes:
- schema serialization;
- structured generation;
- SQL validation;
- safe read-only execution;
- schema linking;
- model comparison;
- execution-based evaluation;
- responsible fallback behavior.
Project 02 — Image Caption Generation combines a visual encoder with an autoregressive text decoder to produce image captions and compare decoding strategies.
Key themes:
- vision encoder + language decoder;
- image preprocessing;
- multi-caption generation;
- greedy versus beam search;
- caption diversity;
- attention/token-generation inspection;
- vision-language evaluation;
- browser-oriented deployment.
Project 03 — Streaming Speech Recognition with Whisper applies an encoder-decoder speech model to microphone and uploaded audio, exposing transcription, timestamps, detected language, chunk-level progress, and robustness evaluation.
Key themes:
- audio feature extraction;
- Whisper encoder-decoder modeling;
- transcription;
- timestamps;
- chunking;
- noisy-audio evaluation;
- latency and real-time factor;
- static application deployment.
Project 04 — Data-to-Text Executive Report Generator converts structured KPI data into an executive narrative while checking whether generated statements are numerically supported by the source table.
Key themes:
- table-to-text generation;
- controlled business language;
- content selection;
- source-cell attribution;
- numerical verification;
- hallucination control;
- executive communication;
- Vercel deployment.
Project 05 — Grammar Error Correction with FLAN-T5 fine-tunes a text-to-text encoder-decoder model for correction while explicitly measuring over-correction, semantic preservation, and preservation of important entities.
Key themes:
- FLAN-T5 fine-tuning;
- ERRANT F0.5;
- GLEU;
- semantic similarity;
- tracked edits;
- error categories;
- over-correction benchmarking;
- preservation of numbers, entities, and technical terms.
An end-to-end natural-language-to-SQL application where users provide a relational schema and business question, generate SQL, validate it in a read-only safety layer, execute approved queries, and inspect the result.
Key evaluation: execution accuracy · exact match · valid-SQL rate · schema-linking quality · unsafe-query rejection · latency
Selected model repository: anmol-unitmole/schema-aware-text-to-sql-codet5p-770m
A vision-language captioning system that generates multiple captions for uploaded images and exposes decoding controls and model-quality evidence.
Key evaluation: BLEU-1/4 · METEOR · ROUGE-L · CIDEr · CLIPScore · caption diversity · model size · latency
Model repository: anmol-unitmole/image-caption-generation-vision-encoder-decoder-model
A browser-oriented Whisper speech-recognition application supporting microphone recording and audio-file upload with timestamps, detected language, chunk-level progress, and robustness analysis.
Key evaluation: Word Error Rate · Character Error Rate · real-time factor · latency · memory usage · robustness across noise/accent slices
Model repository: anmol-unitmole/streaming-speech-recognition-whisper-encoder-decoder-model
A controlled data-to-text application that accepts structured KPI tables and produces an executive report covering movement, top contributors, anomalies, and risks while validating numerical claims against source data.
Key evaluation: PARENT · BLEU · ROUGE-L · BERTScore · content-selection F1 · numerical factuality · hallucination rate · coverage · latency
Model repository: anmol-unitmole/data-to-text-executive-report-generator-flan-t5-base
A precision-first grammatical error correction system that generates corrected text while surfacing tracked edits, error categories, preservation checks, batch correction, and over-correction warnings.
Key evaluation: ERRANT F0.5 · precision · recall · GLEU · semantic similarity · sentence accuracy · over-correction · latency
Selected model repository: anmol-unitmole/grammar-correction-flan-t5-base
| Architecture / Pattern | Demonstrated Through |
|---|---|
| Code-oriented encoder-decoder generation | Project 01 |
| Vision encoder + autoregressive text decoder | Project 02 |
| Speech encoder-decoder Transformer | Project 03 |
| T5 / FLAN-T5 controlled data-to-text generation | Project 04 |
| FLAN-T5 text-to-text correction | Project 05 |
| Beam-search generation | Multiple projects |
| Parameter-efficient / controlled fine-tuning | Task-dependent experiments |
| Mixed-precision GPU training | Portfolio training workflows |
| ONNX / quantization experiments | Deployment-oriented projects |
| Static browser applications | Projects 02, 03, 05 |
| Full-stack web applications | Projects 01, 04 |
Each project uses evaluation methods aligned with the task rather than relying on one universal metric.
| Project | Evaluation Methods |
|---|---|
| Text-to-SQL | Execution accuracy, exact match, valid-SQL rate, schema linking, unsafe-query rejection, latency |
| Image Captioning | BLEU, METEOR, ROUGE-L, CIDEr, CLIPScore, diversity, latency |
| Speech Recognition | WER, CER, real-time factor, latency, memory, noise/accent robustness |
| Data-to-Text | PARENT, BLEU, ROUGE-L, BERTScore, content selection, factuality, hallucination, coverage |
| Grammar Correction | ERRANT F0.5, precision, recall, GLEU, semantic similarity, sentence accuracy, over-correction, latency |
- Exact match can be too strict for many generation tasks.
- A fluent generated sentence can still be factually unsupported.
- Valid syntax does not guarantee correct execution semantics.
- Caption similarity does not fully measure visual grounding.
- Low WER does not describe all robustness conditions.
- High grammar-correction recall can come at the cost of harmful over-editing.
- Latency and model size matter when deployment is part of the engineering goal.
- Baseline comparison is necessary to show whether adaptation creates meaningful value.
The repository demonstrates the complete path from problem definition to public application:
- problem formulation;
- data acquisition or generation;
- data validation;
- preprocessing;
- deterministic splitting;
- model selection;
- training and fine-tuning;
- baseline comparison;
- evaluation;
- error analysis;
- GPU optimization;
- reusable inference code;
- model publishing;
- ONNX / quantization experiments;
- testing;
- CI validation;
- frontend/application development;
- public deployment;
- responsible-use documentation.
The projects do not assume that the newest or largest model is automatically the best.
Across the portfolio, selection decisions consider combinations of:
- task accuracy;
- precision and recall;
- execution behavior;
- semantic similarity;
- factuality;
- preservation metrics;
- robustness;
- model size;
- latency;
- deployment feasibility;
- failure analysis.
Repository practices include:
- modular source code;
- reusable preprocessing;
- deterministic experiments;
- saved configuration and metadata;
- test suites;
- project-specific GitHub Actions;
- large-model artifacts separated from source control;
- model weights published on Hugging Face;
- public deployment assets separated from local training environments;
- explicit limitation and fallback disclosures.
| Platform | Projects | Purpose |
|---|---|---|
| Hugging Face Static Spaces | 3 | Image captioning, speech recognition, grammar correction |
| Vercel | 2 | Text-to-SQL and data-to-text applications |
| Hugging Face Model Hub | 5 | Versioned trained/fine-tuned model artifacts |
This demonstrates selecting a deployment platform according to runtime requirements instead of forcing every model into the same hosting pattern.
The repository is organized as a five-project monorepo:
encoder-decoder-projects/
├── .github/
│ └── workflows/
│ └── project-specific CI workflows
│
├── 01-schema-aware-text-to-sql-encoder-decoder/
├── 02-image-caption-generation-vision-encoder-decoder/
├── 03-streaming-speech-recognition-whisper-encoder-decoder/
├── 04-data-to-text-executive-report-generator/
├── 05-grammar-error-correction-t5-encoder-decoder/
│
├── .gitignore
├── LICENSE
└── README.md
A typical project may contain:
project-folder/
├── configs/
├── data/
├── docs/
├── models/
├── notebooks/
├── outputs/
├── scripts/
├── src/
├── tests/
├── deployment assets
├── DATASET_CARD.md
├── MODEL_CARD.md
├── README.md
├── requirements.txt
└── supporting metadata and reports
The exact files differ by task, but the standards remain consistent:
- reproducible workflows;
- modular code;
- task-appropriate evaluation;
- public deployment;
- automated validation;
- safe repository practices;
- transparent limitations;
- portfolio-quality documentation.
The repository uses project-specific GitHub Actions workflows so that each project can validate the technologies and artifacts relevant to its own stack.
Depending on the project, CI validates:
- Python syntax;
- Python unit tests;
- JavaScript/frontend tests;
- static application builds;
- required deployment files;
- JSON/JSONL validity;
- model configuration;
- README assets;
- accidental oversized files;
- accidental model-weight inclusion;
- safe project structure.
Every completed project has two public endpoints:
| Project | Live Application | Hugging Face Model |
|---|---|---|
| 01 — Text-to-SQL | Vercel | CodeT5+ Model |
| 02 — Image Captioning | HF Space | Captioning Model |
| 03 — Whisper ASR | HF Space | Whisper Model |
| 04 — Executive Report Generator | Vercel | FLAN-T5 Model |
| 05 — Grammar Correction | HF Space | FLAN-T5 Model |
Large trained-model weights are intentionally hosted on Hugging Face Model Hub rather than duplicated in normal Git history.
Each project contains detailed setup instructions in its own README.
git clone https://github.com/unit-mole/encoder-decoder-projects.git
cd encoder-decoder-projectscd 05-grammar-error-correction-t5-encoder-decoderReplace the folder name with whichever project you want to run.
Windows
py -3.12 -m venv .venv
call .venv\Scripts\activate.batmacOS / Linux
python3.12 -m venv .venv
source .venv/bin/activatepython -m pip install --upgrade pip
python -m pip install -r requirements.txtProjects may run through:
- Python scripts;
- Jupyter notebooks;
- local static web servers;
- JavaScript applications;
- Vercel applications;
- Hugging Face Static deployment assets.
The individual project README is the source of truth for project-specific setup.
This repository is intended for education, experimentation, technical demonstration, and portfolio presentation.
General limitations include:
- generated SQL can be syntactically valid but semantically incorrect;
- image captions can omit or misidentify visual details;
- speech-recognition models can fail on noise, accents, domain terms, or poor audio;
- generated executive narratives can misstate source data if not verified;
- grammar correction can alter meaning or unnecessarily rewrite correct text;
- model confidence indicators are not guaranteed probabilities;
- benchmark results should not be generalized beyond the evaluated configuration;
- portfolio models are not automatically production-ready;
- public fallback paths are disclosed when the full trained model is not executed by the deployed frontend;
- no application should be used as the sole basis for safety-critical, legal, medical, financial, or compliance decisions.
Important outputs should be reviewed against trusted data, domain knowledge, and additional validation.
| Area | Demonstrated Through |
|---|---|
| Structured generation | Project 01 |
| Schema-aware prompting | Project 01 |
| Read-only execution safety | Project 01 |
| Vision-language generation | Project 02 |
| ViT / visual encoder workflows | Project 02 |
| Autoregressive text decoding | Project 02 |
| Automatic speech recognition | Project 03 |
| Whisper encoder-decoder modeling | Project 03 |
| Audio chunking and timestamps | Project 03 |
| Controlled table-to-text generation | Project 04 |
| Numerical factuality checks | Project 04 |
| Executive narrative generation | Project 04 |
| Grammatical error correction | Project 05 |
| ERRANT / GLEU evaluation | Project 05 |
| Over-correction evaluation | Project 05 |
| Model publishing | All five projects |
| GPU training and optimization | All five projects |
| Public application deployment | All five projects |
| CI/CD | Project-specific workflows across the repository |
Encoder-Decoder Transformers · PyTorch · Hugging Face Transformers · CodeT5 · CodeT5+ · FLAN-T5 · VisionEncoderDecoderModel · ViT · GPT-2 · Whisper · Natural Language Processing · Computer Vision · Vision-Language Modeling · Automatic Speech Recognition · Text-to-SQL · Data-to-Text Generation · Grammatical Error Correction · Structured Generation · Beam Search · Schema Linking · SQLite Safety · BLEU · METEOR · ROUGE-L · CIDEr · CLIPScore · WER · CER · PARENT · BERTScore · ERRANT F0.5 · GLEU · Semantic Similarity · Over-Correction Evaluation · Model Ablation · BF16 · TF32 · Gradient Accumulation · Gradient Checkpointing · ONNX · INT8 Quantization · Hugging Face Model Hub · Hugging Face Spaces · Vercel · JavaScript · HTML · CSS · Testing · GitHub Actions · CI/CD · Responsible AI Communication
One-line description: Five end-to-end encoder-decoder projects spanning Text-to-SQL, image captioning, Whisper speech recognition, controlled data-to-text generation, and FLAN-T5 grammatical error correction—with public applications and published Hugging Face models.
Pinned repository description: Professional encoder-decoder portfolio featuring five deployed projects across structured SQL generation, vision-language captioning, Whisper ASR, executive data-to-text reporting, and grammar correction, with task-specific evaluation, RTX GPU training, ONNX optimization, Hugging Face model publishing, Vercel/Hugging Face deployment, and project-specific CI.
This portfolio connects naturally to a Quality Data Scientist background because the same modeling and evaluation principles can support:
- quality-case analytics;
- complaint and CAPA data;
- supplier-quality reporting;
- manufacturing and inspection narratives;
- technical documentation;
- speech-to-text documentation workflows;
- image and inspection descriptions;
- safe analytics interfaces;
- executive quality reporting;
- professional communication quality.
This repository is distributed under the MIT License.
Individual models, datasets, and third-party libraries remain subject to their original licenses and usage conditions.
Anmol Tripathi
Quality Data Scientist | Data Science | Machine Learning | Applied AI | Natural Language Processing | Computer Vision | Analytics Engineering | Quality Analytics