Gen AI Engineer · LLM Researcher
Studying reasoning mechanisms & tokenization in Large Language Models
- 🔬 Gen AI Engineer & Researcher , focused on the inner workings of LLMs.
- 🧩 I study reasoning mechanisms and tokenization — building tokenizers and generative models from scratch to understand why they work, not just that they work.
- 🛠️ I've implemented from-scratch BPE tokenizers, diffusion models, and benchmarked AI reasoning paradigms across multiple agent frameworks.
- 📚 Always reading papers and turning them into working, reproducible code.
| Project | Description | Tech |
|---|---|---|
| 🔤 BPE Tokenization & Compression Analysis | From-scratch Byte Pair Encoding tokenizer with a multilingual compression-ratio study (EN/DE/ES/FR) benchmarked against GPT-2/3.5/4. Includes a full research report. | Python tiktoken NLP |
| 🌀 Yi Syllable Diffusion | Unconditional DDPM that generates Yi-script syllables — U-Net with v-prediction + zero-SNR, with a research report and pretrained model. | PyTorch Diffusion |
| ⚖️ Indian Criminal Law RAG Agent | Fully local, open-source Retrieval-Augmented Generation assistant for Indian criminal law (IPC, CrPC, BNS, BNSS). Multi-agent CrewAI pipeline delivering grounded, citation-aware answers, with a full technical report. | CrewAI Qdrant Ollama |
| 📊 TabuQuest | Natural-language question answering over structured/tabular datasets using LLMs. | Pandas LLMs |
Focus areas: LLM tokenization · Generative models (diffusion) · Agentic AI · Reasoning in LLMs
- 📧 Email: pratikrsonawane7@gmail.com
- 💼 Open to collaboration on LLM research, tokenization, and generative AI projects.
Building AI from first principles — one tokenizer at a time.