Skip to content
cksajilPublic

About

No description, website, or topics provided.

Resources

Stars

1 star

Watchers

1 watching

Forks

Latest commit

 

History

13 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 

Repository files navigation

Hi, I'm Sajil 👋

Applied ML Engineer · Speech AI for Malayalam · LLM systems that run on a CPU

I build ML systems end to end and measure them honestly: a Malayalam voice assistant (speech in, speech out), the ASR benchmark behind it, local LLM apps, and vision and IoT models running on edge hardware. PhD in Computational Biology · ML Engineer & Educator at ICT Academy of Kerala · writing at intuitivetutorial.com

LinkedIn Hugging Face Kaggle Blog Email


🗣️ Malayalam speech AI, end to end

Most Malayalam speakers will never have a GPU, so everything here is built and measured on 2 CPU cores.

🎙️ speech ─► ASR (my Whisper fine-tune / IndicConformer) ─► LangGraph agent + hybrid RAG ─► my VITS voice ─► 🔊 speech

A voice assistant that answers Malayalam questions about panchayat services (K-SMART), the weather and the news.

  • LangGraph agent with explicit routing, BM25 + BGE-M3 hybrid retrieval, and Gemma running locally through llama.cpp
  • Reliability by design: fees are calculated in code, not by the LLM; the model must answer UNSURE when the retrieved passage doesn't contain the answer, and that becomes a refusal; weather and news numbers go out through templates
  • Ingestion pipeline: downloads panchayat citizen charters, runs Malayalam OCR with Tesseract and turns them into ~600 passages with page citations, plus a checker that verifies knowledge-base entries against the source pages
  • Error-driven: routing, retrieval and ASR-correction fixes come from reading real conversation logs, and are tested by an eval suite (routing, hit@3, refusals, 100-question answer eval, multi-turn conversations); a whole spoken turn takes 1.7–6.2 s on a shared CPU

malayalam-asr-eval · Benchmark

A reproducible, leakage-free benchmark of 6 Malayalam ASR models on accuracy and 2-core CPU speed.

  • WER/CER with bootstrap 95% CIs, a column that excludes each model's own training corpora, RTF and peak RAM; one transcribe() interface over Whisper, CTranslate2, IndicConformer and wav2vec2; CI-tested adapters
  • Findings: IndicConformer matches my Whisper fine-tune's accuracy at ~14× the speed; one model's 39.7% WER becomes 52.9% on corpora it didn't train on; a CTranslate2 run silently truncated transcripts at 224 tokens
  • Choosing per clip between two recognisers with a character LM: 31.2% → 26.4% WER (significant, no retraining). I deployed it, then reverted it after measuring that it slowed the voice assistant

Also in the stack: my fine-tuned whisper-small-malayalam-v2 and its training corpus, the assistant's voice (mms-tts-malayalam-sajil, a VITS fine-tune), and IndicF5-TTS-Pipeline: zero-shot voice cloning in 11 Indian languages, sped up on CPU (EPSS step schedules, cached preprocessing, core pinning, bf16) from a real-time factor of ~83× down to single digits. Write-ups: leakage in my test set · a bug in my own evaluation

PyTorch Transformers Whisper CTranslate2 IndicConformer ONNX Runtime LangGraph llama.cpp BM25 BGE-M3 VITS Tesseract


🤖 LLM apps and models from scratch

Project What it shows
🎓 AI Assignment Grader · video Grades Jupyter notebooks against a rubric with a local LLM (Qwen2.5-Coder via Ollama): typed JSON output, score guardrails in Python, optional sandboxed code execution; Docker + GitHub Actions deploy to HF Spaces
🔍 OmniSense Semantic search inside videos: faster-whisper transcription → MiniLM embeddings → FAISS; click a hit and the player jumps to that moment
💬 ICTSahayi GPT-2 implemented in plain PyTorch and instruction fine-tuned on an FAQ, evaluated against the base model and a retrieval baseline. An honest result: fine-tuning learned the format, not the facts, which makes the case for RAG
✍️ MiniGPT-JAX · demo Decoder-only transformer built from scratch in JAX / Flax NNX and trained on 200k stories on a free GPU, with pre-tokenised, memory-mapped batches to remove the input bottleneck

👁️ Vision, audio and edge ML

Project What it shows
🚗 Dash-Cam Segmenter DeepLabV3+ and U-Net for Indian road scenes (20 classes, mean IoU 0.659) on my own Kaggle dataset; video or YouTube URL in, segmented video out
🌧️ Acoustic Rain Gauge Estimates rainfall from sound (ICFOSS R&D team): Raspberry Pi data acquisition, LSTM on STFT features (R² 0.97), LoRaWAN uplink (LoraWANPi) and a live Grafana dashboard. Published at IEEE RAICS 2024
✋ Hand Gesture Recognition Touch-free control of an information display: a 3D CNN trained on Jester (8 gestures) reads live webcam clips and drives the pages over Flask-SocketIO; ONNX export for lighter runtimes

Dash-cam segmentation output

🎓 Teaching ML


🧰 Toolkit

Area Tools
LLMs & agents LangGraph, Ollama, llama.cpp, Hugging Face Transformers, RAG (BM25, dense, hybrid), FAISS, structured JSON output
Speech Whisper fine-tuning, CTranslate2 / faster-whisper, IndicConformer, wav2vec2, VITS and F5-TTS, WER/CER evaluation
Deep learning PyTorch, JAX / Flax, TensorFlow / Keras, scikit-learn
Vision & edge OpenCV, segmentation, 3D CNNs, Raspberry Pi, LoRaWAN
Shipping Docker, Gradio, Flask, Hugging Face Spaces, GitHub Actions, pytest, Linux

📚 Research

  • Edge Analytics Enabled Acoustic Rain Gauge, IEEE RAICS 2024
  • Active Noise Control, International Journal of Biomedical and Clinical Engineering

🔭 Right now

  • Making the Malayalam voice assistant faster and more accurate on a shared CPU
  • Measuring, not guessing: leakage-free test sets, confidence intervals, error analysis from real logs
  • Open to Applied ML / Speech AI / GenAI engineering roles and to collaborations on Indic-language AI

About

No description, website, or topics provided.

Resources

Stars

1 star

Watchers

1 watching

Forks

Releases

Packages

Contributors