Ankahi (अनकही) — Backend, Core System & Testing Specification
Model Deployment · Pipeline Conversion · Robustness Testing · Benchmarking
AGENT INSTRUCTION: This document contains NO finished code. It contains precise, actionable instructions, resource pointers, search queries, and architectural decisions that you must execute step by step. For every section marked [SEARCH REQUIRED], you must query the listed resources and documentation before proceeding — do not rely on memory or assumptions, as MediaPipe, LiteRT, and Unsloth APIs change frequently. For every benchmark, you must produce actual measured numbers — no fabricated figures. This document also defines the exact format for the results report that must accompany the Kaggle submission.
PART 1: THE CRITICAL PATH — .litertlm CONVERSION
This is the single biggest blocker to a working demo. Without the on-device binary, the Flutter app has nothing to run. Do this before anything else.
1.1 Overview: What the conversion pipeline does
Unsloth fine-tuned Gemma 4 E4B
(stage1 + stage2 adapter, merged)
↓
[bitsandbytes INT8 quantization]
(merge_and_quantize.py)
↓
Hugging Face safetensors checkpoint
(float32 or int8, standard format)
↓
[MediaPipe Model Maker conversion]
(convert_to_litertlm.py)
↓
.task or .litertlm flatbuffer file
(~2.3 GB for E4B INT8)
↓
Sideloaded to Android via adb push
(or bundled in APK assets for small adapters)
↓
MediaPipe GenAI / LiteRT-LM runtime
running on Android CPU/GPU/NPU
1.2 Environment Setup
[SEARCH REQUIRED] Before running anything, read these resources:
- Search:
"mediapipe model maker" "LLM inference" "gemma" site:ai.google.dev - URL to read:
https://ai.google.dev/edge/mediapipe/solutions/genai/llm_inference - URL to read:
https://ai.google.dev/edge/mediapipe/solutions/genai/llm_inference/android - Search:
"mediapipe-model-maker" pip install python version compatibility 2025 2026 - Search:
"LiteRT" "LiteRT-LM" gemma conversion site:github.com/google
Why this matters: mediapipe-model-maker has strict Python and CUDA version dependencies that change between releases. You must verify the current requirements before installing. Installing the wrong version wastes hours.
Recommended environment (verify against current docs):
Python: 3.10 or 3.11 (verify with mediapipe docs — NOT 3.12)
CUDA: 11.8 or 12.1 (check current compatibility matrix)
PyTorch: The version compatible with your CUDA
mediapipe: Latest stable (search: "mediapipe pypi latest version")
mediapipe-model-maker: Must match mediapipe version exactly
Recommended: Use a fresh conda environment
conda create -n ankahi_convert python=3.10
conda activate ankahi_convert
# Then follow exact installation from current mediapipe docs
[SEARCH REQUIRED] For the H100 server specifically:
- Search:
mediapipe-model-maker H100 installation cuSPARSELt compatibility - Cross-reference with the environment cascade issue documented in the Ankahi handover — the H100 server had pyannote/CUDA/cuSPARSELt conflicts. The conversion environment should be kept SEPARATE from the training environment.
- Run conversion in its own conda environment. Do not mix with the Unsloth training environment.
1.3 Step-by-Step Conversion Pipeline
[SEARCH REQUIRED] For each step below, search for the current API before running:
Step A: Verify the merged checkpoint
The merge_and_quantize.py script should have already produced a merged safetensors checkpoint. Verify:
# Check the output directory — you need to confirm these files exist:
# - config.json
# - tokenizer.json / tokenizer_config.json / tokenizer.model
# - model-00001-of-XXXXX.safetensors (and remaining shards)
# - generation_config.json
# Verification check:
from transformers import AutoModelForCausalLM, AutoTokenizer
# Load the checkpoint to verify it's valid before conversion
# If this fails, the merge step needs to be rerun
[SEARCH REQUIRED] Search: "Gemma 4" "safetensors" "merge lora" "unsloth" verification checklist
Step B: INT8 Quantization verification
The handover doc specifies bitsandbytes INT8. Verify the quantization is correctly applied:
# Check quantization was applied:
# model.config should show quantization_config with load_in_8bit=True
# Run a quick forward pass to confirm no NaN outputs:
# inputs = tokenizer("WANT WATER COLD", return_tensors="pt")
# outputs = model.generate(**inputs, max_new_tokens=20)
# print(tokenizer.decode(outputs[0]))
# Expected: Hindi sentence, no garbage tokens, no infinite repetition
Known issue from handover: INT8 quantization causes repetition loops on short inputs. Confirm repetition_penalty=1.2 is in generation_config.json. If not, add it before conversion — it must be baked into the model config for on-device inference.
Step C: Convert to LiteRT-LM format
[SEARCH REQUIRED] The exact API for this changes between MediaPipe releases. Before running convert_to_litertlm.py, search:
"mediapipe model maker" "llm" "gemma" "convert" "python" site:ai.google.dev 2025 OR 2026"LiteRT" "flatbuffer" ".task" gemma 4 conversion tutorial- Check:
https://github.com/google-ai-edge/mediapipe-samplesfor the most recent LLM inference examples
The conversion call will look approximately like this (verify against current docs — do not run this verbatim):
# APPROXIMATE — verify exact API from current mediapipe docs before running
import mediapipe as mp
# OR: from mediapipe.tasks.python.genai import converter as genai_converter
# Key parameters to understand and configure:
# - model_path: path to your merged safetensors checkpoint directory
# - output_path: where to write the .litertlm or .task file
# - output_type: "TFLITE" or "LITERTLM" — check current options
# - quantization_type: "w8a8" (weights 8-bit, activations 8-bit) or "w4a8" — w8a8 matches your bitsandbytes INT8
# - backend: "CPU" or "GPU" — verify Android GPU support for your target devices
# - lora_rank: 8 (must match Stage 2 adapter rank)
[SEARCH REQUIRED] Critical constraint documented in handover: LoRA rank must be 4 or 8 for MediaPipe GenAI compatibility. This is already set. But verify the conversion tool's current LoRA rank support:
- Search:
"mediapipe" "lora" "rank 8" android "llm inference" compatibility 2025 OR 2026
Step D: Adapter conversion (separate files)
The base model and persona adapters are separate files. The base model is the large .litertlm file. Each 30MB adapter must also be converted to a format MediaPipe can load at runtime.
[SEARCH REQUIRED]:
- Search:
mediapipe genai "lora adapter" android "load at runtime" ".task" OR ".bin" format - Search:
"mediapipe-model-maker" "lora" "convert adapter" python example - Search:
LiteRT-LM "hot-swappable" adapter loading android kotlin
Expected output structure:
ankahi_bundle/
├── model/
│ ├── ankahi_base_e4b_int8.litertlm (~2.3 GB)
│ └── symlinks → point to above (use cp -L when moving across filesystems!)
├── adapters/
│ ├── arjun_v1.ankahi (~30 MB, converted LoRA)
│ ├── ananya_v1.ankahi
│ ├── priya_v1.ankahi
│ ├── rohan_v1.ankahi
│ └── zara_v1.ankahi
└── voices/
└── (AI4Bharat TTS voice clone files, one per persona)
Step E: Quick validation on-device
After conversion, validate before spending time on the Flutter integration:
# Use adb to push and test directly
adb push ankahi_base_e4b_int8.litertlm /sdcard/ankahi/model/
adb push arjun_v1.ankahi /sdcard/ankahi/adapters/
# [SEARCH REQUIRED] Search for:
# "mediapipe" "llm inference" "android" "command line test" OR "standalone test"
# There may be a MediaPipe demo APK you can sideload for quick validation
# before building the full Flutter app
[SEARCH REQUIRED]:
- Search:
"mediapipe" "llm_inference" android demo app github - Search:
google-ai-edge mediapipe-samples llm inference android kotlin example - URL:
https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/llm_inference/android
1.4 AI4Bharat svara-TTS Integration
[SEARCH REQUIRED] This is listed as "Next Step #3" in the handover but should be treated as highest priority after model conversion.
Resources to read:
- Search:
"AI4Bharat" "Indic TTS" OR "IndicTTS" OR "svara" python pip install 2025 2026 - URL:
https://github.com/AI4Bharat/Indic-TTS— check current state, branches, releases - Search:
AI4Bharat TTS "voice cloning" python inference minimal example - Search:
AI4Bharat TTS android deployment OR "mobile" OR "on-device"
Key questions to answer from the search:
- Does AI4Bharat's current TTS support on-device Android inference, or does it require a server?
- If server-required: Is there a small model variant suitable for on-device use?
- What format does the voice cloning model accept? (wav file? sampling rate? duration?)
- What languages are reliably supported right now?
Fallback plan if AI4Bharat TTS cannot run on-device:
- Option A: Run TTS on a lightweight server (FastAPI) that the device calls over local WiFi only (not cloud). This still works for the demo.
- Option B: Use a pre-synthesized voice library with the cloned voice (record 50–100 phoneme combinations, stitch on-device). Cruder, but zero-latency.
- Option C: Use Android's built-in TTS engine (
TextToSpeech) as a fallback only — it won't sound like the parent, but it ensures the app works.
[SEARCH REQUIRED] For Option A (if needed):
- Search:
AI4Bharat TTS "FastAPI" OR "flask" serve REST API example - Search:
AI4Bharat TTS model size "hindi" inference latency
PART 2: TESTING STRATEGY & ROBUSTNESS
2.1 Testing Philosophy
The Ankahi system has three distinct testing domains:
Domain 1: ML Pipeline Tests (pytest)
→ Model output quality, adapter isolation, safety layer
Domain 2: Inference Pipeline Tests (device benchmarking)
→ Latency, memory, battery, hardware compatibility
Domain 3: Application Tests (Flutter)
→ Widget tests, integration tests, accessibility tests
The handover reports 36/36 unit tests passing. This section expands that into a full test strategy for Kaggle submission evidence.
2.2 ML Pipeline Tests (src/ankahi/eval/)
The existing test suite covers: BLEU-4 4-gram potential issue (fixed), BLEU-4 smoothing, chrF++ calculation, IndicSBERT heatmap generation. Expand it as follows:
Test Suite A: Output Quality Tests
# File: tests/test_output_quality.py
# Test 1: Language accuracy per language
# For each of 5 supported languages, run 20 test inputs from test.jsonl
# Assert chrF++ score > threshold (set threshold based on Stage 1 training result)
# Test 2: Code-switching fidelity
# Input: Hinglish pictogram sequences (e.g., "WANT + MOVIE + NIGHT + WITH + FRIENDS")
# Expected output: natural Hinglish ("Yaar, raat ko movie dekhni hai")
# Assert: output contains both Hindi and English words (simple heuristic check)
# Test 3: Persona isolation (the 5×5 heatmap test)
# For each of 5 adapters: run 10 persona-specific inputs
# Compute IndicSBERT similarity between:
# - Arjun adapter output for Arjun-specific input
# - Ananya adapter output for same input
# Assert: diagonal scores > 0.85, off-diagonal < 0.65
# This is the "no leakage" test — adapters must not sound like each other
# Test 4: Repetition loop detection
# Run 50 short inputs (1–3 pictograms)
# Assert: no output repeats any phrase more than 2 times
# Assert: all outputs are < 25 tokens (AAC sentences should be short)
# Test 5: chrF++ vs BLEU-4 comparison
# For morphologically rich languages (Hindi, Tamil, Bengali):
# Assert: chrF++ score > BLEU-4 score (validates the metric choice)
# Test 6: Safety layer completeness
# 50 self-harm trigger inputs (from the safety test set)
# Assert: 100% refusal rate (0 completions on forbidden content)
# 20 legitimate anger/pain inputs
# Assert: 100% completion rate (safety layer must NOT block these)
Test Suite B: Adapter Loading Tests
# File: tests/test_adapter_switching.py
# Test 1: Cold load time
# Measure time to load each adapter from disk to GPU memory
# Assert: load time < 5 seconds per adapter on H100
# Test 2: Hot swap correctness
# Load Arjun adapter → generate sentence
# Swap to Ananya adapter → generate same sentence
# Assert: sentences are different (adapter is actually being swapped)
# Test 3: Memory after swap
# After 5 consecutive adapter swaps:
# Assert: GPU memory usage returns to baseline (gc.collect + empty_cache working)
# Test 4: Rank constraint validation
# Programmatically verify each Stage 2 adapter has rank=8
# Assert: all adapters fail to load if rank > 8 (simulated)
Test Suite C: Audio Tower Tests
# File: tests/test_audio_tower.py
# Test 1: Dysarthric speech disambiguation
# Scenario: pictogram input says "DOG", audio signal suggests "DRINK" (fricative sounds)
# Assert: output sentence is drink-related, not dog-related
# Use pre-recorded test audio clips from the TORGO corpus augmentation
# Test 2: No audio input baseline
# Assert: system works correctly with audio input disabled
# The audio tower is an enhancement, not a requirement
# Test 3: Ambient noise robustness
# Inject synthetic noise (café noise, TV audio) into audio input
# Assert: output quality does not degrade by more than 10% chrF++ vs clean audio
2.3 Device Benchmarking Tests
This is the most important section for the Kaggle submission. You need real numbers.
Hardware Setup
[SEARCH REQUIRED] Before benchmarking:
- Search:
"MediaPipe" "LLM inference" Android benchmark "prefill speed" "decode speed" measurement - Search:
LiteRT android profiling tool GPU CPU NPU benchmark script - URL:
https://ai.google.dev/edge/mediapipe/solutions/genai/llm_inference#performance_benchmarks
Target devices for benchmarking (use what you have, document what you used):
Tier 1 (target market): Any device with Snapdragon 680 or similar (e.g., Realme Narzo 50, ~₹12,000)
Tier 2 (mid-range): Any device with Snapdragon 778G or similar (e.g., Poco X5 Pro, ~₹25,000)
Tier 3 (development): Any available Android tablet/phone you have access to
Note: Document the exact device model, Android version, and RAM in results.
If no physical device is available:
- Use Android emulator with GPU acceleration (note this in results as "emulated, not representative")
- Or use MediaPipe's published benchmarks and cite them as reference
Benchmark Test Suite D: Latency
# File: benchmarks/latency_benchmark.py
# Metric 1: Time to First Token (TTFT)
# = Time from input submission to first output token
# Measure across 3 input lengths: 1 tile (short), 3 tiles (medium), 6 tiles (long)
# Run 20 trials each, report mean ± std
# Metric 2: Total generation time
# = Time from input to complete sentence output
# Target: < 200ms (from handover spec)
# If > 200ms: document and explain (model size, quantization level)
# Metric 3: TTS synthesis time
# = Time from text ready to audio playing
# Measured separately from inference
# Target: < 300ms
# Metric 4: End-to-end latency
# = Tap "Speak" → audio begins playing
# Target: < 500ms total (inference + TTS)
Benchmark Test Suite E: Memory & Battery
# File: benchmarks/resource_benchmark.py
# Metric 1: RAM usage during inference
# Measure with adb shell dumpsys meminfo package.name
# Record: base app RAM, + model loaded RAM, + during inference peak RAM
# Assert: total does not exceed 4GB (leaves headroom for Android OS on 6GB devices)
# Metric 2: Battery consumption
# Run continuous inference for 30 minutes (simulating active communication session)
# Measure battery % delta
# Target: < 5% per hour
# Tool: adb shell dumpsys battery OR Android Battery Historian
# Metric 3: Thermal performance
# Monitor device temperature during 30-minute sustained use
# Note: thermal throttling will increase latency — document the curve
# adb shell cat /sys/class/thermal/thermal_zone*/temp
# Metric 4: Cold start time
# Time from app icon tap to communication screen ready (model loaded)
# Target: < 5 seconds (model should stay resident after first load)
Benchmark Test Suite F: Hardware Compatibility
# File: benchmarks/compatibility_check.py
# Check which inference backends are available on the device:
# - CPU (always available)
# - GPU (via MediaPipe GPU delegate)
# - NPU/DSP (via NNAPI delegate)
# [SEARCH REQUIRED]:
# Search: mediapipe genai "gpu delegate" OR "nnapi delegate" android compatibility
# Search: LiteRT android "backend fallback" cpu gpu npu strategy
# Test each backend:
# 1. Run 10 inference trials on each available backend
# 2. Record latency and memory for each
# 3. Report which backend is recommended for the target device class
2.4 Flutter Application Tests
Widget Tests
// test/widget_test.dart
// Test 1: PictogramTile renders correctly
// - Check tile displays Hindi label
// - Check tile displays English label
// - Check tile is accessible (Semantics widget present with correct label)
// - Check tile responds to tap
// Test 2: PhraseBar state management
// - Start empty → assert SPEAK button is disabled
// - Add 1 tile → assert SPEAK button is enabled
// - Clear → assert returns to empty state
// Test 3: Result card appearance
// - Mock inference result
// - Assert Hindi text is displayed
// - Assert "Play again" button is present
// - Assert "Save to history" button is present
// Test 4: Category tab switching
// - Tap second category tab
// - Assert grid content changes
// - Assert animation completes without error
Integration Tests
// integration_test/communication_flow_test.dart
// [SEARCH REQUIRED]: Search: flutter integration_test mediapipe method channel mock
// Test 1: Full communication flow (with mocked inference)
// Mock InferenceService to return "मुझे पानी चाहिए" for any input
// 1. Tap three pictogram tiles
// 2. Assert tiles appear in phrase bar
// 3. Tap SPEAK button
// 4. Assert result card appears with correct text
// 5. Assert TTS is called (mock TTS service)
// Test 2: Persona switching
// 1. Start with Arjun persona active
// 2. Navigate to Personas screen
// 3. Tap "Set Active" on Ananya
// 4. Assert adapter switch is triggered
// 5. Navigate back to communication screen
// 6. Assert persona indicator shows Ananya
// Test 3: Offline resilience
// Disable network (can be done in integration test setup)
// Assert: app works identically — no network errors, no degraded functionality
Accessibility Tests
// test/accessibility_test.dart
// Test 1: All tiles meet contrast requirements
// Programmatically check color contrast ratio for each category color + text
// Assert: all ratios >= 4.5:1
// Test 2: Touch target sizes
// Assert: all interactive widgets have minimum 72×72 logical pixels
// Test 3: TalkBack labels present
// Assert: every PictogramTile has non-empty Semantics.label
// Assert: SPEAK button has Semantics with state description
// Test 4: Animation disable respect
// Simulate AccessibilityFeatures.disableAnimations = true
// Assert: no AnimationController is started when flag is set
PART 3: RESULTS REPORT FORMAT
3.1 The Results Document Structure
This section defines the exact format for the benchmarking results document that will accompany the Kaggle submission. This document should be a single, well-formatted Markdown file named RESULTS.md in the repository root, AND should be mirrored as a well-designed section of the Kaggle notebook.
The document must contain:
Template: RESULTS.md
# Ankahi — System Evaluation Results
**Version:** 1.0.0
**Date:** [Run date]
**Hardware (training):** NVIDIA H100 (40GB VRAM)
**Hardware (inference benchmark):** [Device model, Android version, RAM]
---
## Section 1: Training Metrics
### Stage 1: Base AAC SFT
| Metric | Value |
|--------|-------|
| Final training loss | 0.6028 |
| Training pairs | 16,500 |
| Languages | Hindi, Punjabi, Tamil, Bengali, Hinglish |
| LoRA rank | 16 |
| Training time | [X hours on H100] |
| Tokens packed (MemoryPackCollator) | 2048-token blocks |
| Training speedup (vs. non-packed) | ~3× |
### Stage 2: Persona Adapters
| Adapter | Language | Rank | Size (MB) | Vocab coverage |
|---------|----------|------|-----------|----------------|
| Arjun | Hindi+Punjabi | 8 | [X] | [N] items |
| Ananya | Tamil+English | 8 | [X] | [N] items |
| Priya | Bengali+Hindi | 8 | [X] | [N] items |
| Rohan | Hindi+Marathi+English | 8 | [X] | [N] items |
| Zara | Urdu+Hindi+Telugu | 8 | [X] | [N] items |
---
## Section 2: Language Quality Evaluation
### BLEU-4 by Language (Stage 1 outputs)
[INSERT BAR CHART: horizontal bars, one per language, showing BLEU-4 score 0.0–1.0]
| Language | BLEU-4 | chrF++ |
|----------|--------|--------|
| Hindi | [X] | [X] |
| Punjabi | [X] | [X] |
| Tamil | [X] | [X] |
| Bengali | [X] | [X] |
| Hinglish | [X] | [X] |
| **Average** | **[X]** | **[X]** |
**Note:** chrF++ is the primary metric — it is better suited to morphologically rich Indian languages than BLEU-4. BLEU-4 is included for comparison.
### BLEU-4 vs. Zero-Shot GPT-4 Baseline
[INSERT GROUPED BAR CHART: Ankahi vs GPT-4 zero-shot on same test set, per language]
Context: GPT-4 zero-shot with a "translate this pictogram sequence to natural Hindi" prompt was used as a baseline. Ankahi's fine-tuned system should outperform it on code-switched and persona-specific outputs where GPT-4 has no grounding.
---
## Section 3: Adapter Isolation (No-Leakage Verification)
### IndicSBERT Cross-Adapter Similarity Heatmap
[INSERT 5×5 HEATMAP: rows = query adapter, cols = reference adapter]
[Color scale: low similarity = white/cream, high similarity = forest green]
[Diagonal = 1.0 (perfect self-similarity)]
Arjun Ananya Priya Rohan Zara
Arjun [ 1.00 X.XX X.XX X.XX X.XX ] Ananya [ X.XX 1.00 X.XX X.XX X.XX ] Priya [ X.XX X.XX 1.00 X.XX X.XX ] Rohan [ X.XX X.XX X.XX 1.00 X.XX ] Zara [ X.XX X.XX X.XX X.XX 1.00 ]
**Interpretation:** Off-diagonal values should be < 0.65. Values above this threshold would indicate persona "bleed" — one adapter sounding too much like another. A well-isolated adapter system protects each child's communication identity.
---
## Section 4: Safety Layer Verification
| Test Category | Inputs Tested | Refusals | Refusal Rate |
|---------------|---------------|----------|--------------|
| Self-harm triggers | 50 | [X] | [X]% |
| Inappropriate content | 50 | [X] | [X]% |
| Legitimate anger | 20 | 0 (correct) | 0% |
| Pain expression | 20 | 0 (correct) | 0% |
| Frustration | 20 | 0 (correct) | 0% |
**Target:** 100% refusal on harmful content, 0% refusal on legitimate emotional expression.
---
## Section 5: On-Device Inference Benchmarks
### Test Device
| Property | Value |
|----------|-------|
| Device | [Model name] |
| Processor | [SoC model] |
| RAM | [X] GB |
| Android version | [X] |
| Backend used | [CPU / GPU / NPU] |
| Backend selection rationale | [Why this backend was chosen] |
### Latency Breakdown
[INSERT BAR CHART: Stacked horizontal bars showing time components]
| Component | Mean (ms) | Std Dev (ms) | P95 (ms) |
|-----------|-----------|--------------|----------|
| Pictogram → tokens | [X] | [X] | [X] |
| LLM prefill (TTFT) | [X] | [X] | [X] |
| LLM decode (full sentence) | [X] | [X] | [X] |
| Adapter swap overhead | [X] | [X] | [X] |
| TTS synthesis | [X] | [X] | [X] |
| **End-to-end (tap → audio)** | **[X]** | **[X]** | **[X]** |
Target: end-to-end < 500ms.
[INSERT LINE CHART: Latency over 100 consecutive inferences — shows warmup curve and steady state]
### Latency by Input Length
[INSERT LINE CHART: x-axis = number of pictograms selected (1–8), y-axis = inference time ms]
Expected: roughly linear scaling. Deviations should be noted and explained.
---
## Section 6: Memory & Battery
### Memory Profile
[INSERT ANNOTATED SCREENSHOT or BAR CHART of memory usage at key states]
| State | RSS Memory (MB) |
|-------|----------------|
| App launched, no model | [X] |
| Base model loaded | [X] |
| Base model + Arjun adapter | [X] |
| During active inference | [X] (peak) |
| After inference, idle | [X] |
### Battery Consumption
| Session duration | Battery consumed | Implied drain/hour |
|-----------------|------------------|--------------------|
| 10 minutes active | [X]% | [X]% / hour |
| 30 minutes active | [X]% | [X]% / hour |
Target: < 5% per hour sustained.
### Thermal Behavior
[INSERT LINE CHART: Device temperature (°C) over 30-minute sustained use session]
Note: At what temperature does thermal throttling begin? What is the latency impact?
---
## Section 7: Unit Test Coverage
| Test Suite | Tests | Passed | Failed | Coverage |
|------------|-------|--------|--------|----------|
| Output quality | [X] | [X] | [X] | [X]% |
| Adapter switching | [X] | [X] | [X] | [X]% |
| Audio tower | [X] | [X] | [X] | [X]% |
| Safety layer | [X] | [X] | [X] | [X]% |
| Flutter widget tests | [X] | [X] | [X] | [X]% |
| Flutter integration tests | [X] | [X] | [X] | [X]% |
| **Total** | **[X]** | **[X]** | **[X]** | **[X]%** |
Previously confirmed: 36/36 passing in the original test suite. All new tests must achieve ≥ 95% pass rate.
---
## Section 8: Comparison vs. Commercial AAC Baseline
[INSERT RADAR CHART: 6 axes, comparing "Commercial AAC (best available)" vs "Ankahi"]
Axes:
1. Hindi / North Indian language support (0–5 scale)
2. South Indian language support (0–5)
3. Code-switching / Hinglish support (0–5)
4. Offline capability (0–5)
5. Dysarthric speech tolerance (0–5)
6. Affordability under ₹20,000 (0–5)
Expected result: Ankahi fills or approaches the full radar on all 6 axes. Commercial AAC (designed for English-speaking markets) scores poorly on axes 1–4 and 6.
---
## Section 9: Open Research Artifacts
| Artifact | Location | Description |
|----------|----------|-------------|
| Training dataset | HuggingFace: ankahi/Ankahi-AAC-SFT | 16,500 code-switched AAC pairs |
| Base model weights | HuggingFace: ankahi/ankahi-gemma4-e4b-stage1 | Stage 1 fine-tuned Gemma 4 E4B |
| Persona adapters | HuggingFace: ankahi/ankahi-personas | 5 Rank-8 LoRA adapters |
| Eval benchmark | GitHub: ankahi/ankahi/benchmarks/ | chrF++, BLEU-4, IndicSBERT evaluation code |
| Source code | GitHub: [repo link] | Apache 2.0 |
---
## Appendix A: Chart Generation Code
All charts in this document were generated with the code in `benchmarks/generate_charts.py`. To reproduce:
```bash
pip install matplotlib seaborn pandas numpy
python benchmarks/generate_charts.py --results benchmarks/results.json --output charts/
Input format for results.json:
{
"bleu4": { "hindi": X.XX, "punjabi": X.XX, "tamil": X.XX, "bengali": X.XX, "hinglish": X.XX },
"chrf": { "hindi": X.XX, ... },
"adapter_similarity": [[1.0, X.XX, ...], [X.XX, 1.0, ...], ...],
"latency_ms": { "prefill_mean": X, "prefill_std": X, "decode_mean": X, ... },
"memory_mb": { "idle": X, "model_loaded": X, "peak_inference": X },
"battery_pct_per_hour": X.X,
"safety": { "harmful_refusal_rate": 1.0, "legitimate_expression_rate": 1.0 }
}
---
### 3.2 Chart Generation Scripts
**[SEARCH REQUIRED]** For the benchmark visualization:
- Search: `seaborn heatmap "IndicSBERT" similarity matrix annotation python`
- Search: `matplotlib radar chart python "fill" multiple series comparison`
Implement the following in `benchmarks/generate_charts.py`:
**Chart 1: BLEU-4 by Language (horizontal bar chart)**
```python
# Use matplotlib, horizontal bars
# Colors: forest green (#2D6A4F) for Ankahi, gray for baseline
# Sort bars by score (highest at top)
# Add data labels on bars
# Save as: charts/bleu4_by_language.png (300 DPI)
Chart 2: IndicSBERT Heatmap (5×5)
# Use seaborn.heatmap
# Color: white → forest green (#2D6A4F), range 0.0–1.0
# Annotate cells with 2-decimal values
# Title: "Cross-Adapter Similarity (IndicSBERT)"
# A clean diagonal of 1.00 and low off-diagonal proves no leakage
# Save as: charts/adapter_heatmap.png (300 DPI)
Chart 3: Latency Breakdown (stacked horizontal bar)
# One bar per latency component
# Stacked to show total
# Colors: one per component (from Ankahi color palette)
# Target line at 500ms (red dashed vertical line)
# Save as: charts/latency_breakdown.png (300 DPI)
Chart 4: Training Loss Curve
# Line chart, loss vs. training step
# Smooth curve (rolling average window=50)
# Forest green line on white background
# Annotate the final loss value
# Save as: charts/training_loss_curve.png (300 DPI)
Chart 5: Radar (Capability Comparison)
# Use matplotlib polar plot
# Two filled areas: Ankahi (forest green, 40% alpha) vs Commercial AAC (gray, 30% alpha)
# 6 axes as specified above
# Legend outside the radar
# Save as: charts/capability_radar.png (300 DPI)
Chart 6: Latency over time (line chart for 100 inferences)
# x: inference number (1–100)
# y: latency in ms
# Show individual points (small dots) + rolling average line
# Annotate: "warmup period" (first 5 inferences), "steady state" (rest)
# Save as: charts/latency_over_time.png (300 DPI)
Chart 7: Cost Comparison (horizontal bar chart)
# y-axis: device/solution names
# x-axis: cost in INR (₹)
# Bars (low to high): Ankahi, Low-cost imported AAC, Mid-range AAC, Tobii Dynavox
# Add annotation: "ADIP scheme benefit: ₹6,000" as a vertical reference line
# Use log scale for x-axis (range is too wide for linear)
# Save as: charts/cost_comparison.png (300 DPI)
PART 4: KAGGLE SUBMISSION CHECKLIST
Use this checklist before the May 18 deadline:
PRE-SUBMISSION
□ .litertlm binary produced and validated on at least one Android device
□ At least one persona adapter converted and loading successfully
□ AI4Bharat TTS integrated (or fallback TTS confirmed working)
□ Flutter app builds successfully in release mode
□ APK sideloads and runs on a physical device without crash
□ All unit tests passing (target: 36+ tests, ≥ 95% pass rate)
□ Benchmarks run and results.json populated with real numbers
□ All 7 charts generated from real data (no placeholder numbers)
□ RESULTS.md complete with all tables and chart embeds
GITHUB REPO
□ Repository is PUBLIC
□ README.md leads with: problem statement → solution → demo link → how to run
□ "Built with Unsloth" badge prominent in README (Unsloth special mention)
□ "Gemma 4 E4B" model badge prominent
□ Apache 2.0 LICENSE file present
□ requirements.txt or pyproject.toml present
□ Installation steps tested on a clean environment
□ Demo video linked in README
KAGGLE NOTEBOOK
□ Notebook runs without errors (test with "Run All")
□ All output cells have saved outputs (don't require actual H100 to view)
□ All charts render in notebook output
□ Section headers clearly label each training stage
□ Unsloth integration is highlighted and explained
□ A "Why Gemma 4?" section explicitly mentions E4B multimodal features used
□ RESULTS.md is linked from the notebook
VIDEO
□ Video is ≤ 5 minutes
□ Opens with problem statement (the 2.5 million CP children figure)
□ Shows a real Android tablet (not a laptop screen)
□ Shows a child (or hand) tapping pictogram tiles
□ Shows the Hindi sentence appearing
□ Shows audio playing (parent's voice)
□ Closes with the GitHub link and "free, offline, open-source" statement
FINAL SUBMISSION
□ Kaggle submission form completed
□ Track selected: Digital Equity (primary)
□ "Uses Unsloth" checkbox marked (if present in submission form)
□ GitHub link submitted
□ Video link (YouTube unlisted or Google Drive) submitted
□ Team members added to submission (if applicable)
PART 5: RESOURCES MASTER LIST
[SEARCH REQUIRED before starting any section]
All resources below should be re-verified for current status — URLs and APIs may have changed.
MediaPipe / LiteRT
- Main docs:
https://ai.google.dev/edge/mediapipe - LLM inference guide:
https://ai.google.dev/edge/mediapipe/solutions/genai/llm_inference - Android LLM guide:
https://ai.google.dev/edge/mediapipe/solutions/genai/llm_inference/android - Samples (search):
site:github.com/google-ai-edge mediapipe-samples llm_inference - PyPI:
mediapipe,mediapipe-model-maker— always check latest version before installing
Unsloth
- Main:
https://github.com/unslothai/unsloth - Docs:
https://docs.unsloth.ai - Gemma 4 specific: Search
"unsloth" "Gemma 4" fine-tuning notebook 2026
AI4Bharat TTS
- GitHub: Search
AI4Bharat Indic-TTS github - HuggingFace: Search
AI4Bharat TTS huggingface model - Demo: Search
AI4Bharat TTS voice cloning demo
IndicSBERT (for eval)
- Search:
"IndicSBERT" OR "Indic SBERT" huggingface sentence similarity - Likely:
sentence-transformerslibrary + an IndicNLP model
Flutter + MediaPipe Integration
- Search:
flutter mediapipe genai "method channel" llm inference kotlin dart - Search:
flutter ffi mediapipe native library android
TORGO / UA-Speech (audio training data)
- TORGO: Search
"TORGO database" download dysarthric speech corpus - UA-Speech: Search
"UA-Speech" database download dysarthric
chrF++ implementation
- Search:
"sacrebleu" python "chrf" "chrf++" implementation - Library:
pip install sacrebleu→sacrebleu.corpus_chrf()
End of Backend & System Specification. All three documents together constitute the complete Ankahi technical brief for the Kaggle Gemma 4 Good Hackathon submission.