all-MiniLM-L6-v2-tflite

all-MiniLM-L6-v2-tflite

A TensorFlow Lite version of the popular sentence-transformers/all-MiniLM-L6-v2 model, optimized for mobile and embedded devices.

Model Overview

This model is a lightweight sentence transformer that maps sentences and paragraphs to a 384-dimensional dense vector space. It's designed for efficient on-device inference while maintaining good semantic understanding capabilities.

Key Features

Optimized for Mobile: Converted to TensorFlow Lite format for fast inference on Android, iOS, and embedded devices
Lightweight: Significantly reduced model size compared to the original PyTorch version
High Performance: Maintains strong semantic similarity performance with ~22M parameters
Cross-Platform: Compatible with Android Studio, Flutter, and other mobile development frameworks
Privacy-First: Enables completely offline text embeddings without sending data to external servers

Use Cases

Semantic Search: Find relevant documents or passages based on meaning rather than keywords
Text Clustering: Group similar sentences or documents together
Similarity Detection: Compare semantic similarity between text passages
Information Retrieval: Build recommendation systems based on text content
Mobile NLP Apps: Add intelligent text processing to mobile applications
Edge AI: Deploy text understanding capabilities on IoT and embedded devices

Model Specifications

Property	Value
Input Sequence Length	Up to 256 word pieces (longer text is truncated)
Output Dimensions	384-dimensional dense vectors
Model Size	~22M parameters (TFLite optimized)
Base Architecture	6-layer MiniLM encoder
Pooling Strategy	Mean pooling with attention mask
Normalization	L2 normalized embeddings

Quick Start

Android/Kotlin Integration

// Load the TFLite model in your Android app
val interpreter = Interpreter(loadModelFile())

// Prepare input tensors
val inputShape = intArrayOf(1, maxSequenceLength)
val inputBuffer = TensorBuffer.createFixedSize(inputShape, DataType.INT32)

// Run inference
interpreter.run(inputBuffer.buffer, outputBuffer.buffer)

Flutter Integration

// Use with tflite_flutter package
import 'package:tflite_flutter/tflite_flutter.dart';

// Load and run inference
final interpreter = await Interpreter.fromAsset('all_minilm_l6_v2.tflite');
final output = List.filled(384, 0.0).reshape([1, 384]);
interpreter.run(input, output);

Python Validation

# Verify model outputs match original
import tensorflow as tf

# Load TFLite model
interpreter = tf.lite.Interpreter(model_path="model.tflite")
interpreter.allocate_tensors()

# Get input and output tensors
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

# Run inference
interpreter.set_tensor(input_details[0]['index'], input_data)
interpreter.invoke()
output_data = interpreter.get_tensor(output_details[0]['index'])

Input/Output Format

Input

Tokenized text sequences as integer arrays
Attention masks for proper sequence handling
Maximum length: 256 tokens (longer sequences are truncated)

Output

384-dimensional float vectors representing sentence embeddings
L2 normalized for direct cosine similarity calculations
Ready for downstream tasks like clustering or search

Performance Benchmarks

Metric	Value
Embedding Speed (CPU)	~15ms per 1K tokens
Model Size	<50MB (TFLite format)
Memory Usage	Low memory footprint for mobile deployment
Accuracy	Maintains >95% similarity to original model outputs