Instructions to use rarfileexe/FlameLedger-Encoder with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use rarfileexe/FlameLedger-Encoder with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("rarfileexe/FlameLedger-Encoder") sentences = [ "That is a happy person", "That is a happy dog", "That is a very happy person", "Today is a sunny day" ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [4, 4] - Notebooks
- Google Colab
- Kaggle
FlameLedger Encoder: EmbeddingGemma 2 fine-tuned for NASA combustion research retrieval
A text embedding model for finding passages in NASA microgravity-combustion and spacecraft fire-safety literature. It is google/embeddinggemma-2 with its text encoder fine-tuned on synthetic search queries over public NASA Technical Reports Server (NTRS) records.
Built by Team Qubit for the NASA Space Apps Challenge 2026 (Flame in Freefall) and used in production by FlameLedger, an evidence research console that ranks NASA/NIST combustion experiments against a proposed cabin environment.
- Live demo: https://flameledger.agenticerastudio.tech
- Code: https://github.com/Muhammad-Hassan12/FlameLedger
- Embedding size: 768 (Matryoshka: 768 / 512 / 256 / 128)
- Language: English
- License: Apache 2.0 (see License and attribution)
Read this first. This is a small, domain-specific fine-tune (1,025 training triplets, 486 passages). Gains over the base model are measured on teacher-written synthetic queries and are modest. On a small teacher-drafted question set (n = 23, unreviewed) no improvement was detectable. BM25 is a strong baseline on this corpus, and this model is intended to be used together with BM25. See Evaluation and Limitations.
Model details
| Base model | google/embeddinggemma-2 (Google DeepMind, 740M total parameters; 270M text backbone) |
| What was fine-tuned | Text tower only (about 271M trainable parameters). Vision and audio towers were frozen and are unused. |
| This repository | Text-only encoder export (FP32). Image and audio inputs are not supported. |
| Output | 768-dimensional embeddings, truncatable to 512 / 256 / 128 (Matryoshka) |
| Similarity | Cosine similarity (L2-normalize embeddings) |
| Max sequence length used in training | 512 tokens (base model supports up to 8K) |
| Precision | Use FP32 or BF16. The base model card warns against FP16. |
| Developed by | Syed Muhammad Hassan (Team Qubit) |
Intended use
Intended:
- Semantic and hybrid (dense + BM25) retrieval over NASA combustion, microgravity, and spacecraft fire-safety technical reports.
- First-stage candidate retrieval in a RAG or research-assistant pipeline where results are shown with their sources.
Not intended:
- Fire-safety decision-making, material qualification, or any safety-critical judgment. Retrieval relevance is not scientific correctness, and the system must not be used as a substitute for expert review.
- General-purpose or multilingual embedding. Use the base model for that.
- Answer verification. This model ranks passages. It does not check claims.
How to use
Prefixes are required
The model was trained with fixed text prefixes. Use exactly these formats for queries and documents, or retrieval quality will silently degrade.
QUERY_PREFIX = "task: search result | query: "
def fmt_query(q: str) -> str:
return QUERY_PREFIX + " ".join(str(q).split())
def fmt_doc(title: str, text: str) -> str:
t = " ".join((title or "").split()) or "none"
return f"title: {t} | text: {str(text).strip()}"
Sentence Transformers
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("YOUR_HF_USERNAME/flameledger-embeddinggemma2-nasa-combustion")
model.max_seq_length = 512
queries = [fmt_query("At what oxygen concentration does flame spread stop in low-speed opposed flow?")]
docs = [
fmt_doc("Combustion of solid fuel in very low speed oxygen streams",
"Flame spread over thin solid fuel in quiescent and low-speed flows ..."),
fmt_doc("Particle and Smoke Detection on ISS for Next Generation Smoke Detectors",
"Smoke detector response depends on particle size distribution ..."),
]
# Prefixes are added manually above. Use .encode() so no extra prompt is prepended.
q = model.encode(queries, normalize_embeddings=True)
d = model.encode(docs, normalize_embeddings=True)
scores = q @ d.T # cosine similarity
print(scores)
Using a smaller dimension (Matryoshka)
import numpy as np
def truncate(emb, dim=256):
emb = emb[:, :dim]
return emb / np.linalg.norm(emb, axis=1, keepdims=True)
q256, d256 = truncate(q, 256), truncate(d, 256)
Re-normalize after truncating. See the dimension results before going below 256.
Hybrid retrieval (recommended)
The best configuration in our evaluation fuses dense and lexical rankings with Reciprocal Rank Fusion:
- Dense: cosine similarity at 768 dimensions.
- Lexical: BM25 (k1 = 1.5, b = 0.75) over
title + " " + text, tokenized as lowercase alphanumerics (keeping internal.and-, e.g.6061-t6). - Fusion: RRF with k = 60 over the top 100 of each ranking.
A reference implementation (HybridRetriever, BM25, rrf_scores) is in the FlameLedger repository.
Training
Data
Training data is synthetic. No human-written training questions were used.
| Stage | Count |
|---|---|
| NTRS records fetched | 416 |
| On-topic papers after de-duplication and topic filter | 399 |
| Passages (chunks) | 486 (399 abstracts + 87 body chunks from 10 full-text PDFs) |
| Valid teacher queries after rule-based validation | 1,412 (train 1,128 / dev 134 / test 150) |
| Training triplets after round-trip filter | 1,025 (dev: 134) |
Most papers are represented by their abstract only, because scanned PDFs were skipped rather than guessed.
Query generation. Gemma 4 12B instruction-tuned (google/gemma-4-12B-it) wrote three queries per passage in three styles: keyword, question, and scenario (an engineering need the passage helps with). Teacher output was kept only if it passed rule checks:
- length limits per style,
- no meta-references such as "the passage" or "this paper",
- no verbatim copying (no 7-word run copied from the passage),
- every multi-digit or decimal number in the query must appear in the passage (blocks invented numbers),
- de-duplication.
Round-trip filter (train split only). Queries for which the base model ranked the source passage outside its top 15 were dropped (kept 1,025 of 1,128, or 91%).
Hard negatives. Three per query, mined with the base model from the same split only, never from the same paper, skipping near-duplicates (cosine โฅ 0.92) and candidates scoring within 5% of the positive, to reduce false negatives.
Leak-free split. Train / dev / test are split by paper (321 / 39 / 39 papers), with papers sharing a normalized title kept in the same split.
Procedure
| Loss | MatryoshkaLoss (dims 768, 512, 256, 128) wrapping CachedMultipleNegativesRankingLoss |
| Epochs | 3 (17 steps per epoch) |
| Batch size | 64 (cached MNRL mini-batch 16) |
| Learning rate | 2e-5, warmup ratio 0.1, weight decay 0.01 |
| Precision | FP32 master weights with BF16 autocast |
| Batch sampler | NO_DUPLICATES |
| Checkpoint selection | Best dev nDCG@10 (evaluated every 8 steps) |
| Max sequence length | 512 |
| Seed | 42 |
| Hardware | 1ร NVIDIA L4 (24 GB), Google Colab |
| Software | sentence-transformers 6.2.0, transformers 5.22.1, PyTorch 2.11.0 |
Dev-set results before and after fine-tuning (dev was used to pick the checkpoint, so treat these as optimistic):
| Dev metric | Base | Fine-tuned |
|---|---|---|
| nDCG@10 | 0.877 | 0.916 |
| MRR@10 | 0.844 | 0.890 |
| Accuracy@1 | 0.769 | 0.821 |
| Accuracy@5 | 0.948 | 0.993 |
Evaluation
All numbers below are from the training notebook's scorecard. Recall@k is the fraction of queries whose labeled source passage appears in the top k results. Confidence intervals are 95% paired bootstrap.
Full scorecard output from the training notebook: scorecard.md.
Evidence quality
| Query set | n | Origin | Strength |
|---|---|---|---|
| Synthetic test | 150 | Teacher-written, same pipeline and style as training queries | Weakest. Optimistic. |
| Teacher-drafted gold | 23 | Harder, less keyword-matching questions drafted by the teacher. Not human-reviewed. | Weak (small n, unreviewed) |
| Human-written gold | 0 | none | Not available |
No human-written evaluation questions exist for this model. The numbers measure retrieval on teacher-written queries and should not be read as deployed accuracy.
Synthetic test, held-out papers only (n = 150)
Candidates are the 51 passages from the 39 test papers. This is the cleanest setting, but it is easy: scores saturate.
| Method | Recall@1 | Recall@5 | Recall@10 | MRR@10 |
|---|---|---|---|---|
| BM25 | 0.800 | 0.967 | 1.000 | 0.874 |
| Base, dense 768 | 0.713 | 0.967 | 0.993 | 0.811 |
| Fine-tuned, dense 768 | 0.793 | 0.993 | 1.000 | 0.871 |
| Base + BM25 (RRF) | 0.787 | 0.987 | 1.000 | 0.873 |
| Fine-tuned + BM25 (RRF) | 0.813 | 1.000 | 1.000 | 0.891 |
Synthetic test, full archive (n = 150)
Candidates are all 486 passages. This includes passages from training papers that the fine-tuned model saw during training, so it is less clean than the table above and likely favors the fine-tuned model.
| Method | Recall@1 | Recall@5 | Recall@10 | MRR@10 |
|---|---|---|---|---|
| BM25 | 0.647 | 0.900 | 0.940 | 0.752 |
| Base, dense 768 | 0.527 | 0.840 | 0.893 | 0.652 |
| Fine-tuned, dense 768 | 0.647 | 0.920 | 0.953 | 0.747 |
| Base + BM25 (RRF) | 0.673 | 0.913 | 0.967 | 0.776 |
| Fine-tuned + BM25 (RRF) | 0.713 | 0.940 | 0.987 | 0.807 |
Teacher-drafted gold, unreviewed (n = 23)
| Method | Recall@1 | Recall@5 | Recall@10 | MRR@10 |
|---|---|---|---|---|
| BM25 | 0.304 | 0.609 | 0.739 | 0.436 |
| Base, dense 768 | 0.391 | 0.609 | 0.739 | 0.506 |
| Fine-tuned, dense 768 | 0.478 | 0.609 | 0.696 | 0.537 |
| Base + BM25 (RRF) | 0.261 | 0.565 | 0.696 | 0.422 |
| Fine-tuned + BM25 (RRF) | 0.391 | 0.696 | 0.696 | 0.505 |
Fine-tuned minus base (95% paired bootstrap CI)
| Query set | Comparison | Recall@5 | MRR@10 |
|---|---|---|---|
| Synthetic, held-out papers | dense 768 | +0.027 [+0.007, +0.060] | +0.060 [+0.031, +0.092] |
| Synthetic, held-out papers | + BM25 (RRF) | +0.013 [+0.000, +0.033] | +0.018 [-0.002, +0.037] |
| Synthetic, full archive | dense 768 | +0.080 [+0.033, +0.127] | +0.095 [+0.053, +0.141] |
| Synthetic, full archive | + BM25 (RRF) | +0.027 [+0.000, +0.060] | +0.031 [-0.001, +0.065] |
| Teacher-drafted gold (n = 23) | dense 768 | +0.000 [-0.130, +0.130] | +0.031 [-0.021, +0.099] |
| Teacher-drafted gold (n = 23) | + BM25 (RRF) | +0.130 [+0.000, +0.261] | +0.083 [+0.014, +0.164] |
How to read these results
- Fine-tuning gives a statistically supported improvement to dense retrieval on synthetic queries (confidence intervals exclude zero for Recall@5 and MRR@10).
- Once BM25 is added, the improvement over base + BM25 is small, and several intervals touch or cross zero.
- On the harder drafted-gold set, dense retrieval shows no detectable difference (n = 23 is too small to conclude either way).
- BM25 alone is competitive: on held-out synthetic queries its MRR@10 (0.874) matches the fine-tuned dense model (0.871). The practical recommendation is hybrid retrieval, not dense alone.
- Synthetic queries share the teacher model's phrasing, so all synthetic scores are optimistic.
Effect of embedding dimension
Fine-tuned model, synthetic test (n = 150):
| Dimension | Held-out Recall@5 | Held-out MRR@10 | Full-archive Recall@5 | Full-archive MRR@10 |
|---|---|---|---|---|
| 768 | 0.993 | 0.871 | 0.920 | 0.747 |
| 512 | 0.987 | 0.880 | 0.907 | 0.752 |
| 256 | 0.973 | 0.871 | 0.893 | 0.734 |
| 128 | 0.967 | 0.865 | 0.833 | 0.720 |
768 and 512 are effectively equivalent here. 256 is a reasonable storage trade-off. 128 loses noticeable recall on the full archive, and the base model card also advises validating 128-d on your own workload.
Limitations
- Very small training set. 1,025 triplets from 486 passages (399 of them abstracts). The model has seen a narrow slice of the literature.
- Synthetic supervision and evaluation. Training and test queries come from one teacher model. There is no human-written evaluation set, and the 23-question drafted set was not human-reviewed.
- Modest, partly uncertain gains. See How to read these results.
- Narrow domain. Trained only on NASA combustion and fire-safety text. Out-of-domain behavior was not evaluated and may be worse than the base model.
- English only. Multilingual ability of the base model was not evaluated after fine-tuning.
- Abstract-heavy corpus. Most papers contribute only their abstract, so retrieval of detailed in-body conditions (tables, run parameters) is largely untested.
- Sequence length. Fine-tuned at 512 tokens. Long-document behavior was not evaluated.
- Text only. The vision and audio towers were removed from this export.
- Retrieval is not correctness. A relevant passage can still be misread, mismatched to a condition, or superseded. Always show sources and keep a human in the loop.
Responsible use
This model supports literature search. It is not a NASA product, is not endorsed by NASA or Google, and must not be used as a fire-safety decision system.
AI and tool disclosure
We fine-tuned Google's EmbeddingGemma 2 to retrieve relevant NASA combustion research. Gemma 4 12B generated synthetic queries for training. FlameLedger combines embedding-based retrieval with BM25 keyword search.
AI-assisted engineering tools (Codex, Antigravity) were used for software development and deployment. Gemini (gemini-3.1-flash-lite) is used in the FlameLedger application, not in this model, to draft cited briefings from retrieved evidence.
License and attribution
- This model is a derivative of
google/embeddinggemma-2, released by Google DeepMind under Apache 2.0, and is released here under Apache 2.0. The upstream license page links Google's Gemma 4 license terms. Please review the base model card for the terms that apply to the original weights. - Training queries were generated with
google/gemma-4-12B-it. - The training corpus is derived from public NASA NTRS records. No corpus text is redistributed with this model.
Citation
@misc{flameledger_encoder_2026,
title = {FlameLedger Encoder: EmbeddingGemma 2 fine-tuned for NASA combustion research retrieval},
author = {{Syed Muhammad Hassan}, {Team Qubit}},
year = {2026},
howpublished = {Hugging Face model repository},
note = {NASA Space Apps Challenge 2026, Flame in Freefall. Code: https://github.com/Muhammad-Hassan12/FlameLedger}
}
Please also cite the base model, EmbeddingGemma 2 (Google DeepMind).
Contact
Open an issue on the FlameLedger repository.
- Downloads last month
- 14
Model tree for rarfileexe/FlameLedger-Encoder
Base model
google/embeddinggemma-2