Spaces:

AaronTekle
/

RAG_Wikipedia

Running on Zero

Output	Meaning
Source number	citation number used in the answer
Passage title	show name for the retrieved passage
Score	ranking score from retrieval or reranking
Passage text	exact text retrieved from the dataset
Source ID	original dataset row or passage identifier

Example:

1
Wikipedia passage 362
9.686

The Legal Tender Act of 1862 established the United States Note,
the first paper currency in United States history. This was done
to increase the money supply to pay for fighting the war

Source ID: 361

the answer citation [1] refers to this source

Retrieval Score (meaning)

displayed score is used to order passages

higher score means the passage ranked more strongly for the current question

score is not:

a probability
a percentage
a factual confidence score
a measure of answer correctness

CrossEncoder scores can be positive or negative

negative score does not automatically (always) mean that a passage is invalid it usually means the passage ranked less strongly than other candidates.

scores should only be compared with other passages from the same request and retrieval configuration

Retrieval Controls

Sources Returned

Controls how many passages are returned after retrieval and reranking
Higher values provide more context but may also add unrelated information

Setting	Value
Default	`5`
Minimum	`3`
Maximum	`8`
Step	`1`

Semantic Retrieval Weight

controls the balance between semantic retrieval and TF-IDF keyword retrieval

Setting	Value
Default	`0.72`
Minimum	`0.00`
Maximum	`1.00`
Step	`0.05`

Semantic Retrieval Weight Formula:

TF-IDF weight = 1 - semantic retrieval weight

Examples:

Semantic weight	TF-IDF weight	Behavior
`1.00`	`0.00`	Semantic retrieval only
`0.72`	`0.28`	Default hybrid retrieval
`0.50`	`0.50`	Equal semantic and keyword weighting
`0.00`	`1.00`	TF-IDF keyword retrieval only

Semantic retrieval is useful when the question and passage use different wording
TF-IDF is useful when the question contains exact names, dates, phrases, or technical terms

Use Neural Reranker

(when enabled), the CrossEncoder compares the full question with each retrieved candidate and reorders the passages
note: disabling the reranker can reduce processing time, but retrieval quality may decrease

Setting	Default
Enabled or disabled	Enabled

Generation Controls

Temperature:

controls variation in generated text

Setting	Value
Default	`0.20`
Minimum	`0.00`
Maximum	`1.00`
Step	`0.05`

Temperature	Expected behavior
`0.00` to `0.20`	more consistent and direct
`0.25` to `0.50`	moderate variation
Above `0.50`	more varied output with greater risk of unsupported content

lower values are recommended for factual RAG answers

Maximum Output Tokens

Setting	Value
Default	`700`
Minimum	`200`
Maximum	`1200`
Step	`50`

limits the maximum generated answer length
(this setting does not control the number of retrieved passages)

Diagnostics:

diagnostics panel shows JSON returned by the RAG engine

can be used to inspect:

pipeline status
retrieval settings
number of retrieved passages
retrieval results
reranker results
generation settings
processing details

exact fields depend on the values returned by RAGEngine.diagnostics()

App Inputs:

Input	Data Type	Default	Range	Purpose
Question	Text	Empty	Up to 5 lines	question sent to the RAG pipeline
Sources returned	Integer	`5`	`3` to `8`	number of passages returned
Semantic retrieval weight	Float	`0.72`	`0.0` to `1.0`	balance between semantic retrieval and TF-IDF
Use neural reranker	Boolean	`True`	On or off	enables CrossEncoder reranking
Temperature	Float	`0.20`	`0.0` to `1.0`	controls generation variation
Maximum output tokens	Integer	`700`	`200` to `1200`	limits answer length

App Outputs:

each successful request updates four outputs

1. Retrieved Evidence:

displays the passages selected by the retrieval pipeline

2. Diagnostics:

displays pipeline information as JSON

3. Question Field:

clears the submitted question

Request Flow:

when a question is submitted, the RAG app:

removes leading and trailing whitespace
checks whether the question is empty
initializes the RAG engine if needed
reads the current conversation history
runs semantic retrieval
runs TF-IDF keyword retrieval
combines the retrieval scores
applies the CrossEncoder reranker when enabled
selects the requested number of sources
sends the selected passages and chat history to Qwen
generates a citation-based answer
adds question and answer to the conversation
shows the retrieved source cards

How Retrieval Works:

the app loads the Wikipedia dataset
dataset rows are cleaned and split into chunks
MiniLM creates an embedding for each chunk
TF-IDF creates a keyword index
the user question is compared with both indexes
semantic and TF-IDF scores are combined
the strongest candidates are selected
the CrossEncoder reranks the candidates when enabled
the highest-ranked passages are sent to Qwen
the generated answer and supporting passages are displayed

Limits:

wikipedia corpus (small amount of data)
free HFSpace restarts may require me to manually rebuild the cache
some dataset text (wikipedia corpus) may have character encoding errors

title: RAG Wikipedia emoji: 🐠 colorFrom: indigo colorTo: purple sdk: gradio sdk_version: 6.22.0 python_version: '3.12' app_file: app.py pinned: false license: apache-2.0 short_description: RAG with Wikipedia Corpus

Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference

space/main

<<<<<<< HEAD

title: RAG Wikipedia emoji: 😊 colorFrom: blue colorTo: indigo sdk: gradio sdk_version: "5.44.1" python_version: "3.10" app_file: app.py pinned: false

RAG (with Wikipedia Corpus)

ML (model) Stack:

Features (app workflow)

Interface Features

Retrieved Evidence