ryanadoremos/government-procurement-rag-assistant

Government Procurement RAG Assistant

A Retrieval-Augmented Generation (RAG) system that answers questions about U.S. federal procurement opportunities using semantic search and a Large Language Model (LLM).

---

Table of Contents

• Introduction
• Project Overview
• Dataset
• Methodology
• System Architecture
• Evaluation
• Example Usage
• Repository Structure
• Future Improvements
• Requirements
• References

---

Introduction

Government procurement opportunities contain lengthy descriptions, technical terminology, and agency-specific language that can make locating relevant information difficult. Traditional keyword search often returns unrelated results, while large language models (LLMs) alone may hallucinate information that is not present in the underlying data.

This project develops a Retrieval-Augmented Generation (RAG) assistant that combines semantic retrieval with a large language model to produce grounded responses using publicly available procurement data from the USAspending.gov API.

Instead of relying solely on an LLM's internal knowledge, the assistant retrieves the most relevant procurement records first and then generates answers based only on those retrieved documents.

---

Project Overview

The project consists of four major components:

1. Download procurement opportunities from the USAspending.gov API.
2. Convert procurement records into retrieval-ready text chunks.
3. Retrieve the most relevant procurement documents using semantic search.
4. Generate grounded responses using Qwen2.5-3B-Instruct.

The complete pipeline minimizes hallucinations by ensuring responses are based on retrieved procurement records rather than model memorization.

---

Dataset

The knowledge base is built from publicly available procurement records collected through the USAspending.gov API.

Each procurement record contains information including:

• Award ID
• Recipient Name
• Award Amount
• Awarding Agency
• Procurement Description

Each record is:

• Cleaned
• Formatted
• Converted into a retrieval document
• Stored as a searchable text chunk

The final corpus contains approximately 100 procurement opportunities collected between January 2024 and December 2025.

---

Methodology

1. Data Collection

The Government Procurement RAG Assistant follows a Retrieval-Augmented Generation (RAG) workflow that combines semantic search with a Large Language Model (LLM) to answer questions about federal procurement opportunities. Rather than relying solely on the language model's internal knowledge, the system first retrieves the most relevant procurement records and then uses those records as context for response generation. This retrieval-first approach helps reduce hallucinations and ensures that generated answers remain grounded in publicly available procurement data. The complete pipeline consists of data collection, document preprocessing, semantic embedding, vector indexing, retrieval, prompt construction, and response generation. Each stage was designed to improve the accuracy, relevance, and transparency of the final responses while maintaining an efficient workflow suitable for real-world applications. The following sections describe each stage of the pipeline in greater detail, from collecting procurement records through generating grounded responses using the Qwen2.5 language model.

Relevant fields include:

• Award ID
• Recipient Name
• Award Amount
• Awarding Agency
• Description

---

2. Document Processing

Each procurement record is converted into a structured retrieval document.

Text cleaning includes:

• HTML removal
• Whitespace normalization
• Character cleanup
• Structured formatting

Each document becomes one searchable chunk stored inside

government_procurement_chunks.csv

---

3. Semantic Retrieval

Semantic embeddings are generated using:

sentence-transformers/all-MiniLM-L6-v2

Embeddings are indexed using Facebook AI Similarity Search (FAISS).

For every user question:

1. The query is embedded.
2. FAISS performs nearest-neighbor search.
3. The top three procurement documents are retrieved.

---

4. Response Generation

Retrieved procurement records are inserted into the prompt provided to

Qwen/Qwen2.5-3B-Instruct

The model is instructed to:

• Answer only from retrieved procurement documents.
• Avoid unsupported claims.
• State when insufficient information is available.
• Produce concise, grounded responses.

---

System Architecture

User Question
      │
      ▼
Sentence Transformer
      │
      ▼
Query Embedding
      │
      ▼
FAISS Vector Search
      │
      ▼
Top 3 Procurement Documents
      │
      ▼
Prompt Construction
      │
      ▼
Qwen2.5-3B-Instruct
      │
      ▼
Grounded Response

---

Evaluation

The RAG system was evaluated using multiple procurement-related benchmark questions.

Example evaluation questions include:

• Which federal agency awarded the largest healthcare contract?
• Which recipient received the highest award?
• What contracts were awarded to the Department of Health and Human Services?
• Summarize this procurement opportunity.
• Explain why vendors should monitor procurement opportunities.

Evaluation focused on:

• Retrieval relevance
• Semantic similarity
• Grounded response generation
• Hallucination avoidance
• Procurement information accuracy

The evaluation demonstrated that retrieval substantially improved response quality by grounding generated answers in procurement records.

---

Example Usage

from rag_pipeline import GovernmentProcurementRAGAssistant

assistant = GovernmentProcurementRAGAssistant(
    chunks_path="government_procurement_chunks.csv"
)

response, sources = assistant.answer(
    "Which federal agency awarded the largest healthcare contract?"
)

print(response)

print(sources)

---

Repository Structure

government-procurement-rag-assistant/

│
├── README.md
├── rag_pipeline.py
├── example_usage.py
├── requirements.txt
├── Adoremos_Ryan_Final.ipynb
├── government_procurement_chunks.csv
│
├── results/
│   ├── pre_rag_eval_results.csv
│   ├── post_rag_eval_results.csv
│   ├── pre_post_rag_comparison.csv
│   ├── comparison_model_detailed_results.csv
│   ├── comparison_model_benchmark_summary.csv
│   └── final_huggingface_model_comparison.csv
│
└── data/

---

Prompt Format

The Government Procurement RAG Assistant follows a Retrieval-Augmented Generation (RAG) prompting strategy in which the user's question is combined with the most relevant procurement records before being sent to the language model. Instead of allowing the model to answer from its pre-trained knowledge alone, the prompt includes procurement information retrieved through semantic search, providing factual context for response generation.

Each prompt contains:

• The user's original procurement question
• The top three semantically retrieved procurement documents
• Award metadata (Award ID, Recipient, Agency, Award Amount)
• Procurement descriptions
• Instructions directing the model to answer only from the retrieved documents
• Guidance to acknowledge when insufficient information is available

A simplified prompt structure is shown below:

Question:

Retrieved Procurement Context: <Document 1>

<Document 2>

<Document 3>

Instructions: • Answer only using the retrieved procurement information. • Do not invent procurement records or regulations. • Clearly state when additional information is required.

Expected Output Format

The assistant produces structured responses designed to clearly separate factual procurement information from explanatory text. This formatting improves readability while ensuring that responses remain grounded in the retrieved procurement records.

Each generated response follows the structure below:

1. Direct Answer

   • A concise response to the user's question.

2. Supporting Procurement Evidence

   • Relevant Award IDs
   • Awarding Agency
   • Recipient
   • Award Amount
   • Procurement details retrieved from the knowledge base

3. Important Considerations

   • Additional context or assumptions supported by the retrieved records.

4. Additional Context

   • Supplementary information that helps explain the procurement data.

5. Insufficient Information (when applicable)

   • If the retrieved procurement documents do not fully answer the question, the assistant explicitly states what additional information would be required rather than generating unsupported content.

This structured output format promotes transparency, improves interpretability, and helps reduce hallucinations by ensuring that generated responses remain closely aligned with the retrieved procurement records.

Future Improvements

While the current system demonstrates that Retrieval-Augmented Generation (RAG) can effectively answer questions about U.S. federal procurement data, several enhancements could further improve both retrieval quality and the overall user experience.

Larger Procurement Knowledge Base

The current knowledge base contains approximately 100 procurement opportunities collected from USAspending.gov. Expanding the corpus to include thousands of procurement records across multiple years and agencies would improve retrieval coverage and allow the assistant to answer a broader range of procurement-related questions with greater accuracy.

Hybrid Retrieval (BM25 + Semantic Search)

The current implementation relies solely on semantic embeddings generated by Sentence Transformers. Future work could combine traditional keyword search (BM25) with semantic retrieval to better capture both exact keyword matches and contextual meaning. Hybrid retrieval is widely used in modern production RAG systems because it often improves retrieval precision for specialized terminology and government-specific language.

Metadata Filtering

Procurement records contain valuable structured metadata such as awarding agency, recipient organization, award amount, award year, contract type, and industry classification. Allowing users to filter retrieval by these fields would significantly narrow the search space and produce more targeted responses for complex procurement queries.

Cross-Encoder Reranking

After FAISS retrieves the most similar procurement documents, a cross-encoder reranking model could be applied to reorder the retrieved results based on a deeper understanding of the relationship between the user query and each document. This additional ranking stage has the potential to improve retrieval accuracy, especially for ambiguous or highly detailed procurement questions.

Source Citations and Explainability

Future versions of the assistant could include explicit citations to the retrieved procurement records used during response generation. Displaying Award IDs, recipient names, agencies, and links to the original procurement records would improve transparency, increase user trust, and make it easier to verify generated responses.

Interactive Web Application

The project could be deployed as an interactive web application using Gradio or Hugging Face Spaces. A web interface would allow users to submit procurement questions, view the retrieved procurement documents alongside generated responses, and explore the retrieval process without requiring programming knowledge.

Continuous Data Updates

Government procurement information changes frequently as new awards are published. A future enhancement would automate data collection from the USAspending.gov API on a scheduled basis so that the knowledge base remains current without requiring manual updates. This would enable the assistant to answer questions using more recent procurement information.

Advanced Retrieval Techniques

Additional retrieval improvements could include query expansion, document chunk optimization, adaptive chunk sizing, and multi-query retrieval. These techniques may improve retrieval robustness by capturing procurement information expressed using different terminology while reducing the likelihood of missing relevant documents.

Support for Additional Government Data Sources

Although this project focuses on USAspending.gov procurement records, the assistant could be expanded to integrate other publicly available government datasets such as SAM.gov contract opportunities, Federal Acquisition Regulation (FAR) documentation, agency acquisition forecasts, and historical contract archives. Combining multiple authoritative data sources would create a more comprehensive procurement assistant capable of supporting vendors, researchers, and government analysts.

Requirements

Core libraries used in this project include:

torch
transformers
sentence-transformers
faiss-cpu
numpy
pandas
scikit-learn
requests
accelerate

Install dependencies using:

pip install -r requirements.txt

---

References

1. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. https://arxiv.org/abs/2005.11401

2. Qwen Team. (2024). Qwen2.5 Technical Report. https://arxiv.org/abs/2412.15115

3. Wolf, T., Debut, L., Sanh, V., et al. (2020). Transformers: State-of-the-Art Natural Language Processing. https://arxiv.org/abs/1910.03771

4. Reimers, N., & Gurevych, I. (2019). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. https://arxiv.org/abs/1908.10084

5. Johnson, J., Douze, M., & Jégou, H. (2017). Billion-Scale Similarity Search with GPUs (FAISS). https://arxiv.org/abs/1702.08734

6. USAspending.gov. https://www.usaspending.gov/

7. USAspending API Documentation. https://api.usaspending.gov/docs/

8. Federal Spending Guide. https://www.usaspending.gov/federal-spending-guide

9. Hugging Face Transformers Documentation. https://huggingface.co/docs/transformers

10. Sentence Transformers Documentation. https://www.sbert.net/

11. FAISS Documentation. https://faiss.ai/

12. Hugging Face Model Hub. https://huggingface.co/

---

Author

Ryan Adoremos

University of Virginia
MS Data Science

DS 5002 — Final Project

Summer 2026