Use Spanish FastText embeddings with pgvector

.dockerignore

@@ -12,4 +12,5 @@ htmlcov/
 .agents/
 .codex/
 chroma_data/
+.models/
 *.log

.env.example

@@ -28,13 +28,20 @@ PGVECTOR_WRITER_USER=nlp_writer
 PGVECTOR_WRITER_PASSWORD=CAMBIA_ESTA_PASSWORD_WRITER
 PGVECTOR_SSL_MODE=require
 
-EMBEDDING_DIMENSION=16
-EMBEDDING_MODEL=mock-embedding
-EMBEDDING_VERSION=v1
+EMBEDDING_PROVIDER=fasttext
+EMBEDDING_DIMENSION=300
+EMBEDDING_MODEL=facebook/fasttext-es-vectors
+EMBEDDING_VERSION=common-crawl-300-v1
+FASTTEXT_MODEL_PATH=.models/fasttext-es/model.bin
+FASTTEXT_MODEL_REPO_ID=facebook/fasttext-es-vectors
+FASTTEXT_MODEL_FILENAME=model.bin
+FASTTEXT_AUTO_DOWNLOAD=true
 
 BM25_K1=1.5
 BM25_B=0.75
 BM25_RELEVANCE_THRESHOLD=3.0
+SEMANTIC_NO_MATCH_THRESHOLD=0.30
+SEMANTIC_RELEVANCE_THRESHOLD=0.50
 
 LOG_LEVEL=INFO
 REQUEST_TIMEOUT_SECONDS=10

.gitignore

@@ -19,3 +19,4 @@ build/
 .vscode/
 chroma/
 chroma_data/
+.models/

Dockerfile

@@ -3,6 +3,7 @@ FROM python:3.12-slim
 ENV PYTHONDONTWRITEBYTECODE=1
 ENV PYTHONUNBUFFERED=1
 ENV PORT=7860
+ENV FASTTEXT_MODEL_PATH=/opt/models/fasttext-es/model.bin
 
 WORKDIR /app
 
@@ -14,6 +15,15 @@ COPY requirements.txt .
 RUN pip install --no-cache-dir --upgrade pip \
     && pip install --no-cache-dir -r requirements.txt
 
+# Keep the 300-dimensional Spanish FastText model in a cached image layer.
+# Normal source-code changes do not download the multi-GB model again.
+COPY app/shared/nlp/embeddings/download_fasttext_model.py /tmp/download_fasttext_model.py
+RUN HF_HOME=/tmp/hf-cache python /tmp/download_fasttext_model.py \
+    --repo-id facebook/fasttext-es-vectors \
+    --filename model.bin \
+    --destination ${FASTTEXT_MODEL_PATH} \
+    && rm -rf /tmp/hf-cache /tmp/download_fasttext_model.py
+
 COPY app ./app
 COPY sql ./sql
 COPY README.md .

README.md

@@ -7,7 +7,7 @@ pinned: false
 
 # Frimeet API NLP
 
-Servicio NLP independiente para recuperacion de candidatos con pgvector, ranking BM25, recomendaciones, embeddings y redaccion conversacional con Llama via Groq.
+Servicio NLP independiente para busqueda semantica con FastText + pgvector, recomendaciones y redaccion conversacional con Llama via Groq.
 
 La API principal sigue siendo la fuente de verdad de lugares, posts, usuarios, sesiones, permisos y reportes. Este servicio NLP solo trabaja con datos derivados para busqueda semantica.
 
@@ -21,8 +21,8 @@ API principal
 Hugging Face API NLP
   |-- usa credenciales nlp_reader
   |-- consulta RDS PostgreSQL + pgvector
-  |-- genera embedding solo del query del usuario
-  |-- ordena candidatos con BM25
+  |-- genera el embedding FastText del query del usuario
+  |-- ordena por similitud coseno en pgvector
   `-- usa Groq/Llama para embellecer recomendaciones y chat
 
 Hugging Face Jobs
@@ -74,13 +74,20 @@ PGVECTOR_WRITER_USER=nlp_writer
 PGVECTOR_WRITER_PASSWORD=CAMBIA_ESTA_PASSWORD_WRITER
 PGVECTOR_SSL_MODE=require
 
-EMBEDDING_DIMENSION=16
-EMBEDDING_MODEL=mock-embedding
-EMBEDDING_VERSION=v1
+EMBEDDING_PROVIDER=fasttext
+EMBEDDING_DIMENSION=300
+EMBEDDING_MODEL=facebook/fasttext-es-vectors
+EMBEDDING_VERSION=common-crawl-300-v1
+FASTTEXT_MODEL_PATH=.models/fasttext-es/model.bin
+FASTTEXT_MODEL_REPO_ID=facebook/fasttext-es-vectors
+FASTTEXT_MODEL_FILENAME=model.bin
+FASTTEXT_AUTO_DOWNLOAD=true
 
 BM25_K1=1.5
 BM25_B=0.75
 BM25_RELEVANCE_THRESHOLD=3.0
+SEMANTIC_NO_MATCH_THRESHOLD=0.30
+SEMANTIC_RELEVANCE_THRESHOLD=0.50
 
 LOG_LEVEL=INFO
 REQUEST_TIMEOUT_SECONDS=10
@@ -132,6 +139,11 @@ El proyecto incluye `Dockerfile` para Hugging Face Spaces. El contenedor expone
 uvicorn app.main:app --host 0.0.0.0 --port ${PORT:-7860}
 ```
 
+Durante el build, Docker descarga `model.bin` desde el repositorio oficial
+`facebook/fasttext-es-vectors` y lo guarda en `/opt/models/fasttext-es/model.bin`.
+La capa queda cacheada, por lo que un cambio normal de codigo no vuelve a descargar
+el modelo de varios GB.
+
 ## Endpoints
 
 ```http
@@ -169,7 +181,8 @@ python -m app.jobs.sync_place_embeddings
 python -m app.jobs.sync_post_embeddings
 ```
 
-Los jobs calculan `content_hash`; si el hash no cambio, omiten regenerar el embedding.
+Los jobs calculan un `content_hash` versionado con el contenido, modelo, version y
+dimension. Un cambio de modelo fuerza la regeneracion aunque el texto no haya cambiado.
 
 ## SQL RDS
 
@@ -188,11 +201,43 @@ docs/pgvector_post_embeddings_schema.md
 
 Ese SQL debe ejecutarse una vez con un rol administrador/DBA fuera de Hugging Face. La API NLP usa solo `nlp_reader`; los jobs usan solo `nlp_writer`.
 
-## Ranking BM25 Y Llama Via Groq
+### Migracion De VECTOR(16) A FastText VECTOR(300)
+
+La guia operativa completa esta en `docs/fasttext_deployment.md`.
+
+Los vectores son datos derivados. Para esta migracion no se intenta convertir los
+16 valores mock en 300 valores semanticos: se vacian ambas tablas y se reconstruyen
+desde la API principal.
+
+Con la API NLP y los jobs pausados, ejecuta como `nlp_owner` o administrador:
+
+```powershell
+psql "host=<host> port=5432 dbname=nlp_vectors user=<admin> sslmode=require" -f sql/migrate_fasttext_300.sql
+psql "host=<host> port=5432 dbname=nlp_vectors user=<admin> sslmode=require" -f sql/aws_pgvector_contract.sql
+```
+
+Despues configura las variables FastText, despliega la nueva imagen y repuebla:
+
+```powershell
+python -m app.jobs.initial_load_place_embeddings
+python -m app.jobs.initial_load_post_embeddings
+psql "host=<host> port=5432 dbname=nlp_vectors user=<admin> sslmode=require" -f sql/verify_fasttext_embeddings.sql
+```
+
+La verificacion debe reportar dimension `300`, modelo
+`facebook/fasttext-es-vectors` y normas cercanas a `1`.
+
+## Ranking Semantico FastText Y Llama Via Groq
 
-`/places/search` y `/places/recommendations` recuperan candidatos reales filtrados desde pgvector y los reordenan con Okapi BM25, siguiendo `Lab3_BM25_y_Evaluacion.ipynb`. Los valores iniciales son `k1=1.5` y `b=0.75`. Las etiquetas se ponderan `x6` y la categoria `x2` antes de calcular BM25.
+`/places/search` y `/places/recommendations` aplican el flujo de
+`Lab5_Embeddings_Busqueda_Semantica.ipynb`: tokenizan el texto, obtienen los vectores
+FastText de cada termino, calculan su promedio, normalizan el documento y consultan
+pgvector mediante similitud coseno. FastText usa subpalabras, por lo que puede relacionar
+variantes morfologicas y palabras fuera de vocabulario.
 
-Las respuestas incluyen metricas de la consulta actual: origen de candidatos (`pgvector` en produccion), parametros BM25, cobertura de terminos, cantidad de candidatos reales, resultados no cero y estadisticas `min`, `max` y `mean`. `match_quality` puede ser `no_match`, `low_confidence` o `confident` segun `BM25_RELEVANCE_THRESHOLD`.
+Las requests y responses HTTP no cambian. Las metricas existentes ahora describen el
+motor `fasttext_mean_embeddings`, similitud coseno y dimension 300. `match_quality`
+usa `SEMANTIC_NO_MATCH_THRESHOLD` y `SEMANTIC_RELEVANCE_THRESHOLD`.
 
 Ambos endpoints aceptan filtro geografico mediante `lat`, `lng` y `radius` en metros. Cuando se proporcionan coordenadas, el servicio NLP consulta `GET /api/v1/places/nearby` en la API principal y limita pgvector a los IDs devueltos. Las coordenadas siguen perteneciendo a la API principal; no es necesario guardarlas en pgvector, truncar tablas ni regenerar embeddings.
 
@@ -208,7 +253,11 @@ Ambos endpoints aceptan filtro geografico mediante `lat`, `lng` y `radius` en me
 
 El bloque `metrics` tambien indica `location_filter_applied`, `nearby_place_count` y `radius_meters` para hacer visible la aplicacion del radio.
 
-`POST /places/recommendations` no mezcla el benchmark fijo con la consulta del usuario. Si el score maximo es `0`, envia a Llama el modo `no_match` y devuelve `places: []`. Si `0 < max_score < BM25_RELEVANCE_THRESHOLD`, envia `low_confidence`; de lo contrario usa `confident`. Llama solo embellece el tono correspondiente y recibe exclusivamente los lugares ya seleccionados.
+`POST /places/recommendations` no mezcla el benchmark fijo con la consulta del usuario.
+Si el score maximo no supera `SEMANTIC_NO_MATCH_THRESHOLD`, envia a Llama el modo
+`no_match` y devuelve `places: []`. Entre ese valor y
+`SEMANTIC_RELEVANCE_THRESHOLD` usa `low_confidence`; por encima usa `confident`.
+Llama solo embellece el tono y recibe exclusivamente los lugares seleccionados.
 
 `GET` o `POST /places/search/metrics?k=5` conserva un benchmark offline separado llamado `built_in_places_v3_bm25`. Contiene doce lugares controlados, diez consultas y qrels graduados para calcular honestamente `Precision@k`, `Recall@k`, `MRR`, `MAP` y `nDCG@k`. Estas metricas requieren juicios de relevancia y por eso no se presentan como si midieran una consulta arbitraria de produccion.
 
@@ -218,7 +267,9 @@ La respuesta incluye `metric_definitions` con etiquetas y descripciones claras,
 GET /places/search/metrics?k=5
 ```
 
-Para habilitar el filtro por IDs cercanos en una base existente, vuelve a ejecutar solamente el bloque `CREATE OR REPLACE FUNCTION match_places` de `sql/aws_pgvector_contract.sql`. La firma no cambia y no se requiere truncar `place_embeddings`.
+Para actualizar funciones o permisos sin cambiar nuevamente la dimension, vuelve a
+ejecutar `sql/aws_pgvector_contract.sql`. No repitas la migracion destructiva una vez
+que las columnas ya sean `VECTOR(300)`.
 
 Groq/Llama se usa en `/places/recommendations` y `/places/chat` para redactar una respuesta conversacional. No decide que lugares recomendar, no hace busqueda y no inventa lugares.
 

app/jobs/sync_place_embeddings.py

@@ -6,10 +6,11 @@ from app.modules.places.infrastructure.main_api_place_source import (
     MainApiPlacesClient,
     PlaceSourceRecord,
 )
-from app.shared.config.settings import get_settings
+from app.shared.config.settings import Settings, get_settings
 from app.shared.logging.config import configure_logging, get_logger
-from app.shared.nlp.embeddings.mock import MockEmbeddingProvider
-from app.shared.nlp.preprocessing.text import prepare_for_embedding
+from app.shared.nlp.embeddings.base import EmbeddingProvider
+from app.shared.nlp.embeddings.factory import create_embedding_provider
+from app.shared.nlp.embeddings.versioning import versioned_embedding_hash
 from app.shared.vector_store.aws_pgvector import AwsPgvectorClient
 from app.shared.vector_store.models import VectorUpsertRecord
 
@@ -34,7 +35,7 @@ async def main() -> None:
 
     source = MainApiPlacesClient(settings)
     vector_client = AwsPgvectorClient(settings, role="writer")
-    embedding_provider = MockEmbeddingProvider(dimension=settings.embedding_dimension)
+    embedding_provider = create_embedding_provider(settings)
     counters = SyncCounters()
     batch: list[PlaceSourceRecord] = []
 
@@ -47,10 +48,24 @@ async def main() -> None:
     ):
         batch.append(place)
         if len(batch) >= args.batch_size:
-            await _flush_batch(batch, vector_client, embedding_provider, counters, args.dry_run)
+            await _flush_batch(
+                batch,
+                vector_client,
+                embedding_provider,
+                settings,
+                counters,
+                args.dry_run,
+            )
             batch = []
 
-    await _flush_batch(batch, vector_client, embedding_provider, counters, args.dry_run)
+    await _flush_batch(
+        batch,
+        vector_client,
+        embedding_provider,
+        settings,
+        counters,
+        args.dry_run,
+    )
     logger.info(
         "Finished place sync processed=%s skipped=%s upserted=%s errors=%s",
         counters.processed,
@@ -63,7 +78,8 @@ async def main() -> None:
 async def _flush_batch(
     batch: list[PlaceSourceRecord],
     vector_client: AwsPgvectorClient,
-    embedding_provider: MockEmbeddingProvider,
+    embedding_provider: EmbeddingProvider,
+    settings: Settings,
     counters: SyncCounters,
     dry_run: bool,
 ) -> None:
@@ -75,17 +91,26 @@ async def _flush_batch(
         existing_hashes = await vector_client.fetch_place_content_hashes(
             [record.id for record in batch]
         )
+        expected_hashes = {
+            record.id: versioned_embedding_hash(
+                source_content_hash=record.content_hash,
+                model=settings.embedding_model,
+                version=settings.embedding_version,
+                dimension=settings.embedding_dimension,
+            )
+            for record in batch
+        }
         changed = [
             record
             for record in batch
-            if existing_hashes.get(record.id) != record.content_hash
+            if existing_hashes.get(record.id) != expected_hashes[record.id]
         ]
         counters.skipped += len(batch) - len(changed)
         if not changed:
             return
 
         embeddings = embedding_provider.embed_batch(
-            [prepare_for_embedding(record.document) for record in changed]
+            [record.document for record in changed]
         )
         upserts = [
             VectorUpsertRecord(
@@ -93,7 +118,7 @@ async def _flush_batch(
                 document=record.document,
                 metadata=record.metadata,
                 embedding=embedding,
-                content_hash=record.content_hash,
+                content_hash=expected_hashes[record.id],
                 is_active=record.is_active,
             )
             for record, embedding in zip(changed, embeddings)

app/jobs/sync_post_embeddings.py

@@ -6,10 +6,11 @@ from app.modules.posts.infrastructure.main_api_post_source import (
     MainApiPostsClient,
     PostSourceRecord,
 )
-from app.shared.config.settings import get_settings
+from app.shared.config.settings import Settings, get_settings
 from app.shared.logging.config import configure_logging, get_logger
-from app.shared.nlp.embeddings.mock import MockEmbeddingProvider
-from app.shared.nlp.preprocessing.text import prepare_for_embedding
+from app.shared.nlp.embeddings.base import EmbeddingProvider
+from app.shared.nlp.embeddings.factory import create_embedding_provider
+from app.shared.nlp.embeddings.versioning import versioned_embedding_hash
 from app.shared.vector_store.aws_pgvector import AwsPgvectorClient
 from app.shared.vector_store.models import VectorUpsertRecord
 
@@ -34,7 +35,7 @@ async def main() -> None:
 
     source = MainApiPostsClient(settings)
     vector_client = AwsPgvectorClient(settings, role="writer")
-    embedding_provider = MockEmbeddingProvider(dimension=settings.embedding_dimension)
+    embedding_provider = create_embedding_provider(settings)
     counters = SyncCounters()
     batch: list[PostSourceRecord] = []
 
@@ -47,10 +48,24 @@ async def main() -> None:
     ):
         batch.append(post)
         if len(batch) >= args.batch_size:
-            await _flush_batch(batch, vector_client, embedding_provider, counters, args.dry_run)
+            await _flush_batch(
+                batch,
+                vector_client,
+                embedding_provider,
+                settings,
+                counters,
+                args.dry_run,
+            )
             batch = []
 
-    await _flush_batch(batch, vector_client, embedding_provider, counters, args.dry_run)
+    await _flush_batch(
+        batch,
+        vector_client,
+        embedding_provider,
+        settings,
+        counters,
+        args.dry_run,
+    )
     logger.info(
         "Finished post sync processed=%s skipped=%s upserted=%s errors=%s",
         counters.processed,
@@ -63,7 +78,8 @@ async def main() -> None:
 async def _flush_batch(
     batch: list[PostSourceRecord],
     vector_client: AwsPgvectorClient,
-    embedding_provider: MockEmbeddingProvider,
+    embedding_provider: EmbeddingProvider,
+    settings: Settings,
     counters: SyncCounters,
     dry_run: bool,
 ) -> None:
@@ -75,17 +91,26 @@ async def _flush_batch(
         existing_hashes = await vector_client.fetch_post_content_hashes(
             [record.id for record in batch]
         )
+        expected_hashes = {
+            record.id: versioned_embedding_hash(
+                source_content_hash=record.content_hash,
+                model=settings.embedding_model,
+                version=settings.embedding_version,
+                dimension=settings.embedding_dimension,
+            )
+            for record in batch
+        }
         changed = [
             record
             for record in batch
-            if existing_hashes.get(record.id) != record.content_hash
+            if existing_hashes.get(record.id) != expected_hashes[record.id]
         ]
         counters.skipped += len(batch) - len(changed)
         if not changed:
             return
 
         embeddings = embedding_provider.embed_batch(
-            [prepare_for_embedding(record.document) for record in changed]
+            [record.document for record in changed]
         )
         upserts = [
             VectorUpsertRecord(
@@ -93,7 +118,7 @@ async def _flush_batch(
                 document=record.document,
                 metadata=record.metadata,
                 embedding=embedding,
-                content_hash=record.content_hash,
+                content_hash=expected_hashes[record.id],
                 is_active=record.is_active,
             )
             for record, embedding in zip(changed, embeddings)

app/modules/places/api/dependencies.py

@@ -19,6 +19,7 @@ from app.modules.places.infrastructure.place_search_benchmark import (
     QRELS_SOURCE,
     get_default_place_search_benchmark,
 )
+from app.modules.places.infrastructure.semantic_place_ranker import SemanticPlaceRanker
 from app.shared.cache.memory import SimpleTTLCache
 from app.shared.config.settings import get_settings
 from app.shared.dependencies import get_embedding_provider, get_llm_provider
@@ -36,9 +37,9 @@ def get_place_repository() -> MockPlaceVectorRepository | AwsPgvectorPlaceReposi
 
 
 @lru_cache
-def get_place_ranker() -> Bm25PlaceRanker:
+def get_place_ranker() -> SemanticPlaceRanker:
     settings = get_settings()
-    return Bm25PlaceRanker(k1=settings.bm25_k1, b=settings.bm25_b)
+    return SemanticPlaceRanker(dimension=settings.embedding_dimension)
 
 
 @lru_cache
@@ -60,7 +61,8 @@ def get_search_places_use_case() -> SearchPlacesUseCase:
         ranker=get_place_ranker(),
         cache=get_place_search_cache(),
         nearby_place_provider=get_nearby_place_provider(),
-        relevance_threshold=get_settings().bm25_relevance_threshold,
+        relevance_threshold=get_settings().semantic_relevance_threshold,
+        no_match_threshold=get_settings().semantic_no_match_threshold,
     )
 
 

app/modules/places/application/use_cases/recommend_places.py

@@ -91,8 +91,11 @@ class RecommendPlacesUseCase:
             places=places,
             metrics=metrics,
             metadata={
-                "strategy": "pgvector_candidates_plus_bm25_ranking",
-                "ranking": "bm25",
+                "strategy": (
+                    f"{search_result.metrics.candidate_retrieval}_candidates_plus_"
+                    f"{search_result.metrics.engine}"
+                ),
+                "ranking": search_result.metrics.engine,
                 "response_mode": response_mode,
                 "relevance_threshold": search_result.metrics.relevance_threshold,
                 "llm_provider": llm_provider,

app/modules/places/application/use_cases/search_places.py

@@ -30,6 +30,7 @@ class SearchPlacesUseCase:
         cache: SimpleTTLCache | None = None,
         nearby_place_provider: NearbyPlaceProvider | None = None,
         relevance_threshold: float = 3.0,
+        no_match_threshold: float = 0.0,
     ) -> None:
         self._embedding_provider = embedding_provider
         self._place_repository = place_repository
@@ -37,6 +38,7 @@ class SearchPlacesUseCase:
         self._cache = cache
         self._nearby_place_provider = nearby_place_provider
         self._relevance_threshold = relevance_threshold
+        self._no_match_threshold = no_match_threshold
 
     async def execute(
         self,
@@ -138,7 +140,11 @@ class SearchPlacesUseCase:
             field_weights=self._ranker.field_weights,
             ranking_parameters=self._ranker.ranking_parameters,
             relevance_threshold=self._relevance_threshold,
-            match_quality=_match_quality(max_score, self._relevance_threshold),
+            match_quality=_match_quality(
+                max_score,
+                self._relevance_threshold,
+                self._no_match_threshold,
+            ),
             query_token_count=len(query_terms),
             matched_query_token_count=len(matched_query_terms),
             query_coverage=(
@@ -179,8 +185,12 @@ class SearchPlacesUseCase:
         return json.dumps(payload, sort_keys=True, ensure_ascii=True)
 
 
-def _match_quality(max_score: float, relevance_threshold: float) -> str:
-    if max_score <= 0:
+def _match_quality(
+    max_score: float,
+    relevance_threshold: float,
+    no_match_threshold: float = 0.0,
+) -> str:
+    if max_score <= no_match_threshold:
         return "no_match"
     if max_score < relevance_threshold:
         return "low_confidence"

app/modules/places/infrastructure/semantic_place_ranker.py

@@ -0,0 +1,25 @@
+from typing import Sequence
+
+from app.modules.places.application.ports.ranker import PlaceRanker
+from app.modules.places.domain.models import PlaceCandidate, PlaceFilters
+
+
+class SemanticPlaceRanker(PlaceRanker):
+    """Preserve the cosine-similarity order returned by PGVector."""
+
+    engine_name = "fasttext_mean_embeddings"
+    score_metric = "cosine_similarity"
+    field_weights = {"document": 1}
+
+    def __init__(self, dimension: int = 300) -> None:
+        self.ranking_parameters = {"dimension": float(dimension)}
+
+    def rank(
+        self,
+        query: str,
+        places: Sequence[PlaceCandidate],
+        filters: PlaceFilters,
+        limit: int,
+    ) -> list[PlaceCandidate]:
+        del query, filters
+        return sorted(places, key=lambda place: place.score, reverse=True)[:limit]

app/shared/config/settings.py

@@ -71,9 +71,29 @@ class Settings(BaseSettings):
     pgvector_ssl_mode: str = Field(default="require", alias="PGVECTOR_SSL_MODE")
     pgvector_places_table: str = "place_embeddings"
     pgvector_posts_table: str = "post_embeddings"
-    embedding_dimension: int = Field(default=16, alias="EMBEDDING_DIMENSION")
-    embedding_model: str = Field(default="mock-embedding", alias="EMBEDDING_MODEL")
-    embedding_version: str = Field(default="v1", alias="EMBEDDING_VERSION")
+    embedding_provider: str = Field(default="fasttext", alias="EMBEDDING_PROVIDER")
+    embedding_dimension: int = Field(default=300, alias="EMBEDDING_DIMENSION")
+    embedding_model: str = Field(
+        default="facebook/fasttext-es-vectors",
+        alias="EMBEDDING_MODEL",
+    )
+    embedding_version: str = Field(default="common-crawl-300-v1", alias="EMBEDDING_VERSION")
+    fasttext_model_path: str = Field(
+        default=".models/fasttext-es/model.bin",
+        alias="FASTTEXT_MODEL_PATH",
+    )
+    fasttext_model_repo_id: str = Field(
+        default="facebook/fasttext-es-vectors",
+        alias="FASTTEXT_MODEL_REPO_ID",
+    )
+    fasttext_model_filename: str = Field(
+        default="model.bin",
+        alias="FASTTEXT_MODEL_FILENAME",
+    )
+    fasttext_auto_download: bool = Field(
+        default=True,
+        alias="FASTTEXT_AUTO_DOWNLOAD",
+    )
 
     bm25_k1: float = Field(default=1.5, gt=0, alias="BM25_K1")
     bm25_b: float = Field(default=0.75, ge=0, le=1, alias="BM25_B")
@@ -82,6 +102,18 @@ class Settings(BaseSettings):
         gt=0,
         alias="BM25_RELEVANCE_THRESHOLD",
     )
+    semantic_no_match_threshold: float = Field(
+        default=0.30,
+        ge=-1,
+        le=1,
+        alias="SEMANTIC_NO_MATCH_THRESHOLD",
+    )
+    semantic_relevance_threshold: float = Field(
+        default=0.50,
+        ge=-1,
+        le=1,
+        alias="SEMANTIC_RELEVANCE_THRESHOLD",
+    )
 
     log_level: str = Field(default="INFO", alias="LOG_LEVEL")
     request_timeout_seconds: int = Field(default=10, alias="REQUEST_TIMEOUT_SECONDS")

app/shared/dependencies.py

@@ -4,7 +4,7 @@ from app.shared.cache.memory import SimpleTTLCache
 from app.shared.config.settings import get_settings
 from app.shared.nlp.embeddings.base import EmbeddingProvider
 from app.shared.nlp.embeddings.cached import CachedEmbeddingProvider
-from app.shared.nlp.embeddings.mock import MockEmbeddingProvider
+from app.shared.nlp.embeddings.factory import create_embedding_provider
 from app.shared.nlp.llm.base import LLMProvider
 from app.shared.nlp.llm.groq_llama import GroqLlamaProvider
 from app.shared.nlp.llm.mock import MockLLMProvider
@@ -14,7 +14,7 @@ from app.shared.nlp.llm.mock import MockLLMProvider
 def get_embedding_provider() -> EmbeddingProvider:
     settings = get_settings()
     return CachedEmbeddingProvider(
-        provider=MockEmbeddingProvider(dimension=settings.embedding_dimension),
+        provider=create_embedding_provider(settings),
         cache=SimpleTTLCache(default_ttl_seconds=settings.embedding_cache_ttl_seconds),
     )
 

app/shared/nlp/embeddings/download_fasttext_model.py

@@ -0,0 +1,58 @@
+import argparse
+from pathlib import Path
+import shutil
+
+
+DEFAULT_REPO_ID = "facebook/fasttext-es-vectors"
+DEFAULT_FILENAME = "model.bin"
+DEFAULT_DESTINATION = "/opt/models/fasttext-es/model.bin"
+
+
+def ensure_fasttext_model(
+    destination: str,
+    repo_id: str = DEFAULT_REPO_ID,
+    filename: str = DEFAULT_FILENAME,
+) -> Path:
+    target = Path(destination).expanduser()
+    if target.is_file():
+        return target
+
+    target.parent.mkdir(parents=True, exist_ok=True)
+    try:
+        from huggingface_hub import hf_hub_download
+    except ImportError as exc:
+        raise RuntimeError(
+            "huggingface-hub is required to download the FastText model"
+        ) from exc
+
+    downloaded = Path(
+        hf_hub_download(
+            repo_id=repo_id,
+            filename=filename,
+            local_dir=target.parent,
+        )
+    )
+    if downloaded.resolve() == target.resolve():
+        return target
+    temporary_target = target.with_suffix(target.suffix + ".part")
+    shutil.copyfile(downloaded, temporary_target)
+    temporary_target.replace(target)
+    return target
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description="Download the Spanish FastText model.")
+    parser.add_argument("--repo-id", default=DEFAULT_REPO_ID)
+    parser.add_argument("--filename", default=DEFAULT_FILENAME)
+    parser.add_argument("--destination", default=DEFAULT_DESTINATION)
+    args = parser.parse_args()
+    path = ensure_fasttext_model(
+        destination=args.destination,
+        repo_id=args.repo_id,
+        filename=args.filename,
+    )
+    print(path)
+
+
+if __name__ == "__main__":
+    main()

app/shared/nlp/embeddings/factory.py

@@ -0,0 +1,19 @@
+from app.shared.config.settings import Settings
+from app.shared.nlp.embeddings.base import EmbeddingProvider
+from app.shared.nlp.embeddings.fasttext import FastTextEmbeddingProvider
+from app.shared.nlp.embeddings.mock import MockEmbeddingProvider
+
+
+def create_embedding_provider(settings: Settings) -> EmbeddingProvider:
+    provider = settings.embedding_provider.casefold()
+    if provider == "fasttext":
+        return FastTextEmbeddingProvider(
+            model_path=settings.fasttext_model_path,
+            expected_dimension=settings.embedding_dimension,
+            repo_id=settings.fasttext_model_repo_id,
+            filename=settings.fasttext_model_filename,
+            auto_download=settings.fasttext_auto_download,
+        )
+    if provider == "mock":
+        return MockEmbeddingProvider(dimension=settings.embedding_dimension)
+    raise ValueError(f"Unsupported EMBEDDING_PROVIDER: {settings.embedding_provider}")

app/shared/nlp/embeddings/fasttext.py

@@ -0,0 +1,107 @@
+import math
+from pathlib import Path
+from typing import Any, Callable, Protocol, Sequence
+
+from app.shared.nlp.embeddings.base import EmbeddingProvider
+from app.shared.nlp.embeddings.download_fasttext_model import ensure_fasttext_model
+from app.shared.nlp.preprocessing.text import tokenize_for_embeddings
+
+
+class FastTextModel(Protocol):
+    def get_dimension(self) -> int: ...
+
+    def get_word_vector(self, word: str) -> Sequence[float]: ...
+
+
+ModelLoader = Callable[[str], FastTextModel]
+
+
+class FastTextEmbeddingProvider(EmbeddingProvider):
+    """Mean-pooled, L2-normalized Spanish FastText document embeddings."""
+
+    def __init__(
+        self,
+        model_path: str,
+        expected_dimension: int = 300,
+        repo_id: str = "facebook/fasttext-es-vectors",
+        filename: str = "model.bin",
+        auto_download: bool = True,
+        model_loader: ModelLoader | None = None,
+    ) -> None:
+        if model_loader is None:
+            resolved_path = self._resolve_model_path(
+                model_path=model_path,
+                repo_id=repo_id,
+                filename=filename,
+                auto_download=auto_download,
+            )
+            model_loader = _load_fasttext_model
+        else:
+            resolved_path = Path(model_path)
+
+        self._model = model_loader(str(resolved_path))
+        self.dimension = int(self._model.get_dimension())
+        if self.dimension != expected_dimension:
+            raise ValueError(
+                "FastText model dimension does not match EMBEDDING_DIMENSION: "
+                f"model={self.dimension}, configured={expected_dimension}"
+            )
+
+    def embed_text(self, text: str) -> list[float]:
+        tokens = tokenize_for_embeddings(text)
+        if not tokens:
+            return [0.0] * self.dimension
+
+        summed = [0.0] * self.dimension
+        for token in tokens:
+            vector = self._model.get_word_vector(token)
+            if len(vector) != self.dimension:
+                raise ValueError(
+                    f"FastText returned dimension {len(vector)} for token {token!r}"
+                )
+            for index, value in enumerate(vector):
+                summed[index] += float(value)
+
+        averaged = [value / len(tokens) for value in summed]
+        return _l2_normalize(averaged)
+
+    def embed_batch(self, texts: list[str]) -> list[list[float]]:
+        return [self.embed_text(text) for text in texts]
+
+    @staticmethod
+    def _resolve_model_path(
+        model_path: str,
+        repo_id: str,
+        filename: str,
+        auto_download: bool,
+    ) -> Path:
+        path = Path(model_path).expanduser()
+        if path.is_file():
+            return path
+        if not auto_download:
+            raise FileNotFoundError(
+                f"FastText model not found at {path}. "
+                "Download it or enable FASTTEXT_AUTO_DOWNLOAD."
+            )
+        return ensure_fasttext_model(
+            destination=str(path),
+            repo_id=repo_id,
+            filename=filename,
+        )
+
+
+def _load_fasttext_model(path: str) -> Any:
+    try:
+        import fasttext
+    except ImportError as exc:
+        raise RuntimeError(
+            "fasttext-wheel is required for FastText embeddings"
+        ) from exc
+    return fasttext.load_model(path)
+
+
+def _l2_normalize(vector: list[float]) -> list[float]:
+    norm = math.sqrt(sum(value * value for value in vector))
+    if norm == 0:
+        return vector
+    return [value / norm for value in vector]

app/shared/nlp/embeddings/versioning.py

@@ -0,0 +1,18 @@
+from app.shared.content_hash import stable_content_hash
+
+
+def versioned_embedding_hash(
+    source_content_hash: str,
+    model: str,
+    version: str,
+    dimension: int,
+) -> str:
+    """Invalidate derived vectors when content or embedding configuration changes."""
+    return stable_content_hash(
+        {
+            "source_content_hash": source_content_hash,
+            "embedding_model": model,
+            "embedding_version": version,
+            "embedding_dimension": dimension,
+        }
+    )

app/shared/nlp/preprocessing/text.py

@@ -2,6 +2,66 @@ import re
 import unicodedata
 
 
+SEMANTIC_TOKEN_PATTERN = re.compile(r"[a-z0-9]+")
+SEMANTIC_STOPWORDS = {
+    "a",
+    "al",
+    "algo",
+    "algun",
+    "alguna",
+    "algunas",
+    "algunos",
+    "con",
+    "cual",
+    "cuando",
+    "de",
+    "del",
+    "donde",
+    "el",
+    "ella",
+    "en",
+    "es",
+    "esta",
+    "este",
+    "hay",
+    "la",
+    "las",
+    "lo",
+    "los",
+    "me",
+    "mi",
+    "mis",
+    "para",
+    "pero",
+    "por",
+    "que",
+    "quiero",
+    "se",
+    "ser",
+    "su",
+    "sus",
+    "te",
+    "tener",
+    "tu",
+    "tus",
+    "un",
+    "una",
+    "unas",
+    "uno",
+    "unos",
+    "ver",
+    "y",
+    "ya",
+    "yo",
+    "busco",
+    "buscar",
+    "lugar",
+    "lugares",
+    "necesito",
+    "puedo",
+}
+
+
 def clean_text(text: str) -> str:
     text = text or ""
     text = re.sub(r"[\r\n\t]+", " ", text)
@@ -26,3 +86,17 @@ def strip_accents(text: str) -> str:
 
 def prepare_for_embedding(text: str) -> str:
     return normalize_text(text, remove_accents=True)
+
+
+def tokenize_for_embeddings(text: str) -> list[str]:
+    """Tokenize Spanish text for mean FastText document embeddings."""
+    normalized = prepare_for_embedding(text)
+    tokens = [
+        token
+        for token in SEMANTIC_TOKEN_PATTERN.findall(normalized)
+        if token not in SEMANTIC_STOPWORDS
+        and (len(token) > 1 or token.isdigit())
+    ]
+    if tokens:
+        return tokens
+    return SEMANTIC_TOKEN_PATTERN.findall(normalized)

app/shared/nlp/prompts/place_chat.py

@@ -9,7 +9,7 @@ Eres un redactor conversacional para una app de planes y recomendaciones de luga
 Tu trabajo es embellecer la respuesta final usando solo los lugares proporcionados por el sistema.
 
 Reglas obligatorias:
-- No decidas que lugares recomendar; la lista ya fue seleccionada por PGVector, filtros y BM25.
+- No decidas que lugares recomendar; la lista ya fue seleccionada por FastText, PGVector y filtros.
 - No inventes nombres de lugares.
 - No inventes horarios, precios, direcciones, calificaciones ni promociones.
 - No digas que un lugar esta abierto si el contexto no lo indica.

docs/fasttext_deployment.md

@@ -0,0 +1,117 @@
+# Migracion Y Despliegue De FastText
+
+Esta migracion cambia los vectores derivados de 16 dimensiones mock a embeddings
+FastText reales de 300 dimensiones. No cambia ningun contrato HTTP.
+
+## 1. Preparar Y Validar Localmente
+
+```powershell
+cd C:\Users\aleja\Desktop\Uni\9o\MID\C2\Frimeet-API-NLP
+.\.venv\Scripts\python.exe -m pip install -r requirements.txt
+.\.venv\Scripts\python.exe -m pytest -q
+.\.venv\Scripts\python.exe -m app.shared.nlp.embeddings.download_fasttext_model `
+  --repo-id facebook/fasttext-es-vectors `
+  --filename model.bin `
+  --destination .models/fasttext-es/model.bin
+```
+
+Configura el `.env` local con:
+
+```env
+EMBEDDING_PROVIDER=fasttext
+EMBEDDING_DIMENSION=300
+EMBEDDING_MODEL=facebook/fasttext-es-vectors
+EMBEDDING_VERSION=common-crawl-300-v1
+FASTTEXT_MODEL_PATH=.models/fasttext-es/model.bin
+FASTTEXT_MODEL_REPO_ID=facebook/fasttext-es-vectors
+FASTTEXT_MODEL_FILENAME=model.bin
+FASTTEXT_AUTO_DOWNLOAD=true
+SEMANTIC_NO_MATCH_THRESHOLD=0.30
+SEMANTIC_RELEVANCE_THRESHOLD=0.50
+```
+
+Antes de tocar la base, valida una pagina real sin escribir datos:
+
+```powershell
+.\.venv\Scripts\python.exe -m app.jobs.sync_place_embeddings `
+  --dry-run --max-pages 1 --page-limit 5
+.\.venv\Scripts\python.exe -m app.jobs.sync_post_embeddings `
+  --dry-run --max-pages 1 --page-limit 5
+```
+
+Ambos comandos deben terminar con `errors=0`. Este ensayo comprueba la descarga y
+carga de FastText, la lectura de la API principal y las credenciales de lectura del
+contrato pgvector, pero no modifica RDS.
+
+## 2. Publicar Codigo En GitHub
+
+```powershell
+git add .
+git commit -m "Use Spanish FastText embeddings with pgvector"
+git push origin hf-deploy
+```
+
+## 3. Pausar El Space Y Migrar RDS
+
+Pausa el Space antes de cambiar la dimension; la API anterior genera vectores de 16
+dimensiones y no puede consultar una columna `VECTOR(300)`.
+
+Haz un snapshot de RDS y ejecuta con un usuario administrador o `nlp_owner`:
+
+```powershell
+psql "host=<host> port=5432 dbname=nlp_vectors user=<admin> sslmode=require" `
+  -f sql/migrate_fasttext_300.sql
+psql "host=<host> port=5432 dbname=nlp_vectors user=<admin> sslmode=require" `
+  -f sql/aws_pgvector_contract.sql
+```
+
+La migracion trunca `place_embeddings` y `post_embeddings` porque son indices
+derivados incompatibles. No toca la base transaccional de la API principal.
+
+## 4. Repoblar PGVector
+
+Con las credenciales writer en `.env`:
+
+```powershell
+.\.venv\Scripts\python.exe -m app.jobs.initial_load_place_embeddings
+.\.venv\Scripts\python.exe -m app.jobs.initial_load_post_embeddings
+psql "host=<host> port=5432 dbname=nlp_vectors user=<admin> sslmode=require" `
+  -f sql/verify_fasttext_embeddings.sql
+```
+
+La verificacion debe mostrar filas, dimension 300, el modelo FastText configurado y
+normas cercanas a 1.
+
+## 5. Actualizar Hugging Face
+
+En Settings del Space conserva `VECTOR_STORE_PROVIDER=aws_pgvector`, las credenciales
+actuales de RDS/Groq y configura estas variables de embedding:
+
+```env
+EMBEDDING_PROVIDER=fasttext
+EMBEDDING_DIMENSION=300
+EMBEDDING_MODEL=facebook/fasttext-es-vectors
+EMBEDDING_VERSION=common-crawl-300-v1
+FASTTEXT_MODEL_PATH=/opt/models/fasttext-es/model.bin
+FASTTEXT_MODEL_REPO_ID=facebook/fasttext-es-vectors
+FASTTEXT_MODEL_FILENAME=model.bin
+FASTTEXT_AUTO_DOWNLOAD=false
+SEMANTIC_NO_MATCH_THRESHOLD=0.30
+SEMANTIC_RELEVANCE_THRESHOLD=0.50
+```
+
+El Dockerfile descarga el modelo durante el build. Publica la rama local en `main`
+del Space:
+
+```powershell
+git push hf hf-deploy:main
+```
+
+Cuando el build termine, reinicia el Space y revisa `/ready` y una consulta real a
+`POST /places/recommendations`.
+
+## Rollback
+
+No intentes insertar vectores de 16 dimensiones en las columnas nuevas. Para volver
+al motor anterior se requiere restaurar el snapshot o ejecutar una migracion inversa,
+repoblar los indices de 16 dimensiones y desplegar el commit anterior.

docs/pgvector_place_embeddings_schema.md

@@ -13,7 +13,7 @@ CREATE TABLE IF NOT EXISTS place_embeddings (
     external_id TEXT PRIMARY KEY,
     document TEXT NOT NULL,
     metadata JSONB NOT NULL DEFAULT '{}'::jsonb,
-    embedding VECTOR(16) NOT NULL,
+    embedding VECTOR(300) NOT NULL,
     content_hash TEXT NOT NULL,
     embedding_model TEXT NOT NULL,
     embedding_version TEXT NOT NULL,
@@ -22,19 +22,8 @@ CREATE TABLE IF NOT EXISTS place_embeddings (
 );
 ```
 
-`VECTOR(16)` corresponde al `MockEmbeddingProvider` actual.
-
-Cuando se cambie a embeddings reales, hay que cambiar:
-
-```env
-EMBEDDING_DIMENSION=<dimension_real>
-```
-
-y tambien:
-
-```sql
-embedding VECTOR(<dimension_real>)
-```
+`VECTOR(300)` corresponde al modelo preentrenado
+`facebook/fasttext-es-vectors` usado por `FastTextEmbeddingProvider`.
 
 ## Columnas
 
@@ -43,8 +32,8 @@ embedding VECTOR(<dimension_real>)
 | `external_id` | `TEXT` | ID del lugar en la API principal. |
 | `document` | `TEXT` | Texto construido para generar el embedding. |
 | `metadata` | `JSONB` | Datos estructurados utiles para filtros y respuesta. |
-| `embedding` | `VECTOR(16)` | Embedding del `document`. |
-| `content_hash` | `TEXT` | SHA-256 estable de document + metadata + is_active. |
+| `embedding` | `VECTOR(300)` | Promedio normalizado de embeddings FastText del `document`. |
+| `content_hash` | `TEXT` | SHA-256 del contenido y la configuracion/version del embedding. |
 | `embedding_model` | `TEXT` | Nombre del modelo usado para generar embeddings. |
 | `embedding_version` | `TEXT` | Version logica del embedding. |
 | `is_active` | `BOOLEAN` | Estado derivado desde la API principal. |
@@ -85,13 +74,14 @@ Cafe Centro cafe Tuxtla Gutierrez Chiapas osm cafe tranquilo Un lugar para plati
 
 ## Content Hash
 
-`content_hash` se calcula con SHA-256 sobre:
+`content_hash` se calcula con SHA-256 sobre el hash del contenido mas:
 
 ```json
 {
-  "document": "...",
-  "metadata": {...},
-  "is_active": true
+  "source_content_hash": "...",
+  "embedding_model": "facebook/fasttext-es-vectors",
+  "embedding_version": "common-crawl-300-v1",
+  "embedding_dimension": 300
 }
 ```
 
@@ -102,7 +92,7 @@ Si el hash no cambia, el job omite regenerar embedding.
 La API NLP usa:
 
 ```sql
-match_places(query_embedding VECTOR(16), match_count INTEGER, filters JSONB)
+match_places(query_embedding VECTOR(300), match_count INTEGER, filters JSONB)
 ```
 
 `filters.place_ids` acepta un arreglo de IDs generado por la consulta geografica a la API principal. `match_places` limita los resultados a esos `external_id`; no almacena coordenadas en esta tabla.
@@ -114,7 +104,7 @@ upsert_place_embedding(
     p_external_id TEXT,
     p_document TEXT,
     p_metadata JSONB,
-    p_embedding VECTOR(16),
+    p_embedding VECTOR(300),
     p_content_hash TEXT,
     p_embedding_model TEXT,
     p_embedding_version TEXT,

docs/pgvector_post_embeddings_schema.md

@@ -13,7 +13,7 @@ CREATE TABLE IF NOT EXISTS post_embeddings (
     external_id TEXT PRIMARY KEY,
     document TEXT NOT NULL,
     metadata JSONB NOT NULL DEFAULT '{}'::jsonb,
-    embedding VECTOR(16) NOT NULL,
+    embedding VECTOR(300) NOT NULL,
     content_hash TEXT NOT NULL,
     embedding_model TEXT NOT NULL,
     embedding_version TEXT NOT NULL,
@@ -22,19 +22,8 @@ CREATE TABLE IF NOT EXISTS post_embeddings (
 );
 ```
 
-`VECTOR(16)` corresponde al `MockEmbeddingProvider` actual.
-
-Cuando se cambie a embeddings reales, hay que cambiar:
-
-```env
-EMBEDDING_DIMENSION=<dimension_real>
-```
-
-y tambien:
-
-```sql
-embedding VECTOR(<dimension_real>)
-```
+`VECTOR(300)` corresponde al modelo preentrenado
+`facebook/fasttext-es-vectors` usado por `FastTextEmbeddingProvider`.
 
 ## Columnas
 
@@ -43,8 +32,8 @@ embedding VECTOR(<dimension_real>)
 | `external_id` | `TEXT` | ID del post en la API principal. |
 | `document` | `TEXT` | Texto construido para generar el embedding. |
 | `metadata` | `JSONB` | Datos estructurados utiles para filtros y respuesta. |
-| `embedding` | `VECTOR(16)` | Embedding del `document`. |
-| `content_hash` | `TEXT` | SHA-256 estable de document + metadata + is_active. |
+| `embedding` | `VECTOR(300)` | Promedio normalizado de embeddings FastText del `document`. |
+| `content_hash` | `TEXT` | SHA-256 del contenido y la configuracion/version del embedding. |
 | `embedding_model` | `TEXT` | Nombre del modelo usado para generar embeddings. |
 | `embedding_version` | `TEXT` | Version logica del embedding. |
 | `is_active` | `BOOLEAN` | Estado derivado desde la API principal. |
@@ -81,13 +70,14 @@ Plan de cafe Tuxtla Gutierrez Chiapas internal cafe amigos Una publicacion para
 
 ## Content Hash
 
-`content_hash` se calcula con SHA-256 sobre:
+`content_hash` se calcula con SHA-256 sobre el hash del contenido mas:
 
 ```json
 {
-  "document": "...",
-  "metadata": {...},
-  "is_active": true
+  "source_content_hash": "...",
+  "embedding_model": "facebook/fasttext-es-vectors",
+  "embedding_version": "common-crawl-300-v1",
+  "embedding_dimension": 300
 }
 ```
 
@@ -98,7 +88,7 @@ Si el hash no cambia, el job omite regenerar embedding.
 La API NLP usa:
 
 ```sql
-match_posts(query_embedding VECTOR(16), match_count INTEGER, filters JSONB)
+match_posts(query_embedding VECTOR(300), match_count INTEGER, filters JSONB)
 ```
 
 Los jobs usan:
@@ -108,7 +98,7 @@ upsert_post_embedding(
     p_external_id TEXT,
     p_document TEXT,
     p_metadata JSONB,
-    p_embedding VECTOR(16),
+    p_embedding VECTOR(300),
     p_content_hash TEXT,
     p_embedding_model TEXT,
     p_embedding_version TEXT,

requirements.txt

@@ -5,5 +5,8 @@ python-dotenv>=1.0,<2.0
 httpx>=0.27,<1.0
 asyncpg>=0.29,<1.0
 groq>=0.9,<1.0
+fasttext-wheel==0.9.2
+huggingface-hub>=0.34,<2.0
+numpy>=1.26,<2.0
 pytest>=8.0,<9.0
 pytest-asyncio>=0.23,<1.0

sql/aws_pgvector_contract.sql

@@ -7,7 +7,7 @@ CREATE TABLE IF NOT EXISTS place_embeddings (
     external_id TEXT PRIMARY KEY,
     document TEXT NOT NULL,
     metadata JSONB NOT NULL DEFAULT '{}'::jsonb,
-    embedding VECTOR(16) NOT NULL,
+    embedding VECTOR(300) NOT NULL,
     content_hash TEXT NOT NULL,
     embedding_model TEXT NOT NULL,
     embedding_version TEXT NOT NULL,
@@ -19,7 +19,7 @@ CREATE TABLE IF NOT EXISTS post_embeddings (
     external_id TEXT PRIMARY KEY,
     document TEXT NOT NULL,
     metadata JSONB NOT NULL DEFAULT '{}'::jsonb,
-    embedding VECTOR(16) NOT NULL,
+    embedding VECTOR(300) NOT NULL,
     content_hash TEXT NOT NULL,
     embedding_model TEXT NOT NULL,
     embedding_version TEXT NOT NULL,
@@ -40,7 +40,7 @@ CREATE INDEX IF NOT EXISTS post_embeddings_metadata_gin_idx
 ON post_embeddings USING gin (metadata);
 
 CREATE OR REPLACE FUNCTION match_places(
-    query_embedding VECTOR(16),
+    query_embedding VECTOR(300),
     match_count INTEGER,
     filters JSONB DEFAULT '{}'::jsonb
 )
@@ -77,7 +77,7 @@ AS $$
 $$;
 
 CREATE OR REPLACE FUNCTION match_posts(
-    query_embedding VECTOR(16),
+    query_embedding VECTOR(300),
     match_count INTEGER,
     filters JSONB DEFAULT '{}'::jsonb
 )
@@ -109,7 +109,7 @@ CREATE OR REPLACE FUNCTION upsert_place_embedding(
     p_external_id TEXT,
     p_document TEXT,
     p_metadata JSONB,
-    p_embedding VECTOR(16),
+    p_embedding VECTOR(300),
     p_content_hash TEXT,
     p_embedding_model TEXT,
     p_embedding_version TEXT,
@@ -157,7 +157,7 @@ CREATE OR REPLACE FUNCTION upsert_post_embedding(
     p_external_id TEXT,
     p_document TEXT,
     p_metadata JSONB,
-    p_embedding VECTOR(16),
+    p_embedding VECTOR(300),
     p_content_hash TEXT,
     p_embedding_model TEXT,
     p_embedding_version TEXT,

sql/aws_pgvector_full_setup.psql.sql

@@ -3,7 +3,7 @@
 -- psql "host=<host> port=5432 dbname=postgres user=<admin> sslmode=require" -f sql/aws_pgvector_full_setup.psql.sql
 --
 -- Replace the passwords before running.
--- VECTOR(16) must match EMBEDDING_DIMENSION=16 in the API environment.
+-- VECTOR(300) must match EMBEDDING_DIMENSION=300 in the API environment.
 
 \set ON_ERROR_STOP on
 
@@ -41,7 +41,7 @@ CREATE TABLE IF NOT EXISTS public.place_embeddings (
     external_id TEXT PRIMARY KEY,
     document TEXT NOT NULL,
     metadata JSONB NOT NULL DEFAULT '{}'::jsonb,
-    embedding VECTOR(16) NOT NULL,
+    embedding VECTOR(300) NOT NULL,
     content_hash TEXT NOT NULL,
     embedding_model TEXT NOT NULL,
     embedding_version TEXT NOT NULL,
@@ -53,7 +53,7 @@ CREATE TABLE IF NOT EXISTS public.post_embeddings (
     external_id TEXT PRIMARY KEY,
     document TEXT NOT NULL,
     metadata JSONB NOT NULL DEFAULT '{}'::jsonb,
-    embedding VECTOR(16) NOT NULL,
+    embedding VECTOR(300) NOT NULL,
     content_hash TEXT NOT NULL,
     embedding_model TEXT NOT NULL,
     embedding_version TEXT NOT NULL,
@@ -74,7 +74,7 @@ CREATE INDEX IF NOT EXISTS post_embeddings_metadata_gin_idx
 ON public.post_embeddings USING gin (metadata);
 
 CREATE OR REPLACE FUNCTION public.match_places(
-    query_embedding VECTOR(16),
+    query_embedding VECTOR(300),
     match_count INTEGER,
     filters JSONB DEFAULT '{}'::jsonb
 )
@@ -111,7 +111,7 @@ AS $$
 $$;
 
 CREATE OR REPLACE FUNCTION public.match_posts(
-    query_embedding VECTOR(16),
+    query_embedding VECTOR(300),
     match_count INTEGER,
     filters JSONB DEFAULT '{}'::jsonb
 )
@@ -143,7 +143,7 @@ CREATE OR REPLACE FUNCTION public.upsert_place_embedding(
     p_external_id TEXT,
     p_document TEXT,
     p_metadata JSONB,
-    p_embedding VECTOR(16),
+    p_embedding VECTOR(300),
     p_content_hash TEXT,
     p_embedding_model TEXT,
     p_embedding_version TEXT,
@@ -191,7 +191,7 @@ CREATE OR REPLACE FUNCTION public.upsert_post_embedding(
     p_external_id TEXT,
     p_document TEXT,
     p_metadata JSONB,
-    p_embedding VECTOR(16),
+    p_embedding VECTOR(300),
     p_content_hash TEXT,
     p_embedding_model TEXT,
     p_embedding_version TEXT,

sql/migrate_fasttext_300.sql

@@ -0,0 +1,60 @@
+-- One-time destructive migration from the old 16-dimensional mock vectors
+-- to the 300-dimensional Spanish FastText vectors.
+--
+-- The tables contain derived search indexes. The main API remains the source
+-- of truth, so old incompatible vectors are intentionally discarded.
+-- Run as nlp_owner or the RDS administrator while the NLP API/jobs are paused.
+
+BEGIN;
+
+LOCK TABLE public.place_embeddings IN ACCESS EXCLUSIVE MODE;
+LOCK TABLE public.post_embeddings IN ACCESS EXCLUSIVE MODE;
+
+DO $$
+DECLARE
+    place_type TEXT;
+    post_type TEXT;
+BEGIN
+    SELECT format_type(attribute.atttypid, attribute.atttypmod)
+    INTO place_type
+    FROM pg_attribute attribute
+    WHERE attribute.attrelid = 'public.place_embeddings'::regclass
+      AND attribute.attname = 'embedding'
+      AND NOT attribute.attisdropped;
+
+    SELECT format_type(attribute.atttypid, attribute.atttypmod)
+    INTO post_type
+    FROM pg_attribute attribute
+    WHERE attribute.attrelid = 'public.post_embeddings'::regclass
+      AND attribute.attname = 'embedding'
+      AND NOT attribute.attisdropped;
+
+    IF place_type <> 'vector(16)' OR post_type <> 'vector(16)' THEN
+        RAISE EXCEPTION
+            'Expected vector(16) columns, found place=% and post=%',
+            place_type,
+            post_type;
+    END IF;
+END $$;
+
+DROP INDEX IF EXISTS public.place_embeddings_embedding_hnsw_idx;
+DROP INDEX IF EXISTS public.post_embeddings_embedding_hnsw_idx;
+
+TRUNCATE TABLE public.place_embeddings, public.post_embeddings;
+
+ALTER TABLE public.place_embeddings
+    ALTER COLUMN embedding TYPE VECTOR(300);
+
+ALTER TABLE public.post_embeddings
+    ALTER COLUMN embedding TYPE VECTOR(300);
+
+CREATE INDEX place_embeddings_embedding_hnsw_idx
+ON public.place_embeddings USING hnsw (embedding vector_cosine_ops);
+
+CREATE INDEX post_embeddings_embedding_hnsw_idx
+ON public.post_embeddings USING hnsw (embedding vector_cosine_ops);
+
+COMMIT;
+
+-- Immediately run sql/aws_pgvector_contract.sql after this migration so the
+-- match/upsert functions also declare VECTOR(300).

sql/verify_fasttext_embeddings.sql

@@ -0,0 +1,21 @@
+SELECT
+    'place_embeddings' AS table_name,
+    count(*) AS row_count,
+    min(vector_dims(embedding)) AS min_dimensions,
+    max(vector_dims(embedding)) AS max_dimensions,
+    min(embedding_model) AS embedding_model,
+    min(embedding_version) AS embedding_version,
+    min(vector_norm(embedding)) AS min_norm,
+    max(vector_norm(embedding)) AS max_norm
+FROM public.place_embeddings
+UNION ALL
+SELECT
+    'post_embeddings' AS table_name,
+    count(*) AS row_count,
+    min(vector_dims(embedding)) AS min_dimensions,
+    max(vector_dims(embedding)) AS max_dimensions,
+    min(embedding_model) AS embedding_model,
+    min(embedding_version) AS embedding_version,
+    min(vector_norm(embedding)) AS min_norm,
+    max(vector_norm(embedding)) AS max_norm
+FROM public.post_embeddings;

tests/conftest.py

@@ -1,4 +1,8 @@
 import os
 
 os.environ["VECTOR_STORE_PROVIDER"] = "mock"
+os.environ["EMBEDDING_PROVIDER"] = "mock"
+os.environ["EMBEDDING_DIMENSION"] = "16"
+os.environ["EMBEDDING_MODEL"] = "mock-embedding"
+os.environ["EMBEDDING_VERSION"] = "test-v1"
 os.environ["GROQ_API_KEY"] = ""

tests/test_api_endpoints.py

@@ -20,11 +20,11 @@ def test_places_search_endpoint() -> None:
     payload = response.json()
     assert payload["query"] == "lugares tranquilos para cenar"
     assert payload["places"]
-    assert payload["metrics"]["engine"] == "bm25"
+    assert payload["metrics"]["engine"] == "fasttext_mean_embeddings"
     assert payload["metrics"]["candidate_retrieval"] == "mock_embeddings"
-    assert payload["metrics"]["score_metric"] == "bm25"
-    assert payload["metrics"]["ranking_parameters"] == {"k1": 1.5, "b": 0.75}
-    assert payload["metrics"]["field_weights"]["tags"] == 6
+    assert payload["metrics"]["score_metric"] == "cosine_similarity"
+    assert payload["metrics"]["ranking_parameters"] == {"dimension": 16.0}
+    assert payload["metrics"]["field_weights"]["document"] == 1
     assert payload["metrics"]["returned_count"] == len(payload["places"])
     assert payload["metrics"]["max_score"] >= payload["metrics"]["mean_score"]
 
@@ -87,7 +87,7 @@ def test_places_chat_endpoint_returns_trace_and_structured_places() -> None:
     assert payload["metadata"]["places_used_as_context"]
 
 
-def test_places_recommendations_returns_llm_message_and_bm25_metadata() -> None:
+def test_places_recommendations_returns_llm_message_and_semantic_metadata() -> None:
     client = TestClient(create_app())
 
     response = client.post(
@@ -104,8 +104,8 @@ def test_places_recommendations_returns_llm_message_and_bm25_metadata() -> None:
     payload = response.json()
     assert payload["message"]
     assert payload["places"]
-    assert payload["metrics"]["engine"] == "bm25"
-    assert payload["metrics"]["score_metric"] == "bm25"
+    assert payload["metrics"]["engine"] == "fasttext_mean_embeddings"
+    assert payload["metrics"]["score_metric"] == "cosine_similarity"
     assert payload["metrics"]["returned_count"] == len(payload["places"])
     assert payload["metrics"]["candidate_retrieval"] == "mock_embeddings"
     assert payload["metrics"]["query_token_count"] > 0
@@ -113,18 +113,19 @@ def test_places_recommendations_returns_llm_message_and_bm25_metadata() -> None:
     assert payload["metrics"]["scope"] == "current_query"
     assert payload["metrics"]["ground_truth_available"] is False
     assert "evaluation_metrics" not in payload
-    assert payload["metadata"]["ranking"] == "bm25"
+    assert payload["metadata"]["ranking"] == "fasttext_mean_embeddings"
     assert payload["metadata"]["response_mode"] == "confident"
     assert payload["metadata"]["used_llm"] is True
 
 
-def test_places_recommendations_returns_no_places_without_bm25_matches() -> None:
+def test_places_recommendations_returns_no_places_without_candidates() -> None:
     client = TestClient(create_app())
 
     response = client.post(
         "/places/recommendations",
         json={
             "query": "xqzv blorf 998zz",
+            "city": "Ciudad inexistente",
             "filters": {"is_active": True},
             "limit": 3,
         },

tests/test_fasttext_embeddings.py

@@ -0,0 +1,50 @@
+import math
+
+import pytest
+
+from app.shared.nlp.embeddings.fasttext import FastTextEmbeddingProvider
+from app.shared.nlp.embeddings.versioning import versioned_embedding_hash
+
+
+class FakeFastTextModel:
+    vectors = {
+        "agua": [1.0, 0.0, 0.0],
+        "problemas": [0.0, 1.0, 0.0],
+        "hidrico": [0.8, 0.2, 0.0],
+    }
+
+    def get_dimension(self) -> int:
+        return 3
+
+    def get_word_vector(self, word: str) -> list[float]:
+        return self.vectors.get(word, [0.0, 0.0, 1.0])
+
+
+def test_fasttext_provider_mean_pools_and_normalizes_tokens() -> None:
+    provider = FastTextEmbeddingProvider(
+        model_path="unused.bin",
+        expected_dimension=3,
+        model_loader=lambda _: FakeFastTextModel(),
+    )
+
+    embedding = provider.embed_text("problemas de agua")
+
+    expected = 1 / math.sqrt(2)
+    assert embedding == pytest.approx([expected, expected, 0.0])
+    assert sum(value * value for value in embedding) == pytest.approx(1.0)
+
+
+def test_fasttext_provider_rejects_mismatched_dimension() -> None:
+    with pytest.raises(ValueError, match="does not match"):
+        FastTextEmbeddingProvider(
+            model_path="unused.bin",
+            expected_dimension=300,
+            model_loader=lambda _: FakeFastTextModel(),
+        )
+
+
+def test_embedding_hash_changes_with_model_configuration() -> None:
+    first = versioned_embedding_hash("source", "fasttext", "v1", 300)
+    second = versioned_embedding_hash("source", "fasttext", "v2", 300)
+
+    assert first != second