CSS classes use the 'category-{slug}' prefix, so checking `country_slug not in card_classes`
never matched. Changed to `f"category-{country_slug}" not in card_classes`.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Location filter fixes:
- euraxess: loc_text defaulted to the searched location when the HTML
element was missing, causing all locationless jobs to pass country
filters; now defaults to "" so they are correctly excluded
- mlscientist: filter was skipped entirely for countries not in
_COUNTRY_SLUG (falsy country_slug short-circuited the condition);
added a second filter block for unmapped countries using card loc_text
- mlscientist: added ireland, austria, italy (confirmed on site);
removed canada (category does not exist on mlscientist.com)
New scrapers:
- scholarshipdb.net: worldwide aggregator with 28k+ positions; country
filtering via URL path (/scholarships-in-{Country}); Europe post-filter
via _EUROPEAN_COUNTRIES set; relative-date posted field supported
- nature.com/careers: multidisciplinary global board; keyword search via
URL; location filter client-side via ISO-3166 country codes in location
strings (e.g. "Berlin (DE)"); "New" badge mapped to posted="1 day a…
refactor: reorganize agent/ into functional subfolders + split prompts
- New agent/prompts.py: all LLM prompts in one place (edit to tune behaviour)
- config.py: add default_model, scraper_delay, recent_days, deadline_warn_days,
min_score_default, max_results_per_source
- app.py: remove model dropdown — hardcoded to config.default_model
- README: strip 4x duplicate HF frontmatter, update structure/model sections
- push_to_hf.sh: guard against prepending frontmatter if already present
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix: surface LLM error message instead of silently returning score=0
When _generate_groq/_generate_hf raises RuntimeError (rate limit, auth
error, etc.), the error was silently swallowed and every job got score=0
with no diagnostic info. Now the actual error message is preserved in
_generate_json and shown in the job's Concerns field.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
feat: filter stale jobs and add freshness indicator
- Filter out listings posted in previous years or with expired deadlines
- Add `freshness` field to each job: 🟢 Recent (≤30d), 🟡 Older,
🔴 Closing soon (deadline ≤14d), or "" when no date is available
- Show freshness in results table and job details panel
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
feat: sort jobs by posting date + remove DuckDuckGo
- Delete scrapers/web.py and remove duckduckgo-search dependency
- Add _parse_date() to BaseScraper (handles ISO, DD Month YYYY,
Month DD YYYY, DD/MM/YYYY with prefix stripping)
- Store `posted` field in Euraxess and mlscientist listings
- Sort results by (has_date, date desc, description length):
dated listings appear first, newest to oldest
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
feat: remove DuckDuckGo scraper and 'any' position type; add docs link to README
- Remove WebSearchScraper from active scrapers in JobSearcher._build_scrapers
- Remove 'any' from position type dropdown, default to 'phd'
- Add documentation link to README
- Update all docs pages to reflect changes (sources, web-ui, overview, api, index)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
refactor: full OOP redesign of agent architecture
- Add Groq (OpenAI-compatible) backend to LLMClient and config
- Auto-select Groq when GROQ_API_KEY secret is set, fall back to HF
- Update JobAgent to accept backend + api_key instead of HF-only token
- Raise LLMQuotaError (402) explicitly instead of silently scoring 0
- Switch model list to Groq models (llama-3.3-70b, llama-3.1-8b, gemma2, mixtral)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix: improve match scoring — semantic reasoning, short-description tolerance
- Instruct LLM to rely on title/institution when description is a short excerpt
- Require semantic equivalence (NLP = natural language processing, ML = machine learning)
- Treat adjacent/complementary fields as partial overlap (score >= 55)
- Highlight ALL matching areas including transferable skills and publications
- Be generous on ambiguous evidence rather than penalising missing keywords
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
feat: make all scored positions reviewable, not just those above min_score
The minimum score now acts as a visual filter for the recommended table
only — all positions are available in the Review tab dropdown, sorted
by score. Users can review and approve any position regardless of score.
- job_matcher.score_all(): remove min_score filter, always return all
- pipeline.score_jobs(): drop min_score param (no longer needed)
- app.py: separate scored (all) from recommended (>= min_score);
scored_df shows recommended only, position_selector shows all
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
refactor: remove dead code, extract shared job helpers
agent/utils.py:
- Add job_institution(job) — eliminates repeated .get("institution", .get("company",...))
pattern; the "company" fallback was dead (no scraper ever sets it)
- Add job_description(job, max_chars) — eliminates repeated [:3000] truncation pattern
agent/job_searcher.py:
- Remove _TYPE_QUERY dict (defined but never referenced anywhere)
- Move _MLSCI_NON_COUNTRY from inside the loop body to module-level frozenset
- Fix module docstring (remove stale FindAPhD/jobs.ac.uk-only references)
agent/job_matcher.py, cover_letter.py, cv_tailor.py:
- Use job_institution() and job_description() from utils
app.py:
- Use job_institution() in all 7 display/logic locations
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
mlscientist:
- Fix selector: article.cm-entry-summary → article.type-post
- Fix title selector: .entry-title a (WordPress standard)
- Fix country detection: read from CSS category classes (category-germany)
instead of excerpt text — accurate and reliable
- Fix country filter: check category-{slug} in card CSS classes
- Fix deadline regex: extract just the date, not the full following text
Field post-filter:
- Check title and description independently with AND-word logic
- Euraxess short excerpts no longer cause false negatives
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix: field filter uses AND logic on words instead of OR
For "machine learning": requires both "machine" AND "learning" to appear
(not just one), avoiding false positives. Still falls back to exact
phrase check first for precision.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add "predoctoral" to position type dropdown in UI
- Add detection keywords: predoctoral, pre-doctoral, early-stage researcher,
research trainee, master student, research internship, ESR
- Add DuckDuckGo query label for predoctoral searches
- Add mlscientist category slug (maps to general jobs category)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix: post-filter results to require field keyword in title or description
Scraper results that don't mention the search field anywhere are now
discarded. Supports comma/slash-separated multi-keyword fields
(e.g. "machine learning, NLP" → keeps results matching either keyword).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix: always post-filter Euraxess results by country
job_country[] param is ignored server-side — Euraxess returns results
from all countries regardless. Apply location filter unconditionally
(not only when country_id is missing).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix: retry HF inference on 503/502/529 with exponential backoff
HF Router occasionally returns 503 Service Unavailable for transient
load issues. Retry up to 3 times (delays: 1s, 2s, 4s) before failing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix: replace FindAPhD (403/Cloudflare) with Euraxess scraper
FindAPhD now returns 403 for all requests. Euraxess (euraxess.ec.europa.eu)
returns 200 and has structured HTML.
Changes:
- Remove _search_findaphd(); add _search_euraxess()
- _EURAXESS_COUNTRY_ID maps 40+ locations to Euraxess numeric IDs
used in job_country[] query param for server-side filtering
- Parse article.ecl-content-item cards: title, institution, location
(extracted from Work Locations block), deadline, description
- For locations without a numeric ID (e.g. "Europe"), post-filter
cards by matching country name in Work Locations text
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
feat: add mlscientist.com scraper; remove map from UI
- Add _search_mlscientist(): scrapes mlscientist.com via WordPress
category pages (/category/phd-positions/, /category/postdoc-positions/)
combined with keyword search (?s=); falls back to geographic category
slugs when location matches a known country
- Remove Leaflet map and _MAP_HTML from app.py; location input is now
a plain dropdown (allow_custom_value=True)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Replace hardcoded "2025" with dynamic _CURRENT_YEAR = "2026"
- Add _looks_like_job_posting() signal filter to discard non-job results
- Post-filter: only keep results mentioning the current year AND matching
at least one job-posting signal (deadline, vacancy, apply now, etc.)
- Use more precise query templates: "call for applications", "open position",
"funded deadline" with quoted location and year
- Increase max_results per query from 6 to 8 to compensate for filtering
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
feat: location dropdown with interactive Leaflet map (Nominatim reverse geocoding)
