education-data-lab / METHODOLOGY.md
Hugdrigo's picture
Upload 19 files
3939ca3 verified
|
Raw History Blame Contribute Delete
4.39 kB

Methodology and reproducibility

Scope

This is an exploratory, source-aware comparative education intelligence dashboard. The eight indicators are selected examples, not a complete education system assessment or an official SDG4 index. The application avoids causal interpretations.

Data ingestion

Clicking Load World Bank data requests all eight series from the WDI source (ID 2):

https://api.worldbank.org/v2/country/ARG;BOL;BRA/indicator/SE.PRM.CMPT.ZS;SE.SEC.ENRR;...?source=2&date=2010:2025&format=json&per_page=20000

The actual URL is assembled from selected economies, indicators and years; users can inspect the exact link in the Data quality view or exported research brief. All retrieval is initiated by the browser. The app never silently substitutes synthetic data after a failed request.

Overview

Each country is represented by its latest non-missing observation within the selected window. Latest values can be from different years; year and reporting lag are shown. A ranking here is not a same-year ranking. Source-specific measurement definitions remain relevant.

Trends

Points show unmodified observations; connecting segments are displayed only for consecutive reported years, with gaps remaining visually disconnected. A change compares first and last available values, possibly from different years per country.

Country gaps

For two selected countries A and B in year t:

gap(A,B,t) = reported_value(A,t) - reported_value(B,t).

Both observations must exist for the exact same year. This is a between-country level difference, not an inequality or socioeconomic equity gap inside either country.

Associations

A pair is included only if X and Y exist for the same country and calendar year. The displayed sample size counts country-year pairs. Pearson's correlation coefficient is the conventional covariance divided by the product of centered sums of squares. It is not causal, and the application does not estimate statistical significance, confidence intervals, fixed effects or corrected clustered standard errors. Repeated years from each country violate independent-sample assumptions; do not treat r as a confirmed structural relationship.

Coverage and reporting lag

coverage = valid_observations / (selected_countries × selected_years × 8_indicators).

Mean reporting lag averages (selected_end_year - latest_available_year) for each observed country-indicator combination. A country-indicator combination without any reported value is excluded from the lag mean but remains missing in the coverage ratio. This is a reporting timeliness statistic, not a data quality score.

Research brief

A deterministic descriptive Markdown narrative is assembled directly from currently loaded observations; it does not use an LLM or claim independent verification. Exported CSV and session JSON include a status marking LIVE / SYNTHETIC / IMPORTED and preserve source-derived years and metadata. Source footnotes can be incomplete because the upstream API may not return them for every observation.

Missing data

No imputation, interpolation, invented country values, or silent fill-forward of missing data. Number of points and exact reporting years remain visible. Nulls are never converted to zero. Year and observation count constraints are validated.

Synthetic demo

demo-data.js generates illustrative values deterministically from a fixed mathematical formula, with intentionally missing observations. Demo output is always marked SYNTHETIC DEMO and must never be presented as official data.

Reproducibility

  • Deterministic analysis helpers: core.js
  • Live API adapter: worldbank.js
  • Clearly synthetic generator: demo-data.js
  • Interactive UI: app.js, index.html, style.css
  • Unit tests: node --test tests.mjs
  • Browser QA: browser_test.py (requires Python Playwright and Chromium)
  • Data dictionary, limitations and privacy: other root Markdown documents.

Limitations

World Bank data might have missing observations, definition changes and revision cycles. Gross enrollment can exceed 100%. GDP per capita in current US dollars is not purchasing-power-adjusted. Country-to-country and time-to-time differences do not identify policy impacts. Review original provider metadata and seek specialist input for high-stakes interpretations.