Spaces:
Running
Running
| # Data Sources | |
| ================================================================================ | |
| INDEX | |
| ================================================================================ | |
| | Data | Sources | | |
| |------|----------| | |
| | OHLCV daily | DuckDB (cached), Alpaca (live), Yahoo (live), DefeatBeta (cached) | | |
| | minute bars | Alpaca (live), DuckDB cache (cached) | | |
| | hour bars | Alpaca (live) | | |
| | gainers/losers | Webull (live, API + scraper), Finviz (live), TradingView (live) | | |
| | premarket/afterhours | Webull (live, API + scraper) | | |
| | Nueva: | sector | NASDAQ API (live), DuckDB (cached), Finviz (live), StockAnalysis (live) | | |
| | Nueva: | industry | NASDAQ API (live), DuckDB (cached), Finviz (live), StockAnalysis (live) | | |
| | Nueva: | market_cap | NASDAQ API (live), DuckDB (cached), Finviz (live), StockAnalysis (live) | | |
| | symbols list | NASDAQ API (live) | symbol, name, price, change, volume, market_cap, country, ipoyear, industry, sector | | |
| | deficient/non-compliant listing | NASDAQ API (live) | non-compliant issuers, deficiency type, market tier, notification date | | |
| | catalyst (why moving) | DefeatBeta news (cached, primary), Finviz API (live, rate-limited) | | |
| | low float stocks | LowFloat (live) | | |
| | high short interest | HighShortInterest (live) | | |
| | hot penny stocks | PennyStockFlow (live) | | |
| | reddit mentions | ChartExchange (live) | | |
| | Nueva: | social sentiment (bullish/bearish %) | StockTwits (live) | | |
| | Nueva: | message volume / social buzz | StockTwits (live) | | |
| | Nueva: | trending symbols | StockTwits (live) | | |
| | Nueva: | watchlist count (social interest) | StockTwits (live) | | |
| | earnings date | Yahoo (live), SEC 10-K (live), DefeatBeta (cached) | | |
| | earnings history | DefeatBeta (cached) | | |
| | news | Yahoo (live), DefeatBeta (cached) | | |
| | news headline | StockTitan overview (live) | | |
| | short interest | StockTitan (live), StockAnalysis (live) | | |
| | press releases | DefeatBeta (cached) | | |
| | insider trades | SEC Form 4 (live), OpenInsider (live), StockTitan (live), DefeatBeta (cached) | | |
| | Nueva: | employees | Finviz (live), StockAnalysis (live) | | |
| | Nueva: | company description | Finviz (live), StockAnalysis (live) | | |
| | fundamentals | Yahoo (live), Finviz (live), StockAnalysis (live), DefeatBeta (cached) | | |
| | Nueva: | valuation ratios (PE, PB, PS, PEG...) | StockAnalysis (live), DefeatBeta (cached) | | |
| | Nueva: | financial position (current ratio, debt/equity...) | StockAnalysis (live) | | |
| | Nueva: | efficiency (ROE, ROIC, WACC...) | StockAnalysis (live), DefeatBeta (cached) | | |
| | Nueva: | balance sheet (cash, debt, equity, book value...) | StockAnalysis (live), DefeatBeta (cached) | | |
| | Nueva: | cash flow (OCF, FCF, capex...) | StockAnalysis (live), DefeatBeta (cached) | | |
| | Nueva: | share stats (insider %, institution %, float) | StockAnalysis (live) | | |
| | Nueva: | dividends (yield, growth, payout) | StockAnalysis (live), Yahoo (live), Alpaca (cached), DefeatBeta (cached) | | |
| | Nueva: | stock splits (date, ratio, type) | StockAnalysis (live), Yahoo (live), Alpaca (cached), SEC 8-K (live), DefeatBeta (cached) | | |
| | Nueva: | scores (Altman Z, Piotroski F) | StockAnalysis (live) | | |
| | Nueva: | fair value (Lynch, Graham) | StockAnalysis (live) | | |
| | Nueva: | key executives | StockAnalysis (live), DefeatBeta (cached) | | |
| | Nueva: | contact (address, phone, website) | StockAnalysis (live) | | |
| | Nueva: | stock metadata (CUSIP, ISIN, SIC, fiscal year...) | StockAnalysis (live), SEC (live) | | |
| | 8-K | SEC (live); también vía DefeatBeta `stock_sec_filing` (cached) con `acceptance_date_time` = hora NY exacta | | |
| | S-1/S-3/S-8 | SEC (live) — offerings / dilution; ⚠️ DefeatBeta NO trae estos form_types para micro-caps (ver nota abajo) | | |
| | 10-K/10-Q | SEC (live) | | |
| | Form 4 | SEC (live) | | |
| | 13D/13G | SEC (live) — activist filings | | |
| | reverse splits | Yahoo (live), Alpaca (cached), SEC 8-K (live) | | |
| | Nueva: | short sale restriction (SSR / Rule 201 list) | NASDAQ Trader (live) — ssrlist.com is a viewer | | |
| | dividends | Yahoo (live), Alpaca (cached) | | |
| | Nueva: | CIK lookup | SEC (live), StockAnalysis (live) | | |
| | trading calendar | DuckDB (cached), Alpaca (live) | | |
| | supports / resistances | DuckDB (cached) + Alpaca minute (live) | | |
| | cost to borrow (fee, available, history) | iBorrowDesk (live) | | |
| | offering risk rating | AskEdgar (live) | | |
| | dilution risk (rating, ATM, capital structure, runway) | SmallCapLab (live, API + scraper) | | |
| | pump timing (pump→filing correlation) | SmallCapLab (live, API) | | |
| | institutional ownership | AskEdgar (live) | | |
| | share float / outstanding shares | AskEdgar (live) | | |
| **Access types:** | |
| - **cached**: obtenido periódicamente y almacenado en local | |
| - **live**: obtenido al vuelo cuando se necesita | |
| ================================================================================ | |
| SOURCE DETAILS | |
| ================================================================================ | |
| ## DuckDB (Local — Cached) | |
| | File | Data | | |
| |------|------| | |
| | `data/alpaca_merged.parquet` | Daily OHLCV + computed | | |
| | `data/market_calendar.csv` | Trading days | | |
| | `data/corporate_actions.csv` | Historical corporate actions | | |
| | `data/symbol_metadata.csv` | Stock metadata | | |
| | `data/metadata.db` (table: tickers) | CIK mapping | | |
| | `data/cache/{SYMBOL}_{DATE}.parquet` | Minute bars cache (TTL: 24h) | | |
| --- | |
| ## Alpaca (Live) | |
| | Endpoint | Data | | |
| |----------|------| | |
| | `/v2/stocks/bars` | OHLCV daily/minute/hour | | |
| | `/v2/corporate_actions` | splits, dividends, mergers, spin-offs | | |
| | `/v1/calendar` | trading calendar | | |
| --- | |
| ## Yahoo Finance (Live) | |
| | Data | Endpoint / Resource | | |
| |------|---------------------| | |
| | OHLCV, volumes, pre/post market | `/v8/finance/chart/{symbol}` | | |
| | recent news | news feed | | |
| | earnings calendar | earnings history | | |
| | splits, dividends | actions feed | | |
| | press releases | press releases feed | | |
| --- | |
| ## Webull (Live — Scraper + API) | |
| Webull expone dos tipos de datos: **scraping HTML** (páginas públicas) y **API interna** (endpoints JSON). | |
| **Base URL:** `https://quotes-gw.webullfintech.com` | |
| ### HTML Scraper | |
| | Endpoint | Data | | |
| |----------|------| | |
| | `/quote/us/gainers` | today gainers | | |
| | `/quote/us/gainers/1d` | 1-day gainers | | |
| | `/quote/us/gainers/5d` | 5-day gainers | | |
| | `/quote/us/gainers/1m` | 1-month gainers | | |
| | `/quote/us/gainers/pre` | premarket | | |
| | `/quote/us/gainers/after` | afterhours | | |
| | `/quote/us/actives/rvol10d` | relative volume 10d | | |
| | `/quote/us/actives/turnover_ratio` | turnover ratio | | |
| | `/quote/us/actives/range` | range % | | |
| ### Internal API — Rankings | |
| | Endpoint | Data | | |
| |----------|------| | |
| | `/api/wlas/ranking/v9/rise` | gainers | | |
| | `/api/wlas/ranking/v9/fall` | losers | | |
| **Parameters:** `regionId=6` (US market), `rankType` (timeframe: 1d, 5d, 1m, 3m, 52w, 3min, 5min, gainers.preMarket, gainers.afterMarket), `pageIndex`, `pageSize` | |
| ### Internal API — Micro Trend | |
| | Endpoint | Data | | |
| |----------|------| | |
| | `/api/bgw/quote/microTrend?ids={tickerIds}` | Intraday micro-trend (minute-by-minute price data for current session) | | |
| --- | |
| ## Finviz (Live — Scraper + API) | |
| **Base URL:** `https://finviz.com` | |
| ### Views | |
| | Vista ID | Vista | Descripción | | |
| |----------|-------|--------------| | |
| | `v=110` | Overview | Datos básicos (Ticker, Company, Sector, Market Cap, etc.) | | |
| | `v=120` | Valuation | P/E, Forward P/E, PEG, P/S, P/B, etc. | | |
| | `v=160` | Financial | EPS, Revenue, Margins, ROE, ROA, etc. | | |
| | `v=130` | Ownership | Insider Ownership, Institutional Ownership, Short Interest, etc. | | |
| | `v=140` | Performance | Perf Week, Perf Month, Perf YTD, Perf Year, etc. | | |
| | `v=170` | Technical | RSI, SMA20/50/200, ATR, Volatility, etc. | | |
| | `v=180` | ETF | ETF-specific data (AUM, Holdings, Flows, Expense Ratio) | | |
| | `v=150` | **Custom** | Columnas personalizadas vía parámetro `c=` | | |
| ### Endpoints | |
| | Endpoint | Data | | |
| |----------|------| | |
| | `/screener.ashx?v=111` | overview signals | | |
| | `/screener.ashx?v=171` | technical signals | | |
| | `/screener.ashx?v=150&c={cols}&t={tickers}` | custom screener | | |
| | `/api/stocks-why-moving/boxover/{symbol}` | catalyst (headline, summary, sentiment) | | |
| | `/quote.ashx?t={symbol}` | employees, sector, industry, company description | | |
| ### Custom Screener (v=150) | |
| **URL:** `https://finviz.com/screener?v=150&t={tickers}&c={column_ids}` | |
| **Parameters:** `v=150` (custom), `t=` (tickers, comma-separated), `c=` (column IDs, comma-separated), `r=` (optional signal filters) | |
| --- | |
| ## SEC EDGAR (Live) | |
| **Base URL:** `https://www.sec.gov` | |
| | Endpoint | Data | | |
| |----------|------| | |
| | `/files/company_tickers.json` | symbol → CIK lookup | | |
| | `/submissions/CIK{cik}.json` | all filings | | |
| **Form types:** Form 4 (insider trades), 8-K (material events), 6-K (foreign events), S-1/S-3/S-8/S-4 (offerings), F-1/F-3 (foreign offerings), 13D/13G (activist filings), 10-K/10-Q/20-F (periodic reports), DEF 14A (proxy statements). | |
| **Filing HTML URL format:** `https://www.sec.gov/Archives/edgar/data/{CIK}/{accessionNumberWithoutDashes}/{primaryDocument}` | |
| --- | |
| ## OpenInsider (Live — Scraper) | |
| | Endpoint | Data | | |
| |----------|------| | |
| | `/screener` | insider trades (symbol filtered from page) | | |
| | `/cluster-buy` | cluster buys | | |
| | `/ceo-trades` | CEO-only trades | | |
| --- | |
| ## NASDAQ API (Live) | |
| **Base URL:** `https://api.nasdaq.com` | |
| | Endpoint | Data | | |
| |----------|------| | |
| | `/api/screener/stocks?tableonly=false&download=true` | Full symbols list with metadata | | |
| | `/api/quote/list-type-extended/listing?queryString=deficient` | Non-compliant / deficient companies | | |
| ### Screener — `/api/screener/stocks` | |
| Returns: symbol, name, lastsale (price), netchange, pctchange, volume, marketCap, country, ipoyear, industry, sector, url. | |
| Note: NASDAQ only. NYSE/AMEX not included. | |
| ### Non-Compliant / Deficient — `/api/quote/list-type-extended/listing` | |
| Lists NASDAQ issuers non-compliant with listing requirements. Parameter: `queryString=deficient` (required). No pagination — returns full list (~350 issuers). | |
| **Deficiency types observed:** Bid Price, Equity, Market Value of Listed Securities, Delinquent filings, Market Value of Publicly Held Shares, Shareholders, Annual Shareholder Meeting, Audit Committee Composition, Net Income, Board Independence, and others. | |
| **Market tiers:** NCM (Capital Market), NGM (Global Market), NGS (Global Select). | |
| --- | |
| ## SSR — Short Sale Restriction (NASDAQ Trader / Rule 201) | |
| **Data:** Lista de valores sujetos a la restricción de venta en corto de la SEC **Rule 201** (la "Alternative Uptick Rule"). Se activa cuando una acción cae ≥10% respecto al cierre previo; aplica el resto de ese día + el siguiente día hábil. **No prohíbe** vender en corto: solo restringe shorts en dowtick (baja de precio). Útil para filtrar nombres "en play" y detectar presión de cortos reducida en caídas. | |
| **Origen:** NASDAQ Trader (oficial). **ssrlist.com es un viewer de terceros** sobre este feed — JS-rendered, sin API pública. No scrapear ssrlist.com; traer de NASDAQ directo. | |
| **Base URL:** `https://www.nasdaqtrader.com` | |
| | Endpoint | Data | Access | Notes | | |
| |----------|------|--------|-------| | |
| | `/trader.aspx?id=shortSaleCircuitBreaker&symbol=ALL` | SSR list completa (tabla HTML) | live | Página **ASP.NET postback**: la tabla NO está en el HTML estático; hay que replicar el postback (`__VIEWSTATE`/`__EVENTVALIDATION`) o manejar sesión. Frágil. | | |
| | `/rss.aspx?feed=SSRCircuitBreaker` | SSR list (RSS 2.0) | live | Feed canónico machine-readable, pero devuelve la página de error de NASDAQ en un GET simple (requiere sesión/postback). Verificar desde el server de deploy antes de construir sobre esto. | | |
| | `/dynamic/symdir/shorthalts/shorthaltsYYYYMMDD.txt` (relacionado) | Volatility-halt list (CSV) | live | Prueba que los `.txt` CSV estáticos SÍ funcionan en este host. Formato: `Symbol,Security Name,Market Category,Trigger Time`. El path equivalente de SSR aún no está confirmado/descubierto. | | |
| **Columnas SSR esperadas** (según NASDAQ / ssrlist.com): Symbol, Security Name, Market Category (Q=NASDAQ, N=NYSE, A=AMEX, Z=BATS, V=IEX, R=OTC/Regional), Trigger Time. | |
| **Access type:** live (re-fetch por día hábil; la lista cambia intraday a medida que se disparan nombres). | |
| **Integración recomendada** (espeja providers existentes): | |
| - `core/ssr_provider.py` → `SSRProvider.fetch(date?)` devolviendo `[{symbol, name, market, trigger_time, date}]`, con caché TTL (como `SplitsProvider`/`SmallCapLabProvider`). | |
| - Registro en `core/shared/deps.py`. | |
| - Endpoint `GET /api/ssr` en `modules/lists/routes.py` (count + lista), igual que `/api/reverse-splits`. | |
| **Gotchas (verificados 2026-07-13):** | |
| 1. ssrlist.com no tiene API y es client-rendered → scrapearlo exige headless browser. Capa equivocada. | |
| 2. La página SSR de NASDAQ requiere postback ASP.NET para devolver la tabla (un fetch estático solo trae el chrome de la página). | |
| 3. El feed `rss.aspx?feed=SSRCircuitBreaker` tira 404 / error page en GET simple desde sandbox; podría funcionar desde el server de deploy con sesión correcta — confirmar antes de usarlo. | |
| 4. Los `.txt` CSV estáticos SÍ funcionan en `nasdaqtrader.com` (verificado con `shorthalts20260713.txt`), así que un `.txt` de SSR probablemente existe — hallar el path/nombre exacto antes de elegir la estrategia de parser. | |
| --- | |
| ## TradingView (Live) | |
| | Data | Description | | |
| |------|-------------| | |
| | premarket signals | premarket change and volume | | |
| | postmarket signals | postmarket change and volume | | |
| **Columns:** close, premarket_change, postmarket_change, premarket_volume, volume, market_cap_basic, float_shares_outstanding, relative_volume_10d_calc, average_volume_30d_calc | |
| --- | |
| ## StockTitan (Live — Scraper) | |
| | URL | Data | | |
| |-----|------| | |
| | `/overview/{symbol}/` | news headline, SEC filing title, price impact | | |
| | `/sec-filings/{symbol}/form-4.html` | insider trades | | |
| | `/sec-filings/{symbol}` | filings summary | | |
| | `/short-interest/{symbol}` | short_percent, days_to_cover | | |
| | `/momentum` | stocks with % change (momentum alerts) | | |
| --- | |
| ## LowFloat (Live — Scraper) | |
| | URL | Data | Frequency | | |
| |-----|------|-----------| | |
| | `/all/` | all exchanges | Weekly | | |
| | `/all_with_otcbb/` | all + OTCBB | Weekly | | |
| | `/nasdaq/` | Nasdaq only | Weekly | | |
| | `/nyse/` | NYSE only | Weekly | | |
| | `/amex/` | AMEX only | Weekly | | |
| | `/otcbb/` | OTCBB only | Weekly | | |
| **Columns:** Ticker, Company, Exchange, Float, Outstd, ShortInt, Industry | |
| **Filter:** Float < 10M shares | |
| --- | |
| ## HighShortInterest (Live — Scraper) | |
| **Base URL:** `https://www.highshortinterest.com` | |
| | URL | Data | Frequency | | |
| |-----|------|-----------| | |
| | `/all/` | all exchanges | Weekly | | |
| | `/nasdaq/` | Nasdaq only | Weekly | | |
| | `/nyse/` | NYSE only | Weekly | | |
| | `/amex/` | AMEX only | Weekly | | |
| **Columns:** Ticker, Company, Exchange, ShortInt, Float, Outstd, Industry | |
| **Filter:** Short interest > 20% | |
| --- | |
| ## PennyStockFlow (Live — Scraper) | |
| | URL | Data | Frequency | | |
| |-----|------|-----------| | |
| | `/` | Today's hot penny stocks | Daily / Intradía | | |
| **Columns:** Ticker, Price, Change, $ Volume, Volume, # Trades | |
| --- | |
| ## ChartExchange (Live — Scraper) | |
| | Endpoint | Data | | |
| |----------|------| | |
| | `/trends/reddit/mentions/{subreddits}/` | Reddit mentions and sentiment from 232 subreddits | | |
| **Subreddit groups:** pennystocks (default), wallstreetbets, crypto, stocks, all, or custom comma-separated list. | |
| **Data fields:** Ticker, Type (stock/crypto), Name, Mention Count, Mention Change %, Sentiment, 1d Price Change % | |
| --- | |
| ## StockAnalysis (Live — Scraper) | |
| **Base URL:** `https://stockanalysis.com/stocks/{symbol}/` | |
| ### Pages | |
| | Page | URL Path | Data | | |
| |------|----------|------| | |
| | Overview | `/stocks/{symbol}/` | price, market cap, volume, PE, EPS, shares, beta, dividends, analyst rating, price target, earnings date | | |
| | Statistics | `/stocks/{symbol}/statistics/` | valuation, financial position, efficiency, margins, dividends, FCF, share stats, short interest, splits, scores, fair value | | |
| | Company | `/stocks/{symbol}/company/` | description, country, industry, sector, employees, CEO, contact, stock metadata (CIK, CUSIP, ISIN, SIC, fiscal year) | | |
| | Financials | `/stocks/{symbol}/financials/` | income statement, balance sheet, cash flow (annual/quarterly/TTM) | | |
| | Forecast | `/stocks/{symbol}/forecast/` | analyst price targets, estimates | | |
| | Ratings | `/stocks/{symbol}/ratings/` | analyst ratings detail | | |
| | Filings | `/stocks/{symbol}/filings/` | SEC filings summary | | |
| | Employees | `/stocks/{symbol}/employees/` | employee count | | |
| | Market Cap | `/stocks/{symbol}/market-cap/` | market cap breakdown | | |
| | History | `/stocks/{symbol}/history/` | historical price data | | |
| | Transcripts | `/stocks/{symbol}/transcripts/` | earnings call transcripts | | |
| ### Statistics — Fields | |
| | Category | Fields | | |
| |-----------|--------| | |
| | **Valuation** | PE, Forward PE, PS, PB, PEG, P/FCF, P/OCF, EV/EBITDA, EV/EBIT, EV/FCF | | |
| | **Financial Position** | Current Ratio, Quick Ratio, Debt/Equity, Debt/EBITDA, Interest Coverage | | |
| | **Efficiency** | ROE, ROA, ROIC, ROCE, WACC, Asset Turnover, Inventory Turnover, Revenue/Employee, Profits/Employee | | |
| | **Balance Sheet** | Cash, Total Debt, Net Cash, Net Cash Per Share, Equity (Book Value), Book Value Per Share, Working Capital | | |
| | **Cash Flow** | Operating Cash Flow, FCF, FCF Per Share, Net Borrowing, Depreciation & Amortization | | |
| | **Share Stats** | Shares Outstanding, Shares Change YoY/QoQ, Insider %, Institution %, Float | | |
| | **Short Selling** | Short Interest, Short Previous Month, Short % of Shares Out, Short Ratio | | |
| | **Dividends** | Dividend Per Share, Yield, Growth YoY, Payout Ratio, Buyback Yield, Shareholder Yield, Earnings Yield, FCF Yield | | |
| | **Stock Splits** | Last Split Date, Split Type, Split Ratio | | |
| | **Scores** | Altman Z-Score, Piotroski F-Score | | |
| | **Fair Value** | Lynch Fair Value, Graham Number, Lynch/Graham Upside | | |
| | **Stock Price** | Beta, 52-Week Change, 50/200-Day MA, RSI (14), Avg Volume (20D) | | |
| ### Company — Fields | |
| Name, Description, Country, Industry, Sector, Employees, CEO, Address, Phone, Website, Ticker Symbol, Exchange, Fiscal Year, Reporting Currency, CIK Code, CUSIP Number, ISIN Number, Employer ID, SIC Code. | |
| ### Financials — Fields | |
| | Statement | Line items | | |
| |-----------|------------| | |
| | **Income Statement** | Revenue, SG&A, R&D, Operating Expenses, EBIT, Interest Income/Expense, Pretax Income, Net Income, EPS, Shares Outstanding, EBITDA, Free Cash Flow | | |
| | **Balance Sheet** | Cash, Total Debt, Net Cash, Equity (Book Value), Working Capital | | |
| | **Cash Flow** | Operating Cash Flow, Capital Expenditures, Depreciation & Amortization, Net Borrowing, Free Cash Flow | | |
| Modes: Annual, Quarterly, TTM | |
| --- | |
| ## iBorrowDesk (Live — API) | |
| | URL | Data | | |
| |-----|------| | |
| | `https://iborrowdesk.com/api/ticker/{ticker}` | cost to borrow: fee, available, updated timestamp, 5-day history | | |
| --- | |
| ## AskEdgar (Live — API) | |
| | URL | Data | | |
| |-----|------| | |
| | `https://api.askedgar.io/v1/fmp/company/{ticker}/profile` | company profile: price, change, market cap, sector, industry, exchange, range, volAvg | | |
| | `https://api.askedgar.io/v1/fmp/company/{ticker}/share-float` | float shares, outstanding shares | | |
| | `https://api.askedgar.io/v1/fmp/form13f/inst-ownership?symbol={ticker}` | institutional ownership % | | |
| | `https://api.askedgar.io/v1/screener/{ticker}/overview-page` | short interest (shares), short float % | | |
| | `https://app.askedgar.io/ticker/{ticker}` (POST) | offering risk rating | | |
| --- | |
| ## SmallCapLab (Live — API + Scraper) | |
| **Base URL:** `https://www.smallcaplab.com` | |
| Specialized dilution risk analysis and pump-timing detection for small-cap stocks. | |
| ### Internal API | |
| In addition to the public HTML page, SmallCapLab exposes internal JSON endpoints: | |
| | Endpoint | Data | | |
| |----------|------| | |
| | `/ticker/{ticker}` | Public HTML page — dilution risk, capital structure, filings, offering mechanisms | | |
| | `/api/pump-timing/{ticker}` | Pump-and-dump pattern detection correlated to SEC filings | | |
| #### `/api/pump-timing/{ticker}` | |
| Detects price pumps and correlates them with subsequent dilution filings (ATM, offerings). Returns: | |
| | Field | Type | Description | | |
| |-------|------|-------------| | |
| | `totalPumps` | int | Total pump events detected | | |
| | `matchedPumps` | int | Pumps followed by a dilution filing within matchWindow | | |
| | `hitRatePct` | int | % of pumps that preceded a filing | | |
| | `medianDays` | int | Median days from pump to filing | | |
| | `avgDays` | int | Average days from pump to filing | | |
| | `fastestDays` / `slowestDays` | int | Min/max gap | | |
| | `links[]` | array | Each pump with `pump` (startDate, peakDate, returnPct, durationDays, peakVolMult) and optional `match` (date, form, label, confidence) | | |
| | `allFilings[]` | array | All dilution-relevant SEC filings (date, form, label, confidence) | | |
| | `priceHistory[]` | array | Daily close prices (date, close) — ~2yr lookback | | |
| | `cfg` | object | Detection config: thresholdPct, lookbackBars, volumeMultiple, cooldownBars, matchWindowDays, maxPumps, priceHistoryDays | | |
| **Confidence levels for matches:** `high`, `medium`, `low` | |
| **Form types detected:** 424B1 (Prospectus), 424B3 (Resale Prospectus), 424B4 (Follow-on Offering), 424B5 (ATM Filing), S-8 (Stock Plan), 13G (>5% Holder) | |
| ### HTML Page Data Fields | |
| Scraped from the public ticker page at `/ticker/{ticker}`: | |
| | Category | Fields | | |
| |----------|--------| | |
| | **Dilution Risk Score** | Overall rating (Low/Moderate/High), Structural score (1-10), Near-Term score (1-10), Severity score (1-8), ATM Drip flag | | |
| | **Survival / Financing Pressure** | Cash on hand (SEC), quarterly burn, estimated runway (months), 30d ADTV, ATM capacity ($/day) | | |
| | **Dilution Activity** | YoY share growth %, Share CAGR (2Y), capital raises count (12mo/24mo/total), last raise (days ago) | | |
| | **Capital Structure** | Charter authorized shares, shares outstanding (SEC), unissued authorized headroom, authorized÷outstanding ratio | | |
| | **Active Offering Mechanisms** | ATM offering (Active/Inactive + 424B5 on file), Shelf Registration (S-3), Resale Prospectus (424B3), Private Placement (Form D), Reverse Split Signal | | |
| | **Trading Relevance** | Shares outstanding, 30d ADTV | | |
| | **Historical Capital Raises** | Prospectus/offering filings timeline (date, transaction type, SEC document link) | | |
| | **Catalyst Timeline** | 6-month lookback / 12-month outlook with filing dates | | |
| | **SEC Filing History** | Dilution-relevant filings (424B, S-1/S-3/S-8, 13G, Form D) with dates and document links | | |
| **Source:** SEC EDGAR XBRL data (cash, burn, shares outstanding, authorized shares, filing history) + market data (ADTV). | |
| **Note:** Free tier shows current ticker data. Some historical data and API access may require subscription. | |
| --- | |
| ## StockTwits (Live — Internal v2 API) | |
| Social network for traders/investors. The distinctive value vs. every other source here is **retail sentiment**: an aggregated bullish/bearish poll per symbol returned as a daily historical series. Also exposes a public message stream (social buzz), trending symbols, and per-symbol watchlist counts. | |
| **Web base URL:** `https://stocktwits.com` | |
| **API base URL:** `https://api.stocktwits.com/api/2/` | |
| > ⚠️ These are StockTwits' **internal v2 endpoints** (the ones the web app itself calls), NOT the official developer API. The legacy v1 developer API (`api.stocktwits.com/api/1/...`) is deprecated and now requires an approved token. The v2 endpoints currently need **no API key**, but: | |
| > - You MUST send a browser `User-Agent` header, or they return 404. | |
| > - Stability is not guaranteed — internal endpoints can change without notice and may be rate-limited. Treat as fragile (same risk class as a scraper). | |
| ### Endpoints | |
| | Endpoint | Data | | |
| |----------|------| | |
| | `/api/2/symbols/{symbol}/sentiment.json` | **Social sentiment**: daily `bullish`% / `bearish`% poll series (most recent first) | | |
| | `/api/2/streams/symbol/{symbol}.json` | Message stream for a symbol → body, user, `created_at`, `symbols`, `prices`; `cursor` for pagination | | |
| | `/api/2/streams/trending.json` | Trending symbols stream (messages referencing currently-trending tickers) | | |
| | `/api/2/symbols/{symbol}.json` | Symbol metadata (returns 404 for some symbols — use the `symbol` object inside stream responses instead) | | |
| ### Sentiment — `/api/2/symbols/{symbol}/sentiment.json` | |
| Returns `data[]` (newest first), each element: | |
| | Field | Type | Description | | |
| |-------|------|-------------| | |
| | `bullish` | float | % of bullish votes (0–100) | | |
| | `bearish` | float | % of bearish votes (0–100) | | |
| | `timestamp` | string | ISO date (`YYYY-MM-DD`) | | |
| Example (SKYQ, 2026-07-12): `bullish: 94.03, bearish: 5.97`. The series gives a sentiment trend over time — useful as a contrarian or momentum social signal. | |
| ### Streams — fields of interest | |
| Within `/api/2/streams/symbol/{symbol}.json` and `/api/2/streams/trending.json`: | |
| | Field | Use | | |
| |-------|-----| | |
| | `messages[].body` / `tokenized_body` | Post text (cashtags preserved) | | |
| | `messages[].created_at` | Timestamp (ISO 8601, UTC) — count posts per window to derive **message volume / social buzz** | | |
| | `messages[].user` | Author (`username`, `followers`, `classification`, `badges`) | | |
| | `messages[].symbols[]` | Referenced tickers (`symbol`, `title`) | | |
| | `messages[].prices[]` | Price tagged at post time | | |
| | `symbol.watchlist_count` | How many users watch the symbol — **social-interest** metric | | |
| | `cursor` | Opaque pagination token for the next page | | |
| ### Gotchas (verified) | |
| 1. **Per-message `sentiment` is `null`** in public streams — users rarely tag bull/bear on individual posts. Do NOT use per-message sentiment. The reliable signal is the **aggregated `/sentiment` poll**. | |
| 2. **Crypto tickers use a `.X` suffix** (e.g. `ETH.X`, `XRP.X`) in streams and trending. | |
| 3. **No auth today, but fragile** — internal endpoint, browser UA required, rate limits possible. Wrap in retry/backoff + cache. | |
| 4. `watchlist_count` and message volume are the best proxies for retail attention; the poll is the best proxy for retail *direction*. | |
| --- | |
| ## DefeatBeta API (Cached — Parquet via DuckDB HTTPFS) | |
| **GitHub:** https://github.com/defeat-beta/defeatbeta-api | |
| **PyPI:** `defeatbeta-api` | |
| **Dataset:** https://huggingface.co/datasets/defeatbeta/yahoo-finance-data | |
| **License:** Apache 2.0 | |
| **Python:** 3.11+ | |
| **Stars:** ~683 (Jul 2026) | |
| ### Overview | |
| El **DefeatBeta API** es una alternativa open-source a Yahoo Finance con mayor confiabilidad. En lugar de depender de scraping de Yahoo (frágil, rate-limited), hostea todo el dataset de Yahoo Finance transformado en **Parquet files** en Hugging Face, y usa **DuckDB + cache_httpfs** para consultarlos con SQL directo sobre HTTP. No necesita API key, no tiene rate limits, y da respuestas sub-segundo. | |
| ### Arquitectura Interna | |
| ``` | |
| ┌──────────────┐ ┌────────────────────┐ ┌──────────────────────┐ | |
| │ Ticker() │───→│ HuggingFaceClient │───→│ HF dataset (Parquet) │ | |
| │ │ │ (HTTP metadata) │ │ (read-only, hosted) │ | |
| │ price() │ └────────────────────┘ └────────┬─────────────┘ | |
| │ news() │ │ | |
| │ info() │ ┌────────────────────┐ │ HTTP range requests | |
| │ transcripts │───→│ DuckDBClient │←───────────┘ (cache_httpfs) | |
| │ statements │ │ (in-memory OLAP) │ | |
| │ ... │ │ SQL on parquet │ | |
| └──────────────┘ └────────────────────┘ | |
| ``` | |
| **Flujo de datos:** | |
| 1. `HuggingFaceClient` obtiene metadatos (update_time desde `spec.json`) y genera URLs de Parquet vía API REST de HuggingFace. | |
| 2. `DuckDBClient` conecta DuckDB en memoria y ejecuta SQL directamente sobre los archivos `.parquet` via HTTP range requests. | |
| 3. `cache_httpfs` (extensión DuckDB) cachea los bytes localmente → segundas consultas son instantáneas. | |
| 4. Las SQL queries viven en archivos `.sql` en `defeatbeta_api/data/sql/`, cargadas por `sql_loader.py` con templates tipo `{ticker}`, `{url}`. | |
| ### Caché Local | |
| - **Ubicación:** `~/.cache/httpfs/` (configurable) | |
| - **Validación automática:** Al iniciar, compara `update_time` del `spec.json` remoto vs local. Si cambió, limpia el caché. | |
| - **TTL:** Determinado por el update_time del dataset (se actualiza cada ~24h) | |
| ### Datasets Disponibles (15 tablas Parquet) | |
| Cada tabla es un archivo `.parquet` en Hugging Face datasets (`defeatbeta/yahoo-finance-data`): | |
| | Constante Interna | Dataset (Parquet) | Contenido | Período | | |
| |---|---|---|---| | |
| | `stock_prices` | `stock_prices.parquet` | OHLCV diario (open, high, low, close, volume) | ~15+ años | | |
| | `stock_profile` | `stock_profile.parquet` | Perfil empresa: sector, industry, employees, descripción, website | Actual | | |
| | `stock_officers` | `stock_officers.parquet` | Ejecutivos clave (CEO, CFO, etc.) | Actual | | |
| | `stock_earning_calendar` | `stock_earning_calendar.parquet` | Calendario de earnings (fecha reporte, estimados) | Próximos quarters | | |
| | `stock_tailing_eps` | `stock_tailing_eps.parquet` | TTM EPS histórico | ~15+ años | | |
| | `stock_statement` | `stock_statement.parquet` | Financial statements (income, balance, cash flow) en formato largo con template_type | ~10+ años | | |
| | `stock_dividend_events` | `stock_dividend_events.parquet` | Dividendos históricos (monto, frecuencia) | ~15+ años | | |
| | `stock_split_events` | `stock_split_events.parquet` | Stock splits (ratio, fecha) | ~15+ años | | |
| | `stock_earning_call_transcripts` | `stock_earning_call_transcripts.parquet` | Transcripciones earnings calls (texto completo) | ~15+ años | | |
| | `stock_news` | `stock_news.parquet` | Noticias financieras (título, publisher, cuerpo, uuid) | ~5+ años | | |
| | `stock_revenue_breakdown` | `stock_revenue_breakdown.parquet` | Revenue por segmento/geografía/producto | ~5+ años | | |
| | `stock_shares_outstanding` | `stock_shares_outstanding.parquet` | Shares outstanding histórico | ~15+ años | | |
| | `exchange_rate` | `exchange_rate.parquet` | FX rates diarios | ~15+ años | | |
| | `daily_treasury_yield` | `daily_treasury_yield.parquet` | Treasury yields diarios | ~15+ años | | |
| | `stock_sec_filing` | `stock_sec_filing.parquet` | SEC filings metadata | ~15+ años | | |
| ### API Python - Todas las Métricas Disponibles | |
| #### Categoría 0 - Precio | |
| | Método | Returns | Descripción | | |
| |---|---|---| | |
| | `ticker.price()` | DataFrame | OHLCV diario completo | | |
| | `ticker.beta(period, benchmark)` | DataFrame | Beta vs benchmark (5y default) | | |
| #### Categoría 1 - Financieros (Statements) | |
| | Método | Returns | Descripción | | |
| |---|---|---| | |
| | `ticker.quarterly_income_statement()` | `Statement` object | Quarterly income statement | | |
| | `ticker.annual_income_statement()` | `Statement` object | Annual income statement | | |
| | `ticker.quarterly_balance_sheet()` | `Statement` object | Quarterly balance sheet | | |
| | `ticker.annual_balance_sheet()` | `Statement` object | Annual balance sheet | | |
| | `ticker.quarterly_cash_flow()` | `Statement` object | Quarterly cash flow | | |
| | `ticker.annual_cash_flow()` | `Statement` object | Annual cash flow | | |
| | `ticker.calendar()` | DataFrame | Earnings calendar | | |
| | `ticker.splits()` | DataFrame | Split events | | |
| | `ticker.dividends()` | DataFrame | Dividend events | | |
| | `ticker.shares()` | DataFrame | Shares outstanding | | |
| | `ticker.ttm_eps()` | DataFrame | TTM EPS | | |
| | `ticker.ttm_revenue()` | DataFrame | TTM Revenue (conversión USD) | | |
| | `ticker.ttm_fcf()` | DataFrame | TTM Free Cash Flow (conversión USD) | | |
| | `ticker.ttm_net_income_common_stockholders()` | DataFrame | TTM Net Income (conversión USD) | | |
| | `ticker.ttm_ebitda()` | DataFrame | TTM EBITDA (conversión USD) | | |
| | `ticker.quarterly_revenue_by_breakdown()` | DataFrame | Revenue breakdown (segment/geo/product) largo | | |
| | `ticker.trailing_revenue_by_breakdown()` | DataFrame | TTM revenue breakdown largo | | |
| #### Categoría 2 - Valoración (Valuation) | |
| | Método | Returns | Descripción | | |
| |---|---|---| | |
| | `ticker.ttm_pe()` | DataFrame | TTM P/E (merge asof price + TTM EPS) | | |
| | `ticker.market_capitalization()` | DataFrame | Market cap histórico (price × shares) | | |
| | `ticker.ps_ratio()` | DataFrame | P/S ratio histórico | | |
| | `ticker.pb_ratio()` | DataFrame | P/B ratio histórico | | |
| | `ticker.peg_ratio()` | DataFrame | PEG ratio (P/E ÷ EPS growth) | | |
| | `ticker.enterprise_value()` | DataFrame | EV = market cap + debt + minority + preferred - cash | | |
| | `ticker.enterprise_to_revenue()` | DataFrame | EV/Revenue | | |
| | `ticker.enterprise_to_ebitda()` | DataFrame | EV/EBITDA | | |
| | `ticker.net_debt_ttm()` | DataFrame | Net debt TTM | | |
| | `ticker.debt_to_equity()` | DataFrame | Debt/Equity | | |
| | `ticker.rope()` | DataFrame | ROE calculado | | |
| | `ticker.roa()` | DataFrame | ROA calculado | | |
| | `ticker.roic()` | DataFrame | ROIC calculado | | |
| | `ticker.roce()` | DataFrame | ROCE calculado | | |
| | `ticker.wacc()` | DataFrame | WACC (con treasury, beta, S&P500) | | |
| | `ticker.equity_multiplier()` | DataFrame | Equity multiplier (ROE/ROA) | | |
| | `ticker.asset_turnover()` | DataFrame | Asset turnover (ROA/net margin) | | |
| #### Categoría 3 - Crecimiento (Growth) | |
| | Método | Returns | Descripción | | |
| |---|---|---| | |
| | `ticker.quarterly_revenue_yoy_growth()` | DataFrame | Revenue YoY % | | |
| | `ticker.annual_revenue_yoy_growth()` | DataFrame | Revenue YoY % anual | | |
| | `ticker.quarterly_operating_income_yoy_growth()` | DataFrame | Operating income YoY % | | |
| | `ticker.annual_ebitda_yoy_growth()` | DataFrame | EBITDA YoY % anual | | |
| | `ticker.quarterly_net_income_yoy_growth()` | DataFrame | Net income YoY % | | |
| | `ticker.annual_fcf_yoy_growth()` | DataFrame | FCF YoY % anual | | |
| | `ticker.quarterly_eps_yoy_growth()` | DataFrame | EPS YoY % | | |
| | `ticker.quarterly_ttm_eps_yoy_growth()` | DataFrame | TTM EPS YoY % | | |
| #### Categoría 4 - Rentabilidad (Profitability) | |
| | Método | Returns | Descripción | | |
| |---|---|---| | |
| | `ticker.quarterly_gross_margin()` | DataFrame | Gross margin % | | |
| | `ticker.annual_gross_margin()` | DataFrame | Gross margin % anual | | |
| | `ticker.quarterly_operating_margin()` | DataFrame | Operating margin % | | |
| | `ticker.annual_net_margin()` | DataFrame | Net margin % anual | | |
| | `ticker.quarterly_ebitda_margin()` | DataFrame | EBITDA margin % | | |
| | `ticker.annual_fcf_margin()` | DataFrame | FCF margin % anual | | |
| #### Categoría 5 - Fundamentales (Info & News) | |
| | Método | Returns | Descripción | | |
| |---|---|---| | |
| | `ticker.info()` | DataFrame | Company profile completo | | |
| | `ticker.sec_filing()` | DataFrame | SEC filings metadata | | |
| | `ticker.officers()` | DataFrame | Key executives | | |
| | `ticker.currency(symbol)` | DataFrame | FX rates históricos | | |
| | `ticker.earning_call_transcripts()` | `Transcripts` object | Earnings call transcripts (get_transcripts_list, get_transcript) | | |
| | `ticker.news()` | `News` object | Financial news (get_news_list, get_news by uuid) | | |
| #### Categoría 6 - Comparación por Industria | |
| | Método | Descripción | | |
| |---|---| | |
| | `ticker.industry_ttm_pe()` | Industry TTM P/E | | |
| | `ticker.industry_ps_ratio()` | Industry P/S | | |
| | `ticker.industry_pb_ratio()` | Industry P/B | | |
| | `ticker.industry_roe()` | Industry ROE | | |
| | `ticker.industry_roic()` | Industry ROIC | | |
| | `ticker.industry_quarterly_gross_margin()` | Industry gross margin | | |
| #### Batch - `Tickers` Class | |
| `Tickers(['AAPL', 'TSLA', 'NVDA'])` ejecuta queries en paralelo vía `ThreadPoolExecutor`. Cada método retorna o un DataFrame combinado o un dict `{symbol: object}`. | |
| ### Uso en LT (Nuestro Proyecto) | |
| **Wrapper:** `core/defeatbeta_provider.py` — `DefeatBetaProvider` como wrapper DI. | |
| **Endpoints expuestos:** | |
| - `GET /api/news/{symbol}` → `ticker.news().get_news_list()` (news headlines) | |
| **Uso actual:** | |
| - Enriquecimiento de calendario con `has_news` + headline. | |
| - Enriquecimiento de scan results con headline + publisher. | |
| - Startup pre-warm: `fetch_news_batch` (WHERE symbol IN (...)) vía DuckDB directo. | |
| - **Catalizador primario:** `fetch_news_batch` (268/277 símbolos cubiertos sin rate-limit en el análisis de jul-2026; Finviz rate-limitó a 89). Devuelve `title` + `report_date`. | |
| - **Hora exacta de evento de dilución/catalizador:** `fetch_sec_filings_batch` → columna `acceptance_date_time` (ISO con hora, convertir a `America/New_York`). Es la marca de tiempo de aceptación SEC del 8-K, buena proxy de la hora del evento material. | |
| - **Patrón híbrido de dilución (verificado jul-2026):** SmallCapLab da el *mecanismo* (ATM/Shelf/PrivatePlacement/ReverseSplitSignal) y DefeatBeta da la *hora* del 8-K que anuncia el evento. Son complementarios. | |
| **⚠️ Caveat verificado (2026-07):** para el universo de micro-caps (movers de jul-2026, close<$5), `fetch_sec_filings_batch` **NO trajo filings S-1/S-3/S-8/D** — solo 8-K, 10-Q, 10-K, 4, 6-K. Por eso el *mecanismo* de oferta se obtiene de SmallCapLab, no de DefeatBeta SEC. Si necesitás S-1/S-3 de micro-caps, ir a SEC EDGAR directo. | |
| **Instalación / Dependencias:** | |
| ```toml | |
| # pyproject.toml | |
| [project.dependencies] | |
| defeatbeta-api = ">=0.0.43" | |
| ``` | |
| **Override de pandas** (por conflicto con tradingview-screener): | |
| ```toml | |
| [tool.uv] | |
| override-dependencies = [ | |
| "pandas>=3.0.1", | |
| ] | |
| ``` | |
| ### Ventajas sobre Yahoo Finance Directo | |
| | Aspecto | Yahoo Finance (directo) | DefeatBeta API | | |
| |---|---|---| | |
| | Rate limits | Sí (~200 req/min) | No (Parquet estáticos) | | |
| | Confiabilidad | Baja (scraping, caídas frecuentes) | Alta (HF datasets) | | |
| | Velocidad | 200-500ms por request | <100ms (cached) / 2-5s (cold, primer query) | | |
| | Datos históricos | Limitado a 2-3 años (intraday) | 15+ años completo | | |
| | Earnings transcripts | No disponible nativo | Sí, completos | | |
| | Revenue breakdown | No disponible | Sí, por segmento/geo/producto | | |
| | DCF / Excel report | No | Sí (integrados) | | |
| | LLM analysis | No | Sí (OpenAI integrado) | | |
| | Windows support | N/A | Sí (v0.0.60+) | | |
| ### Limitaciones Conocidas | |
| 1. **Cold start:** La primera query después de instalar o limpiar caché tarda 2-5s (descarga bytes del parquet via HTTP range). | |
| 2. **DuckDB in-memory:** Solo singleton; no recomendado para workloads multi-process pesados. | |
| 3. **Datos diarios únicamente (precios):** No hay barras intraday (minute, hour) en los datasets de precios. PERO los SEC filings (`stock_sec_filing`) SÍ incluyen `acceptance_date_time` (timestamp ISO con hora, convertible a NY) — útil como hora exacta del evento material. No confundir: la limitación es de *precios*, no de *filings*. | |
| 4. **Sin streaming:** No soporta datos en tiempo real; es un batch histórico actualizado ~24h. | |
| 5. **Ticker único por instancia:** Cada `Ticker` se instancia con un símbolo; usar `Tickers` para batch. | |
| 6. **Windows:** Nativo desde v0.0.60; versiones anteriores requerían WSL. | |