Yahoo Coverage + Charts Audit
Date: 2026-05-21
Scope: Company table coverage, PriceHistory, CompanyNews, EU_strict V14e cohort, chart rendering
Auditor: data-engineer agent
Note: PriceHistory is used exclusively for backtest compute. Live charts fetch from Yahoo Finance API directly via /api/stock. These are separate coverage concerns.
A. Companies basics
| Metric | Count | Pct |
|---|---|---|
| total_companies | 29,201 | 100% |
| yahooSymbol | 21,901 | 75.0% |
| analystAt | 21,488 | 73.6% |
| marketCap | 18,737 | 64.2% |
| currentPrice | 17,107 | 58.6% |
| descriptionEn/Fr | 8,561 | 29.3% |
| sectorTag | 7,947 | 27.2% |
| trailingPE | 6,007 | 20.6% |
| beta | 8,692 | 29.8% |
| isin | 4,932 | 16.9% |
Note: description (legacy field) = 0. All descriptions are in descriptionEn/descriptionFr columns.
B. PriceHistory coverage by market
| Market | Companies | With prices | Pct |
|---|---|---|---|
| XNAS | 6,759 | 4,476 | 66.2% |
| XTKS | 6,174 | 566 | 9.2% |
| XKRX | 3,782 | 635 | 16.8% |
| XSTO | 2,199 | 102 | 4.6% |
| XSHE | 1,716 | 389 | 22.7% |
| XMIL | 1,445 | 232 | 16.1% |
| XSHG | 1,069 | 302 | 28.3% |
| XASX | 783 | 9 | 1.1% |
| XBOM | 729 | 323 | 44.3% |
| XHKG | 589 | 117 | 19.9% |
| XPAR | 583 | 510 | 87.5% |
| BVMF | 538 | 150 | 27.9% |
| XMAD | 441 | 104 | 23.6% |
| XETR | 292 | 208 | 71.2% |
| XCSE | 257 | 91 | 35.4% |
| XPHS | 257 | 202 | 78.6% |
| XHEL | 228 | 63 | 27.6% |
| XWAR | 191 | 102 | 53.4% |
| XOSL | 181 | 66 | 36.5% |
| XWBO | 170 | 26 | 15.3% |
| XAMS | 123 | 114 | 92.7% |
| XBRU | 25 | 23 | 92.0% |
EU_strict aggregate: 1,112 / 3,801 = 29.3% companies with PriceHistory
Critical gap: XSTO has 4.6% PriceHistory coverage (102/2,199) with 1,590 companies having declarations but no backtest prices. 1,520 of these have a yahooSymbol. This is a backtest quality gap, not a chart rendering gap.
C. PriceHistory depth
| Metric | Value |
|---|---|
| total_symbols | 7,358 (growing, backfill running) |
| total_rows | 3,445,134+ |
| avg_rows_per_symbol | 458 |
| global_earliest | 1962-01-01 |
| global_latest | 2026-05-21 |
| deepest_symbol | BPN.WA.BIZRD (3,000 rows, 2013-2026) |
| deepest_par | EN.PA (2,552 rows, 1991-2026) |
D. News coverage
CompanyNews (Yahoo Finance news, primary)
| Metric | Value |
|---|---|
| table | CompanyNews |
| total_rows | 15,911 (growing, backfill running) |
| distinct_companies | 1,642 |
| pct_companies_with_news | 5.6% |
| oldest | 2026-02-17 |
| newest | 2026-05-21 |
CompanyNewsItem (translated news, secondary)
| Metric | Value |
|---|---|
| total_rows | 5,035 |
| distinct_companies (by slug) | 566 |
| oldest | 2001-08-06 |
| newest | 2026-05-14 |
E. EU_strict V14e cohort focus
Per-market fundamentals coverage
| Market | Companies | marketCap pct | sectorTag pct | currentPrice pct | yahooSymbol pct |
|---|---|---|---|---|---|
| XSTO | 2,199 | 62.5% | 1.4% | 63.2% | 73.8% |
| XPAR | 583 | 85.2% | 99.7% | 87.5% | 87.8% |
| XETR | 292 | 81.5% | 99.7% | 42.1% | 99.3% |
| XHEL | 228 | 83.3% | 18.0% | 75.0% | 88.6% |
| XOSL | 181 | 78.5% | 49.2% | 60.2% | 96.1% |
| XWBO | 170 | 41.2% | 8.2% | 44.7% | 57.1% |
| XAMS | 123 | 93.5% | 65.0% | 62.6% | 98.4% |
| XBRU | 25 | 100.0% | 68.0% | 28.0% | 100.0% |
Description coverage (descriptionEn) EU strict
| Market | Total | Desc pct |
|---|---|---|
| XPAR | 583 | 99.7% |
| XETR | 292 | 99.3% |
| XBRU | 25 | 80.0% |
| XAMS | 123 | 56.1% |
| XHEL | 228 | 50.9% |
| XWBO | 170 | 24.1% |
| XSTO | 2,199 | 0.7% |
| XOSL | 181 | 0.6% |
EU companies with declarations but NO PriceHistory (backtest gap)
| Market | decls_no_prices | has_yahoo_no_prices |
|---|---|---|
| XSTO | 1,590 | 1,336 |
| XHEL | 157 | 135 |
| XWBO | 121 | 65 |
| XOSL | 115 | 108 |
| XETR | 84 | 82 |
| XPAR | 73 | 2 |
| XAMS | 9 | 7 |
| XBRU | 2 | 2 |
| TOTAL | 2,151 | 1,735 |
News coverage EU strict
| Market | Total | With yahooSymbol | With news |
|---|---|---|---|
| XSTO | 2,199 | 1,622 | 2 |
| XPAR | 583 | 512 | 270 |
| XETR | 292 | 290 | 25 |
| XHEL | 228 | 202 | 0 |
| XOSL | 181 | 174 | 0 |
| XWBO | 170 | 97 | 0 |
| XAMS | 123 | 121 | 3 |
| XBRU | 25 | 25 | 2 |
F. Chart rendering test
Architecture clarification
The live chart (/api/stock) fetches directly from Yahoo Finance API at render time. PriceHistory is NOT used for chart rendering. The StockChart component shows a fallback trade table when Yahoo returns no data for a symbol.
5 companies WITH PriceHistory (backtest-capable + live chart)
| Company | Market | API result | HTTP |
|---|---|---|---|
| tikehau-capital-6396 | XPAR | 200, symbol=TKO.PA, 63 points | 200 |
| totalenergies-se-3321 | XPAR | 200, live chart served | 200 |
| les-constructeurs-du-bois-7233 | XPAR | 200, live chart served | 200 |
| konecranes-oyj | XHEL | 200, live chart served | 200 |
| calibre-3420 | XPAR | 200, live chart served | 200 |
5 companies WITHOUT PriceHistory
| Company | Market | Yahoo live API | HTTP | Status |
|---|---|---|---|---|
| nordea-bank-abp | XHEL | 200, NDA-SE.ST, 62 points | 200 | Chart renders (Yahoo has data) |
| kontron-ag | XWBO | 200, KTN.DE, 63 points | 200 | Chart renders (Yahoo found XETR symbol) |
| g-city-europe-limited | XWBO | 200, GYC.F found by name | 200 | Chart renders |
| bajaj-mobility-ag | XWBO | 404 (Symbol not found AT0000A1MBL8) | 200 | Fallback table shown (328 decls visible) |
| smart-good-things-holding-sa-7760 | XPAR | no ISIN/symbol - likely fallback | 200 | Fallback table shown (166 decls visible) |
Finding: Fallback table (declared in StockChart lines 359-430) is confirmed operational. The message "Donnees prix en cours d'ingestion" renders correctly for symbols Yahoo cannot resolve.
G. Backfill jobs launched
| Job | PID | Log | Notes |
|---|---|---|---|
| Yahoo news backfill | 64560 | /tmp/backfill-eu-news-2026-05-21.log | 3,000 companies queued, 429 rate limiting, progressing |
| PriceHistory backfill | 65794 | /tmp/backfill-price-history-eu-2026-05-21.log | 2,000 symbols, 1,000d lookback, ~50 bars/s |
| Fundamentals SE | 66102 | /tmp/backfill-fundamentals-se-reset-2026-05-21.log | 250 XSTO companies, 206 noSym (micro-caps no ISIN), 1 ok |
ETA estimates
- PriceHistory: 2,000 symbols x 0.2s throttle = ~7 minutes for first pass. Adds ~1.3M rows for EU markets.
- News: Rate-limited by Yahoo 429s. With 90s backoffs, throughput ~10-20 companies/minute. Full EU coverage (3,000 companies) = 3-5 hours.
- Fundamentals SE: Most XSTO micro-caps fail symbol resolution (noSym=82%). Small improvement possible.
Action items
XSTO sectorTag = 1.4% (critical): 2,168/2,199 XSTO companies have no sector. The
backfill-company-fundamentals-yahoo.tsscript returns no sector for XSTO micro-caps because Yahoo'sassetProfilemodule is not returningsectorfor Swedish small-caps. Need to check if Yahoo v11 or alternative (e.g. SWEGVS classification) can fill this.XSTO PriceHistory = 4.6%: PriceHistory backfill now running. After completion, backtest coverage should reach ~50-60% for XSTO (symbols already resolved). Key issue: 1,520 XSTO companies have yahooSymbol but no PriceHistory rows.
XHEL/XOSL/XWBO news = 0%: News backfill running. These 3 markets have 0 news items today. With 174+202+97=473 companies having yahooSymbol, expected coverage post-backfill: ~50% (Yahoo news sparse for Nordic/Austrian exchanges).
XETR PriceHistory = 71.2% but currentPrice = 42.1%: 82 XETR companies have yahooSymbol and declarations but no PriceHistory. PriceHistory backfill will address. The currentPrice gap is separate (fundamentals stale).
XWBO marketCap = 41.2%: Only 57.1% of XWBO companies have yahooSymbol resolved. 73 XWBO companies have no symbol at all. Manual mapping or Stooq fallback needed.
XOSL/XHEL description = <1%: Descriptions for 178 XOSL + 112 XHEL companies missing.
backfill-desc-top.tsor_backfill-yahoo-fundamentals.ts --country=NO --country=FIcan fill via assetProfile.CompanyNews coverage = 5.6% globally: Only 1,642/29,201 companies have news. 24,759 companies with yahooSymbol have no news. News is primarily populated for XPAR/XNAS. Running
_yahoo-news-scrape.ts --limit=10000 --workers=8in a sustained overnight job would expand coverage significantly.bajaj-mobility-ag (AT0000A1MBL8, BMAG.VI): Yahoo returns no data for this ISIN/symbol. Check if the Vienna exchange ticker has changed or if Stooq has it (
.VIsuffix).