Futures backfill — live mrežni test

Date: 2026-05-15. First end-to-end exercise of tools.history_cache.build_futures_cache against the Binance public futures API. Pre-test state: funding_rates and open_interest_history tables existed in database/klines_cache.db but were empty.

Findings

1. Funding endpoint works as-is

$ python -m tools.history_cache.build_futures_cache --symbol BTCUSDT --since 2026-04-01
{'symbol': 'BTCUSDT', 'funding_rows': 132}

132 funding records across 44 days (3 funding intervals per day × 44 ≈ 132). Range 2026-04-01 00:00 UTC → 2026-05-14 16:00 UTC. Rate values plausible (~3-4e-05 per 8h funding). Mark prices align with spot.

2. Open Interest required a 30-day clamp

First run failed with:

oi: APIError(code=-1130): parameter 'startTime' is invalid.

Root cause: Binance's /futures/data/openInterestHist endpoint only retains the last ~30 days of aggregated OI snapshots. Passing startTime older than that triggers -1130. The docstring acknowledged the constraint but the code did not enforce it.

Fix in src/binance_futures.py: _OI_HISTORY_MAX_DAYS = 29 constant plus params['startTime'] = max(start_ms, now - 29d) clamp inside fetch_open_interest_history. After fix:

{'symbol': 'BTCUSDT', 'funding_rows': 132, 'oi_rows': 500, 'errors': []}

500 OI rows × 1h period = ~21 days back from now. Matches the documented window.

3. Idempotency confirmed

Re-running with the same --since 2026-04-01 returns identical row counts (132 / 500). persist_funding_rates and persist_open_interest use INSERT OR REPLACE on the natural keys.

4. Non-perpetual symbol degrades cleanly

$ python -m tools.history_cache.build_futures_cache --symbol BTCBTC
{'symbol': 'BTCBTC', 'funding_rows': 0, 'oi_rows': 0, 'errors': []}

Binance returns empty arrays (not 4xx) for symbols without a USDT-margined perpetual contract. Both endpoints handle this without raising.

5. Windows asyncio warning suppressed

Python 3.9 ProactorEventLoop on Windows logged a noisy but harmless "Event loop is closed" traceback at GC time. Added WindowsSelectorEventLoopPolicy guard at script entry. Data flow unaffected; output is now clean.

Final state (post-test)

How to apply

• For long history (> 30d) of OI, multiple calls with rolling endTime cursors are needed (script doesn't loop yet; current behavior gives the most recent 500 rows after the clamp). Add a paginated fetcher if a 6+ month OI window is ever required for backtests.
• Funding has full retention (~3 years on Binance) — --since works as expected for any reasonable backtest window.
• Default schedule: not part of any cron / scheduler yet. Tabela je populated samo na ručno pokretanje. Ako se počne koristiti u meta-score features (regime/cohort task #4), treba dodati u backfill pipeline.

Reproduce

# Funding only, custom range
.venv39/Scripts/python.exe -m tools.history_cache.build_futures_cache `
  --symbol BTCUSDT --since 2026-04-01 --skip-oi

# OI only, last 500 hourly snapshots
.venv39/Scripts/python.exe -m tools.history_cache.build_futures_cache `
  --symbol BTCUSDT --skip-funding

# All symbols already in klines cache
.venv39/Scripts/python.exe -m tools.history_cache.build_futures_cache `
  --all-cached-symbols