Power Markets
Five Years of New York Grid Data, One Probabilistic Price Forecast
A market-data warehouse feeds LEAR, LightGBM and structural congestion models, all scored under time-block cross-validation.
Electricity prices in New York move with load, weather and transmission congestion, and those forces change from one hour to the next. The nyiso-grid project collects five years of NYISO market data into a DuckDB warehouse. On top of it sit locational marginal price forecasts that give a full distribution instead of a single point estimate.
The models are a LEAR regression, LightGBM, and a structural congestion model, each tested on time-block cross-validation so that no future data leaks into training.
“A forecast that only gives one number can't tell you how wrong it might be.”
Overfitting diagnostics are part of the pipeline, not an afterthought. A dashboard is included. Joint DA/RT scenarios now feed TCC and DART pricers; neither beats its baseline yet, so the DART pricer is being tested forward on live data.
Python · DuckDB · LightGBM
