Data Corner: Better forecasting leads to lower costs, greater reliability

By Taj Ait-Laoussine and Titiaan Palazzi

Previously published in PowerGrid International

On two consecutive days in August 2020, temperatures swelled across much of California, leading to the hottest August day on record since 1985. Electricity demand also rose rapidly, forcing the California Independent System Operator (CAISO) to initiate rotating outages that affected more than a million people. It was the first time in nearly 20 years that Californians experienced such outages. What happened?

In January 2021, the CAISO and other agencies published their final analysis of the event in response to a request from Governor Newsom. They identified three root causes for the rotating outages, with under-scheduling of demand in the day-ahead market a primary culprit. For August 14 and August 15, California scheduling coordinators (SCs) under-scheduled peak demand (net of customer generation) by 2,164 MW (4.6%) and 2,023 MW (4.5%), respectively. As a result, CAISO did not have enough generation or demand response capacity on hand to maintain its required reserves, leading to the proactive triggering of rotating blackouts.

East Bay Community Energy (EBCE) is a community choice aggregator (CCA) that provides electricity to more than 550,000 homes and businesses in the East Bay Area. As a Load Service Entity (LSE) in California, it must generate and submit a load forecast to the CAISO. To support its day-ahead forecasting needs, EBCE has partnered with Myst AI (Myst), a Google-backed technology company focused on accurate near-term forecasting. Since 2019, EBCE has collaborated with Myst to reduce its load forecast error and associated procurement costs. As part of this process, EBCE feeds Myst with a historical bottoms-up aggregate of its customers’ meter data on an ongoing basis. Myst then uses this data, in conjunction with machine learning techniques, to generate a load forecast for each hour of the next seven days, refreshed every hour.

During typical days, Myst generates highly accurate day-ahead load predictions that are less than 3% of the settled demand for EBCE for any given hour. However, for the hours of the rotating outages on August 14 and 15, EBCE’s load forecast errors on system peaks were on par with system errors reported in the CAISO report, but less accurate than typical for EBCE. This caught both EBCE and Myst by surprise.

In exploring the root cause of this deviation, Myst and EBCE quickly uncovered that there was a significant error in the underlying weather forecasts that were used to predict demand. Weather forecasts are an important input into day-ahead electricity forecasting models, and inaccurate weather forecasts can lead to significant errors. To confirm this finding, we performed a back-cast by running our models with actual weather, and found that with such perfect weather data, the day-ahead load forecast errors dropped well below 3%, with a reduction in error of nearly 70% for August 15. In other words, inaccurate weather forecasts explained most of the unusual load forecast error for EBCE on the days where CAISO triggered the rotating outages. Indeed, the weather in California on those days was a 1-in-30-year event according to the CAISO, which may explain why weather forecasts failed to keep up – another probable consequence of climate change.

To minimize the impact of inaccurate weather forecasts going forward, Myst and EBCE are now including forecasts from multiple weather providers. We are also using more weather forecasts with more geographic granularity. And we are incorporating air quality data in our load forecasts, in anticipation of future wildfires and extreme weather events, and their effect on load and solar production.

Further improvements are possible still. As a CCA, EBCE’s total load cannot be directly metered as a whole but must instead be constructed by a bottom-up aggregation of the meter data for all its customers. That’s because EBCE does not serve all customers within a specific geography and does not have access to a system-level meter that records aggregated usage. Therefore, EBCE relies on aggregating the interval data for each of the meters it serves to create a system load shape.

To ensure highly accurate forecasts, EBCE needs access to meter data as quickly, reliably and completely as possible. EBCE has deployed advanced technology and complex processes to achieve this, and currently succeeds in having 97% of its customers’ data within five days or less. Myst can therefore train its model on electricity load data up to five days ago, which is better than the official settlement load data lag of 60 days, or even the standard 30-day bill cycle.

Still, five days is a long time when performing day ahead forecasts. In fact, considering this limitation, and given the relatively small size of EBCE, we find our forecast performance during the heat wave remarkable, especially on August 15, when we did not have the benefit of knowing what had happened to our system on the days leading up to the extreme events, yet still managed to generate a forecast error during peak hours on the order of 5%.

EBCE is now doubling down on its efforts to improve on those 5-day lag / 97% completeness metrics. While the source system in our service territory can provide data with a lag of 2 days under ideal conditions, it is not considered mission critical for CCAs and suffers from data availability and completeness issues that present significant hurdles that result in our current performance metrics. Until those hurdles are addressed, we are exploring technical and process improvements to further work around them.

We suspect other CCAs have even greater challenges than EBCE. Some CCAs do not even have access to interval meter data until bill cycle completion, well past forecasting usefulness. California has invested billions in a smart grid infrastructure that enables meters to provide near-real-time data, but CCAs, however, currently experience significant hurdles in accessing that data quickly, reliably and comprehensively, because access to such data has not been considered critical to CCA operations to date. Unlike climate change, this challenge should be relatively easy to solve and would yield significant system reliability benefits.

The events of August 2021 have demonstrated the importance of accurate load forecasting, and the impact that forecasting errors can have on the entire California economy. While weather was a primary driver of the error that EBCE experienced, we believe that faster, comprehensive and reliable access to meter data would help us and other CCAs further improve. Myst and EBCE have invested significantly in technology and processes to minimize their errors, and we encourage other market participants to do the same. At the same time, we urge regulatory agencies to push for faster, comprehensive and reliable CCA access to smart meter data. Such access won’t just improve system reliability; it will foster innovations in rate design and product offerings.

Taj Ait-Laoussine is VP of Technology and Data Analytics at East Bay Community Energy. Titiaan Palazzi is Co-Founder and COO of Myst AI. This article also ran in PowerGrid International magazine.