EXECUTIVE SUMMARY
The Bonneville Power Administration is developing a Strategic Energy Management (SEM) program for commercial buildings served by its customer utilities. BPA conducted a pilot test of the program, including a test of the program’s savings estimation procedure. We reviewed the program design, the pilot projects’ data and energy savings estimation models, and identified factors relevant to future evaluations of the program.
There will be two program delivery tracks: “SEM Lite” and “SEM Heavy.” The former is more flexible and is designed for Option 2 and large Option 1 utilities, who have technical staff that can take the lead in providing SEM services to program participants. SEM Heavy will provide additional resources from BPA via training and individual support to utilities and participants.
Both program tracks require screening of candidate commercial buildings for availability of energy data, potential for an accurate statistical model of energy use, and savings potential. The energy models are based on regression, typically using ASHRAE change-point models, and include a calculation of uncertainty, which is used to judge the accuracy of the saving estimates.
There are reporting requirements, including: a site assessment; the baseline model and a Baseline Model Analysis Report; general M&V Plan; and the Custom Project Proposal and Completion Report describing the site, baseline, implemented measures, and final estimated savings.
An energy model is created for each participating building. The energy modeling is adherent to IPMVP and follows recommendations found in the BPA Verification by Energy Modeling Protocol and the BPA Regression for M&V: Reference Guide.
Factors Important in Future Evaluation
- Pre-implementation savings estimates can be helpful to evaluators. They provide corroboration of the evaluated savings, or an indication of a need for review of the evaluation calculations. Providing information about what times or under what conditions the SEM actions are expected to save energy can also be useful to evaluators when reviewing savings models. Further research is needed to determine how to handle buildings where pre-implementation savings estimates are not within the uncertainty bounds of the regression model estimate of savings.
- The regression models should provide accurate and transparent estimates of savings and precision. Models with physical significance facilitate the comparison of results with expectations. Models that follow industry guidelines and common practice will be transparent and readily evaluated.
- Non-routine adjustments may best be estimated using statistical methods. When engineering calculations are used, the calculations should reside in the same workbook as the regression model so that the adjustments can easily be accessed by evaluators. If records can be kept regarding the timing of SEM actions or other program activities, they will be very helpful in verifying that the energy use changes visible in the data can be attributed to the program.
- Even more so than for other program changes, it is important to know the timing of implementation for capital measures. This will facilitate reconciliation of the capital measure savings with the regression model estimates of savings.
- BPA needs to have a policy and provide clear direction on how negative savings estimates are treated for both the setting of incentives and the estimation of program savings. We recommend that negative savings be treated the same as positive savings in estimating program savings. When program models show negative savings, the program should document any reasons why the savings estimates are negative.
- Since BPA’s SEM program may cover a wide geographical area, encompassing many utility and regulatory jurisdictions, it is very important that the program document whether the applicable baseline is pre-conditions or current practice. The need for a current practice, instead of a pre-conditions baseline, needs to be established as part of project screening.
Pilot Project Review and Recommendations
The pilot projects were well-analyzed using the energy models developed by BPA. Our suggestions for improvement were mostly toward using the simplest models possible that fit the data, but the choices made for the pilot project models were all reasonable. The overall conclusion is that projects can be evaluated using the proposed process. Savings can be estimated with sufficient precision except for sites and meters that have very low savings, and most of these types of sites are not expected make it through screening.
Four important recommendations were made as a result of this review. These recommendations will help ensure that evaluation is efficient by reducing the changes needed to be made to the program’s models and analyses.
- We recommend that engineers use models with the fewest number of parameters that still fit the data.
- We recommend that a model of the reporting period data always be created.
- Modelers should ensure that significant modeling choices are traceable.
- Analysts should document analyses and decision-making rationales.