EXECUTIVE SUMMARY
This report describes the results of the impact evaluation of Energy Trust of Oregon’s Path to Net Zero (PTNZ) pilot. The primary goal of the pilot was to understand if net zero energy could be achieved and to understand the process required by design teams to make decisions that lead to a net zero outcome or are critical to creating a path to net zero program approach in commercial buildings. Increased technical and financial support was provided to support decision‐making in pursuit of the goals. A process evaluation was conducted alongside the program’s implementation of the pilot and concluded in 2012.1 With all pilot buildings now complete, and enough post‐occupancy data available, this impact evaluation was conducted to characterize the effectiveness of many measures pursued through the New Buildings program (the program)’s Path to Net Zero Pilot.
From the separate process evaluation, we know that the pilot was met with great interest from owners and the design community, providing information to the program. A white paper was also published through ACEEE that describes the early energy use of the buildings along with major energy efficiency features and measures selected, and the program’s assessment of major net zero design strategies.2 This evaluation builds on extensive information gained by program staff, design teams, and building owners; and provides a final assessment of savings impact.
Background
The pilot was designed to significantly advance major renovation and new construction projects beyond energy code. It referenced the 2007 Oregon Structural Specialty code and applied program requirements about building layout to determine baseline. Pilot requirements included a commitment to target 50% energy savings in design, but allowed projects to pursue 60% energy savings if using a combination of efficiency and on‐site renewable energy generation, which provided the program with a learning opportunity as far as incorporating renewable energy measures.
To achieve these high energy savings targets, the pilot focused on early decision making and was geared to provide technical support to assess various energy‐savings considerations in the very early stages. With early design assistance, the program engaged with building owners and teams to discuss energy‐savings strategies; the program also provided technical support for studies and energy modeling, and later addressed installation and commissioning, as well as providing optional monitoring and reporting support. By 2014, eight projects completed the pilot.
Energy Trust’s New Buildings program incorporated early lessons learned into the standard program, such as early design assistance, and then launched Path to Net Zero in 2014 as a standard offering with updated requirements and reference energy code, making the requirement a target of 40% energy savings. The impact evaluation summarized in this report aims to provide important information to program staff about the performance of this highest tier of energy efficient building design.
Due to the unique nature of PTNZ pilot project, the more extensive commissioning, monitoring and reporting and post‐occupancy engagement that was required, the eight pilot projects had not been included in previous impact evaluations of the NB program. Most PTNZ projects were evaluated with two years of post‐occupancy data compared to the regular program’s one‐year timeline.
The goals of this impact evaluation were to:
- Measure actual savings compared to program estimated savings for these projects by determining the gas and electric energy savings associated with each measure implemented at six sampled sites. Energy Trust uses this information for program savings projections and budget developments, and incorporates it into their annual true‐up of program savings.
- Report observations and make recommendations to help Energy Trust improve the effectiveness of future engineering studies and impact evaluations of its commercial new construction projects, particularly buildings designed for the highest energy performance targets. These include findings that help explain substantial deviations from the claimed savings, and recommendations for changes to gross savings calculation methods and/or other program processes that will enhance future realization rates.
Methodology
The three primary steps of this evaluation included a review of previous engineering estimates, site data collection, model review, and an impact analysis. Energy Trust provided project files for each site, which the evaluation team reviewed and examined to assess the reasonableness of modelling assumptions, such as hours of operation and equipment specification. In particular, we examined the baseline model definition to determine if the baseline methodology complied with code requirements. The 2007 Oregon Structural Specialty Code was applicable to five sites, with ASHRAE 90.1‐2007 applied to one site.
Following this, we performed site visits and collected trend data to inform revisions to the simulation models for each site. The revised simulation models were in turn used to analyze energy savings impacts.
Findings
- How well did the pilot program predict savings for each project?
- How well did the PTNZ pilot program meet its goals of percent savings over code?
- What program‐wide observations were made and what issues were uncovered during the course of the evaluation?
Prediction of Savings
Four projects were well within their estimated electric savings goals (100% realized savings) or exceeding them; one project was slightly lower than the target, only realizing 86% of original savings estimated; and one saved far less than expected, realizing only 68% of the original electric savings estimate. Gas realization rates varied more widely compared to electric energy savings; in two of the four projects with gas savings measures, the gas savings were around 20% higher than predicted, and two projects saved 30‐50% less gas energy than their target. Renewable generation, in the form of photovoltaic (PV) arrays, was incorporated into five projects. Four of the five projects with solar PV installed were within range of original generation estimates and one large project exceeded its goal with additional investments made to extend its PV system.
Figure 1 displays the impact evaluation findings as percent realization rates, indicating what percent of the original savings estimates were actually achieved. Savings realization rates are broken out among the three main sources‐‐electric and natural gas savings, and also renewable energy generation‐‐followed by the total site‐level savings realization rates presented at the bottom of the chart. Across all evaluated sites, the electric realization rate was 105%, the gas realization rate was 65%, and the renewable generation realization rate was 113%. When considering all fuels savings and renewable energy generation, the realization rate across the evaluated sites was 85%.
Percent Savings Over Code
Overall Observations/Issues
The following observations and issues were encountered during the course of the evaluation:
PTNZ Program Issues/Observations
- Data integrity issues were encountered at each site. These included the following:
- Lack of historical trends
- Historical trends set to record for too short a term (typically less than a week)
- Large chronological gaps in data
- Issues with intermediate data handling, especially for sites with web “dashboards” that import data from control systems
- At all sites we visited, we found that the Monitoring and Reporting (M&R) system was either inactive or experienced data issues. Sites that indicated a high level of involvement in site energy monitoring demonstrated better savings realization rates. The one site with an inactive M&R system exhibited the lowest realization rate.
- In addition to the energy impact analysis presented above, SBW surveyed site building management staff to improve understanding of their energy management and monitoring behaviors. The results of the survey are shown in the appendices provided with the internal version of this report.
- Electric measures are generally performing well, with a few exceptions performing slightly below expectations.
- There appears to be much more variability amongst the gas measures. Some of these measures are performing well against code, while others are demonstrating significantly lower realization rates. Actual gas usage was found to be higher than expected.
- Renewables are generally performing better than expected, and at some sites are offsetting under-performance by the electric efficiency measures.
- Energy models were found to be well‐developed and of excellent quality. Although we did find modeling errors which impacted savings, the relative number of errors we found was small. We believe that the high level of technical review required for the models was a major reason for success in this area.
Recommendations
This section describes recommendations developed as a result of the issues encountered during the engineering review and impact analysis processes.
Data Integrity
Data plays a critical part in meeting the goals of any energy efficiency program. It is especially important for net zero programs. PTNZ program energy savings goals are far more aggressive than for conventional programs, and even a small increase in energy can significantly impact
whether or not these goals are met. Data allows building owners to quickly detect rises in energy, identify the causes, and ensure that applied solutions are working.
Data issues were discovered at each of the six evaluated sites. The following recommendations are meant to improve how the program handles both M&R data (which pertains to whole building and end use energy such as lighting, HVAC, etc.) and control system trend data (which
pertains to detailed system operation such as fan speeds, temperature setpoints, daylighting, control, etc.).
- Currently, although the program requires periodic checks (sometimes quarterly) of M&R data throughout the 18 month reporting period, data quality issues were encountered with data stored during this period. We recommend improving the methodology used to check M&R data during this period. This could include more frequent checks to ensure that valid data is being stored, a closer examination of the reasonableness of the data, or comparison of accumulated M&R data with concurrent utility billing data. These activities could also be incorporated into the site commissioning plan in order to facilitate implementation.
- We also recommend strongly encouraging customers to keep the M&R system active and functioning beyond the current 18 month requirement in order to facilitate the maintenance of energy savings. This could be encouraged by educating customers about the advantage in energy savings of maintaining the system. The same methodology used by Energy Trust to ensure data validity during the 18 month period (with the improvements noted above) could be adopted by the customer.
- In general, sites should be encouraged to regularly check all historical trend system data (M&R and otherwise) to ensure continued integrity. Sites should also be encouraged to maintain historical control system trends of 15‐minute interval data for at least a year in order to capture building operation during all conditions. Sites should be educated as to why this is important, and the benefits of monitoring their historical trend systems regularly (as well as the negative impacts of not doing so).
- M&R does not extend to natural gas. Due to underperformance of gas measures and data issues at many of the sites, consider offering an M&R option for gas, which would install gas meters on major equipment such as boilers and domestic hot water heaters, and would record gas usage at 15 minute intervals similar to how electric M&R is currently being accomplished.
Integrated Building Management System Practices
- Identify and support advanced system maintenance trainings specifically tailored to new technologies for building maintenance staff to ensure sufficient site expertise throughout the life of the measures installed. This is especially important for sites that experience building management changeover.
- Encourage site staff to conduct periodic walkthroughs that check on key measures; this ensures that tenants continue to use best practices. Some of the projects have involved measures whose effectiveness relies on good tenant education and regular communication with building management staff.
- Encourage site staff to collaborate with their controls contractors in actively managing building performance. This may include the controls contractor directly monitoring performance and reporting regularly to site staff, or setting up a system that alerts staff to abnormally high energy usage and other performance indicator abnormalities. Enlisting the help of controls contractors is important because in most cases they are more familiar with the complexities of building operation than the site staff.
- Support and encourage building design teams to minimize building maintenance complexity and requirements. One important design feature would be maintenance accessibility to key systems. In one site in particular, maintenance of system performance was inhibited by a building design that made the system difficult to physically access, resulting in costly and time‐consuming maintenance that required specialized vendors.
Maintenance of Effective Practices
- Technical reviews were found to be a thorough review of energy calculation methods, inputs, and assumptions. The reviews ensured that pre‐occupancy energy models and calculations were of good quality. The format and layout of the technical review memos presented information clearly and helped in isolating specific issues and responses.
Energy Modeling Methodology
Energy modeling methodologies employed by the pilot were found to be sound. The one area for potential improvement relates to calculation of both project level and measure level savings. The pilot used a rolling baseline method to determine measure level savings, in which measures are added one at a time to the baseline model, and savings are calculated in steps as each measure is added.
This could be improved upon by using the full interactive model (all measures included) to calculate both total project and individual measure savings. Savings by measure can be accomplished with the interactive model using a “last in” approach in which the measure of interest is removed from the full interactive model to create the baseline condition for the measure. This ensures that interaction with all other measures is accounted for. Note that this method could potentially impact measure cost‐effectiveness.