farms

EXECUTIVE SUMMARY

This report describes the results of the impact evaluation of Energy Trust of Oregon’s Path to Net Zero (PTNZ) pilot. The primary goal of the pilot was to understand if net zero energy could  be achieved and to understand the process required by design teams to make decisions that  lead to a net zero outcome or are critical to creating a path to net zero program approach in  commercial buildings. Increased technical and financial support was provided to support  decision‐making in pursuit of the goals. A process evaluation was conducted alongside the  program’s implementation of the pilot and concluded in 2012.1 With all pilot buildings now  complete, and enough post‐occupancy data available, this impact evaluation was conducted to  characterize the effectiveness of many measures pursued through the New Buildings program  (the program)’s Path to Net Zero Pilot.  

From the separate process evaluation, we know that the pilot was met with great interest from   owners and the design community, providing information to the program. A white paper was  also published through ACEEE that describes the early energy use of the buildings along with  major energy efficiency features and measures selected, and the program’s assessment of  major net zero design strategies.2 This evaluation builds on extensive information gained by  program staff, design teams, and building owners; and provides a final assessment of savings  impact. 

Background

The pilot was designed to significantly advance major renovation and new construction projects beyond energy code. It referenced the 2007 Oregon Structural Specialty code and applied  program requirements about building layout to determine baseline. Pilot requirements  included a commitment to target 50% energy savings in design, but allowed projects to pursue  60% energy savings if using a combination of efficiency and on‐site renewable energy  generation, which provided the program with a learning opportunity as far as incorporating  renewable energy measures.

To achieve these high energy savings targets, the pilot focused on early decision making and  was geared to provide technical support to assess various energy‐savings considerations in the  very early stages. With early design assistance, the program engaged with building owners and  teams to discuss energy‐savings strategies; the program also provided technical support for  studies and energy modeling, and later addressed installation and commissioning, as well as  providing optional monitoring and reporting support. By 2014, eight projects completed the  pilot.

Energy Trust’s New Buildings program incorporated early lessons learned into the standard program, such as early design assistance, and then launched Path to Net Zero in 2014 as a  standard offering with updated requirements and reference energy code, making the  requirement a target of 40% energy savings. The impact evaluation summarized in this report  aims to provide important information to program staff about the performance of this highest  tier of energy efficient building design.

Due to the unique nature of PTNZ pilot project, the more extensive commissioning, monitoring  and reporting and post‐occupancy engagement that was required, the eight pilot projects had  not been included in previous impact evaluations of the NB program. Most PTNZ projects were  evaluated with two years of post‐occupancy data compared to the regular program’s one‐year  timeline. 

The goals of this impact evaluation were to:

  1. Measure actual savings compared to program estimated savings for these projects by determining the gas and electric energy savings associated with each measure implemented  at six sampled sites. Energy Trust uses this information for program savings projections and  budget developments, and incorporates it into their annual true‐up of program savings.
  2. Report observations and make recommendations to help Energy Trust improve the effectiveness of future engineering studies and impact evaluations of its commercial new  construction projects, particularly buildings designed for the highest energy performance  targets. These include findings that help explain substantial deviations from the claimed  savings, and recommendations for changes to gross savings calculation methods and/or  other program processes that will enhance future realization rates. 

Methodology

The three primary steps of this evaluation included a review of previous engineering estimates,   site data collection, model review, and an impact analysis. Energy Trust provided project files  for each site, which the evaluation team reviewed and examined to assess the reasonableness  of modelling assumptions, such as hours of operation and equipment specification. In  particular, we examined the baseline model definition to determine if the baseline  methodology complied with code requirements. The 2007 Oregon Structural Specialty Code  was applicable to five sites, with ASHRAE 90.1‐2007 applied to one site.

Following this, we performed site visits and collected trend data to inform revisions to the  simulation models for each site. The revised simulation models were in turn used to analyze  energy savings impacts. 

Findings

Findings are presented to answer three questions:
  1. How well did the pilot program predict savings for each project?
  2. How well did the PTNZ pilot program meet its goals of percent savings over code?
  3. What program‐wide observations were made and what issues were uncovered during the  course of the evaluation?

Prediction of Savings

Four projects were well within their estimated electric savings goals (100% realized savings) or   exceeding them; one project was slightly lower than the target, only realizing 86% of original  savings estimated; and one saved far less than expected, realizing only 68% of the original  electric savings estimate. Gas realization rates varied more widely compared to electric energy  savings; in two of the four projects with gas savings measures, the gas savings were around 20%  higher than predicted, and two projects saved 30‐50% less gas energy than their target.  Renewable generation, in the form of photovoltaic (PV) arrays, was incorporated into five  projects. Four of the five projects with solar PV installed were within range of original  generation estimates and one large project exceeded its goal with additional investments made  to extend its PV system.

Figure 1 displays the impact evaluation findings as percent realization rates, indicating what  percent of the original savings estimates were actually achieved.  Savings realization rates are  broken out among the three main sources‐‐electric and natural gas savings, and also renewable  energy generation‐‐followed by the total site‐level savings realization rates presented at the  bottom of the chart. Across all evaluated sites, the electric realization rate was 105%, the gas  realization rate was 65%, and the renewable generation realization rate was 113%. When  considering all fuels savings and renewable energy generation, the realization rate across the  evaluated sites was 85%. 

project sites figure
Figure 1: PTNZ Project Realization Rates by Site, Type of Savings and Generation

Percent Savings Over Code

Six of the eight pilot projects completed by 2014 were evaluated, three of which are meeting their Path to Net Zero energy savings goals.
Figure 2 below presents the site‐level evaluation estimates of percent savings over the 2007 Oregon Structural Specialty Code (ASHRAE 90.1‐2007 for Site 2) broken down by savings component. The percent savings goal for each site is shown as a dotted red line. Ex ante and ex post savings are presented side‐by‐side to illustrate changes between the original pilot estimate and the evaluated estimate. It should be noted that the PTNZ pilot sought to achieve 50% savings over code with projects that utilized only energy efficiency measures and 60% savings over code for projects that utilized efficiency and renewable energy measures. With exception of Site 1, all of the projects evaluated in this study installed renewable generation, in the form of solar PV.
project sites
Figure 2: PTNZ Project Percent Savings over 2007 Oregon Structural Specialty Code

Overall Observations/Issues

The following observations and issues were encountered during the course of the evaluation:

PTNZ Program Issues/Observations

  • Data integrity issues were encountered at each site. These included the following:
    • Lack of historical trends
    • Historical trends set to record for too short a term (typically less than a week)
    • Large chronological gaps in data
    • Issues with intermediate data handling, especially for sites with web “dashboards” that import data from control systems
    • At all sites we visited, we found that the Monitoring and Reporting (M&R) system was either inactive or experienced data issues. Sites that indicated a high level of involvement in site energy monitoring demonstrated better savings realization rates. The one site with an inactive M&R system exhibited the lowest realization rate.
  • In addition to the energy impact analysis presented above, SBW surveyed site building management staff to improve understanding of their energy management and monitoring behaviors. The results of the survey are shown in the appendices provided with the internal version of this report.
Savings Related Issues/Observations
  • Electric measures are generally performing well, with a few exceptions performing slightly below expectations.
  • There appears to be much more variability amongst the gas measures. Some of these measures are performing well against code, while others are demonstrating significantly lower realization rates. Actual gas usage was found to be higher than expected.
  • Renewables are generally performing better than expected, and at some sites are offsetting under-performance by the electric efficiency measures.
  • Energy models were found to be well‐developed and of excellent quality. Although we did find modeling errors which impacted savings, the relative number of errors we found was small. We believe that the high level of technical review required for the models was a major reason for success in this area.

Recommendations

This section describes recommendations developed as a result of the issues encountered during the engineering review and impact analysis processes.

Data Integrity

Data plays a critical part in meeting the goals of any energy efficiency program. It is especially important for net zero programs. PTNZ program energy savings goals are far more aggressive than for conventional programs, and even a small increase in energy can significantly impact
whether or not these goals are met. Data allows building owners to quickly detect rises in energy, identify the causes, and ensure that applied solutions are working.

Data issues were discovered at each of the six evaluated sites. The following recommendations are meant to improve how the program handles both M&R data (which pertains to whole building and end use energy such as lighting, HVAC, etc.) and control system trend data (which
pertains to detailed system operation such as fan speeds, temperature setpoints, daylighting, control, etc.).

  • Currently, although the program requires periodic checks (sometimes quarterly) of M&R data throughout the 18 month reporting period, data quality issues were encountered with data stored during this period. We recommend improving the methodology used to check M&R data during this period. This could include more frequent checks to ensure that valid data is being stored, a closer examination of the reasonableness of the data, or comparison of accumulated M&R data with concurrent utility billing data. These activities could also be incorporated into the site commissioning plan in order to facilitate implementation.
  • We also recommend strongly encouraging customers to keep the M&R system active and functioning beyond the current 18 month requirement in order to facilitate the maintenance of energy savings. This could be encouraged by educating customers about the advantage in energy savings of maintaining the system. The same methodology used by Energy Trust to ensure data validity during the 18 month period (with the improvements noted above) could be adopted by the customer.
  • In general, sites should be encouraged to regularly check all historical trend system data (M&R and otherwise) to ensure continued integrity. Sites should also be encouraged to maintain historical control system trends of 15‐minute interval data for at least a year in order to capture building operation during all conditions. Sites should be educated as to why this is important, and the benefits of monitoring their historical trend systems regularly (as well as the negative impacts of not doing so).
  • M&R does not extend to natural gas. Due to underperformance of gas measures and data issues at many of the sites, consider offering an M&R option for gas, which would install gas meters on major equipment such as boilers and domestic hot water heaters, and would record gas usage at 15 minute intervals similar to how electric M&R is currently being accomplished.

Integrated Building Management System Practices

Discussions with site property management personnel and the survey given to site personnel demonstrated a wide range of building management practices and engagement levels that inherently affect the persistence of savings measures. At one site, the controls contractor and building management worked closely together to maintain system performance, while at another site, the building management was not fully aware of typical system controls, resulting in unnecessary summer boiler operation. Additionally, some features of these advanced buildings provide unusual challenges for maintenance staff in their day‐to‐day work, resulting in specialized vendors having to be called in to facilitate standard maintenance.
We recommend developing measures and resources that support building management in sustaining building performance, including the following:
  • Identify and support advanced system maintenance trainings specifically tailored to new technologies for building maintenance staff to ensure sufficient site expertise throughout the life of the measures installed. This is especially important for sites that experience building management changeover.
  • Encourage site staff to conduct periodic walkthroughs that check on key measures; this ensures that tenants continue to use best practices. Some of the projects have involved measures whose effectiveness relies on good tenant education and regular communication with building management staff.
  • Encourage site staff to collaborate with their controls contractors in actively managing building performance. This may include the controls contractor directly monitoring performance and reporting regularly to site staff, or setting up a system that alerts staff to abnormally high energy usage and other performance indicator abnormalities. Enlisting the help of controls contractors is important because in most cases they are more familiar with the complexities of building operation than the site staff.
  • Support and encourage building design teams to minimize building maintenance complexity and requirements. One important design feature would be maintenance accessibility to key systems. In one site in particular, maintenance of system performance was inhibited by a building design that made the system difficult to physically access, resulting in costly and time‐consuming maintenance that required specialized vendors.

Maintenance of Effective Practices

This recommendation is based on which practices evaluators identified as being the most effective during the PTNZ pilot. We recommend that these practices be maintained in the future to help ensure the success of the program going forward.
  • Technical reviews were found to be a thorough review of energy calculation methods, inputs, and assumptions. The reviews ensured that pre‐occupancy energy models and calculations were of good quality. The format and layout of the technical review memos presented information clearly and helped in isolating specific issues and responses.

Energy Modeling Methodology

Energy modeling methodologies employed by the pilot were found to be sound. The one area for potential improvement relates to calculation of both project level and measure level savings. The pilot used a rolling baseline method to determine measure level savings, in which measures are added one at a time to the baseline model, and savings are calculated in steps as each measure is added.

This could be improved upon by using the full interactive model (all measures included) to calculate both total project and individual measure savings. Savings by measure can be  accomplished with the interactive model using a “last in” approach in which the measure of interest is removed from the full interactive model to create the baseline condition for the measure. This ensures that interaction with all other measures is accounted for. Note that this method could potentially impact measure cost‐effectiveness.