The Public Utility Commission of Texas (PUCT) hired the EM&V contractor through a Request for Proposals.
In 2011, the Texas Legislature enacted SB 1125, which required the PUCT to develop an EM&V framework that promotes effective program design and consistent and streamlined reporting. The EM&V framework is embodied as P.U.C. SUBST. R. 25.181, relating to Energy Efficiency Goal (Project No. 39674).
The PUCT is committed to a thorough and independent evaluation of utility energy efficiency and load management programs to document load impacts and their compliance with regulatory and statutory objectives, understand why those effects occurred, and identify ways to improve current portfolios.
The PUCT selected Tetra Tech to perform the third-party evaluation, measurement and verification of Texas utilities’ energy efficiency and load management portfolios.
The Tetra Tech team also includes a subcontractor, Texas Energy Engineering Services, Inc. (TEESI).
Evaluation, Measurement and Verification (EM&V) is a catch-all phrase used in the energy industry to refer to a systematic, third-party review of the delivery and impacts of DSM programs.
A confidence interval is a range of values that describes the uncertainty surrounding an estimate. Confidence intervals are one way to represent how “good” an estimate is; the larger a confidence interval for a particular estimate, the more caution is required when using the point estimate.
Energy efficiency program evaluations routinely employ 90% confidence intervals with +/-10 percent as the industry standard. The “90%” in the confidence interval represents a level of certainty about our estimate. If we were to repeatedly obtain new estimates using exactly the same procedure (by drawing a new sample, conducting new interviews, calculating new estimates and new confidence intervals), the confidence intervals would contain the average of all the estimates 90% of the time.
The EM&V results will report confidence intervals by its endpoints to show the uncertainty of the estimates.
The EM&V activities are designed to result in a minimum confidence interval of 90 percent +/-10 percent (90/10) at the utility portfolio level. A precision of 90/10 is the industry standard as discussed above.
A critical first step in the EM&V process is to prioritize evaluation efforts and the associated level of rigor. The prioritization process is important in allocating resources required to meet evaluation needs while ensuring cost-effective use of evaluation funds. There are a number of components that feed into evaluation prioritization decisions, which include:
Magnitude of savings—percentage of contribution to the portfolio of programs’ impacts
Level of relative uncertainty in estimated savings
Level and quality of existing quality assurance and verification data from on-site inspections completed by utilities or their contractors
Stage of program or programmatic component (e.g., pilot, early implementation, mature)
Importance to future portfolio performance
PUCT and Texas utilities’ priorities.
The EM&V team provides a due-diligence review of impacts across each program in every utility’s portfolio. This broad reach review of all programs is then supplemented with more rigorous evaluation methods if deemed a higher priority during the prioritization process.
Most commonly EM&V results are applied prospectively to improve the accuracy of reported impacts and program designs and processes. However, the EM&V team may recommend utilities adjust claimed savings based on the evaluation findings if a calculation error or other major factor that affects savings is found.
To-date, the PUCT has conducted annual EM&V since the 2012 program year.
The objectives of the EM&V effort are to:
Document gross and net energy and demand impacts of utilities’ individual energy efficiency and load management portfolios
Determine program cost-effectiveness
Prepare and maintain a statewide Technical Reference Manual (TRM)
Provide feedback to the PUCT, utilities, and other stakeholders on program portfolio performance.
The EM&V team meets periodically with utilities and implementation contractors to understand the programs’ design and delivery, data capture and tracking processes and savings calculations. The EM&V team has formal data requests for information to support claimed savings calculations. The EM&V team requests reviews of draft data collection instruments and other EM&V deliverables and to have periodic meetings and discussions throughout the evaluation process.
Tetra Tech maintains a secure FTP site for transferring program tracking data and supporting documentation. The site is accessible only to Tetra Tech team members, relevant utilities, contractors, and PUCT staff. Data is deleted from the site immediately after the transfer is complete.
Tetra Tech has created a statewide EM&V database that allows the primary requirements of the evaluation to be achieved. The EM&V database is securely stored with restricted access.
Yes, Tetra Tech may conduct telephone surveys and on-site inspections with a sample of customers from each utility. Utilities have the opportunity to review all data collection instruments and will be informed of data collection field periods.