Who hired the EM&V team and why?
The Public Utility Commission of Texas (PUCT) hired the EM&V contractor through a Request for Proposals.
In 2011, the Texas Legislature enacted SB 1125, which required the PUCT to develop an EM&V framework that promotes effective program design and consistent and streamlined reporting. The EM&V framework is embodied as P.U.C. SUBST. R. 25.181, relating to Energy Efficiency Goal (Project No. 39674).
The PUCT is committed to a thorough and independent evaluation of utility energy efficiency and load management programs to document load impacts and their compliance with regulatory and statutory objectives, understand why those effects occurred, and identify ways to improve current portfolios.
Who is doing the EM&V?
The PUCT selected Tetra Tech to perform the third-party evaluation, measurement and verification of Texas utilities’ energy efficiency and load management portfolios.
The Tetra Tech team also includes a subcontractor, Texas Energy Engineering Services, Inc. (TEESI).
What is EM&V?
Evaluation, Measurement and Verification (EM&V) is a catch-all phrase used in the energy industry to refer to a systematic, third-party review of the delivery and impacts of DSM programs.
What are confidence intervals and how are they used in the EM&V effort?
A confidence interval is a range of values that describes the uncertainty surrounding an estimate. Confidence intervals are one way to represent how “good” an estimate is; the larger a confidence interval for a particular estimate, the more caution is required when using the point estimate.
Energy efficiency program evaluations routinely employ 90% confidence intervals with +/-10 percent as the industry standard. The “90%” in the confidence interval represents a level of certainty about our estimate. If we were to repeatedly obtain new estimates using exactly the same procedure (by drawing a new sample, conducting new interviews, calculating new estimates and new confidence intervals), the confidence intervals would contain the average of all the estimates 90% of the time.
The EM&V results will report confidence intervals by its endpoints to show the uncertainty of the estimates.
How rigorous is the EM&V?
The EM&V activities are designed to result in a minimum confidence interval of 90 percent +/-10 percent (90/10) at the utility portfolio level. A precision of 90/10 is the industry standard as discussed above.
A critical first step in the EM&V process is to prioritize evaluation efforts and the associated level of rigor. The prioritization process is important in allocating resources required to meet evaluation needs while ensuring cost-effective use of evaluation funds. There are a number of components that feed into evaluation prioritization decisions, which include:
Magnitude of savings—percentage of contribution to the portfolio of programs’ impacts
Level of relative uncertainty in estimated savings
Level and quality of existing quality assurance and verification data from on-site inspections completed by utilities or their contractors
Stage of program or programmatic component (e.g., pilot, early implementation, mature)
Importance to future portfolio performance
PUCT and Texas utilities’ priorities.
The EM&V team provides a due-diligence review of impacts across each program in every utility’s portfolio. This broad reach review of all programs is then supplemented with more rigorous evaluation methods if deemed a higher priority during the prioritization process.
How are the EM&V results applied?
Most commonly EM&V results are applied prospectively to improve the accuracy of reported impacts and program designs and processes. However, the EM&V team may recommend utilities adjust claimed savings based on the evaluation findings if a calculation error or other major factor that affects savings is found.
What program years will be evaluated?
To-date, the PUCT has conducted annual EM&V since the 2012 program year.
What are the objectives of the EM&V effort?
The objectives of the EM&V effort are to:
Document gross and net energy and demand impacts of utilities’ individual energy efficiency and load management portfolios
Determine program cost-effectiveness
Prepare and maintain a statewide Technical Reference Manual (TRM)
Provide feedback to the PUCT, utilities, and other stakeholders on program portfolio performance.
What does the EM&V team need from utilities and implementation contractors?
The EM&V team meets periodically with utilities and implementation contractors to understand the programs’ design and delivery, data capture and tracking processes and savings calculations. The EM&V team has formal data requests for information to support claimed savings calculations. The EM&V team requests reviews of draft data collection instruments and other EM&V deliverables and to have periodic meetings and discussions throughout the evaluation process.
How do program data transfers occur? Are transfers be secure and data remain confidential?
Tetra Tech maintains a secure FTP site for transferring program tracking data and supporting documentation. The site is accessible only to Tetra Tech team members, relevant utilities, contractors, and PUCT staff. Data is deleted from the site immediately after the transfer is complete.
What is the EM&V team doing with all of the data they receive?
Tetra Tech has created a statewide EM&V database that allows the primary requirements of the evaluation to be achieved. The EM&V database is securely stored with restricted access.
Does Tetra Tech contact customers directly?
Yes, Tetra Tech may conduct telephone surveys and on-site inspections with a sample of customers from each utility. Utilities have the opportunity to review all data collection instruments and will be informed of data collection field periods.