Lessons Learned After 30 Years of Process Evaluation

This paper documents lessons learned from 30 years of process evaluations of energy efficiency programs. The lessons were collected from interviews with 18 evaluators, most with two decades of experience. Findings related to program design and implementation include lessons about technologies, lessons about motivating with rewards and incentives, and lessons about implementers. Findings related to program administration include lessons learned about databases and lessons about the program process.

Summary of Lessons Learned - Program Design and Implementation

• Lesson: Conduct research on new technologies to gauge customer response and identify problems with the technology prior to large-scale introductions. Collaborate with manufacturers to fix problems.
• Lesson: Rewards work - but they don't always work as expected. Metrics for programs, whether for third parties or utilities, should be carefully designed. Look behind the explicit reward to see what is really being rewarded.
• Lesson: Program managers and organizations with a commitment to adapt and grow a program can make otherwise lackluster efforts work.
• Lesson: Programs are more effective in targeting their services when they have access to consumption data.

Summary of Lessons Learned - Program Administration

• Lesson: There is no easy answer to databases. They are needed, they are costly, and the first one that is built will likely need to be rebuilt as the program evolves. Effective database creation requires skilled people who understand the program and its regulatory and evaluation requirements.
• Lesson: Simplify, simplify! Complex application processes and requirements for detailed inforamtion can lead to non-participation and missed opportunities.