Training: Designing for Lasting Change

The five cases studies were for courses offered in 2003 by four investor owned utilities in California. Each Center independently identified the courses to include in the case study analysis and the associated issues. The selected case study courses addressed a range of targeted audiences, from small business owners or managers, to energy trades and professionals such as architects and engineers. All focused on increasing attendance. For some of the courses, the Centers found that they were not filling the seats available for the course. Other Centers thought that demand for the course would be higher and were hoping to offer more workshops on the topic. One Center wanted more market segments represented in the workshops. The overall concern expressed by the Centers, however, was increasing the “butts in chairs (BIC),” and not on results from the courses.

In April 2006, we provided a Technical Trainers Toolbox training to 12 staff from the California utility Energy Centers. At the beginning of the two day class, we asked the participants to write down one thing that they hoped to get from the class to consider it a success. Three of the participants put “increasing attendance” or “butts in chairs” as the single most important item. In discussions with these and other staff at California energy centers, we learned that the Centers’ goals, as approved by the California Public Utilities Commission, are based upon the number of attendees at the various centers. Regulators use attendance as a performance measure because it is objective, and easy to track.

In late 2005, we interviewed the Center Directors again, discussing their funding levels goals, and overall objectives. During the interviews in 2003, one Center Director had discussed the wish to demonstrate energy savings from his Center’s offerings. In 2005, two additional Directors said that they felt an increasing need to demonstrate energy savings. They saw this need not as one to meet the requirements of the CPUC for the statewide ETS program, but instead, in anticipation of greater competition for other sources of funding for their Centers. Most of the Centers obtain funding from multiple sources, of which the Public Goods Charge for ETS funding is only one. They perceive increased competition for the other funding sources and believe that if they are able to demonstrate energy savings from some of their courses, they will be better able to continue obtaining these alternative funds.

The Centers have been able to meet their goals for attendance levels. Some of the Centers, however, are increasingly interested in changing attendee behavior in ways that produce demonstrable energy savings. This objective is consistent with a need to implement Best Practices in adult education. The centers are well positioned to achieve the goal of demonstrating savings from some of their course offerings.

A key focus of Best Practices is identifying approaches that lead to high learner retention and change in attitudes and behavior. This approach is “learner focused” and identifies teaching techniques that engage the learner so that action is more likely to result from the learning. “A frequent complaint about training programs is ‘poor retention.’ And this will always be the case if we do not allow time for a [practice or integration phase.]” (Meier 2000, 92). . .”learner-centered and performance-based efforts result in training and transformation.” Stolovitch and Keeps 2002, 14).

This is an essential construct for the energy efficiency educators to embrace in moving forward. Courses will be more attractive to potential attendees if they are explicitly designed to motivate changes in behavior. Behavioral changes can be as simple as exploring energy efficiency opportunities or as sophisticated as changing HVAC installation practices.

We discuss the Best Practices in adult education in three main areas. The first is characterized as marketing. This involves knowing who the desired audience is and attracting them to the course. The second is design and implementation. Once you have identified the course composition and the desired changes, the instructional design can focus on best meeting these objectives through learner activities that allow for practice of the desired outcome. Finally, Best Practice involves evaluating the course on multiple levels (Kirkpatrick 1994) over time to both test for results (behavioral change) and identify necessary changes to achieve these results.

We found that the courses offered through the various Energy Centers rarely had clearly stated learner objectives – that is a desired course outcome that was included in the course description or marketing materials. Through interviews with Center staff we determined, however, that the courses offered do have underlying objectives beyond obtaining attendance. These objectives are described in Table 1 below, with behavioral changes identified by the paper’s authors.

So, what are the Best Practices to achieve the Center objectives? We discuss these below under three areas; marketing, instructional design and evaluation.

Energy centers are clearly reaching a market that is interested enough to attend the classes. Evaluations have shown that many of the classes are attended by people who have attended other classes at the Center and that marketing efforts are not designed to draw substantial numbers of new attendees. It is not clear, however, that the attendees at the courses have similar learning objectives as the Centers. The Centers need to expand their reach to draw more attendance from persons and firms that have not previously partaken in center activities and that include more learners interested in achieving goals consistent with the Center’s goals. (The actual goal may not be same, but the behavioral change may be. For example, an end-user may want to lower business operating costs and the Center may want to increase the adoption of higher SEER air conditioning.) Applying Best Practices in marketing would increase attendance at the Centers, and perhaps more importantly, attract attendees who are more likely to take action as a result of what they have learned.

We identified the following Best Practices in marketing that would help achieve these objectives.

Provide marketing messages that emphasize to potential attendees direct benefits to them from attending courses. Effective marketing must convey the specific value of the course to the target market. The message should emphasize the specific benefits that attendees will come away with. While the goal of energy efficiency programs may be to generate energy savings, it must be recognized that that message is not always of greatest value to potential attendees. Centers will need to tailor different messages depending upon the course content and the audience. For example, a course that certifies contractors to participate in an energy-efficiency program could lead with “Stay ahead of the competition . . .,” mention “business profit and growth”, and talk about helping customers solve energy-related problems and increasing comfort and safety.

Include in the course description the specific outcomes from the course. Course descriptions should include the “learner objectives,” a list of the skills or tools that the learner will obtain by attending this course. Malcolm Knowles, the father of adult learning theory, identified as one of his adult learning principles “Adults have a need to know why they should learn something before investing time in a learning event. Trainers must ensure that the learners know the purpose for training as early as possible.” (Biech 2005, 26) For example, a description of a lighting course for designers might include the following statement. “Attendees will be able to specify energy efficient lighting for a variety of commercial settings, with particular emphasis on retail clothing and office space.”

Increase the reach of course-specific marketing efforts by partnering with leading trade groups. There are excellent examples of centers using local and regional trade associations for more effective target marketing; however, we found other examples where the centers did not take enough responsibility for maintaining the relationship and addressing the trade associations’ needs. Courses can also be marketed at professional conferences and trade shows. To be successful with trade groups and professional organizations, the Centers must work regularly with these groups and demonstrate to the organization the value energy education brings to the organization and its members.

Identify and obtain certification to offer additional educational credits. Adult learners have limited time in which to attend courses during the work week. Some are required to obtain, and are therefore motivated to attend courses that offer, continuing education. This gets “butts in chairs” and the instructor must then provide the motivation for learning and behavioral change.

Target e-mail marketing to only those parties likely to be interested in the course. The California Energy Centers have made great use of email lists (of former attendees) to market courses. Refining these lists so that recipients receive only course offerings that are targeted toward their interests or profession will increase the likelihood that the emails are read.

Make course content specific to one (or possibly two) professions (or target audiences). The Best Practice principles suggest that successful marketing starts with development of courses geared to the needs of a specific audience and marketed directly to them. “Adults are willing to devote energy to learning those things that they believe help them perform a task or solve a problem. Trainers who determine needs and interests and develop content in response to these needs are most helpful to adult learners.” (Biech 2006, 27) One case study of poorly attended lighting courses illustrated this principle. The courses were designed and marketed for a broad range of clients, ranging from lighting designers to architects and engineers. The educational needs of these professions are different enough that a single course targeted to the three groups proved unattractive to all, as they perceived the courses as covering material not relevant to their specific needs.

Bring some courses physically closer to the customers. Some of the Centers offer courses through mobile units, at trade shows or conferences, or at remote locations. This is essential for reaching some customers who are located too far from Energy Centers to make the trip for a half or single day workshop.
Charge for courses to demonstrate that they have value. Courses offered for free are perceived as having less value than those for which there is a charge. Several Centers had courses for which registration was high, but attendance low. Participants (and their employers) will have higher expectations for courses with fees and will be more selective regarding those classes that they attend.

The Best Practice examination demonstrated a series of principles in adult learning that should be a part of every energy efficiency course. These principles begin with a need to ensure that the focus of the classes moves beyond one that is only a transfer of information from knowledgeable instructor to attendees, to one that empowers attendees to take specific actions. The lecture type of instruction is not very effective in communicating information that will be retained by attendees. More importantly, this approach is less likely to lead to the desired actions being taken. One of the trainers pointed out the “Cone of Experience and Learning” by Edgar Dale, shown below, which correlates remembering with level of activity. As shown in Figure 1, passive activities result in low levels of retention.

Best Practices in design an implementation incorporate learning activities that are likely to increase learning, retention and results. The activities are targeted to the specific objectives of the course and provide opportunities for learners to practice the desired behavior. We discuss key components of course design and implementation below. Best Practices garnered through the case studies, expert interviews and secondary research, include:

Focus courses on obtaining actions not just transmitting knowledge. This means identifying the desired actions before developing the course content and materials. Describe the outcomes in the marketing materials.

Design the course materials to meet the objectives outlined in the course description. The course materials and activities should be designed specifically to meet the learner objectives. All content and class activities should be focused on providing (or reinforcing) the knowledge and skills to the learners to result in the desired behavioral change outcomes.

Structure course content so that it is practical and applicable to the participants in their jobs. Adult learners are less interested in obtaining pure knowledge than in obtaining skills and tools that they can use in their job. “Adults have a strong readiness to learn those things that help them cope with daily life effectively. Training that relates directly to situations adults face is viewed as relevant.” (Biech 2006, 26) Employers are more likely to send staff to trainings that result in positive changes in how employees do their jobs.

Limit course (or section) content to teaching of three major objectives. Do not overwhelm attendees with too much information at once. Adults can only process so much before all the material, including the objectives is lost. This caution also speaks to the need for frequent changes in approach. If lectures are needed, they should be short and followed by an active activity that reinforces the message of the lecture portion. Many courses include a lot of content, but an increase in course content does not translate into an increase in the amount of information that is learned. Classes can result in more behavioral change if the content is limited, but the material is learned and retained.

Provide opportunities for attendees to participate and exchange ideas. Adult learners bring experience and knowledge to the classroom. Learning is increased if participants recognize their own expertise, build upon it, and share it with others in the classroom.

Structure courses so that they engage the attendees in active participation in order to retain information conveyed. Several of the experts pointed out the need to move from passive learning to actively engaging the students in the learning. As one respondent pointed out, “Lecturing is easy, but the instructor needs to engage the student in the learning.” “(The practice or integration ) phase of the learning cycle can account for 70% o(or more) of a total learning experience. It’s in this phase that the learning actually takes place.” (Meier 2000, 91) Engineers may prefer problem-solving activities that require a calculator. Other student groups may be more extroverted and prefer interactive activities. Courses should provide attendees with opportunities to actively engage in the learning, not only by sharing what they already know, but by practicing what is being taught.

Build in opportunities for post-training reinforcement. Learning is achieved when lessons are reinforced over time. “…”even though something may be well-learned, if it is not used it can be forgotten. Consequently, once something has been learned, we need to increase the probability of its retention by changing the practice schedule from ‘massed’ to ‘distributed’ practice. Distributed practice means that material is periodically reviewed but with longer and longer time intervals between reviews. Distributed practice makes for very durable learning that is ‘forgetting resistant.’” (Hunter 1994, 102) The classroom is the massed practice, reviewing or using class lessons in the workplace is distributed. Some instructors incorporate student action plans (as discussed above) as one method to increase the likelihood of practice. Follow-up emails, and telephone calls can also facilitate this learning.
Structure each workshop so that each attendee leaves with an action plan developed by that attendee. Learners are more likely to change their behavior if they have a plan for how to implement what they have learned. Developing an action plan allows the learner to apply what they learned in the class to their job and determine what they need to do to make it happen. An additional benefit is follow-up evaluations can address the learners action plan to determine what changes resulted from the class and identify the barriers to implementing the plan.

Best Practice includes a heavy reliance on evaluation as an integral part of any course. Evaluation should be used to determine if actions are being taken as a result of course attendance. Kirkpatrick identifies four levels of evaluation that progress from participant reaction to results. (Kirkpatrick 1994).

The first level addresses how the participant liked the experience. A more positive experience is likely to result in more learning and the evaluation demonstrates to the learner that the instructor cares about their reactions. It can be used to modify courses for improvement.

Level 2 evaluation focuses on the learning. This is often incorporated into the classroom activities, in which the learner exhibits their new knowledge by demonstrating new skills or answering questions.

Level 3 addresses behavior change when the learner returns to the work-site. It is used to determine if the course skills are being transferred to the workplace. There are challenges associated with when and how to determine this and it is rarely performed.

Level 4 evaluation focuses on results. In the energy industry we would call this an impact evaluation to measure energy savings, or an evaluation to identify difference in how an organization makes decisions, purchases or sells equipment. Again, this is rarely done.
Information from each prior level serves as a base for the next level’s evaluation. Thus, each successive level represents a more precise measure of the effectiveness of the training program, but at the same time requires a more rigorous and time-consuming analysis.

The Best Practices for evaluation include:

Design evaluations to collect more than just satisfaction information. Evaluations should include marketing source, instruction quality, issues with course content, issue with setting, and helpfulness in moving to actions.

Conduct evaluations early and often, especially for courses that span several days. Waiting until the end of a course does not help current attendees. Conduct follow-up evaluations for key courses several months after the workshop to determine changes in behavior.
Design evaluations to address the specific objectives of the course. The evaluation should be designed to assess the desired changes in behavior the course was designed to address. Check to see how well the learner felt the courses specific objectives were met and what could be improved to better meet these objectives.

Act on evaluation findings. Evaluations can reveal weaknesses in course content and delivery. They are only useful, however, if the findings are communicated to those who can make changes, and if the recipients of the information are motivated to make changes. Course instructors, even those from outside the provider’s staff, should be informed of both the positive and negative evaluation results, and assisted in refining course materials to better meet objectives. Best Practices include comparing evaluation results both over time (for the same class and instructor) and across classes.