M. Theresa Breining

The Phillips ROI Methodology™ utilizes five levels of evaluation, which are essential in determining the return on investment.

At Level 1 - Reaction and Planned Action, attendee and stakeholder satisfaction from the meeting can be measured. Almost all organizations evaluate at Level 1, usually with a generic, end-of-meeting questionnaire. While this level of evaluation is important as a “stakeholder” satisfaction measure, a favorable reaction does not ensure that attendees have acquired new skills, knowledge, opinions or attitudes from the meeting.

At Level 2 - Learning, measurements focus on what participants learned during the meeting using tests, skill practices, role-plays, simulations, group evaluations, and other assessment tools. A learning check is helpful to ensure that attendees have absorbed the meeting material or messages and know how to use or apply it properly. It is also important at this level to determine the quantity and quality of new professional contacts acquired and whether existing professional contacts were strengthened due to the meeting. However, a positive measure at this level is no guarantee that what was learned or whether the professional contacts acquired will be used on the job.

At Level 3 - Job Applications, a variety of follow-up methods can be used to determine if attendees applied on the job what they learned or acquired at the meeting. The frequency and use of skills are important measures at Level 3. While Level 3 evaluations are important to gauge the success of the meeting, it still does not guarantee that there will be a positive business impact in the organization or for the attendee.

At Level 4 - Business Results, the measurement focuses on the actual business results achieved by meeting participants as they successfully apply the meeting material or messages. Typical Level 4 measures include output, sales, quality, costs, time and customer satisfaction. Although the meeting may produce a measurable business impact, there is still a concern that the meeting may cost too much.

At Level 5 - Return on Investment, this ultimate level of measurement compares the monetary benefits from the meeting with the fully-loaded meeting costs as expressed in the ROI formula.

All levels of evaluation must be conducted in order to determine the ROI of a meeting or event. The data collected should show a chain of impact occurring through the levels as the skills and knowledge learned (Level 2) are applied on the job (Level 3) to produce business results (Level 4).