Knowledge Infusion Center of Excellence: Consultant's Corner : Tags : measurement

In speaking with a lot of learning and development professionals lately, I hear that a lot of them are not concerned with 'butts-in-seats' for classroom training or 'eyeballs' or completions for online courses. Rather they are concerned with more strategic measures such as knowledge transfer and ROI. Some of them have even relegated attendance to a lowly Level 0 on Kirkpatrick's famous 4-level impact scale, which starts at level 1, stating that such reports are useless.

I think this view is a little extreme. In fact, I think that while 'butts-in-seats' as a measure is transactional, it is foundational in that it can lend strategic insights into the value of learning investments and lead to better decisions on how to allocate those investments going forward. For instance:

It matters to a Chief Compliance Officer that every manager took part in the sexual harassment training. Without any needed proof of knowledge transfer or understanding in any state of which I am aware (sad), attendance is all you really have.
Several senior business leaders with whom I've spoken want to know that their potential successors are taking part in the accelerated leadership development programs they were involved in designing for the good of the long-term health of the company.
The head of product marketing wants to know that all salespeople and channel partners have sat through training on the new black-box, which is now available for sale 4 weeks ahead of the company's nearest competitors' black-box.
The head of customer training is concerned about butts-in-seats because its directly tied to the top line of her P&L. She is also interested in the average price charged per learner.
The CLO cares because he wants to know what courses are in high-demand and which ones should be put out to pasture.

While I don't subscribe to obsessing over transactional measures like attendance or training departmental measures such as how much you spend on catering, I do feel that you do need spend some time tracking them in order to tie training to business value via quality and efficiency. I'd love to hear how you're approaching your T&D measurement strategy.

1 Comments Permalink Tags: kirkpatrick, measurement, analytics, learning_management, intelligence, scorecard

This entry is a build on a great post from Jason Averbook today on the Knowledge Infuser, which asserts, "Talent Management has specific industry characteristics that MUST be taken into account when addressing all aspects of people, process and technology." It stands to reason that an organization engaged in one kind of commercial enterprise - say, health insurance - would have different talent demands than another - say, fashion retail. Furthermore, while the ultimate success of any company is measured by its ability to deliver long-term value to shareholders, the metrics to gauge this success are certainly not identical. After all, does a health insurer care about same-store sales?

If you take a popular measurement framework such as The Balanced Scorecard, there is perhaps no category of measurement that differs from one industry to the next than the one that focuses on people. Some refer to this category as ‘learning and growth;' others call it, ‘human capital.' The uniqueness driven by industry is driven by distinctive operations and differences in what roles are critical in driving those operations. As Mike Shoemaker pointed out in a response to Heidi Spirgi's post last month, "measurement in talent management...is clearly critical, provided the measures are tied to the industry and functional context, as well as the strategic objectives of the company."

Furthermore, measurement is essential to diagnosing business issues, which HR executives and their business partners should use to drive the creation and revision of talent management strategies and tactics. Take the following:

Hiring salespeople with consulting backgrounds to combat the commoditization of your retail business.
Developing employee relations training programs for first-time managers who your biotech company is counting on to retain talent and scale its business now that a drug in your pipeline has been approved for commercialization.

To help you think through how you should measure your people and the investments your organization makes in them (e.g., pay and benefits, training, incentives, attracting candidates outside the organization, etc.), try using the following framework. What you will find as you work down this list is that it is progressive in several ways: It becomes more specific to industry; the type of metrics becomes more strategic to the organization; and the type of metrics becomes more difficult to measure. In addition to the measures becoming more industry-specific, it is important to point out that the benchmarks (e.g., voluntary turnover rate) established using these measures often differs.

Transactional: These measures quantify the activity in various HR functions. When taken in isolation, none of these figures is very strategic. However, many of them act as building blocks (i.e., variables used to calculate) for HR Operational, Workforce Management and Workforce Effectiveness Metrics.
HR Operational: These are meant to monitor and continually improve HR functions' performance in terms of process efficiencies and outcomes and are meant to better focus efforts and resources. Examples include time-to-hire, time to submit performance reviews and # of courses per instructor.
Workforce Management Metrics: These metrics demonstrate the soundness of the organizational policies, procedures, and programs administered by HR as well as the competence of business managers and leaders in applying them. An example would include voluntary turnover within 90 days, which may reflect opportunities to improve hiring decision-making, development of new managers, and/or on-boarding improvements.
Talent Intelligence: These metrics are products of statistical analysis meant to demonstrate the relationships between the three categories above and KPIs outside HR. As such, they yield the intelligence to support strategic decision making - by showing what has happened and modeling scenarios of what the future might bring. Examples include the total cost of new-hire turnover and the effect of product knowledge training on sales productivity.

This was a mouthful, even for those who may be focused on Talent Management strategy at the moment. Any and all questions/comments are welcome.