White Paper

Some Performance Improvement Learning ROI Principles and Ideas

All business and government Performance Improvement (PI) program leaders must understand and manage the ROI “input-output” modeling process for the application of all resources and assets. It’s all about resources allocation among competing programs and related resources needs. This is true for PIsupporting learning programs as well, even if top management intrinsically understands that improving human capital assets’ skills excellence is a baseline requirement to continuously improve, be competitive, to grow and to be successful.

For personal career growth the quality of the ROI preparation process will often be a gauge for career potential for the process improvement leader. Top management is continually evaluating potential.

Absolute and relative Return On Investment (ROI) is the key resourcing criteria used by top management (especially the CFO), so ROI must be understood and managed. ROI includes:

• Payback Period (the time in months before an investment’s gains pay back the investment)
• Net Present Value of gains (future gains "discounted" to today’s value per a "hurdle rate" percent)
• IRR (Internal Rate of Return) (the net gains percent return on initial investments)

Several dimensions exist, and where the "art" of ROI forecasting resides:

1. The metrics type(s) to be used.
2. Before vs. after results (present metrics and goals should be set first, to compare against).
3. Study group vs. Control group comparative results, if possible.
4. The calculated value of the gains over a defined time frame, and the clarity of the causative relationship (e.g., “R2”). This includes the cause-effect timings (e.g., leading, coincidental, lagging).
5. The costs over a defined period (direct and opportunity costs, if the latter can be reasonably calculated).
6. Intangible, "off the balance sheet" gains (that might be the most important even if hard to quantify).
7. The resulting net gain benefits value vs. the costs.
8. The internal NPV/IRR "hurdle rate" the company uses to gauge a program’s success.

The #4 correlation/causation element is very important, and the one that proves the most confounding (e.g., some actions are leading factors, coincidental, or lagging). Luckily, many bivariate or multi-variate online statistical type-tools exist to explore the inputs-to-outputs and timing relationships to define R (correlation) or R2 (causation) relationships as they may (or may not) exist; including for timing relationships.

Carefully eliminating “noise” factors (surrounding forces not directly involved), autocorrelations and other distorting factors is important to the clarity and veracity of the forecasting relationships being established.

The metric type(s) to be used are important, and many possible ones exist. The metrics measurement bases can be: absolute, trends/slopes, before vs. after, actual vs. goals, training group vs. control group results, etc.; and/or a combination. Also, HR, ERP, Accounting, CRM, WFM and other central systems will need to be accessed to provide the business numerical data to track (and calculate) returns vs. costs.

Each Process Improvement project will have its own ROI elements. For Learning and for Knowledge programs sub-elements, here are just a few metrics types from which to choose:

Learning costs efficacy:

• Cost/student, including travel costs
• Training $/employees
• Training $/employees $
• Training costs/revenues $
• Learning time
• Learning/paid time
• Voluntary efforts/ vs. required efforts
• The value of saved “opportunity costs” due to blended vs. live-only learning.

1

Knowledge programs effectiveness:

• Pre and post tests results
• Regular retention Mastery tests
• Training group results vs. control group results comparisons
• Surveys
  • Learners
  • Managers
  • Peers
  • Subordinates
  • Customers
• Orientation time and costs impacts
• Proficiency rates (specific measurements needed, such as assembly time, error rates, sales results)
• Work process improvements, such as in the Productivity section below.

Productivity improvement (many numerical examples can be used, and will vary by department and/or program):

• Relative speed of the below results being realized vs. the past (“time value of money”)
• Sales Revenues (growth timing, growth slope)
• Gross Margin
• Operating Expenses
• EBITDA
• Cash flow
• Production units
• Higher inventory turns; less carrying costs
• Errors reduction; Zero-defect rates
• Scrap rates
• Proficiency scores (can include 360 assessment-type scores)
• Certifications rates
• Re-certifications
• Skills levels, or skills gaps
• Time to productivity
• Slope of productivity improvement
• Escalations improvement
• Time-to-resolve
• Results per time frame
• Results/costs ratio
• Results/other resources used ratio
• Promotion rates
• Demotion rates
• Terminations
• Retention rates
• Turnover rates
• Performance/Remediation volume
• Quality ratings
• Compliance percent
• Violations, citations
• Legal costs (losses, reserves, fees)
• Litigation levels and costs
• Employee satisfaction ratings
• Management satisfaction ratings
• Emergency response times
• Emergency response quality
• Many others are possible from which to choose.

2

White Paper

Pre-project ROI calculations are forecasts/estimates only, are forward-looking, and “what if” in nature. In such instances several levels of What If results can be presented and a Most Likely, Bad Case and Good Case identified. They need to stand the relevancy, accuracy, logic and timing tests.

“Hard” results are mandatory to identify, and it is advisable to also identify “Soft” benefits that are hard to measure (and should not be for credibility usually), but are valued none-the-less.

In Advertising, the common term DAGMAR (Define Advertising Goals, Measure Advertising Results) provides a parallel framework to be used. The goals need to be pre-agreed to by all relevant parties. Inaddition, how results are to be measured needs to be pre-agreed to and resourced. This includes top management enabling data access from central systems if needed.

Summary
ROI modeling is critical to project or program approval and a reflection of leadership skills. Many possible elements are involved, and many tools can be used. The needed “input-output” model components will vary by project and part of the “art” is choosing the optimal components to use. Multiple What If options should be provided, and Good, Bad, Most Likely projections provided. Goals should be established beforehand, and actual results measured against goals.

About The Quality Group (http://www.thequalitygroup.net/)

For over a decade, and for hundreds of thousands of learners in corporations and higher education, with over 1,000,000 modules completed, The Quality Group (TQG) has been producing innovative “Performance Improvement” e-Learning solutions and blended learning that improve lives, organizations, and communities. All TQG e-Learning is powered by the OpusWorks®, a proprietary Web 2.0 e-Learning platform that streamlines e-Learning creation, customization, development, and delivery. LSS blended learning, Project Management, Change Management and other courses exist. These can be used of all aspects of “xBelt” training, retention training, job aids, project management, change management and ISO 9001 preparation.

TQG’s Advisory Services Group embodies all the needed skills and experiences to assist in the Business Process Improvement (BPI) planning, delivery, evaluation and ROI calculation stages. Some Excel models are available if helpful. Just contact TQG.

The Quality Group also provides advisory services for all aspects of learning strategy, ROI analyses, learning designs, learning content, MBB-level training services, and more.

3

TQG Solutions URLs:
Online Demo

• Key Success Factors
• Differentiations and Unique Value
• Products Library
• OpusWorks®
• Healthcare Lean Six Sigma
• Government

4