What do participants think and feel about the training

Forum #1

The four level evaluation model introduced in 1959 by Donald Kirkpatrick has served as a valuable tool in the evaluation process for training programs. It allows practitioners and researchers alike to focus their efforts by classifying evaluations into reactions, learning, behavior, and results. It has however endured some scrutiny concerning the terminology it utilizes and its shortcomings. Holton (1996b) argues that it is more appropriately identified as a taxonomy rather than a model because it classifies evaluation rather than serving the true purpose of a model which he identifies as “specifying outcomes correctly, accounting for the effects of intervening variables that affect outcomes, and indicating causal relationships” (p. 5).

Interestingly, in the issue of Human Resource Development Quarterly in which Holten’s article was published, a response from Kirkpatrick was published, followed by a final response from Holten . Kirkpatrick (1996), in the face of criticism for his choice of terminology, maintains the perspective that his “framework” as he calls it, still is useful to practitioners and that the argument over whether it should be called a taxonomy rather than a model is beside the point. Holten makes several interesting points in his response that I believe characterize the nature of solid HRD research. He gives credit for Kirkpatrick’s important contributions but notes how wide use does not indicate correctness (Holten, 1996a). Kirkpatrick’s taxonomy constitutes an “intermediate stage of theory development” that can be improved.

As someone who is just entering the field of HRD, it is interesting to me to see the evolution of a model that comes about through debate, research, and constructive criticism. When I first learned about the model/taxonomy, I immediately found it helpful as it allows me to ask very pointed practical questions. Our text (Russ-Eft & Preskill, 2009) suggests particular questions that correspond with each level:

· “What do participants think and feel about the training?”

· “What do participants know now that they didn’t know before?”

· “In what ways has performance on the job improved?

· “How has the organization benefitted?” among other questions

Throughout my graduate studies, I have often included research on academic advising and see how each of these questions are addressed in on various studies that evaluate advising. Often, the research has focused on reactions–that is, student satisfaction–of advising. It is of critical importance however, to focus also on the other questions, particularly how the organization benefits as we as academic advisors seek to demonstrate the value of our “training” or rather advising students. If we can start with Kirkpatrick’s model and then expand it as Holten (1996b) suggests by seeking out causation and influential factors, it seems to be a useful model for evaluation.

References

Holton III, E. F. (1996a). Final word: Response to reaction to Holton article. Human Resource Development Quarterly, 7(1), 27-29.

Holton III, E. F. (1996b). The flawed four-level evaluation model. Human Resource Development Quarterly, 7(1), 5-21.

Kirkpatrick, D. L. (1996). Invited reaction: Reaction to Holten article. Human Resource Development Quarterly, 7(1), 27-29.

Russ-Eft, D., & Preskill, H. (2009). Evaluation in organizations: A systematic approach to enhancing learning, performance, and change [Kindle version]. Retrieved from www.amazon.com

Forum #2

Kirkpatrick’s Four Level Evaluation Model is a tool that can be used to objective analyze the impact and effectiveness of a training program.  The model consists of 4 progressive testing parameters (Levels) as the name suggest.  First level, the reaction of each trainee is evaluated.  This evaluation allows the organization to know or gauge how well the training is being received.  Second level – Learning, this level seeks to answer the question of, “how much has the trainee’s knowledge increase because of the training?”.  Third level – Behavior, here, the impact of such training on behavior changes is evaluated.  Essentially, “…an organization that takes the time to evaluate its training programs would want to know if the workers actually learned and used the desired knowledge…” (Kennedy, P., Chyung, S., et al., 2013).  And the forth level –  here, the final results of the training outcomes as it relates to the bottom line (organization) is evaluated.

Unlike Kirkpatrick’s Four Level, Learning Domains and Bloom’s Taxonomy takes an approach that evaluates learning/training based on 3 domains.  These domains are Affective, Cognitive and Psychomotor.  After reading and conducting further research on these three domains, it seems to be that cognitive learning is something I understand.  The Cognitive domain in of itself consists of 6 cumulative hierarchy of cognitive skills:  Knowledge, Comprehension, Application, Analysis, Synthesis and Evaluation (Ursani, A., Memon, A., 2014).  The premise here is that learning involves taking these steps.  This learning domain deals with logic derivation, as a HR and IT professional who writes codes, this is a style that I can relate.

References

Kennedy, P., Chyung, S., et al. (2013). Training professionals’ usage and understanding of Kirkpatrick’s

Level 3 and Level 4 evaluations. International Journal of Training and Development 18:1 ISSN

1360-3736

Ursani, A., Memon, A. (2014). Bloom’s taxonomy as a pedagogical model for signals and systems.

         International Journal of Electrical Engineering Education, Volume 51, Number 2