Chapter 6 Training Evaluation

Reasons for Evaluating Training Perusahaan telah menginvestasikan dana yang besar dalam program pelatihan untuk membantu mendapatkan keuntungan kompetitif Perusahaan berinvestasi di training karena terbukti dapat meningkatkan pengetahuan yang dapat mebedakannya dengan perusahaan lain.

Evaluasi Pelatihan menyediakan data yang diperlukan untuk menunjukkan pelatihan yang tidak memberikan manfaat bagi perusahaan.

Formative Evaluation Evaluasi formatif - evaluasi yang dilakukan untuk meningkatkan proses pelatihan Membantu untuk memastikan bahwa: program pelatihan yang terorganisasi dengan baik dan berjalan lancar peserta pelatihan belajar dan puas dengan program tersebut Memberikan informasi tentang bagaimana membuat program yang lebih baik

Summative Evaluation Evaluasi sumatif - Evaluasi dilakukan untuk mengetahui sejauh mana peserta pelatihan telah berubah sebagai hasil dari keikutsertaan dalam program pelatihan Dapat mengukur keuntungan monetary atau biasa disebut return on investment (ROI). Menunjukkan bahwa perusahaan mendapatkan keuntungan dari program pelatihan.

Why Should A Training Program Be Evaluated? (1 of 2) Untuk mengidentifikasi kekuatan dan kelemahan program. Untuk menilai apakah program training mendukung proses belajar dan dapat digunakan di tempat kerja. Untuk mengidentifikasi apakah peserta pelatihan mendapatkan keuntungan atau tidak dari program pelatihan tersebut.

Why Should A Training Program Be Evaluated? (2 of 2) Untuk mengumpulkan data untuk membantu dalam program pelatihan pemasaran Untuk menentukan keuntungan finansial dan biaya program Untuk membandingkan biaya dan manfaat dari investasi pelatihan versus non-pelatihan Untuk membandingkan biaya dan manfaat dari program pelatihan yang berbeda untuk memilih program terbaik

The Evaluation Process Conduct a Needs Analysis Develop Measurable Learning Outcomes and Analyze Transfer of Training Develop Outcome Measures Choose an Evaluation Strategy Plan and Execute the Evaluation

Training Outcomes: Kirkpatrick’s Four-Level Framework of Evaluation Criteria Focus 1 Reactions Trainee satisfaction 2 Learning Acquisition of knowledge, skills, attitudes, behavior 3 Behavior Improvement of behavior on the job 4 Results Business results achieved by trainees

Outcomes Used in Evaluating Training Programs: (1 of 4) Affective Outcomes Results Return on Investment Cognitive Skill-Based The following suggestions specify the types of changes in jobs that are most likely to lead to improvements in each of the five core dimensions. (1) Combine tasks - managers should put existing fractionalized tasks back together to form a new, larger module of work. This increases skill variety and task identify. (2) Create natural work units - managers should design tasks that form an identifiable and meaningful whole. This increases employee “ownership” of the work and encourages employees to view their work as meaningful and important rather than as irrelevant and boring. (3) Establish client relationships - the client is the user of the product or service that the employee works on. Whenever possible, managers should establish direct relationships between workers and their clients. This increases skill variety, autonomy, and feedback for the employee. (4) Expand jobs vertically - vertical expansion means giving employees responsibilities and controls that were formerly reserved for management. It partially closes the gap between the “doing” and “controlling” aspects of the job, and it increases employee autonomy. (5) Open feedback channels - by increasing feedback, employees not only learn how well they are performing their jobs but also whether their performances are improving, deteriorating, or remaining at a constant level. Ideally, employees should receive performance feedback directly as they do their jobs rather than from management on an occasional basis. 9

Outcomes Used in Evaluating Training Programs: (2 of 4) Hasil kognitif Menentukan sejauh mana peserta pelatihan yang akrab dengan prinsip-prinsip, fakta, teknik, prosedur, atau proses ditekankan dalam program pelatihan Mengukur apa peserta pelatihan pengetahuan yang dipelajari dalam program Hasil keterampilan Berbasis Menilai tingkat keterampilan teknis atau motorik Sertakan perolehan atau belajar keterampilan dan penggunaan keterampilan pada pekerjaan

Outcomes Used in Evaluating Training Programs: (3 of 4) Hasil afektif Sertakan sikap dan motivasi Persepsi peserta pelatihan 'program termasuk fasilitas, pelatih, dan konten Hasil Tentukan program pelatihan hadiah untuk perusahaan

Outcomes Used in Evaluating Training Programs: (4 of 4) Return on Investment (ROI) Membandingkan manfaat moneter pelatihan dengan biaya pelatihan biaya langsung biaya tidak langsung manfaat

How do you know if your outcomes are good? Hasil pelatihan yang baik harus: relevan handal diskriminatif praktis

Good Outcomes: Relevance Kriteria Relevansi - sejauh mana program pelatihan terkait dengan kemampuan belajar yang ditekankan dalam program pelatihan Kontaminasi Kriteria – bahwa hasil pelatihan mengukur kemampuan yang tidak pantas atau dipengaruhi oleh kondisi asing Defisiensi Kriteria - kegagalan untuk mengukur hasil pelatihan yang ditekankan dalam tujuan pelatihan

Criterion deficiency, relevance, and contamination: Outcomes Identified by Needs Assessment and Included in Training Objectives Outcomes Related to Training Objectives Outcomes Measured in Evaluation Contamination Relevance Deficiency

Good Outcomes (continued) Reliability - sejauh mana hasil dapat diukur secara konsisten dari waktu ke waktu Diskriminasi - sejauh mana kinerja peserta pelatihan pada hasil yang sebenarnya, hal ini mencerminkan perbedaan dalam kinerja Kepraktisan/practicality - mengacu pada kemudahan mendapatkan hasil yang terukur.

Training Program Objectives and Their Implications for Evaluation: Learning Transfer Outcomes Reactions: Did trainees like the program? Did the environment help learning? Was material meaningful? Skill-Based: Ratings by peers or managers based on observation of behavior Cognitive: Pencil-and-paper tests Affective: Trainees’ motivation or job attitudes Performance on a work sample Results: Did company benefit through sales, quality, productivity, reduced accidents, and complaints? Performance on work equipment

Evaluation Designs: Threats to Validity Ancaman terhadap validitas merujuk pada faktor yang akan membawa kita untuk pertanyaan : Kepercayaan hasil penelitian (validitas internal), atau Sejauh mana hasil evaluasi digeneralisasikan untuk kelompok lain pelatihan dan situasi (validitas eksternal)

Threats To External Validity Threats to Validity Threats To Internal Validity Company Persons Outcome Measures Threats To External Validity Reaction to pretest Reaction to evaluation Interaction of selection and training Interaction of methods

Methods to Control for Threats to Validity Pre- and Posttests Use of Comparison Groups Random Assignment

Types of Evaluation Designs Posttest – only Pretest / Posttest Posttest – only with Comparison Group Pretest / Posttest with Comparison Group Time Series Time Series with Comparison Group and Reversal Solomon Four–Group

Comparison of Evaluation Designs (1 of 2) Measures Design Groups Pre-training Post-training Cost Time Strength Posttest Only Trainees No Yes Low Pretest / Posttest Medium Posttest Only with Comparison Group Trainees and Comparison Pretest / Posttest with Comparison Group High

Comparison of Evaluation Designs (2 of 2) Measures Design Groups Pre-training Post-training Cost Time Strength Time Series Trainees Yes Yes, several Medium Time Series with Comparison Group and Reversal Trainees and Comparison High Solomon Four-Group Trainees A Trainees B Comparison A Comparison B No

Example of a Pretest / Posttest Comparison Group Design: Pre-training Training Post-training Time 1 Post-training Time 2 Lecture Yes Self-Paced Behavior Modeling No Training (Comparison) No

Example of a Solomon Four-Group Design: Pretest Training Posttest Group 1 Yes IL-based Group 2 Traditional Group 3 No Group 4

Factors That Influence the Type of Evaluation Design How Factor Influences Type of Evaluation Design Change potential Can program be modified? Importance Does ineffective training affect customer service, product development, or relationships between employees? Scale How many trainees are involved? Purpose of training Is training conducted for learning, results, or both? Organization culture Is demonstrating results part of company norms and expectations? Expertise Can a complex study be analyzed? Cost Is evaluation too expensive? Time frame When do we need the information?

Conditions for choosing a rigorous evaluation design: (1 of 2) Hasil evaluasi dapat digunakan untuk mengubah program Program pelatihan yang sedang berlangsung dan memiliki potensi untuk mempengaruhi banyak karyawan (dan pelanggan) Program pelatihan melibatkan beberapa kelas dan sejumlah besar peserta latihan Biaya pembenaran untuk pelatihan didasarkan pada indikator numerik

Conditions for choosing a rigorous evaluation design: (2 of 2) Anda atau orang lain memiliki keahlian untuk merancang dan mengevaluasi data yang dikumpulkan dari studi evaluasi Biaya pelatihan menciptakan kebutuhan untuk menunjukkan bahwa itu bekerja Ada waktu yang cukup untuk melakukan evaluasi Ada minat dalam mengukur perubahan dari tingkat pra-pelatihan atau dalam membandingkan dua atau lebih program yang berbeda

Importance of Training Cost Information Untuk memahami total pengeluaran untuk pelatihan, termasuk biaya langsung dan tidak langsung Untuk membandingkan biaya program pelatihan alternatif Untuk mengevaluasi proporsi uang yang dihabiskan untuk pengembangan pelatihan, administrasi, dan evaluasi serta untuk membandingkan uang yang dihabiskan untuk pelatihan untuk berbagai kelompok karyawan Untuk mengontrol biaya

To calculate return on investment (ROI), follow these steps: (1 of 2) Identifikasi hasil (s) (misalnya, kualitas, kecelakaan) Tempatkan nilai pada hasil (s) Tentukan perubahan dalam kinerja setelah menghilangkan pengaruh potensial lainnya pada hasil pelatihan. Mendapatkan jumlah manfaat tahunan (hasil operasi) dari pelatihan dengan membandingkan hasil setelah pelatihan untuk hasil sebelum pelatihan (dalam dolar)

To calculate return on investment (ROI), follow these steps: (2 of 2) Tentukan biaya pelatihan (biaya langsung + biaya tidak langsung + biaya pengembangan + biaya overhead + kompensasi bagi peserta pelatihan) Hitung total tabungan dengan mengurangi biaya pelatihan dari manfaat (hasil operasional) Hitung ROI dengan membagi manfaat (hasil operasional) dengan biaya ROI memberikan perkiraan pengembalian dolar yang diharapkan dari setiap dolar yang diinvestasikan dalam pelatihan.

Determining Costs for a Cost-Benefit Analysis: Development Costs Overhead Compensation for Trainees Direct Costs Indirect Costs The following suggestions specify the types of changes in jobs that are most likely to lead to improvements in each of the five core dimensions. (1) Combine tasks - managers should put existing fractionalized tasks back together to form a new, larger module of work. This increases skill variety and task identify. (2) Create natural work units - managers should design tasks that form an identifiable and meaningful whole. This increases employee “ownership” of the work and encourages employees to view their work as meaningful and important rather than as irrelevant and boring. (3) Establish client relationships - the client is the user of the product or service that the employee works on. Whenever possible, managers should establish direct relationships between workers and their clients. This increases skill variety, autonomy, and feedback for the employee. (4) Expand jobs vertically - vertical expansion means giving employees responsibilities and controls that were formerly reserved for management. It partially closes the gap between the “doing” and “controlling” aspects of the job, and it increases employee autonomy. (5) Open feedback channels - by increasing feedback, employees not only learn how well they are performing their jobs but also whether their performances are improving, deteriorating, or remaining at a constant level. Ideally, employees should receive performance feedback directly as they do their jobs rather than from management on an occasional basis. 9

Example of Return on Investment Industry Training Program ROI Bottling company Workshops on managers’ roles 15:1 Large commercial bank Sales training 21:1 Electric & gas utility Behavior modification 5:1 Oil company Customer service 4.8:1 Health maintenance organization Team training 13.7:1