LECTURE 5 5 MAIN STEPS: 1. WRITE THE LEARNING GOAL 5. WRITE LEARNING OBJECTIVES 2. DETERMINE THE TYPES OF LEARNING OF THE GOAL 3. INFORMATION-PROCESS ANALYSIS 4. PREREQUISITE ANALYSIS AND ITS TYPE OF LEARNING I. ANALYZING THE LEARNING TASK English Study Program-FKIP-UNSRI

WRITE LEARNING GOALS THE LEARNING GOALS ARE STATEMENTS OF PURPOSE OR INTENTIONS (WHAT LEARNINERS SHOULD BE ABLE TO DO AFTER INSTRUCTION): THE FOCUS IS ON CLEAR DESCRIPTIONS OF WHAT CAPABILITIES LEARNERS WILL POSSESS AFTER INSTRUCTION. English Study Program-FKIP-UNSRI

DETERMINING TYPES OF LEARNING USEFUL TO IDENTIFY THE TYPE OF LEARNING OUTCOME COGNITIVE EFFORT LEARNING CONDITIONS WAYS TO TEST FOR ACHIEVEMENT English Study Program-FKIP-UNSRI

DETERMINING TYPES OF LEARNING BLOOM’S: RECALL (KNOWLEDGE) COMPREHENSION APPLICATION ANALYSIS SYNTHESIS EVALUATION English Study Program-FKIP-UNSRI

DETERMINING TYPES OF LEARNING GAGNE’S: DECLARATIVE KNOWLEDGE INTELLECTUAL SKILLS COGNITIVE STRATEGIES ATTITUDES PSYCHOMOTOR SKILLS English Study Program-FKIP-UNSRI

Information- Processing Analysis 01. Read and gather information as much as possible 02. Change the goal into a representative test “question” 03. Give the problem to expert (observe and discuss with them) 04. Review the written steps (no. 03) 05. Identify similar steps conducted by those experts 06. Identify the shortest, least complex way 07. Note factors that may require a more complex way/steps 08. Select the circumstances that best match the goal(s) 09. List the steps appropriate to the goals 10. Confirm the analysis with other experts English Study Program-FKIP-UNSRI

II. ASSESSING LEARNING FROM INSTRUCTION English Study Program-FKIP-UNSRI

PURPOSES OF EVALUATION 1.TO ASSESS PERFORMANCES OF INDIVIDUAL STUDENTS (CAN THEY DEMONSTRATE THEM) 2. TO PROVIDE INFORMATION FOR THE NEEDED REVISION IN THE INSTRUCTIONAL MATERIALS (IS THE INSTRUCTION EFFECTIVE, EFFICIENT, APPEALING?) English Study Program-FKIP-UNSRI

9 Activity of Sequencing Assessment Design 1. Purposes of the assessment instrument and the model type to follow 2. Kinds of assessment necessary 3. Forms of items (performance test, essay, MC, portfolio, etc) 4. Write test items and directions 5. The number of items needed 6. Proportion of objectives appear in the assessment instrument English Study Program-FKIP-UNSRI 2014

Purposes and Model of Assessment 1. To determine competence level - to assess competence or identify gaps - to assess competence or identify gaps in learning (Criterion-referenced, objective- reference, or domain-referenced) in learning (Criterion-referenced, objective- reference, or domain-referenced) - Effective for determining who is competent and where individual weaknesses are; not effective for comparing or ranking learners - Effective for determining who is competent and where individual weaknesses are; not effective for comparing or ranking learners 2. To compare or rank learners’ abilities - to obtain a spread or variation of scores - to obtain a spread or variation of scores (Norm-referenced ) (Norm-referenced ) English Study Program-FKIP-UNSRI

11 TYPE OF ASSESSMENTS  Entry Skills Test CAN BE GIVEN PRIOR TO THE INSTRUCTIONAL LESSON IN COMBINATION WITH THE PRETEST OR AS A WAY OF REVIEWING THE PREREQUISITE SKILLS English Study Program-FKIP- UNSRI 2014

12  PREASSESSMENTS/PRETESTS To find out how much learners already know some of the objectives Given before a lesson or as a way to gain students’ attention and to inform them of the objectives of the lesson Focus on portion of the instruction not yet learned previously English Study Program-FKIP- UNSRI 2014

13  POSTASSESSMENTS/POSTEST To find out if learners have achieved the terminal objective Given at the end of a lesson Given at the end of a lesson Test items must be different from those of pretest or practice test Test items must be different from those of pretest or practice test Test items cover the most critical objectives Test items cover the most critical objectives English Study Program-FKIP- UNSRI 2014

14 CHARACTERISTICS OF INSTRUMENTS ► Criterion-referenced : validity, reliability, practicality Five C’s: Five C’s: Congruence (sesuai), Completeness(sempurna), Consistency (konsisten), Confidence (keyakinan), Congruence (sesuai), Completeness(sempurna), Consistency (konsisten), Confidence (keyakinan), Cost (biaya ) Cost (biaya ) English Study Program-FKIP-UNSRI 2014

15 VALIDITY To measure/assess what is supposed to be measured/assessed Test item must be consistent with the objective (congruence) Items for each objective are representative of the range of items (completeness) Objectives are adequately sampled (completeness) English Study Program-FKIP-UNSRI 2014

16 RELIABILITY  Consistently measures what it is supposed to measure, and  We are very confident with the result/score of the instrument English Study Program-FKIP- UNSRI 2014

17 PRACTICALITY Consider the time in processing the assessment, human resource conducting the test, time between administration of the test and when scores must be reported English Study Program-FKIP-UNSRI 2014

18 ASSESSMENT FORMAT PERFORMANCE ASSESSMENT Observation of on-the-job performance, Simulation, Essays, Portfolios PAPER-AND-PENCIL TESTS Recall items, Recognition items, constructed answer items English Study Program-FKIP-UNSRI 2014

19 ASSESSMENT FORMATS PERFORMANCE ASSESSMENTS (AUTHENTIC ASSESSMENT -Norman & Linn, 1995 ): higher order knowledge and skills in the real-world context in which they are actually used, generally with open- ended tasks that require substantial examinee time to complete e.g.: observation, open-ended problems, micro worlds, essays, simulations, projects, portfolios English Study Program-FKIP-UNSRI 2014

20 PERFORMANCE ASSESSMENTS Observation (dalam situasi pekerjaan) Observation (dalam situasi pekerjaan) Instrumen: checklist, rating scales (menilai kualitas proses saat siswa melakukan keterampilan tertentu; atau menilai produk atau hasil dari performa ini. Checklist/Rating scales: daftar prilaku atau karakteristik yang dapat diamati; Checklist/Rating scales: daftar prilaku atau karakteristik yang dapat diamati; Prilaku/karakterisktik dinilai dengan pernyataan antara lain: ada/tidak ada, ya/tidak; tepat/tidak tepat; benar/salah, dlsb Prilaku/karakterisktik dinilai dengan pernyataan antara lain: ada/tidak ada, ya/tidak; tepat/tidak tepat; benar/salah, dlsb English Study Program-FKIP-UNSRI 2014

21 PERFORMANCE ASSESSMENTS  Simulation Jika tidak praktis atau tidak diinginkan, situasi dapat disimulasikan. Disampaikan melalui bahan ajar cetak, video, interaksi kelompok, komputer, multimedia interaksi. English Study Program-FKIP-UNSRI 2014

22 PERFORMANCE ASSESSMENTS EEssays Jawaban esai yang benar: rangkaian jawaban, bahasa, kekhususan, tata bahasa (sulit menilai secara objektif). Mengurangi kemungkinan subjektivitas penilaian, gunakan checklists, rating scales, model jawaban, atau penilai berlapis. English Study Program-FKIP-UNSRI 2014

23 PERFORMANCE ASSESSMENTS  Portfolio “Kumpulan hasil kerja siswa yang menunjukkan usaha, kemajuan, atau prestasi mereka dalam bidang tertentu Sangat berguna untuk menilai tujuan pemelajaran secara meluas Sangat berguna untuk menilai tujuan pemelajaran secara meluas Isi portoflio: pedoman pemilihan, kriteria manfaat penilaian, bukti dari refleksi diri siswa English Study Program-FKIP- UNSRI 2014

24 Evaluation of Performance Assessments CRITERIA Standards of Validity, Consequences of Assessments on instructional practice, Directness, Transparency, Fairness, transfer and generalizability, adequate cognitive complexity of the tasks, content quality, content coverage, meaningfulness, cost, and efficiemcy English Study Program-FKIP-UNSRI 2014

25 Paper-and-Pencil Assessment Forms Category: MC, T/F, Matching, Completion, Short Answer Used to assess declarative knowledge, intellectual skills, attitudes, cognitive strategies, and portions of motor skills English Study Program-FKIP-UNSRI 2014

26 Paper-and-Pencil Tests Forms: Recall Recognition Constructed answers English Study Program-FKIP-UNSRI 2014

27 PAPER-AND-PENCIL TESTS Recall - Menilai tujuan pengetahuan deklaratif - Menilai komponen hasil pemelajaran lain dari pengetahuan deklaratif Siswa mereproduksi pengetahuan yang mereka peroleh dalam pengajaran, kata per kata seperti yg tertulis/ diucapkan, menggunakan kata-kata lain, meringkas Sangat menuntut daya ingat, tidak memerlukan proses penalaran tinggi Bentuk item tes tertulis: Jawaban Singkat, Mengisi bagian kosong, Menyempurnakan English Study Program-FKIP-UNSRI 2014

28 PAPER-AND-PENCIL TESTS Recognition Siswa harus mengenali atau identifikasi jawaban yang benar dari pilihan yang ada. Menilai pengetahuan deklaratif yang sudah dihafalkan. Dapat saja memerlukan keterampilan lebih tinggi (aplikasi prinsip atau konsep yg diperoleh utk mengenali jawaban yang benar) English Study Program-FKIP-UNSRI 2014

29 PAPER-AND-PENCIL TESTS Constructed Answers Siswa menyusun jawaban: jawaban tertulis selama simulasi, jawaban tertulis dengan pensil/pena/komputer Memerlukan penalaran lebih tinggi dari keterampilan intelektual (petunjuk/pilihan sedikit) English Study Program-FKIP-UNSRI 2014

30 ASSESSMENT ITEM SPECIFICATION (Popham, 1978) 1.Objective 2.Descreation of test form 3.Sample Item 4.Question characteristics 5.Response characteristics 6.Number of item and mastery criterion English Study Program-FKIP-UNSRI 2014

31 SPESIFIKASI ITEM PENILAIAN 1. Tujuan Pernyataan tujuan akan Membantu dalam pengembangan spesifikasi item Tujuan terdiri dari kondisi, performa, kriteria English Study Program-FKIP-UNSRI 2014

32 SPESIFIKASI ITEM PENILAIAN 2. Deskripsi bentuk tes Pilihan Berganda Simulasi dgn menggunakan komputer Pilihan utk prosedur diagnosa Pilihan solusi suatu persoalan English Study Program-FKIP-UNSRI 2014

33 SPESIFIKASI ITEM PENILAIAN 3. Item sampel Harus menilai kemampuan untuk melakukan prilaku yang dinyatakan dalam tujuan sesuai dengan kondisi English Study Program-FKIP-UNSRI 2014

34 SPESIFIKASI ITEM PENILAIAN 4. Karakteristik pertanyaan (Spesifikasi domain isi dan rentang faktor kesulitan item) Merumuskan kualitas pertanyaan, penilaian tes, skenario dalam simulasi English Study Program-FKIP-UNSRI 2014

35 SPESIFIKASI ITEM PENILAIAN 5. Karakteristik Jawaban Desainer membuat garis besar karakteristik jawaban benar (mudah dilakukan), dan jawaban salah (lebih sulit dilakukan) a. Siswa melingkari huruf jawaban benar. Tiap item soal ada 4 opsi jawaban (1 B, 3 S) b. Gunakan 1 distractor (pengalih) pada opsi jawaban. English Study Program-FKIP-UNSRI 2014

36 SPESIFIKASI ITEM PENILAIAN 6. Jumlah item & kriteria penguasaan Penguasaan (mastery): performa lulus atau memadai Identifikasi setiap tujuan persentase item siswa harus menjawab benar untuk dapat kita katakan mereka menguasai tujuan itu. English Study Program-FKIP-UNSRI 2014

37 ASSESSMENT INSTRUMENTS BLUEPRINTS Menilai performa siswa (lebih dari satu tujuan yang dinilai) Komponen: 1.Tujuan5. Petunjuk pelaksanaan 2.Bentuk Item6. Metode penilaian 3.Jumlah item7. Pembobotan item 4.Proporsi item8. Nilai kelulusan English Study Program-FKIP- UNSRI 2014

38 ASSESSMENT INSTRUMENTS BLUEPRINTS Jumlah Item Formatif/diagnostik: buat beberapa Periode penilaian (jika tidak bisa) kurangi jumlah item utk sub tujuan dan jumlah secukupnya utk tujuan akhir English Study Program-FKIP- UNSRI 2014

39 ASSESSMENT INSTRUMENTS BLUEPRINTS Proporsi Item Persentase item utk setiap tujuan erat kaitannya dgn jumlah item keseluruhan Lebih fokus pada keterampilan intelektual ketimbang pengetahuan deklaratif English Study Program-FKIP- UNSRI 2014

ASSESSMENT INSTRUMENTS BLUEPRINTS Petunjuk Pelaksanaan Petunjuk untuk siswa (termasuk batas waktu) Petunjuk untuk pelaksana tes (cara menjawab soal, menentukan area penilaian, bahan yg diperlukan selama penilaian, lama waktu, dan informasi terkait lainnya: realibilitas, validitas, dan kepraktisan) English Study Program-FKIP- UNSRI

ASSESSMENT INSTRUMENTS BLUEPRINTS Scoring Methods - Simple (MC, T/F) - Complete and Detailed (observation, simulation, essays) English Study Program-FKIP- UNSRI

ASSESSMENT INSTRUMENTS BLUEPRINTS Weighting of items Berikan item lebih banyak untuk tujuan penting Beri bobot lebih Siswa harus tahu pembobotan ini English Study Program-FKIP- UNSRI

ASSESSMENT INSTRUMENTS BLUEPRINTS Passing or Cut-Off Score - 80/80 criterion (learners must achieve at least 80% of the items on at least 80% of the objectives) Quiz question : Describe any weakness(es) of using this criterion English Study Program-FKIP- UNSRI

44 Tujuan Evaluasi menilai pencapaian tujuan siswa dan evaluasi efektifitas pengajaran Model Penilaian NRA (ranking siswa) dan CRA (Kompetensi siswa) Karakteristik Instrumen Penilaian Validitas, Reliabilitas, Kepraktisan Spesifikasi Item Penilaian tujuan, item sampel, bentuk pertanyaan, karakteristik pertanyaan dan jawaban, jumlah item dan kriteria penguasaan Pedoman Instrumen Penilaian jumlah item, proporsi item, petunjuk, metode penilaian, pembobotan item, nilai kelulusan English Study Program-FKIP-UNSRI 2014

45English Study Program-FKIP-UNSRI 2014 END OF LECTURE ? QUESTIONS?