CURRICULUM BASED REASONING TEST for UNIVERSITY ENTRANCE EXAMINATION Susanto Imam Rahayu Indonesian Academy of Sciences.

Presentasi berjudul: "CURRICULUM BASED REASONING TEST for UNIVERSITY ENTRANCE EXAMINATION Susanto Imam Rahayu Indonesian Academy of Sciences."— Transcript presentasi:

1 CURRICULUM BASED REASONING TEST for UNIVERSITY ENTRANCE EXAMINATION Susanto Imam Rahayu Indonesian Academy of Sciences

2 UNIVERSITY ENTRANCE EXAMINATION IN INDONESIA Historical Perspective
* In the 1950’s, there were no University Entrance Examination. Everyone having high school certificate were accepted. * Due to high demand for some popular degree programs, in particular engineering and medicine, some faculties began to impose entrance examination * With ever higher demand and limited space, the number of facuties and universities imposing entrance examination spread.

3 - nearly all universities impose entrance examination
* By early 1970’s, the situation is getting chaotic: - nearly all universities impose entrance examination - the dates of examinations and subjects tested varied - many exams were still conducted by faculties, where even in one university different faculties might test different subjects - there were large student intercity movements, as many of them want to enrol to more than one university in different cities - this created hardship for high school leavers from outside Jawa, dues to bad transportation, mainly by ship - to overcome this situation, students from outside Jawa who want to enrol to universities in Jawa usually enrol their last high school clas in Jawa

4 This unhealthy situation got the attention of five prominent universities at that time. In 1975 the five - Universitasf Indonesia (UI), Jakarta - Institut Pertanian Bogor (IPB), Bogor - Institut Teeknologi Bandung (ITB), Bandung - Universitas Gadjah Mada (UGM), Yogyakarta - Universitas Airlangga, (UNAIR), Surabaya established a working group, known by its acronym SKALU (Sekretariat Koordinasi Antara Lima Universitas), Cooperational Secretariate of Five Universities. * Through SKALU, they agreed to hold their entrance examination among them at the same time, where prospective students may put their choice of degree programs in any one of them.

5 SKALU also standardize the types of subject to be tested in the entrance examination, irrespective of the faculties, into just nine subjects: - Indonesia Language - English - Mathematics - Physics - Chemistry - Biology - History - Economics - Geography

6 Considering that all entrance tests that were administered by the various universities were all “home made”, hence not suitable to be used as entrance examination, SKALU began to conduct training programs on how to make test and do educational measurement. Starting from SKALU exam of five universities, by the years, it now becomes a national entrance examination, involving all state universities (now around 50), with the same standardized subject as agreed by SKALU. Establishment of national entrance examination not only rectivies academic problem, also equity problems.

7 Achievement vs Predictive Tests
The problems of testing which were conducted by the universities, prior to SKALU exams, are mostly of the type of achievement test, testing proficiency in one fiels of subject. By doing this, the exams were just some kind of prerequisite tests, testing ability of subjects thought to be prerequisite to unversity courses. Entrance examination belongs to the type of predictive tests, with the purpose of predicting prospective students expected succes in the university.

8 Test after educational process.
mesurement educational process TEST Purpose Measuring how much of the taught subjects are mastered after undergoing an educational/ learning process Knowing how large learning objectives of the course or educational process are met Achievement Test

9 Predictive Test educational process Purpose
Test before entering educational process educational process TEST measurement Purpose Measurng prospective intake’s academic potential needed to undergo the educational process. Predicting the degree of success of applicants in undergoing the educational process. Predictive Test

10 Achievement tests are easy to construct, is it comes after an education process. It is all about what were the subject of the process. Predictive tests, being prior to an education process, cannot be based on subject of the coming process. Following the ETS, it should logically be an aptitude test, as shown in their SAT tests. University entrance exams in Indonesia, historically, are subject based. To change it to become a predictive test, some changes has to be conducted, by introducing concepts related to aptitude tests.

11 OF EDUCATIONAL OBJECTIVES
BLOOM’S TAXONOMY OF EDUCATIONAL OBJECTIVES Cognitive

12 FORMAT OF BLUEPRINT/GRID
Ruang lingkup Know. C1 Comp. C2 Appl. C3 Ana. C4 Synth C5 Eval. C6 Jumlah 2. JUMLAH

13 PHYSICS GRID FOR 2008 Ruang lingkup Mekanika - 4 8 6 18 2. Kalor 2 10
Know. C1 Comp. C2 Appl. C3 Ana. C4 Synth C5 Eval. C6 Jumlah Mekanika - 4 8 6 18 2. Kalor 2 10 3. Getaran, Gelombang dan Bunyi 12 4. Cahaya 5. Listrik & Magnet 6. IPBA 7. Fisika Moderen 16 8. Antar ruang lingkup JUMLAH 28 40 32 100

14 Test pattern before 2009 Using UMPTN/Sipenmaru pattern, consisting of : Basic Ability (Basic Mathematics , Indonesian Language, English), 75 items, 180 min.. Science Ability (Mathematics in Science, Physics, Chemistry, Biology, Integrated Science), 75 items, 120 min.. Social Science Ability (Economy, History, Geography, Integrated Social Science), 75 items, 90 min. Constituted out of individual test items, while keeping the whole test being predictive

15 Apabila pohon pisang dipangkas dekat tanah, maka pada tempat pemangkasan tersebut akan keluar air. Hal ini membuktikan bahwa air yang keluar itu disebabkan oleh adanya proses eliminasi. proses gutasi. peristiwa imbibisi. proses menitik. pengaruh tekanan akar. Dua buah bola lampu masing-masing tertulis 60 watt 120 volt dan 40 watt 120 volt. Jika kedua bola lampu tersebut dihubungkan seri pada tegangan 120 volt maka jumlah daya pada kedua lampu tersebut adalah 100 w 50 w 24 w 20 w 18 w

16 The problem of this type of test here is that it has boceme the meat of coaching industry (bimbel in Indonesia). Their effectiveness as tests became quetionable. The challenge is to construct a new test which could measure more than these single item predictive test, and be imune toward attacks by coaching industry. To that purpose we construct what we call as Curriculum Based Reasoning Test.

17 Test pattern since 2009 Using Curriculum Based Reasoning Test, consisting of the following sections : Mathematics and Language (Mathematics, Indonesian Language, English), 70 items, 120 min. Science (Mathematics in Science, Physics, Chemistry, Biology), 55 items, 90 min. Social Science (Economy, History, Geography), 70 items, 90 min. Except for Mathematics and Language, the test are not divided and evaluated according to its components.

18 Characteristics of Reasoning Test
Questions for test items :  emphasising those needing reasoning to answer  avoiding recall questions Percentage of questions on reasoning ability: Constitute 2/3 of items in tests for Mathematics, Science and Social Science, based on reading paragraphs followed by 3 to 4 questions. Types of reading paragraphs : - representation of data from observations or experiments - conclusions or results of investigations or obervations - exposition of contradictory opinions

19 Reasoning abilities tested in questions consists of:
- reading graphs or tables - using graphs or tables - understanding the meaning of graphs or tables in relation to naration in readings - taking the next steps, e.g. exptrapolation or interpolation, in relation to data in reading - understanding or identifying implications from information - finding and doing analysis on similarities and discrepancies of arguments - identifying special information from reading

20 Di bawah ini adalah contoh data pengukuran tekanan osmotik untuk larutan hemoglobin pada berbagai konsentrasi yang diukur pada dua suhu yang ber­beda. Gambar 2. Tekanan osmotik terhadap konsentrasi hemoglobin pada 25oC dan 50oC.

21 Berdasarkan grafik pada Gambar 2, beda tinggi silinder osmometer (h) ketika osmometer diisi dengan larutan hemoglobin 0,3 g/L pada 25 oC adalah (Pada 25 oC air = 1 g/cm3) (A) 5 cm (B) 10 cm (C) 15 cm (D) 20 cm (E) 25 cm

22 Experiment no 1 Silinder diisi gas argon pada suhu 300 K, volum 5 liter dan tekanan 1 atmosfer. Dengan menjaga suhu tetap ada 300K, tekanan dinaikkan dengan menekan piston. Pengamatan tekanan dan volum gas adalah seperti Tabel 1 berikut. Tabel 1 tekanan volum 1.0 5.0 1.5 3.3 2.0 2.5

23 Experiment no 2 Tabel 2 Silinder diisi dengan gas X yang mengalami dissosiasi , dengan volum awal 5 liter, tekanan awal 1 atmosfer dan suhu 300K. Pada suhu yang dijaga tetap, tekanan gas dibesarkandengan menekan piston. Pengamatan tekanan dan volum adalah seperti pada Tabel 2 berikut. tekanan volum 1.0 5.0 1.5 3.2 2.0 2.3 2.5 1.8

24 (E) pengecilan ukuran dari molekul-molekul
Dari pengamatan di Tabel 2, Percobaan 2 menunjukkan bahwa dibanding data di Tabel 1, pada tekanan dan suhu yang sama, volum sistem reaksi lebih kecil dari volum gas tunggal. Hal ini disebabkan kenaikan tekanan menimbulkan (A) penambahan jumlah molekul, karena pembesaran harga . (B penggeseran kesetimbangan ke kanan. (C) penurunan jumlah molekul, karena pengurangan harga . (D) pengecilan volum, karena kekenyalan campuran tinggi. (E) pengecilan ukuran dari molekul-molekul

25 (61) In one experiment, team leaders were shown different videos designed to put them in certain moods, then they were told to lead groups in tasks. (62) However, it turns out that for emotions to spread, two people do not have to be in the same room. (63) They do not even have to know each other. Emotions can be transmitted through mass media and (64) must extend far beyond their originators. (62) The word “However” should be replaced by the following: __________. (A) unchanged (B) replaced by “Therefore” (C) replaced by “In addition” (D) replaced by “For example” (E) replaced by “Likewise”

26 POPULASI MUSIMAN Populasi suatu spesies yang hidup di suatu lingkung­an selalu berubah sepanjang tahun berjalan. Berdasar­kan data masa lalu puncak populasinya tercapai pada bulan Januari. Kemudian turun dan paling rendah pada bulan Juli. Populasinya naik kembali sampai mencapai puncaknya pada bulan Januari tahun beri­kutnya, Proses ini berlangsung terus dan berulang setiap tahun. Suatu pengamatan dari bulan Januari 2008 sampai Ja­nuari 2009 memberikan data berikut. Waktu Jan April Juli Okt Jan Populasi Pengamat ini memperkirakan suatu fungsi periodik sebagai hampiran untuk populasi pada setiap saat t, yang aturannya adalah P(t) = cos 2  t, 0 ≤ t ≤ 1. Untuk fungsi ini, t = 0 adalah awal Januari 2008 dan t = 1 adalah awal Januari 2009 dengan setiap bulan t naik sebesar.

27 Dengan menggunakan fungsi hampiran pada Bacaan 2, populasi spesies pada bulan Mei 2008 adalah
Pada tahun 2008, laju penurunan yang paling besar terjadi pada bulan (A) Pebruari (B) Maret (C) April (D) Mei (E) Juni

28 Using two types of score:
Scoring of test Using two types of score: RAW SCORE x. Obtained directly from answering the multiple choice, according to: - right +4 - wrong -1 - omit NATIONAL SCORE N. Obtained by assigning score 500 to average raw acore <x> and 100 for every standard deviation  . With z = (x - <x>)/, the national score is defined as N = z Academic decision are based on ranking of national score.

29 Statistical Meaning of National Score
N > : belonging to uppest 2,3% in the national ranking 700 > N > 600 : belonging to the next 13,6% under 600 > N > 500 : belonging to 34,1% above the national average

30 Percentile Distribution of Raw Scores for two test batteries
IPA (Science Set) IPS (Social Science Set)

31 ITEM ANALYSIS on a Test Item
Garis x+y=4 memotong parabol y=4x-x2 di titik A dan B. Panjang ruas garis AB adalah 2 2√3 3√2 4 4√2 N Omit A B C* D E Mtot P+ Rbis MO MA MB MC MD ME 21619 10525 1737 2070 2758 2585 1944 13.00 0.12 0.70 11.97 12.39 12.70 17.58 13.15 12.66

32 Di daerah yang memiliki perbedaan yang besar antara suhu udara terendah dan suhu udara tertinggi, pelapukan batuan secara mekanik menjadi lebih mudah terjadi. SEBAB Batuan umumnya tersusun atas mineralmineral dengan koeffisien muai yang berbeda. N Omit A* B C D E Mtot P+ Rbis MO MA MB MC MD ME 27050 7108 8072 7662 1788 1952 468 13.00 0.29 0.20 12.70 13.93 11.78 12.66 12.04

33 Evaluation of Test Item Effectivenes
Based on two measured quantities, by item analysis p+ = fraction of testees answering right Rbis = point-bisserial correlation, measure of discriminating power Data for all test items are presented in a scatter plot: Rbis more selective 0,25 0,0 p+ difficult easy

34 Rough relation between p+ and Rbis
Items too difficult, nobody can do right, useless as testing item Items too easy, everybody can do right, useless as test item

35 Evaluasi TNBK 2013

36 Evaluasi TNBK 2012

37 Evaluasi TNBK 2013

38 Evaluasi TNBK 2012

39 Evaluasi TNBK 2013

40 Evaluasi TNBK 2012

41 Evaluasi TNBK 2013

42 Evaluasi TNBK 2012

43 Evaluasi TNBK 2013

44 Evaluasi TNBK 2012

45 Distribution of Raw Score
IPA IPS

46 Distribution of National Score

47 Two final question on this test;
How long does it take to produce this kind of test ? * How much does it cost ?

48 THANK YOU

49 Profil Nilai Mentah SET SOAL Mean S.D Min Max IPA-A Semua Anchor IPA-B
12303 49,2 42,37 -69 260 Anchor 11,27 11,37 -25 65 IPA-B 14066 54,93 40,72 -55 329 15,59 12,04 -30 74 IPS-A 11660 38,23 29,59 -50 212 9,71 10,82 -33 64 IPS-B 14527 43,83 28,77 -54 217 12,69 10,96 -28 62

50 Nilai Rata-rata Per Panlok

51 PERBANDINGAN DENGAN KINERJA TAHUN 2012

52 TERIMA KASIH

53 Bacaan 2 POPULASI MUSIMAN Populasi suatu spesies yang hidup di suatu lingkung­an selalu berubah sepanjang tahun berjalan. Berdasar­kan data masa lalu puncak populasinya tercapai pada bulan Januari. Kemudian turun dan paling rendah pada bulan Juli. Populasinya naik kembali sampai mencapai puncaknya pada bulan Januari tahun beri­kutnya, Proses ini berlangsung terus dan berulang setiap tahun. Suatu pengamatan dari bulan Januari 2008 sampai Ja­nuari 2009 memberikan data berikut. Waktu Jan April Juli Okt Jan Populasi Pengamat ini memperkirakan suatu fungsi periodik sebagai hampiran untuk populasi pada setiap saat t, yang aturannya adalah P(t)  2000  1000 cos 2 t, 0  t  1. Untuk fungsi ini, t  0 adalah awal Januari 2008 dan t  1 adalah awal Januari 2009 dengan setiap bulan t naik sebesar. Bacaan 2 untuk menjawab soal nomor 16 sampai 19. 16..Dengan menggunakan fungsi hampiran pada Bacaan 2, populasi spesies pada bulan Mei 2008 adalah (A) 1200 (B) 1250 (C) 1350 (D) 1500 (E) 1600 17. Untuk bulan April 2008, jika p adalah data dari tabel pada Bacaan 2 dan q adalah nilai fungsi hampiran P(t), maka p - q = (A) -150 (B) -100 (C) (D) 100 (E) 150