Evaluating Interface Designs Interaksi Manusia dan Komputer Pertemuan 4

Learning Outcomes Mahasiswa dapat membandingkan hasil rancangan sistem interaktif dengan menerapkan teknik-teknik pengujian usability, ulasan pakar, dan survey.

Topik Bahasan Pendahuluan Ulasan Pakar Usability Evaluation Survey Acceptance Test 3

Pendahuluan Pengujian ekstensif dibutuhkan. Yang perlu diperhatikan dalam rencana evaluasi meliputi: Tahapan perancangan (awal, tengah, akhir). Tingkat kebaruan proyek (terdefinisi atau bersifat eksplorasi). Jumlah pemakai yang diperkirakan. Tingkat kritis antarmuka (mis. sistem medis kritis kehidupan vs. dukungan pameran di museum). Biaya produk dan keuangan yang dialokasikan untuk pengujian. Waktu yang tersedia. Pengalaman perancangan dan tim evaluasi. 4

Pendahuluan (cont..) The range of evaluation plans might be anywhere from an ambitious two-year test with multiple phases for a new national air-traffic-control system to a three-day test with six users for a small internal web site. The range of costs might be from 20% of a project to 5%. Testing should occur at different times in the evaluation cycle, ranging from early to just before release. 5

Ulasan Pakar (Expert Review) Ulasan pakar yang cukup formal telah terbukti efektif. Ulasan pakar dapat dilakukan di awal atau di akhir fase perancangan, dan keluarannya berupa laporan formal dengan masalah yang ditemui atau rekomendasi perubahan. Metode ulasan pakar: Evaluasi heuristik (heuristic evaluation) Ulasan kesesuaian dengan pedoman (guidelines review) Pemeriksaan konsistensi (consistency inspection) Penelusuran kognitif (cognitive walkthrough) Metaphors of human thingking (MOT) Pemeriksaan usability formal (formal usability inspection) 6

Ulasan Pakar (Expert Review) Heuristic Evaluation Sekelompok pakar menguji antarmuka terhadap prinsip-prinsip usability (heuristics) Melibatkan 3-5 pakar– setiap pakar menemukan permasalahan-permasalahan perancangan dan usability yang berbeda 7

heuristic evaluation Box 4.1 - Heuristics for the gaming environment Provide consistent responses to user’s actions. Allow users to customize video and audio setting, difficulty, and game speed. Provide predictable and reasonable behavior for computer controlled units. Provide unobstructed views that are appropriate for the user’s current actions. Allow users to skip non-playable and frequently repeated content. Provide intuitive and customizable input mappings. Provide controls that are easy to manage and that have an appropriate level of sensitivity and responsiveness. Provide users with information on game status. Provide instructions, training, and help. Provide visual representations that are easy to interpret and that minimize the need for micromanagement. From Pinelle et al., 2008.

Ulasan Pakar (Expert Review) Guideline Checklist Review Seorang pakar membandingkan rancangan dengan dokumen pedoman Memeriksa format, navigation, context, workflow, dsb Contoh: Penggunaan font Sans Serif 10pt. seperti: Arial atau Verdana pada halaman Web untuk kemudahan membaca Another approach, getting an bird’s-eye view of an interface by studying a full set of printed screens laid out on the floor or pinned to walls, has proved to be enormously fruitful in detecting inconsistencies and spotting unusual patterns. The bird’s-eye view enables reviewers to quickly see if the fonts, colors, and terminology are consistent and whether the multiple developers have adhered to a common style. 9

Ulasan Pakar (Expert Review) Cognitive walkthroughs Pengujian antarmuka oleh pakar dengan mengambil peran sebagai pengguna yang sedang menggunakan sistem utk menemukan permasalahan. 10

Ulasan Pakar (Expert Review) The danger with expert reviews is that the experts may not have an adequate understanding on the task domain or user communities. Different experts tend to find different problems in an interface, so three to five expert reviewers can be highly productive, as can complementary usablility testing. However, even experienced expert reviewers have difficulty knowing how typical users – especially first-time users – will behave. 11

Uji dan Laboratorium Usability Uji usability (usability test) memberikan konfirmasi kemajuan yang mendukung dan rekomendasi perubahan yang spesifik. Uji usability tidak hanya mempercepat proses, tetapi juga menghasilkan penghematan biaya yang dramatik. Sifat Laboratorium Usability Sederhana Dua ruangan 3x3 meter, dibatasi kaca satu arah. Satu untuk ruang kerja peserta. Satu untuk pengamat (perancang, manajer, pelanggan). 12

Contoh Laboratorium Usability Tata letak Microsoft Usability Lab, Redmond 13

Contoh Laboratorium Usability (Lanj.) Sun Microsystems Usability Lab 14

Contoh Laboratorium Usability (Lanj.) Observer Room Business / Development Team Control Room Facilitator & Application team User Room - participant 15

Usability Testing and Laboratories 4-16

Step-by-Step Usability Guide from http://usability.gov/ 4-17

Memilih Peserta untuk Uji Usability Peserta dipilih mewakili komunitas pemakai dengan memperhatikan: Pemahaman komputer Pengalaman mengerjakan tugas Motivasi dan pendidikan Kemampuan bahasa alami yang digunakan dalam antarmuka. Peserta uji usability harus diberitahu bahwa bukan mereka yang diuji, tetapi software dan antarmuka pemakai. Keikutsertaan dalam uji usability adalah sukarela, dengan perjanjian terlebih dahulu. 18

Usability Testing and Laboratories (cont.) In this eye-tracking setup, the participant wears a helmet that monitors and records where on the screen the participant is looking 4-19

More portable eye-tracking devices Usability Testing and Laboratories (cont.) More portable eye-tracking devices Up until recently, this software was rather expensive, but now it can be supplied as a simple add-on to a personal computer (Fig. 4.4) When testing with small mobile devices like cellphones, special equipment may be needed to capture the user’s screen and associated activities (Fig.4.5) 4-20

Usability Test - Metodologi Libatkan 4-5 pengguna dan evaluasi interaksi mereka dalam menyelesaikan kasus-kasus nyata. Lakukan test sedini mungkin dalam tahapan perancangan – paper prototypes, wireframes, HTML mockups, dsb. Lakukan pencatatan (log) selama pengujian Buat rekaman video Ukur kepuasan pengguna dengan on-line feedback questionnaire Diskusikan permasalahan-permasalahan dan rekomendasi-rekomendasi terkait usability dengan tim pengembang. 21

Usability Test – Metodologi (lanj.) Mengapa hanya melibatkan 4 sampai 5 peserta? Menguji dengan 5 orang yang mewakili pengguna biasanya cukup utk mengungkapkan 90% atau lebih permasalahan-permasalahan usability More users = more time + more expense + repeated usability issues Peserta yang lain bisa dilibatkan dalam pengujian selanjutnya terhadap antarmuka pemakai yang sudah diperbaiki. Pengujian yang dilakukan beberapa kali sangat berguna untuk memberbaiki rancangan dan bukan hanya mendokumentasikan kekurangan-kekurangan. 22

Beberapa Teknik Laboratorium Usability Meminta pemakai mengucapkan apa yang mereka pikirkan dan akan kerjakan (think aloud). Menggunakan dua peserta bekerja bersama untuk mendukung bicara. Memvideokan kegiatan peserta untuk dilihat lagi kemudian. They may hear comments such as “This web page text is too small…so I’m looking for something on the menus to make the text bigger… maybe it’s on the top in the icons… I can’t find it…. So I’ll just carry on.” 23

Beberapa Teknik Laboratorium Usability Uji Usability di Lapangan Uji lapangan berusaha menempatkan antarmuka pemakai dalam lingkungan realistik dalam periode waktu tertentu. Pencatatan (logging) software lebih membantu. 24

Survey Kunci survey yang berhasil: Tujuan yang jelas di awal. Pengembangan hal-hal terfokus yang membantu mencapainya. Tujuan survey dapat dikaitkan dengan model OAI. Pemakai dapat ditanyakan kesan subjektif mereka tentang aspek antarmuka. Tujuan survey yang lain adalah untuk menemukan: Latar belakang (umur, asal, jenis kelamin, pendidikan, penghasilan) Pengalaman dengan komputer Tanggung jawab pekerjaan Gaya kepribadian Alasan tak menggunakan antarmuka Keakraban dengan fitur Perasaan setelah menggunakan antarmuka 25

Uji Penerimaan (Acceptance Tests) Untuk proyek implementasi besar, klien biasanya menentukan tujuan objektif dan terukur untuk kinerja hardware dan software. Jika produk gagal memenuhi kriteria penerimaan, sistem harus diperbaiki sampai berhasil. Kriteria terukur dari antarmuka pemakai adalah kelima faktor manusia terukur. Time to learn specific functions Speed of task performance Rate of errors by users Human retention of commands over time Subjective user satisfaction 26

Lima Faktor Manusia Terukur Faktor-faktor ini menjadi pusat evaluasi: Waktu belajar: berapa lama orang biasa mempelajari cara relevan untuk melakukan suatu tugas? Kecepatan kinerja: berapa lama suatu tugas dilakukan? Tingkat kesalahan: berapa banyak kesalahan dan kesalahan-kesalahan apa saja yang dibuat pemakai? Daya ingat: bagaimana kemampuan pemakai mempertahankan pengetahuannya setelah jangka waktu tertentu? Kepuasan subjektif: bagaimana kesukaan pemakai terhadap berbagai aspek sistem? 27

Uji Penerimaan (Acceptance Tests) Setelah uji penerimaan berhasil, uji lapangan dapat meningkatkan: Metode pelatihan Materi tutorial Prosedur bantuan melalui telepon Metode pemasaran Strategi publikasi 28

Evaluation During Active Use Successful active use requires constant attention from dedicated managers, user-services personnel, and maintenance staff. Perfection is not attainable, but percentage improvements are possible. Interviews and focus group discussions Interviews with individual users can be productive because the interviewer can pursue specific issues of concern. Group discussions are valuable to ascertain the universality of comments. 4-29

Evaluation During Active Use (cont.) Continuous user-performance data logging The software architecture should make it easy for system managers to collect data about The patterns of system usage Speed of user performance Rate of errors Frequency of request for online assistance A major benefit is guidance to system maintainers in optimizing performance and reducing costs for all participants. Online or telephone consultants, e-mail, and online suggestion boxes Many users feel reassured if they know there is a human assistance available On some network systems, the consultants can monitor the user's computer and see the same displays that the user sees 4-30

Evaluation During Active Use (cont.) Online suggestion box or e-mail trouble reporting Electronic mail to the maintainers or designers. For some users, writing a letter may be seen as requiring too much effort. Discussion groups, wiki’s and newsgroups Permit postings of open messages and questions Some are independent, e.g. America Online and Yahoo! Topic list Sometimes moderators Social systems Comments and suggestions should be encouraged. 4-31

Evaluation During Active Use (cont.) Bug report using Google’s Chrome browser (http://www.google.com/chrome/) 4-32

Controlled Psychologically-oriented Experiments Scientific and engineering progress is often stimulated by improved techniques for precise measurement. Rapid progress in the designs of interfaces will be stimulated as researchers and practitioners evolve suitable human-performance measures and techniques. 4-33

Controlled Psychologically-oriented Experiments (cont.) The outline of the scientific method as applied to human-computer interaction might comprise these tasks: Deal with a practical problem and consider the theoretical framework State a lucid and testable hypothesis Identify a small number of independent variables that are to be manipulated Carefully choose the dependent variables that will be measured Judiciously select subjects and carefully or randomly assign subjects to groups Control for biasing factors (non-representative sample of subjects or selection of tasks, inconsistent testing procedures) Apply statistical methods to data analysis Resolve the practical problem, refine the theory, and give advice to future researchers 4-34

Controlled Psychologically-oriented Experiments (cont.) Controlled experiments can help fine tuning the human-computer interface of actively used systems. Performance could be compared with the control group. Dependent measures could include performance times, user-subjective satisfaction, error rates, and user retention over time. 4-35