It seems clear that evaluation of "e-learning content", i.e., learning objects used in information and communication technology (ICT) based learning, is linked to the evaluation of learners' level of understanding and the improvement of classes.
Although various evaluation methods have been proposed, the most standard model is Kirkpatrick's four-step evaluation method [1], which consists of evaluation and measurement of effectiveness on the basis of 1. Behavior, and 4. Results. Depending
on the level of measurement, we can understand the effectiveness of the learning activity (training). The following table shows the evaluation items in the Kirkpatrick model and the methods used to collect these data.
4-step evaluation method: Kirkpatrick model (Suzuki (2006), Table 6-2)
| level |
evaluation item |
Data Collection Tools |
| 1. reaction |
How did participants respond to education? |
Student Survey |
| 2. learning |
What knowledge and skills have you acquired? |
post-test
performance test |
| 3. actions |
How did participants apply their knowledge and skills to their work? |
Follow-up Surveys
Questionnaire for supervisors |
| 4. results |
What effect has education had on the organization and its goals? |
Effectiveness Measurement Checklist
Return On Investment (ROI) Indicator |
Although some extensions of Kirkpatrick's model have been proposed and practiced in corporate training and other fields, the majority of studies on content-based practices and evaluations in Japanese higher education institutions are still
based on simple surveys and analyses (level 1 and 2) using tests and questionnaires, etc. The significance of formative evaluation techniques in actual content development is described in detail by Suzuki (1987) [2]. See Suzuki (1987) for
a detailed description of the significance of formative evaluation techniques in the actual content development [3].
In the evaluation of content and content development itself (quality assessment), many research methods using Rubric indicators and checklists have been adopted. In addition, for quality assurance of learning objects, the MERLOT (The Multimedia
Education Resource for Learning and Online Teaching) project used peer review (peer review), which has been used in many of the other areas of review of published papers (mutual assessment) methodology [4].
The assigned paper presents a practical case study on content evaluation at Nagoya University and Kumamoto University. Critique these references and list your findings and doubts. Conduct research on the questionable points and report your findings.
Make three headings: "What you found out", "What you wondered about", and "Findings". Also include discussion and source information in the Findings section.