Example of
Practice
Presentation

Chemistry IT Workshop 16 November 1998

Using WebMCQ for Formative and Summative Assessment

Overview

WebMCQ is a web-based multiple choice assessment system which can be used to present practice questions (formative assessment), class tests (summative assessment) and surveys over the web. Formative assessment in WebMCQ can be delivered with multiple layers of feedback, while the secure summative assessment mode of WebMCQ provides students with their score at the completion of the test, as well as recording all student marks for examiners. WebMCQ has been run in both modes for First Year Psychology at The University of Sydney (with over 1300 students using WebMCQ in semester 1 of 1998), and evaluations indicate enthusiastic support from students. WebMCQ is now offered as a commercial service to universities, schools and corporations.

Development of WebMCQ

WebMCQ was designed to utilise the many natural advantages of the web for assessment. By using the web as the basis of this system, no paper resources are required, and students may use WebMCQ from any location with access to the Internet, such as classrooms, libraries, home and work. The system is interactive (using cgi scripts) and platform-independent (i.e. it will run on any computer with web access regardless of operating system, such as PC, Macs and Unix-based systems). The system was designed in such a way that in summative mode, WebMCQ can deliver secure classroom tests that provide a final score on completion of the test. In formative mode, students can receive immediate feedback on whether they answered each question correctly or incorrectly, and information regarding why each question option was correct or incorrect. Further, at the end of each initial feedback screen, a "more information" link was provided, which allowed students to go to a further feedback screen which presented a general discussion of the question and the topic area to which the question was related. Students would then proceed to the next question from either of these feedback pages. Figure 1 illustrates the structure of the formative assessment system.

WebMCQ - Formative Assessment

Figure 1. Structure of question and feedback system used in WebMCQ

WebMCQ was first used in 1997 with a large (1200+ students) First Year Psychology course at The University of Sydney. WebMCQ was provided as an additional learning tool in the weeks prior to an end of semester quiz, and sets of questions and feedback for each topic area were developed and made available to students via WebMCQ so as to allow an opportunity for formative assessment prior to the summative quiz. Tutorial rooms (which contained web-connected computers) were left open all week prior to the quiz (there were no classes in this week), with a tutor available to assist with any problems. All students were invited to use WebMCQ, and were informed that a few of the practice questions could be used in the quiz.

Evaluation of WebMCQ in 1997

The response to WebMCQ was exceptional. Over 10,000 hits were recorded on the main WebMCQ page during the month that it was available, and over 1000 First Year Psychology students accessed WebMCQ at least once (students logged in with their student IDs, so multiple uses were only counted as one student). The tutorial rooms were full for much of the week, and external access to WebMCQ occurred at all hours of the day (including the early hours of the morning). The general response of students appeared very positive.

On an evaluation question asking students to rate WebMCQ as a learning resource (on a seven point scale), 61% of students rated WebMCQ as "(1) Excellent", 28% rated it as "(2) Very Good", 9% rated it as "(3) Good" and 2% rated it as "(4) Average". No negative responses (5, 6 or 7) were received. In open questions designed to allow students to state what they found best and worst about WebMCQ, and what they would change, students provided a wealth of comments about the project, the majority of which were very positive. In terms of the best aspect of WebMCQ, the most popular responses were related to "explanation of wrong answers" and "immediate feedback on answers". Other positive comments included:

"Very informative and thorough"
"Great having it on the Internet"
"It emphasises your strengths and weaknesses"
"Easy to use, a fantastic learning tool"

Many students also indicated that using WebMCQ had encouraged them to go back to their class notes and do further study, such as:

"It motivated me to study, since I knew less than I thought I did."

The following three longer quotes provide further general information about how students viewed WebMCQ:

"It gave me a chance to get a perspective on what to study, as well as being an opportunity to test my knowledge."
"It was comprehensive, and very helpful having the 'more information' option and the summaries."
"The 'more information' section [was best], as it helps students to remember the tutorial, and to put questions in context, giving students the opportunity to take notes."

Several suggestions for change were received. These included: a suggestion that a timer would be helpful, comments that feedback in different sections was not always in the same format, and a request that the text be made smaller so that the entire question would fit on one screen (rather than requiring scrolling down on web browsers with large text formats). All of these suggested changes have since been incorporated into the current version of WebMCQ.

Finally, many students offered spontaneous praise for the layout, design, structure and even just the provision of WebMCQ, indicating that they appreciated the system. Quotes such as "THANK YOU! Overall it was great!" were common last comments on the survey. Several students noted that they wished that other courses offered the kind of question and feedback system provided by WebMCQ.

WebMCQ in 1998

During the first and second semester of 1998, using a revised version of the WebMCQ software server, both practice questions and classroom tests were conducted in First Year Psychology at The University of Sydney using WebMCQ. The use of WebMCQ for formative assessment was similar to that reported above for 1997, and student responses were similarly positive. In 1998, over 1250 of the 1350-1400 students enrolled in the course used WebMCQ for practice during the weeks prior to the tests, again with many using it on multiple occasions and throughout the day and night.

In addition, WebMCQ was used to present and score each end of semester test. The newly developed secure mode for WebMCQ was utilised to present questions to students during their classroom quizzes. Password protection, IP address limitations, and encryption of data during transition were among the security features used to protect tests from unauthorised accessing. During the week of the tests, no data was lost, and no security breaches were recorded.

In total, during the month in which WebMCQ was used for formative and summative assessment with First Year Psychology in semester 1 of 1998, over 500,000 web pages of information were served without any errors or software malfunctions. Student evaluations were very positive, with the modal evaluation category once again being "(1) Excellent". Responses to open questions about the formative assessment material again indicated an enthusiastic student reaction to WebMCQ, as illustrated by the following quotations:

"I just wanted to say how FANTASTIC this has been and to say thank you...."
"[WebMCQ] is a great way to practice, and I actually enjoyed studying this way!"
"The feedback on every answer is a brilliant idea! Additional information relating to each topic is a very good study guide. As well as this, access from home encourages and motivates me to study in free time."

The Future of WebMCQ

WebMCQ is now a commercial service offered to universities, schools and corporations both in Australia and internationally. The system includes two main parts WebMCQ Server, the system which presents and scores student usage of quizzes, and WebMCQ Administration Tools which teachers use to create and monitor WebMCQ content. The service (provided on a yearly licence basis) provides access to all tools, hardware and software maintenance, web hosting and email support. A wide range of new features for WebMCQ are currently under development, including questions branching, alternative scoring options, multiple response answers, and alternative language versions of the interface. Later developments will include more general web-based course creation and management tools, and interactive web-based teaching/training modules using WebMCQ's interactive interface. For further details contact info@webmcq.com.au

Summary

The design and construction of WebMCQ was based on sound principles of assessment practice and utilised the inherent potential advantages of the web. Student evaluation of WebMCQ indicated that it made a valuable contribution to their learning, and that the vast majority of students made use of this resource in the context described. Use of WebMCQ in 1998 indicates that it provides a comprehensive assessment package for multiple choice material, allowing both formative and summative assessment options. Due to the generic structure of the software, questions from any discipline area that are adaptable to multiple choice may be presented using WebMCQ, hence its value far beyond the confines of Psychology. For further details about WebMCQ, visit: http://webmcq.com.au/

Note: WebMCQ and all material described here are (c) Copyright, 1/7/98, all rights reserved. An earlier version of this paper was presented at the UniServe Science Physics IT Workshop, The University of Sydney, 1998. See Dalziel, J. R. & Gazzard, S. (1998). Using WebMC for Formative and Summative Assessment. Proceedings of UniServe Science Physics IT Workshop, (Ed. A. Fernandez), 34-37, Sydney: UniServe Science.

© 1997 - 2010 UniServe Science

Page Maintained By: PhySciCH@mail.usyd.edu.au
Last Update: Friday, 12-Feb-1999 16:59:05 EST
URL: http://science.uniserve.edu.au/workshop/chemit/dalziel.html