The Best of Creative Computing Volume 1 (published 1976)
"Very Good. Please Continue." A New Approach to Testing (development of computer-generated examinations)
Very good. Please continue.
A New Approach to Testing
Sandra J. Hershkowitz
The development of computer-generated examinations is fairly recent, but the use
of such testing is spreading rapidly; in fact, the first CATC (computer-assisted
test construction) conference has recently been held in San Diego to provide
information about and experience in this area.
For my master's thesis in education, I devised a computerized testing system,
with the hope of increasing learning and motivation in an introductory computer
science course. With the generous cooperation of Dr. Ronald Wojcik (who was then
affiliated with St. Bonaventure University) and his staff, the exams were used
by 63 undergraduate students in order that the system could be evaluated and
student reaction obtained.1
Basically, the idea is as follows: Students are taught in whatever manner is
usual but are tested so that they are competing with themselves rather than with
the rest of the class. Exams are administered via “terminals”,
typewriter-like machines hooked up to a computer, which may be located in the
school building or may in fact be miles away — in one of the many computer
centers located in colleges, universities, or business corporations throughout
the country.
The student identifies himself on the typewriter, and testing begins. A question
is typed out for the student to answer. Questions are chosen at random from a
storage bank of questions so that each student's exam is unique. Because the
student is competing with himself, he is encouraged to take each exam several
times, with the idea of improving his previous high score. The use of randomly
selected questions means that a student's retests will also be different;
therefore he is being exposed to a large amount of material covering a
particular unit or area and is not merely taking the same exam over and over.
Questions are corrected immediately as they are answered. If the answer typed in
by the student is correct, he is given an encouraging comment and asked to
proceed (“Very good. Please continue.") If the answer is incorrect, an
appropriate comment is typed out (“The correct answer is ....... . Why don‘t
you take a look at this later?”). At the end of the exam, the student is given
his score, along with his previous high score, and the areas needing review are
listed. He may then remove his corrected paper, with comments, and use it for
further study.
In the particular system used by the St. Bonaventure students, there were three
units or areas. Each unit exam contained 25-30 objective-type (true-false,
identification, multiple-choice, fill-in) questions, chosen from a storage bank
of approximately 300 questions; and students were allowed to take each exam up
to nine times, with a two-week period allowed per unit. A questionnaire was
administered to assess student reaction, and the responses were very gratifying.
Students overwhelmingly liked the exams, they liked the interactive approach and
the repeatability option, and they felt they were learning more using the exams.
Such a testing system is a practical reality — it is adaptable to many subject
areas at any level and does not involve great expense. It can be programmed on a
small computer; and, as mentioned earlier, a computer does not have to be on the
premises. Students may retest themselves on a particular unit and try to improve
themselves with each retest. They can use their completed exams as study aids
and receive each exam, graded, immediately as completed.
Another point not to be overlooked is the value of such a testing system to the
teacher. In addition to being freed from administering and grading exams, the
teacher can readily obtain desired statistical information, continually updated.
(For example, for a particular unit: number of times each question is answered
correctly; how many times each student has taken the exam; mean and standard
deviation of high scores for the unit; individual scores for each student; and
so on.)
With the increasing use of individualized learning programs, computer-generated
examinations fit in perfectly. A student may proceed at his own pace and take
the exams appropriate to his level of learning.
1"Interactive Testing-Evaluation in an Introductory Computer Science Course" by
Sandra J. Hershkowitz and Ronald Wojcik, Journal of Educational Data Processing,
Spring, 1975.
***
Eclectic Languages continued —
machine so that it can apply the algorithm to a set of data, and can return the
result of the application.
3. Transmit the knowledge of the algorithm to someone else.
If you write for Reason 1, you write in the available language of your choice.
Many will write in the first language learned, some will write in the second
language learned, one or two will write in the best language available. The best
language for Reason 1 is not necessarily the best language for Reason 2. That is
the dilemma. To transport an algorithm to another computer, you must write in a
language known to it. Will there ever be one language for all of us?
A Universal Language?
Computer scientists are confronted with an astounding decline in the price of
computer hardware. The technology that has generated the dramatic and rapid cost
reductions that you have noticed in the hand calculator market will produce,
probably within five years, very powerful computers at costs within the reach of
anyone who can afford a TV set.
Remember that the curvilinear relationship described at the beginning of this
article is subjective. Even today what one computer user finds difficult, a
"programming linguist" might find easy. When the butcher, the baker and the
candlestick maker have that sophisticated computer hardware available to them,
how will they communicate with it? More and more people will find the effort to
learn even languages like BASIC to be more than they are willing or able to
exert. The pressures on computer scientists to come closer to the ultimate
instruction will intensify.
But wait; further development of eclectic languages like ECL may give the
professional programmer much better tools to develop "easy" specialized
extensions (read "languages") for classes of users: the butcher class, the baker
class,. . . The kind of professional programmer who is equipped to develop
custom languages for others is likely to be willing and capable to exert
whatever mental effort is required to achieve fluency in a sophisticated
language. But the casual user of computers, whose numbers will increase
substantially in the next several years, must be given simple languages — he
will tolerate no other kind. Unlike today's general purpose "simple" languages,
though, the simple language of the future will not have a few constructs to
absorb into the intuition, but a few constructs that are already part of the
intuition.
Few people will learn the full-blown eclectic language, but many will learn the
intuitive extensions.
