The class came up with various important categories in deciding
how to evaluate the multimedia courseware presented by the Needs
Organization. A full list of all of these categories can be found
at the end of this document. For my purposes, I have chosen to
use those criteria which I consider to be most important in dealing
with how best to improve the nature of the vibrating beam assignment
-- its content, delivery, design & presentation, use of the
WWW, and overall efficiency.
- The quality and presentation style of the material was superb.
There are several factors which highlight this: if a non-engineering
major (such as myself) is able to understand the course objectives
clearly, then it should not be a problem for the engineering students
it has been designed for. While I assume that the material content
is correct, it is not a difficult matter to go through the assignment
and gain a basic understanding fairly quickly and easily.
- Having the menu bar is a definite plus. This is a critical
tool so that the user can jump from section to section and in-between
separate sections without using the "forward" and "back"
buttons on the browser. Generally speaking, a good on-line document
is one which is "directionally self-contained" - that
is, the site can be navigated from within without using the browser
- The language was clear and generally concise. No major problems
- There was a good balance between text and graphics, but I
noticed a lack of interactivity: isn't this what the Web is about?
- A good overall design. From an organizational point of view,
the materials were logically and sequentially set. A strength
of the project is that one is able to jump from one topic to another
by using either the image map, the menu bar, or even the clickable-directory-structure-tree
type display presented towards the top of the page (i.e.: "lab/theory/stress/beam/movie").
Linking, mapping and navigation issues are adequately addressed.
- Bandwidth does present somewhat of a problem. While being
on an ethernet backbone causes little concern, anybody doing the
assignment from home may have to wait for longer periods -- especially
if they are intent on watching the QuickTime movies. Thumbnails
of images should be used more liberally, and perhaps an initial
page directing users to a "high" or "low"
bandwidth presentation can be created. Thus, people on a 14.4
KBps modem at home can choose "low" and not have to
waste time downloading non-necessary bits of information. Caching
and memory issues also need to be explored: nice presentations
are indeed appealing, but is it worth it if only few people can
see them because of hardware incompatibilities?
- The graphics themselves are relevant and well produced. The
videos are good, and the inclusion of the size of the file is
indeed very helpful. Icons used throughout the tutorial are consistent.
- A major omission from the presentation was that no audio was
used. This would be a very nice feature (for those with appropriate
bandwidths) since audial learning often reinforces visual learning.
The user could be given a choice as to whether they wanted combination
audio, only audio, or no audio. In addition, it is critical that
the entire presentation (barring the videos) be made accessible
as a text document via. the world wide web. This could be efficiently
done, for example, through the use of software like Adobe Acrobat
by creating a .pdf file. Often times it is very helpful
to look at something and be able to write notes next to it...
this cannot be done on computer screens (yet). In addition, a
user may want to refer to the materials while at a cafe, and in
this case it would be virtually impossible to see the on-line
- Perhaps the greatest weakness of the entire project is its
lack of interactivity. The world wide web is different from paper
text precisely because it has an interactive potential.
Unfortunately, this potential was not realized in the vibrating
beam experiment. Areas where this could be improved:
- Create section by section quizzes which test the amount of
knowledge the user has retained from each section. Although natural
language processing could present significant problems, a simple
multiple choice interface would be adequate provided that
explanations as to why a certain choice was better than the others
- Evaluations need to be prompted. Simply having an empty box
to fill out is not good enough because nobody will want to take
the time and effort to do all of the thinking (as I have done
here!). A focused set of criteria need to be presented to the
user, both with clickable choices and space for narrative. For
example, a question such as "How do you rate the use of video
in our presentation?" could have several options: Very Applicable;
Applicable; Don't Know; Inapplicable; Tangential; Unfocused; Don't
Know. Space for "additional comments" could also be
included. While it is unrealistic to assume that every person
using the multimedia learning tool will answer every single question,
a prompted interface will definitely be more successful because
much of the structural thinking will be done for the user.
- Although the page itself is quite good, this is because many
assumptions are being made by the creators about the audience.
It would behoove the project sponsors to make variation pages
which could be used by novice, regular and expert users. For
example, somebody not knowing anything about Mechanical Engineering
may have to have even some of the elementary concepts explained.
This may require more text, explanatory experiments, graphics
and so on. An expert user, on the other hand, may only want to
reference a certain formula she forgot, so it would be very convenient
if there was a meta-index she could use to easily find that formula
instead of going through the whole experiment. The key concept
here is to customize the interface for as many people as possible.
USABILITY & LONGEVITY
- Relatively uncomplicated and simple to use from a "technical"
standpoint. This is good because a central purpose of the multimedia
program is to make users comfortable with the internet and its
implications as a new medium of learning. The fewer the hassles,
- The meta-design of the course seems quite well thought out.
Looking at the image map
shows a logical connection between lab, apparatus, theory and
assignments. This is a good instructional template which can
be used for other courses also, because no matter what the class,
the basic learning process is often similar. While the content
of what is being taught may vary, the structure and delivery mechanism
will often be reapplicable.
This is a good project overall. The assignments and explanations
are logical and follow a pedagogically sound order. The use of
graphics and videos show that a fair amount of available multimedia
technology is being used to enhance the process of learning.
However, to remain useful (and competitive), new features
need to be included (audio, feedback options, text file availability)
to make the program truly interactive. This can only happen via
iteration and reiteration. In this sense, it is extremely important
that accurate data be collected from users and evaluators with
regards to the strengths and weaknesses of the system.
Evaluation criteria discussed in class
- Useful links
- Uniqueness, originality
- Carefully selected
- Quality of Content
- Browse use ability
- Realistic (non-fiction? fiction?)
- Response Time
- Realistic system requirements
- graphic abilities
- Help functions and guides (printed and/or on-line)
- Adherence to standards familiar to user
- Appropriateness to objectives
- Accessibility to all users (consideration of impairments)
- Appropriateness of modality to content
- Both underlying structure and graphic layout
- Representation options (pictorial, words, audio)
- Iconicness of icons
- keeps one coming back
- adapt to technological changes
- ease of installation
- incorporating new info
- adapting to new instructors and instructional environments
- ability to decontectualize & use MM elements separately
- Legal Issues
- Mechanism to give the author feedback
Objectives of MM Package
- Value Added
- Context w/other content
- Ability/age level
- Is it clear and truthful what user is getting?
Considering the Audience
- Appropriateness to audience
- Use of Appropriate Learning Styles
- Instructional Design
- Feedback options
- Progress feedback
- Mechanism to give the instructor feedback
- public/private annotations