Teaching Multimedia Authoring and Production as a Complex Skill

by Donald T. Payne, Paul D. Henry, James Watson, and Baynon McDowell

Educational Technology and Communications Program, New York University

A growing number of students and professionals are interested in gaining enough mastery of authoring languages and production techniques to pursue work in the many fields that have adopted multimedia as a primary vehicle for communication. These students - and the trainers and teachers who provide this type of instruction - represent a large and heterogeneous group who differ in significant ways from those in the computer science and technology curriculum that typically support the computer programming, systems, and operations professions. Students may be seeking this training as an important component in the work they perform in advertising, marketing, training, scientific research, or as multimedia producers and designers.

We began this curriculum project - a multimedia authoring and production learning environment (MAPLE) - with an underlying premise that multimedia authoring and production is a complex, problem-solving skill. Problem-solving skills are not effectively taught in books and courses that focus attention on learning factual or conceptual knowledge, but not developing skills. We propose an instructional approach based on research about developing complex skills and mental models (Bereiter, 1990).

MAPLE integrates several cognitive theories that are being brought to bear upon this particular instructional situation: teaching and learning multimedia authoring and production using a multimedia authoring system, in this case HyperCard. Recognizing the importance of enabling learners with tools and instructional support that lets them take control of their learning, these premises shape the instructional model:

Multimedia production involves many complex skills and an understanding (mental model) of many different systems. Students are likely to have naive models about some systems (multimedia production is cool), strong emotional reactions to some, like programming, and a lack of knowledge about others. One strategy for facilitating the learning of new information or models is to use analogies of something the student does know to relate the new knowledge to existing knowledge. Other approaches are to monitor student responses that are related to their model of a system, to identify the error in their model and show how a better model resolves the problem. (Brewer, 1987; Carlson, 1990; Norman, 1983 and 1993; Rogers, Rutherford, and Bibby, 1992. ).

The most encompassing learning strategy that we are using to develop this project is the Cognitive Apprenticeship approach as it was developed by Collins, Brown, and Newman (1989). The concepts of increasing complex microworlds and situated cognition, below, are incorporated within this theory.

The learning of complex skills whether it is multimedia production or skiing can be learned in more personally meaningful ways by using situated cognition that posits learning activities within real-world activities where the transfer of these skills can be fairly transparent between learning and doing (Brown, Collins, and Duguid, 1989).

The target audience is composed primarily of learners who are novices to the areas of multimedia authoring and production, hypertext and hypermedia, and scripting/programming. The learning of complex skills can be simplified by organizing instruction such that even though a part of a target skill is central, it is constrained and therefore more easily understood. Gradually these constraints are removed and the full set of skills are elicited. This strategy is know as "increasingly complex microworlds" (Burton, Brown, and Fischer, 1984).

The project is being developed for users who may be working alone; however, we strongly urge that, when possible, the materials be used in group settings and that group sessions be designed to facilitate group interaction and sharing of experiences and projects.

The environment is organized into units. The units are organized as increasingly complex microworlds (ICMs). The first set of units are designed to develop an understanding and use of the HyperCard environment. The second set develops scripting knowledge and skills and relates scripting to the objects of the environment. The third set of units develop more complex aspects of multimedia such as meaningful interactions between the environment and the user, and the use of sounds and video. Individual units within these sets are also ordered on complexity. For example, the set on scripting is divided into three units that correspond to the three different control structures of HyperCard: sequential control, if-then structures, and repeat structures. Each unit also raises increasingly complex design issues accompanied by a discussion of related principles along with examples of alternative solutions of the issues.

Each unit features a multimedia production task to be solved. These projects are somewhat simplified cases of real world projects. The strategy is for the production tasks to emphasize high-level rather than basic skills. Required low-level elements are either provided to the user, or introduced and supported in tutorials, or modelled in programmed demonstrations. Models of similar projects, models of exemplary projects, and programmed instructional materials related to the task are provided. Design issues are also related to these tasks. Other resources are members of the group, provided there is a group, reference guides, and articles on specific topics related to projects or to learning issues.

This project is a work-in-progress. It is being used in a course offered at New York University. We don't have sufficient numbers of students in this course to do accurate comparisons, but many of the effects we are studying are sufficiently large that statistical analysis is not required. We have invited doctoral students in our program to participate in the development of this project. We ask them to define issues of interest to them and make available to them the current version of the project to be modified to incorporate their research. They, in turn, permit us to incorporate their changes in the next version of the project, if we wish.

Collectively, we envision this learning framework as a pyramid structure of interconnected activities, information, and materials.

Our model for acquiring complex problem-solving skills for multimedia authoring and production involves the relationship between environmental resources (including cooperative learning), knowledge, skills, and mental models. Computer-based tutorials provide knowledge and problem-solving exercises. Projects are linked back to the tutorials and the material resources in the environment for support.

Diagram of Multimedia Learning Environment

Following the model of multimedia authoring and production as a complex skill, the highest-level and primary instructional delivery is in the form of a series of problem-solving cases which we call "projects." These projects draw from real-world situations and use resources from more than one topical area to solve the problem. Key to the solution of a problem is the refinement of very general and often incomplete, inconsistent, and inaccurate information that is used to describe the problem and its expected solution. Many cognitive science researchers have acknowledged the need for strategic skills in translating general, abstract information into concrete, manageable procedures to solve complex problems (Clements and Nastasi, 1988). This is typically the initial challenge posed by an initial project description for a multimedia production. The "incomplete" aspect to this initial information is related to the entirety of the description in respect to the depth of detail. Information about a project or assignment that is provided by a teacher, client, or manager often lacks sufficient detail and may also unintentionally omit one or more important details. It may also be incomplete in respect to the perspective that is embedded in a description. For example, the perspective of a designer's description of a problem is likely to be qualitatively different from that of a programmer or from that of a client. Our solution to this is use role models to pose the description of a project within the multiple perspectives of typical constituents of a multimedia project, such as a client, a project manager, a programmer, and a designer.

However typical these multiple perspectives may be in the description of a multimedia project and its associated problems and solutions, there is a distinct, yet commonplace effect that occurs when this information is presented by more than one source: these voices collectively pose descriptions that are inconsistent and often quite contrary. Translating and even mediating these differences is an important task-defining skill that we want to provide an opportunity for practice.

Finally, no matter how much attention is paid to issues of completeness and consistency of the task description, there are bound to be some aspects of this information that are not accurately or reliably stated. Anticipating this problem and analyzing the information carefully should be part of the strategic skills of task definition.

With these issues in mind, we have posed the overall task description in a series of separate communications composed of textual and graphical descriptions characteristic of the usual roles that are assumed in any multimedia project. We believe this is an appropriate mental model of the complex and varied description of a multimedia production

For example, here are brief excerpts from the "dialogs" that are presented to the student-producers for the third project. These dialogs appropriately exploit multimedia resources to communicate information and objectives through text, graphics, animation, digital video and sound. We represent them in this paper in excerpted form only using text and graphics. Note that these dialogs distinguish between information that is perceived as descriptions that resemble objectives or requirements as well as suggestions which require additional mediation by the student-producer.

With some work in HyperCard and graphics under your belt, I think you are ready for this project: an animated corporate logo. I have discussed this project with the client, a high-tech company that wants a corporate logo to use on video and TV. It will use the name of the company as the logotype, so it is pretty straightforward. Here are the details from my vantage point.

Project Description: The client is a new company and they need to establish image recognition for the company name: Integrated Optics. They would like the initial letter of each word in the company name (I and O) to move to the center of the screen in a five to ten second animation sequence and then have the remaining letters of each word appear from left to right as if someone were typing the letters on-screen... Consult with the client for these suggestions and their description of the project. Then check with the Designer and Programmer for some advice on this project as well. For additional information, check the Animation Tutorial in the Knowledge Base, especially the part on using Page Flipping which I think is the easiest method.

I assume the Project Manager has already given you an idea of this project, so I will provide additional description of the project, but also make some suggestions for visual and sound effects to give it some pizzazz!

Project Description: As you probably know, we are a startup venture and do not yet have a corporate logo, so part of your job is to create an animation for using corporate video and TV spots, and the other part is to create an appealing logotype based on our initial specs. It should look good on screen and in print....

The primary goal of a logo is to remember the name of the company -- and what the company represents, not just the product or service, but the image and the qualities it wants to be characterized by. Logotype Design Suggestions: To have objects appear to come towards you, they not only become larger as they come closer, the spaces between each appearance of the letter also becomes larger. This is a rule of perspective. Recall the famous image of a railroad track to illustrate this principle. The horizontal railroad ties both become larger and appear to be farther away from the prior one as they move away from the horizon and "move" forward. So each "rail road tie" in essence may stand for a single card in animation.

The corporate logo project will provide you an opportunity to create your first card-flip animation. According to the project manager and client dialogs, you're to animate the letters "I","O" and "/" for the Integrated Optics corporate logo. One method of simulating movement is to draw the same graphics on successive cards, offsetting their locations by a fixed factor; as the cards are replayed rapidly, the graphics will appear to move across the screen

One method of rapidly displaying a series of cards is to construct a Repeat Loop within a handler. In the provided example stack, a repeat loop is defined in the transparent button covering the entire background layer; review the script and use it as a guide for your own animation.

According to the project manager and client, the "I" and "O" of Integrated Optics are to move from the edges of the screen diagonally toward the center. Once you have selected the font, style, and size; place each letter at its starting point. Select and copy the entire card, create a new card and paste the previous image from the clipboard. Use the existing location of the graphics as the starting point for repositioning the characters closer to the center of the card. Repeat the procedure until the characters are position as desired. On the last card, align fields adjacent to the "I" and "O". Use the Type command to simulate the keying of the remaining letters into each field. You'll find the type command illustrated in the provided example stack.

The customer has requested sound to accompany the animation, from our stock inventory we've mixed and digitized an audio clip labeled "Logo Sound". To invoke the sound in your script, use the Play command followed by the name of the sound resource in quotes. A new stack containing only the sound resource has been provided for your use, it's labeled "Empty Stack".

Tutorials

Our general strategy is to emphasize target, or higher level skills, but, in the beginning, these are presented in situations where performing the skill is simplified and/or in situations where the learner is assisted or provided with lower level support. (Burton, Brown, & Fischer, 1984).

We assume that most students are unable to do the projects assigned to them without additional help. Help is provided by easy access to tutorials, by consulting with members of the group, or with available experts, or by reading available reference materials. In this setting, provided the gap is not perceived as too large, students learn to locate information needed to solve problems, rather than feeling that they must master large amounts of information before they can begin to solve problems. (aside on reading vs working with software???)

We are slowly converting textual presentation of concepts to situations that engage learners in activities from which they abstract the nature of these concepts. We can ensure the formation of such abstractions, by asking learners to respond to questions, analyzing their answers, and directing them to remedial pieces if they fail to form proper concepts. (This approach may work well with students with limited language skills. Current classes include many students whose first language was not English.)

One of the most difficult tasks for novices is converting an abstract description of a problem to a script that solves the problem. The novice lacks understanding of programming, lacks knowledge of specific commands and program structures, and has great difficulty seeing the relation between the problem statement and the program. Some research on understanding programming (a very procedural task) (Bonar and Cunningham, 1988) proposes to get users to write English-like plans as a first step in programming. The steps of these plans are then matched with individual commands. We are experimenting with a similar system that works with HyperTalk, the programming language of HyperCard.

The tutorials of the current version of the project are presented as hypermedia materials. Suggestions are provided about the use of these materials, but students are free to use the materials as they wish. We know that students use these materials very differently, but we have not studied these differences.

We are currently reorganizing tutorial information as a network of knowledge, as suggested by Lesgold. (Lesgold, 1988) The network includes both declarative and procedural knowledge, and can also be differentiated as cognitive and metacognitive knowledge. An important component is knowledge about mental models of the different systems of the multimedia environment. Payne's (1993) argument that mental models for computers as artifacts, must include yoked models of .... and ...

Storing knowledge as a network makes it possible to use different mechanisms for determining when to present information and to alter the mechanism for presenting information.

For example, where learners are more capable of effectively managing their learning activities, we prefer to have them exercise and strengthen these skills. Of course individual student's ability to manage learning varies widely. We are considering the use of an intelligent tutor to coach those users who exhibit ineffective learning strategies. Students in our program have the greatest difficulty learning programming skills. Without an intelligent programming tutor it seems almost certain that they will waste time and experience major frustrations that will severely impede their learning momentum.

Our limited resources, up to this time, have been spent on exploring how to develop an environment for teaching multimedia development. We feel that the goals described above are appropriate as a framework for future development. Our next step is to modify the current system into a more flexible form that will support further experimentation and modification. We then hope to shift our attention to analyzing and improving the performance of the system. In future work we hope to attend to individual differences, to explore variations on the sequencing of different kinds of information, to work on assisting learners to develop better learning strategies.

Even though we have not as yet collected systematic data on the performance of the current system, we are convinced by informal evaluations that the current system is much better than our earlier approaches to teach this subject. Using the time when the group is together in the lab to facilitate interaction among members of the group we believe accounts for much of the observed difference. We feel that the ICM approach to programming improved performance in this area, but large changes depends on getting coaching to students at the moment when they are experiencing difficulties.

Bereiter, Carl. 1990. Aspects of an Educational Learning Theory. Review of Educational Research. (60), 4, pp. 603-624.

Bonar, Jeffrey and Cunningham, R. 1988. Bridge: Tutoring the Programming Process. In J. Psotka, L. Massey, and S. Mutter (Eds) Intelligent Tutoring Systems: Lessons Learned. Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers.

Brewer, W. 1987. Schema versus mental models in human memory. In P. Morris. Modeling Cognition. New York: John Wiley & Sons Ltd.

Brown, J., Collins, A., and Duguid, P. 1989. Situated Cognition and the Culture of Learning. Educational Researcher, (18), 1.

Burton, R., Brown J., and Fischer G., 1984. Skiing as a Model of Instruction, In Barbara Rogoff and Jean Lave (Eds.) Everyday Cognition: Its development in social context. Cambridge, MA: Harvard University Press.

Carlson, David and Ram, Sudha. 1990. HyperIntelligence: The Next Frontier. Artificial Intelligence and Language Processing (33), 3.

Clements, Douglas and Nastasi, Bonnie. 1988. Social and Cognitive Interactions in Educational Computer Environments. American Educational Research Journal (25) 1, pp. 87-106.

Collins, Alan, Brown, J.S., and Newman, Susan. 1989. Cognitive Apprenticeship: Teaching the Crafts of Reading, Writing, and Mathematics. In Lauren B. Resnick (Ed.), Knowing, Learning, and Instruction: Essays in honor of Robert Glaser. Hillsdale, NJ: Erlbaum.

Lesgold, Alan. 1988. Toward a Theory of Curriculum for Use in Designing Intelligent Tutoring Systems, In Alan Lesgold and Heinz Mandl (Eds.) Learning Issues for Intelligent Tutoring Systems. New York: Springer-Verlag

Norman, Donald. 1983. Some Observations on Mental Models. In Dedre Gentner and Albert Stevens (Eds.) Mental Models. Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers.

Norman, Donald. 1993. Cognition in the Head and in the World: An Introduction to the Special Issue on Situated Action. Cognitive Science (17).

Payne, Stephen. 1993. Artifacts and Mental Models. In Yvonne Rogers, Andrew Rutherford, and Peter Bibby (Eds.), Models in the Mind. London: Academic Press.

Rogers, Yvonne, Rutherford, Andrew, and Bibby, Peter. 1992. Models in the Mind. London: Academic Press.