Jabe Wilson, Multimedia Group, Kodak Imaging Research Centre, London College of Printing and Distributive Trades, London, UK.
Jabe Wilson
Multimedia Group, Kodak Imaging Research Centre, School of Media, London College of Printing and Distributive Trades, Back Hill, Clerkenwell, London EC1R 5EN.
Abstract
This paper describes how a model of interactive multimedia design is being developed along with support for design process at the London College of Printing and Distributive Trades. This is involves considering models of the design processes of film, theatre, graphic and industrial design from the perspective of human-computer interaction (HCI). The paper sets out the research which is producing a theoretical framework incorporating artist-design[1] practice (TFAD).
Keywords: Interactive multimedia, human-computer interaction, artist-design, visual form.
1: Artist-design in software development: Creating a sympathetic theoretical framework.
Presenting software in a meaningful form at the computer screen is a difficult task. This is the purpose of the user-interface (multimedia or otherwise), but the demands of its creation go beyond the realms of software engineering. As Brenda Laurel [Laurel (1986)] pointed out the meaningfulness of our interactions with computers are partly governed by our degree of engagement with them. She showed that awareness of the principles of theatre can help the creation of engaging interfaces. Aspects of film grammar have also been used to structure the design of multimedia at MIT [Hodeges and Sasnett (1990)]. Artist-designers have long been employed in the development of computer software, such as the Xerox Star the forerunner of the Apple Macintosh [Verplank (1988)].
The multi-disciplinary nature of such development work can be a problem in itself. To overcome such problems common terminology and mutually understood working practices are needed. Inherent difficulties are often exacerbated by engineering-design traditions and working practices dating back to before the advent of interactive computing. Such organisational structures can offer resistance to new working practices [Grudin (1991)]. One way of addressing this problem is to present the rationale behind such artist-design working practices in a manner that can be understood within the existing traditions of software development.
The competing needs of software engineering and user-interface design within software development have been investigated by HCI research. Although HCI theory is drawn from many areas and disciplines there has been comparatively little work looking at the practices of artist-designers. The design process required to instantiate the plans created in the early stages of software development could well benefit from such work [Madsen (1994)]. One consequence of the lack of awareness of design processes is the fact that in practice user-interface designers have not adopted the methods put forward by HCI theorists [Bellotti (1988)].
HCI has developed theoretical frameworks for several reasons:
The advantage of artist-design contributing to the development of HCI theoretical frameworks will not simply be to the benefit of HCI. By contributing in this way a greater understanding of artist-design processes can be expressed and understood within the predominantly engineering-design based software development process. This offers for the artist-designer potentially both greater enfranchisement within the software development process and the support of theoretical investigations into issues which directly concern artist-designers in their work.
2: What is the role of artist-design in user-interface development?
Artist-design can play an important part in the work of instantiating the plans created by the early stages of software development. There is no necessary link between the form taken by the user-interface on the screen and the functions of the software it triggers into action [Crampton-Smith and Tabor (1994)], because the display on the computer screen is controlled by separate software code to that which provides the functions [2]. In the past there have been two major conventions dictating the form taken by the user-interface we see on our screens: either user interfaces whose form mirrors that of software code as seen by programmers, borrowing form from the most direct, if inappropriate source (i.e. UNIX or MS-DOS), or the Windows, Icons, Menus, Pointer (WIMP) convention (Mac-OS; MS-Windows; X-Windows) which grew out of the work carried out at the Xerox Palo Alto Research Centre, being the work of inspired individuals many of whom were artist-designers [Verplank (1988)].
HCI theoretical frameworks have had little to say regarding artist-design process. This is surprising considering their contribution to a paradigm of user-interface with such undoubtable success (WIMP). We must also look at artist-design outside of the software development process. It has been said that the very nature of software promotes `semantic crises' [Crampton-Smith and Tabor (1994)], which cause the loss of understandable codes of form from the workplace often resulting in difficulty in use of the software. How can user-interface designers successfully cope with these semantic crises? There are examples of artist-design disciplines dealing with things which lack structural constraints in the way software does, modernist architecture and film being two such cases [Crampton-Smith and Tabor (1994)]. Whereas modernist architecture failed to create a new code of forms, film developed its new codes through the innovation of individuals. In the case of user-interface design we cannot wait for such innovation. We need good design now. We need practical guidelines for training future user-interface designers and of informing and supporting their current practice [Winograd (1994)]. The development of a better understanding through a theoretical framework such as TFAD is one way of supporting the work of user-interface designers.
3: The implications for Human-Computer Interaction research
The work of artist-designers in user-interface development seems to address a fundamental issue, that of providing the form available to those intending to use the software. This suggests a clear cut area for investigation by HCI research: the role of form in our communication, perception, cognition and interaction. There are already bodies of knowledge which provide a firm basis for such an investigation and the development of a theoretical framework around artist-design practice (TFAD). These disciplines will be discussed in the next section.
There is another reason to undertake such an investigation. From a purely practical point of view there is a need to consider how to improve the effectiveness of the instantiation stage of software development. Software development can be seen as consisting of two stages, the first stage involves planning what the software should do, and the second involves instantiating that design in working software. There is a great quantity of research which has looked at the planning stage which would be well supported by more work looking at the instantiation stage.
3.1: The theoretical basis of TFAD
User-interfaces have no necessary form. This is an important but easily overlooked fact. If the form is arrived at without due consideration it will cause great problems for those who have to use the software. Research work which has concentrated on the creation of meaningful forms through artist-design has been successful to date [Gaver (1989)]. Where software replaces existing tools in the workplace removal of familiar visual forms from the tools can adversely affect the work of those using the new software. This is because the forms taken by previous tools may have served functions which were not consciously acknowledged by those using them until they have been lost [Hutchins (1994)].
The visual forms which help create the engagement described by Laurel and make up the grammar of Hodges and Sasnett, can also be created by artist-design, but there are other basis for the study of meaning and visual form. Theories of the role of form in our work and day to day activities (our communication, perception, cognition and interactions) must be considered. TFAD is drawing from various sources to develop its framework. Semiotic theory is useful in considering how form is understood in terms of codes [Peirce (1933), Corner and Hawthorn (1980)]. Design theory helps in looking at the nature of artist-design processes and how they differ from the engineering-design of the software development process [Lawson (1990), (1994)]. Valuable information and unexpected insights can be gained from considering psychological evidence of the role of form in our mental processes from Gestalt psychology [Köhler (1947)], direct perception [Gibson (1979)], and theories of meaningful form [Uexküll (1957)]. Such knowledge can then be utilised for the benefit of user-interface design.
3.2: Understanding the design process
The emphasis of Human-Computer Interaction has undergone a major shift in the last ten years. Investigations have moved from looking at individuals working at a single computer to considerations of the whole social environment surrounding the activity [Greenbaum and Kyng (1991)]. Anthropologists have noticed within groups engaged in similar work or activities that there exist shared language forms, common actions and practices. Such a group is said to make up a "community of practice" [Anderson (1983)]. Great attention has been placed on designing computer systems for particular communities of practice. This new social awareness can be extended to consideration of the communities of practice engaged in developing these computer systems themselves [Olson (1994)].
The development of a computer system and its user-interface consists of many different skilled individuals [Grudin (1993)]. Many HCI frameworks have not specified whom they are addressing within the development process and in practice the proposals put forward have not been practical because they have addressed more than one community of practice at once. One distinction that has been disputed in some HCI research is that between the functionality and the user-interface [Norman (1986)]. The workings of the user-interface must influence underlying functions to some degree, but within software development the stages at which user-interface and software functions are decided are often distinct if not independent and consisting of different communities of practice [Grudin (1991), (1993)]. There is a case for suggesting that the organisational structure of software development should change so that the development of functions and the user-interface are the concern of a single community of practice [Bødker (1991)]. However, there is also a strong case for trying to develop methods which can cope with the way software is developed in the industry as it stands. We must therefore be specific about who we are addressing when our research suggests new design procedures. If a procedure is to be shared between more than one community of practice, we have the extra consideration of how members will communicate when carrying it out. Therefore it is best to avoid procedures that are shared between different communities of practice (for instance a procedure used by computer programmers for choosing "objects" in object-oriented programming would not be appropriate for choosing interface forms by artist-designers).
The design of the software code is one major task at the instantiation stage of software development. This consists of writing code which successfully provides the required functions [Pressman (1992)]. This particular work has its own pitfalls and difficulties, but since this community of practice falls within traditional engineering-design work practices it is already well supported by software engineering methods. The other task of the instantiation stage is that of creating the user-interface (the buttons, menus, sliders, dialogue boxes and icons and their respective layout corresponding to the functions provided by the software). This community of practice is often not well integrated with the traditional working practices of engineering-design. It is this group that the procedures drawn from TFAD are intended to support.
4: Can TFAD support the design process?
In order for a theoretical framework to support design practice, procedures which operationalise the framework's findings must be drawn from it. TFAD's procedures are intended to provide the structural design constraints currently absent from software and user-interfaces. The aim is to focus attention on matching the behaviour of the software to appropriate forms. Initial research results indicate that the following procedures are necessary:
1. Characterise the behaviour of the software. This will have already have been done in the previous planning stage of the development process [Pressman (1992)].
2. Distil this information into a usable state. A useful way of doing this is to produce a list of tasks or scenarios which explicitly set out a detailed plan of the possible dialogue between user and system [Carroll & Rosson (1992)].
3. Either gather candidate forms for consideration directly from these documents [Andersen (1990)], or have subjects act out scenarios of how people might use the software and analyse the participants' actions and language from videotape [Muller et al (1992)].
4. The result of this analysis consists of a list of forms which share the behaviour of the software (as a paper folder might share some of the behaviours of a computer desktop folder). Those forms which have the most appropriate behaviours and the least inappropriate ones should be kept for further consideration and possible inclusion in the final design.
The whole procedure is intended to provide the user-interface designer with a set of forms each with a collection of scenarios describing possible and likely behaviours for an object possessing this form [Wilson (in press)]. Each software behaviour in the computer system should have a corresponding set of candidate forms. Even if the final interface does not include forms from those sets produced by the procedure, the act of evaluating and rejecting them should have produced a greater appreciation of the design task than might otherwise be the case.
5: Conclusions.
Software development needs to be able to learn from artist-design be it in film, theatre, graphic or industrial design. A central thread which runs through all these disciplines is a concern for the use of visual forms. Visual form makes up the grammar of film, enables theatrical style engagement with the user-interface and is the primary medium for graphic and industrial design. Greater understanding within HCI of the role played by visual forms in allowing comprehension and successful interaction with the user interface may facilitate the integration of artist-design and software development. In this way it is hoped that a dialogue may be built between the two disciplines. An improved dialogue should also improve work practice cohesion.
It should be emphasised that to avoid divorcing theory from practice, a criticism which was levelled at artist-design theory in the past [Alexander (1964)], this research is developing its framework and procedures closely with practice via the use of case studies. Bibliography.
Alexander, C. (1964).Notes on the Synthesis of Form. Cambridge, MA.: Harvard University Press.
Andersen, P. A. (1990). A Theory of Computer Semiotics. Cambridge University Press, Cambridge, England.
Anderson, B. (1983). Imagined Communities. London: Verso.
Bellotti, V. (1988). Implications of Current Design Practice for the Use of HCI Techniques. In Jones, D & Winder, R. (Eds.) People and Computers IV, pp. 13-34
Bødker, S. (1991). Through the interface: A human activity approach to user interface design. Hillsdale: New Jersey. Lawrence Erlbaum Associates.
Brown, J. and Duguid, . (1994). "Borderline Issues", In Human-Computer Interaction, Vol. 9., No. 1.
Carroll, J. M., Mack, R. L., and Kellogg, W. A. (1988). Interface metaphors and user interface design. In Helander (ed.) Handbook of Human-Computer Interaction. Elsevier Science Publishers. B. V. (North Holland) 1988, pp.67-85.
Carroll, J. M., Kellogg, W.A., & Rosson, M.B. (1991). The task-artifact cycle. In J.M. Carroll (Ed.), Interfacing thought: Psychology at the human-computer interface. (pp. 74-102). New York: Cambridge University Press.
Carroll, J.M., & Rosson, M.B. (1992). Getting around the task-artifact cycle: How to make claims and design by scenario. ACM Transactions on Information Systems, 10, 181-212.
Corner, J. and Hawthorn, J. (eds.) (1980). Communication Studies. London: Arnold.
Crampton-Smith, G. and Tabor, P. (1994). Response to the paper "Borderline Issues", In Human-Computer Interaction, Vol. 9., No. 1.
Gaver, W. (1989). The Sonic Finder: An interface that uses auditory icons. Human-Computer Interaction. Vol. 4, No 1.
Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Boston Mass.: Houghton-Mifflin.
Greenbaum, J., and Kyng, M. (1991). Design at Work - Cooperative design of computer systems (pp. 169-196). Hillsdale, NJ: Lawrence Erlbaum Associates.
Grudin, J. (1990). The computer reaches out: The historical continuity of interface design. Proceedings of CHI '90: 621-268.
Grudin, J. (1991b). Systematic sources of suboptimal interface design in large product development organisations. Human-Computer Interaction, 6, 147-196.
Grudin, J. (1993). Obstacles to Participatory Design in Large Product Development Organisations. In D. Schuler and A. Namioka (eds.) Participatory Design: Principles and Practices. Hillsdale, NJ: Lawrence Erlbaum Associates.
Hodeges, M.E. ,and Sasnett, R.M. (1990)Design for the new medium, in Multimedia computing, case studies from MIT project Athena. MIT Press.
Hutchins, E. (1994). Cognition in the wild. Cambridge, MA: MIT Press.
Itten, J. (1975). Design and Form: The Basic Course at the Bauhaus and Later. Van Nostrand Reinhold.
Köhler, W. (1947). Gestalt Psychology. New York: Liveright.
Lawson, B. (1990). How Designers Think: The design process demystified. Oxford, England: Butterworth Architecture, Butterworth-Heinmann Ltd.
Laurel, B. K. (1986). "Interface as Mimesis". in User Centred System Design: New Perspectives on Human-Computer Interaction. D. A. Norman & S. W. Draper (Eds.) Hillsdale, NJ: Lawrence Erlbaum Associates.
Lawson, B. (1994). Design in mind. Oxford, England: Butterworth Architecture, Butterworth-Heinmann Ltd.
Madsen, K. H. (1994). A Guide to Metaphorical Design. In Communications of the ACM, December 1994, Vol.37, No.12.
Muller, M. J., Wildman, D. M. and White, E. A. (1992). Games and other techniques for group design of user interfaces. Tutorial at CHI '92 (Monterey Calif., Apr. 1992).
Norman, D. A. (1986) Cognitive engineering. In D. A. Norman and S. W. Draper (Eds.) User Centred System Design: New perspectives on human-computer interaction. Hillsdale, New Jersey: Lawrence Erlbaum Associates.
Olson, G. M. (1994). Response to the paper "Borderline Issues", In Human-Computer Interaction, Vol. 9., No.1.
Peirce, C. (1933). Collected Papers,Vol. II, C. Hartshorne and P. Weiss, eds. Cambridge MA: Harvard University Press.
Pressman, R. S. (1992). Software engineering: A practitioner's approach. (Third Edition). Singapore: McGraw-Hill, Inc.
Uexküll, J. von. (1957). A Stroll Through the Worlds of Animal and Men. In, Schiller, C.H. (Ed.) Instinctive Behaviour. New York: International University Press, pp.5-80.
Verplank, W.L. (1988). Graphic challenges in designing object-oriented user interfaces. In (Ed.) M. Helander. Handbook of Human-Computer Interaction, p.365-376.
Wilson, J.G. (in press). Metaphors we design by: language, visual arts and electronic document design. Proceedings of Electronic Imaging and the Visual Arts. July 1995.
Winograd, T. (1994). Response to the paper "Borderline Issues", In Human-Computer Interaction, Vol. 9., No. 1.
For more SYNDICATE SPEAKERS For other SYNDICATES |