Why Usability Problems Still Occur Christelle Harkema TU/e, Industrial Design [email protected]

ABSTRACT

RESEARCH PROBLEM

Gaining information to reduce uncertainty about usability can be done by using usability techniques during the development process, in particular in the design phase. Since Gould [11] introduced the key principles of Designing for Usability many more principles, methods, techniques and standards have been introduced to improve the process of user-centred product design, for example Nielsen [19], Norman [20] and Gulliksen [12] and new techniques are still being developed within different academic fields, among which the field of HCI [23,24].

New product development has rapidly changed over the last few decades. The fierce competition on the global market urges developers to shorten the time-to-market in order to be first on the market to keep their market position and to keep making profit [4]. To be competitive new product functionality is added, because; at least the same functionality as others should be offered, the customer requires new functionality and new functionality gives a competitive advantage. All this new functionality increases the product complexity on several levels i.e.: within the product, the user-product interaction and the interaction among products [21]. This complexity influences the product usability by lowering effectiveness, efficiency or satisfaction of product use [3]. Product usability is: “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” [15].

Summarised, on the one hand the research problem is the existing and ever increasing number of usability problems and on the other hand the availability of existing and recently developed usability techniques. So why do these usability problems still occur despite all the available usability techniques? The aim of this PhD research is to deliver A) insights into why usability problems occur despite existing usability techniques and B) a practical application of these insights to support designers in making better usability decisions. The relevance of improving usability decision making lies in less usability problems. A reduction of usability problems is of importance for industry as it will reduce the after sales costs and will improve brand image. Moreover the economical life span of usable products is longer corresponding with a lower environmental impact.

Consumers experience a growing number of usability problems while on the other hand there are many existing and recently developed usability techniques available. This PhD research aims to deliver: A) insights into why usability problems occur despite existing usability techniques and B) a practical application of these insights to support designers in making better usability decisions.

This increased product complexity leads to a more complex user-product interaction [8] and this results in many usability problems which users experience when using consumer electronics products [1, 24, 26]. These usability problems represent a mismatch between the actual and intended product use [20, 25]. A consequence of them may be user dissatisfaction resulting in complaints [7]. The number of complaints and product returns in consumer electronics are rising [22] and at least half of them can be allocated to non-technical failures [5], i.e. the returned product complies with its technical requirements. Usability problems are part of this group of failures. Most of the user complaints about products can be traced back to design decisions made in the development process [22], so the sources of usability problems can therefore be found in usability related decisions. Decision making is a conscious choice between at least two alternatives [6] and uncertainty is always a major obstacle, because lack of information (uncertainty) will block or delay taking action [16].

THEORETICAL BACKGROUND

Usability problems are a result of a mismatch between how the designer intended the product use and how the user actual uses the product based on his expectations [Norman 1986]. The designer’s expectations about the humanproduct interaction can be based on various aspects, e.g.; everyday life experience, previous project experience, expert opinion, or (well-proven) information [14]. The actual use depends on e.g.; the user goals, context, expectations, mental models, and situation [15, 20, 25]. Making ‘correct’ decisions about the intended use is crucial in developing usable products, as most of the usability problems can be traced back to ‘incorrect’ decisions in the design process [22]. Design decisions take on ill-defined problems which have to be structured and solved [9]. In making such decisions one needs to be supported with information and knowledge which improve the quality of the design decisions [27]. One of the major factors influencing the quality of design decisions is uncertainty, probably caused by insufficient design information [2]. This

can be supported by the definition of Galbraith about uncertainty: “Uncertainty is the difference between the amount of information required to perform a task and the amount of information already possessed by the organisation” [10]. This definition of uncertainty should not be confused with perceived uncertainty, the feeling of uncertainty of the decision maker. Uncertainty can be reduced by collecting the missing information about usability. This information can be gained by the application of usability techniques. Uncertainty is the starting point for this research to investigate whether the information required to make correct usability design decisions can be covered by existing usability techniques to reduce the uncertainty that causes usability problems. Uncertainty literature describes different types of information that is required for correct decision making [28, 18]. The earlier described research problem raises the presumption that available usability techniques are not able to generate all the types of information required for correct usability decision making. So altogether the first question is (research aim A): To which extent are usability techniques able to reduce uncertainty by generating the required types of information in order to make ‘correct’ usability decisions? APPROACH AND WORK PERFORMED SO FAR Research Aim A: Why usability problems still occur?

To analyse whether existing usability techniques cover the uncertainty that causes usability problems, the following questions were or will be answered: 1) Usability techniques: What types of information are generated by the available usability techniques? 2) Usability problems: What types of information were lacking in the usability decision making process that lead to actual usability problems in the field? 3) Comparison: Is there a (mis)match between the required information to prevent usability problems and the generated information by usability techniques? To make this comparison, an uncertainty scale based on literature was defined [13]. This scale addresses the types of information that are generated by usability techniques on and that are required for usability decision making. The type of information in this context encompasses the level of information and the awareness about the level of information. For example, not knowing the usability variables is more uncertain than not knowing the values of these variables. A decision maker being not aware of missing information is even more uncertain. Subsequently the first step of the analysis was to map the existing usability techniques on this uncertainty scale (What types of information are generated by the available usability techniques?). The approach for mapping those techniques is also described in detail by Harkema [13]. Mapping the usability techniques is done by product designers and

developers in the fields of consumer electronics products, consumer products and professional products. The second step is about mapping the usability problems on the uncertainty scale (What types of information were lacking in the usability decision making process that lead to actual usability problems in the field?). This is done based on usability problems of various products and companies. Decision makers of the involved products identified the types of information that were lacking at the moment of decision making. The third step, comparing both mappings, will give the insight whether or not existing usability techniques cover the uncertainty that causes usability problems. Based on these comparison four scenarios are possible for each information type, see figure 1:

Figure 1: Four scenarios of possible outcomes applicable for each information type

Scenario A implies that there are usability techniques available but for some reason they do not generate the required type of information for making proper design decisions. From scenario B we may conclude that the available usability techniques support the developer adequately resulting in ‘correct’ decisions, preventing usability problems. In scenario C no usability techniques are available but usability problems exist, motivating the development of specific usability techniques addressing this specific type of required information. The last scenario D, where no usability techniques and no usability problems occur, show that no techniques are needed since there are no usability problems. Our presumption that existing usability techniques are not able to generate all the types of information required for correct usability decision making indicates the prevalence of scenario C. Research Aim B: Practical application of insights

The identified scenario will determine the further approach for making this insight operational to support designers in making better usability decisions. During this Nordi CHI doctoral colloquium I wish to discuss the approach of making my research insights operational for decision makers in the design process. Experts from the (Nordi) CHI community and fellow PhDs could inspire and support me with their input and contributions in pursuing this research aim B.

BIOGRAPHICAL NOTE

Christelle Harkema studied Industrial Design Engineering at the Delft University of Technology. In 2005 she gained her Masters Degree with a specialisation in ergonomics and research. After her education she worked for two years as a usability expert at Indes, a Dutch design office. To deepen her speciality on usability she started in 2007 her PhD research on Design for Usability at the Eindhoven University of Technology. In the first 3 years of her PhD she performed various case studies to gain insights into why usability problems occur despite existing usability techniques. Usability problems are mostly found during the human product interaction with the user interface of products. First findings support the presumption that available usability techniques are not able to generate all the types of information required for correct usability decision making. In the last year she will focus on validating these results and making a practical application of these insights to support designers in making better usability decisions. ACKNOWLEDGEMENTS

The author gratefully acknowledge the support of the Innovation-Oriented Research Programme ‘Integrated Product Creation and Realization (IOP IPCR)’ of the Netherlands Ministry of Economic Affairs. REFERENCES

1. Accenture. Big Trouble with No Trouble Found: How Consumer Electronics Firms Confront the High Cost of Customer Return. (2008). www.accenture.com. 2. Beheshti, R. Design decisions and Uncertainty. Design Studies 14, 1 (1993), 85-95. 3. Bly, S., Schilit, B., McDonald, D. W., Rosario, B., Saint-Hilaire, Y. Broken Expectations in the Digital Home. In proc. CHI 2006, ACM Press (2006). 4. Brombacher, A.C. Designing Reliable Products in a Cost and Time Driven Market: a conflict or a challenge. Eindhoven University of Technology (Inaugural lecture), 2000. 5. Brombacher, A.C., Sander, P.C., Sonnemans, P.J.M. and Rouvroye, J.L. Managing Product Reliability in Business Processes ‘Under Pressure’. Reliability Engineering and System Safety 88, (2005), 137-146. 6. Brunsson, N. The irrationality of Action and Action Rationality: decisions, ideologies and organizational actions. Journal of Management Studies 19, 1 (1982), 29-44. 7. Cho, Y., Im I., Hiltz, R. and Fjermestad J. The Effects of Post-Purchase Evaluation Factors on Online vs. Offline Customer Complaining Behavior: Implications for Customer Loyalty. Advances in Consumer Research 29, (2002), 318-326.

8. Cooper, A. The inmates are running the asylum: Why high-tech products drive us crazy and how to restore the sanity. Sams publishing, Indianapolis, 1999 9. Cross, N. Expertise in design: an overview. Design studies 25. (2004), 427-441. 10. Galbraith, J.R. Designing complex organizations. Addison-Wesley Publishing Company, Massachusetts, 1973. 11. Gould, J.D. and Lewis, C. Designing for Usability: Key Principles and What Designers Think. Communications of the ACM 28, 3 (1985), 300-311. 12. Gulliksen, J., Goransson, B., Boivie, I., Blomkwist, S., Persson J., Cajander, A. Key Principles for UserCentred Systems Design. Behaviour and information technlolgy 22, 6 (2003), 397-409. 13. Harkema, C.L.E., Sonnemans, P.J.M., Luyk - de Visser, I.M. Usability Techniques in the Design Process. A First Step in Relating Usability and Uncertainty. Submitted but for Proc. Nordi CHI 2010 > Rejected, ACM Press (2010). 14. Hasdogan, G. The role of user models in product design for assessment of user needs. Design Studies 17, (1996), 19-33. 15. ISO 20282-1. Ease of operation of everyday products. 2006. 16. Lipshitz, R. and Strauss, O. Coping with Uncertainty: A Naturalistic Decision-Making Analysis. Organizational Behaviour and Human Decision Processes 69, 2 (1997), 149-163. 17. Maguire, M. Methods to Support Human-Centred Design. International Journal of Human Computer Studies 55, 4 (2001), 587-634. 18. Modica, S. and Rustichini, A. Unawareness and Partitional information structures. Games and Economic Behaviour 27, 2 (1997), 265-298. 19. Nielsen, J. Usability Engineering. Academic Press, San Diego, 1993. 20. Norman, D.A. and Draper S.W. User-Centred System Design – New perspectives on Human-computer interaction. Lawrence Erlboua Associates, New Jersey, 1986. 21. Norman, D.A. The invisible computer. Cambridge, MIT press, 1998. 22. Ouden, den P.H. Development of a Design Analysis Model for Consumer Complaints – Revealing a New Class of Quality Failures. PhD thesis, Technical University Eindhoven, Eindhoven, 2006. 23. Proceedings of the 28th international conference on Human factors in computing systems, ACM Press 2010. 24. Proceedings of the 5th Nordic conference on Humancomputer interaction: building bridges, ACM Press 2008.

25. Suchman. L.A. Plans and Situated Actions – The problem of human-machine communication. Cambridge University Press, Cambridge, 1987. 26. Vinter, O., Poulson, P.M., Lauesen, S., Experience driven software process improvement, Software Process Improvement, 1996. 27. Wang, K.R.L., Tong, S., Eynard, D., Matta N. Design Knowledge for Decision-Making Process in a DFX Product Design Approach. In Global Design to Gain Competitive Edge: a Holistic and Collaborative

Approach Based on Computational Tools. Springer, London, 2008. 28. Wideman R. M. Risk Management, A Guide to Managing Project Risks and Opportunities. Project Management Institute, Upper Darby, USA, 1992.