Applying a Multi-Criteria Approach for the Selection of Usability Patterns in the Development of DTV Applications Kenia Sousa

Hildeberto Mendonça

Elizabeth Furtado

Universidade de Fortaleza Av. Washington Soares, 1321 Fortaleza, Ce, Brazil +55 85 34773079

Mentores Consultoria Ltda. Av. Dom Luis, 300, S.828 Fortaleza, Ce, Brazil +55 85 32424979

Universidade de Fortaleza Av. Washington Soares, 1321 Fortaleza, Ce, Brazil +55 85 34773079

[email protected]

[email protected]

[email protected]

ABSTRACT In this paper we describe a multi-criteria approach in which the execution of its steps integrated to a Software Development Process (SDP) allow the generation of the User Interface (UI) Definition Plan, which is an artifact used for UI design of software. This approach applies techniques from Operational Research (OR), and from Human-Computer Interaction (HCI), considering diverse criteria (functional and non functional requirements) that have an impact on the interaction design and using usability patterns, respectively. In this text, its main goal was to identify the order of attractiveness of a list of usability patterns for a certain interactive task of Digital TV (DTV) applications, thus allowing the selection of the most appropriate pattern in this new communication resource.

These criteria refer to various usability requirements (as usefulness, users’ experience, their context of use, etc.) as well as non usability requirements (as complexity, integration, scale, etc.). We believe designers need to follow a technique that guides them to consider multiple criteria that may have an impact on their

decision, while making such important decisions in a productive manner. The characteristics desirable from a technique to decrease the complexity of choosing among various options are the following: •

Consideration of previous analysis of existing solutions. Software organizations working with interaction design can prepare a list of usability patterns that can be used in specific design situations, such as: providing feedback, presentation layouts, navigation and selection of options, etc. Usability patterns represent best design solutions for known usability problems. One issue that arises is the following: How to choose the best usability pattern from a list of alternative patterns productively?

•

Implementation of strategies to evaluate these criteria by taking into account suggestions, needs, habits of different stakeholders (as users, designers, managers) involved in delivering successful products and services. Usually, there are many people involved in the project who want to make their opinion have an impact on the final result and such opinions can be controversial since the discussions may involve users’ representatives and professionals from the software organization. This fact raises other issues: How can we organize the project participants with different points of views to discuss and choose the best usability pattern? How to deal with the subjectivity from the users’ and the organization’s point of view in order to bring to one sole result?

Categories and Subject Descriptors H.5.2 [User Interfaces]: Interaction styles, prototyping, standardization, style guides, theory and methods, user-centered design.

General Terms Algorithms, Management, Measurement, Documentation, Design, Reliability, Experimentation, Human Factors, Standardization.

Keywords Usability Patterns, usability, multi-criteria decision-analysis.

1. INTRODUCTION In the last decades of HCI research, designers used to design UIs by defining a detailed look-and-feel of a system. More and more criteria concerning the interaction design (to understand the goals users want to achieve with the technology) are being took into account by designers during the requirement elicitation stage.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Conference’04, Month 1–2, 2004, City, State, Country. Copyright 2004 ACM 1-58113-000-0/00/0004…$5.00.

These issues motivated us to research existing techniques in order to help designers in the selection of usability patterns by considering various criteria and needs of different stakeholders. As a result of our researches, we found works, presented in section 7, such as [18], that evaluate options of usability patterns and most of these alternatives only focus on the usability criterion considering the users’ point of view, such as ease of use, etc.

Neither usability issues nor other stakeholders’ point of view are considered. This work is innovative in the sense that it integrates an approach from OR in a SDP in order to solve these mentioned issues for interaction design. This approach, called MACBETH (Measuring Attractiveness through a Category Based Evaluation TecHniques), supports collaborative decision making to be applied during interaction design [2]. It uses a Multi-Criteria Decision Aiding (MCDA) methodology that aims at helping decision-makers in understanding and learning a problem in order to make decisions. It considers both objective and subjective criteria relevant for the judgment of options through the use of a fixed scale (objectivity) and discussions in workshops (subjectivity). This methodology is in accordance to the idea that it is impossible to determine, only through a mathematical model, if a decision is good or bad; therefore, the quality and success of a decision are influenced by individuals and groups and by organizational, cultural, and pedagogical factors of the decision-making process [3]. MCDA is interested in helping decision-makers during the judgment of a problem. Therefore, it represents a methodology that decreases the subjectivity of the opinions of all the people involved in the decision-making process by using pre-defined values for their opinions in an iterative, collaborative and constructive manner. In this approach, several stakeholders with different knowledge background can participate, including project managers, programmers, software architects, usability engineers, designers, users’ representatives, etc. The evaluations of patterns and criteria are supported by meetings in which all of them present their opinion and discuss points of view until they reach a consensus. This approach can contribute with usability and productivity to the interaction design process. Concerning usability, this approach can improve the overall UI usability through the use of selected usability patterns from different points of view. In this text we will call the organization’s point of view as technological view aiming at bringing objectivity for the decision making process through the execution of specific steps that lead to an applicable solution. In addition, its application results in the generation of an artifact, called UI Definition Plan, which lists the usability patterns that are selected. We specified this artifact in order to make it useful as a formal document for users’ acceptance and also to be used by designers while prototyping. Concerning productivity, it has four main advantages: •

•

•

Structured technique – This multi-criteria approach has a structured technique with defined steps that helps designers in selecting usability patterns in an objective manner; Defined responsibilities – This approach allows the evaluation of the importance of non-functional requirements that have a direct impact on usability patterns from the users’ point of view, and it also allows designers to evaluate the usability patterns considering nonfunctional requirements from the technological view; Organized meetings – The decision-makers present their opinions in meetings organized by a facilitator who arranges the discussions toward a consensus with objectivity and;

Reliable algorithm – The use of a tool that processes an algorithm provides a rapid, mathematical, and reliable result. This paper is organized as follows: motivation to apply this approach, description of the MCDA approach; explanation of the application of this approach in a project for developing DTV applications; integration of MACBETH in a process; discussion of the potential contributions of this work to the HCI community with recommendations for future work; state of the art; and conclusion of this paper. •

2. MOTIVATION There are projects in software organizations that represent a new situation for its professionals. Picture a project where they need to develop an interactive system with the following constraints: its UI requires detailed analyses, its UI is essential for the system acceptance, it is intended for a certain user profile that needs special attention, and it needs to work in a peculiar environment or in a new technology. One example that is well suited in this scenario is the DTV, which is in its early research and development stages in Brazil, as well as in other countries in Latin America. Designing applications for DTV requires investigations about user experiences, investments in industrial design for bringing innovation in sophisticated interaction resources, imagination and creativity of the designers, among other factors. Some methods for UI design focus on user centered approaches [10] in order to observe users’ behaviors, their needs, habits, and so on. Users become more active and involved giving suggestions, making choices, etc. Other methods being conducted focus on technology-centered approaches [11] in order to help designers in a deep understanding of industry trends and emerging technologies. Designers identify alternatives of UI design either using their creativity or even existing ideas, expressed in usability patterns, for instance. Intel Organization [22] has developed a hybrid solution, which focuses on a methodology of starting with user problems and needs and translating these problems and needs into a series of working technologies and prototypes, standards, and eventually products in the marketplace. A matrix to each usage scenario describing working technologies that provoke barriers and barrier busters for each criterion (as end user, applications, network communication, devices, etc.) is generated in order to be considered in the prototyping phase. Nokia organization maintains roadmaps of the sorts of device and services that will come on stream over the next years [11]. We suggest organizations to create and maintain a UI Definition Plan of the technology that is available or will be available. Specifically, a plan can be made for each usability pattern also taking into account suggestions, needs, habits of different stakeholders (as users, designers, managers) involved in delivering successful products and services.

3. DESCRIPTION OF MACBETH As we mentioned previously, the approach we are applying is called MACBETH, and it follows the MCDA methodology that considers multiple criteria in order to choose the best option for a certain issue.

MACBETH facilitates communication among decision-makers through discussions over their opinion and points of views presented to solve a given problem. The decision-makers are the project participants, who benefit from the issue being analyzed, with the power to intervene in the construction of the solution, supported by a facilitator [1]. We suggested the decision-makers to be organized in two groups: (1) one that represents the technological view, with the responsibility to evaluate patterns in order to bring their knowledge and experience with patterns into the approach, and; (2) the other one that represents the users’ point of view and evaluates criteria in order to make users’ opinions a relevant aspect of the approach. The facilitator has an important role in this decision-making process (while identifying and evaluating usability patterns and criteria), especially in situations when the opinions differ substantially. The facilitator should not interfere in the judgment of the decision-makers, but he/she should provide an adequate atmosphere for their learning and decision-making process. The facilitator is responsible for letting decision-makers discuss; listing the multiple beliefs and opinions; solving possible arguments or problems; and leading decision-makers to reach a consensus among the different opinions. The facilitator and the decision-makers follow six main steps from MACBETH: identify and generate the patterns, select criteria for the project, group the designers to evaluate the usability patterns, group the users’ representatives to evaluate the criteria that have an impact on the patterns, view and analyze the result, and make changes, when necessary. These six steps are integrated into the Unified Process for Interactive systems, called UPi [20], described in the fifth section. In this process, system analysts elicit requirements, generate task models, and associate tasks to alternatives of usability patterns, which one of the associated usability patterns is selected by applying MACBETH, explained in the next section, and documented in the UI Definition Plan. This application will be supported by the M-MACBETH software that allows qualitative (e.g. subjective) judgments, and facilitates the participation of end-users in a decision-making process through its graphical resources. These resources are tables to visualize the association among criteria and bar graphs to illustrate the results of the decision making process.

4. SELECTION OF USABILITY PATTERNS USING MACBETH AND MMACBETH SOFTWARE The scenario in which we applied MACBETH to generate the UI Definition Plan was on the selection of a usability pattern to attend a usability requirement considered relevant in the project for the Brazilian Digital TV System. This requirement is “to provide guidance for all kinds of users while interacting with the system”. To meet this requirement, we proposed the task “navigate”, which allows users to navigate through different categories of TV programs and applications. According to UPi, during requirements, the system analyst maps the identified usability requirements to usability tasks (interaction tasks that affect the system overall usability and are related to usability requirements), which are in their turn, mapped to a list of usability patterns.

We have applied the mapping of usability requirements to tasks in order to facilitate the identification of usability patterns for interaction design. This mapping of usability requirements, tasks and patterns was initially adapted from Juristo’s usability levels [12] (usability attributes, usability properties, usability patterns, and architectural patterns). In this scenario, we involved four designers to be decisionmakers, who represented the technological view during the evaluation of the usability patterns, and eight project participants, who represented the user’s point of view to evaluate the criteria selected for the project, the first author of this paper was the facilitator. In order to consider the needs and expectations of the Brazilian people, these participants had constant contact with people with varied ages and social-economic constraints. As follows, we describe the six steps that supported their decision.

4.1 First Step – Identify / Generate Patterns The main goal of this step is to identify existing patterns (already designed, like existing patterns for websites) or to design patterns for new technologies, like DTV, Kiosks, etc. The designers researched existing Digital TV systems and identified four different types of usability patterns for the “access applications” use case, more specifically, the usability task “navigate”: menu with icons, retractable menu, fixed menu, and activated menu. According to [23]: Menu with icons allows users to select an item by selecting an image that is in a fixed location. Retractable Menu can be put aside and easily retrieved again. Fixed menu is an always visible menu at a fixed position. Activated menu appears at the mouse pointer location after the users click. Since there were no patterns available for DTV, they designed these four usability patterns in drawing prototypes using Photoshop.

4.2 Second Step – Select the Criteria After the patterns are identified or designed, in this step, the main goal is to define criteria to help in the analysis of the alternative patterns. The criteria can be non-functional requirements, such as usability, security, etc. The software organization can identify any criteria considered as important for the project being developed. Users need to be interviewed to select relevant criteria based on their needs. Criteria can be selected from a pre-defined list of criteria that can be prepared by the software organization as a corporate memory for interaction design. Such corporate memory can include the list of criteria that affect interaction design, in which each criterion is associated to lessons learned with its application in previous interaction design projects, including both points of view. To identify the criteria necessary to choose the most appropriate usability pattern in this case study, the Brazilian Government sent a formal documentation listing constraints for the Brazilian Digital TV System [7]. The selected criteria by project participants were: price, maintenance, performance, efficiency of use, ease of learning, ease of remembering, error rate, and subjective satisfaction. According to [17], efficiency of use allows users to be fast and productive while performing tasks. Ease of learning concerns the

UI being intuitive to novice users. Ease of remembering is concerned with the ability users may have to remember the UI after some time not using it. Error rate is related to the UI being reliable, thus, avoiding users to make mistakes. Subjective satisfaction is related to the UI being able to please and capture the attention of users. Since accessibility and social inclusion were mentioned by the Brazilian Government, we included these criteria in all of our discussions and decisions, but they were not included in this example to simplify the scenario.

4.3 Third Step – Evaluate Patterns The main goal of this step is to define a strategy to evaluate patterns from the technological view: judgment of alternative patterns considering the criteria selected in the previous step. The designers were organized in a meeting monitored by a facilitator, where they ranked the options (usability patterns) for each criterion of the “access applications” use case from the most attractive to the least attractive in the technological view. For instance, for the maintenance criterion, they ranked the options from the easiest to maintain (fixed menu - the most attractive), to the most difficult to maintain (icon menu – the least attractive). Then, they qualitatively evaluated the difference of attractiveness between the options for each criterion using a six-option scale that includes: extreme, very strong, strong, moderate, weak, and very weak (Figure 1). For instance, while the difference between fixed menu and requested menu is moderate, the difference between fixed menu and icon menu is very strong because the fixed menu is a lot easier to maintain than the icon menu.

Figure 1 – Judgment of options for maintenance

4.4 Fourth Step – Evaluate the Criteria The main goal of this step is to know, among the selected criteria, which ones are the most attractive in the users’ point of view. This evaluation directly influences the final results, but this evaluation does not affect the evaluation of patterns, done by the software organization. Therefore, this step can be done in parallel with the previous one. The users’ representatives were organized in a meeting monitored by a facilitator, where they ranked the selected criteria from the most attractive to the least attractive in the users’ point of view. Figure 2 partially illustrates the criteria ranked from the most important for the project (ease of learning), on the left of the figure, to the least important (maintenance), but only the first four criteria are presented for space reasons.

Figure 2 – Judgment of Criteria Then, the users qualitatively evaluated the difference of attractiveness between the criteria using the six-option scale mentioned in section 4.3. For each criterion, users compared it with all the other criteria, two at a time. One example of comparison between two criteria is the following: while the difference of importance between ‘ease of learning’ and ‘efficiency of use’ is very weak, the difference between ‘ease of learning’ and ‘error rate’ is weak because ease of learning is considered more important for the project than error rate in the users’ point of view. Each user has the chance to give an opinion in this evaluation and all of them discuss until they reach a consensus. That is exactly when the work of the facilitator is most important, which is to help users to decide the best evaluation for the criteria. The evaluation of the criteria has a direct impact on the final result because the pattern that is best evaluated for the criteria considered the most important for users is likely to be selected as the most appropriate for the UI.

4.5 Fifth Step – View and Analyze the Result The evaluation of the usability patterns (from the technological view) and of the criteria (from the users’ point of view) are now processed by the MCDA algorithm, implemented by the MMACBETH software. The main goal of this step is to produce a list of usability patterns from the most attractive to the least attractive, combining both points of view. Figure 3 depicts the different options in their order of attractiveness and their associated weight for the “access applications” use case, which is: 1) icon menu, 2) fixed menu, 3) requested menu, and 4) retractable menu. After the result is reached, either the users might change their opinions about the weights they provided for the criteria or the software organization staff about the judgments they made for the usability patterns.

4.6 Sixth Step – Make Changes If the result presented in the previous step is satisfactory for all the decision-makers, then there is no need to make changes. On the other hand, if any decision-maker is not satisfied, for instance, they feel that their opinion is not represented in the final result, then, they can propose changes. The possibility to make changes on the scores of patterns and weights of criteria can be made until the decision-makers are satisfied with the final result. This step leads to higher satisfaction of users since they are active participants of the decision-making process. In this step, to avoid an infinite loop, the project manager needs to balance the value of schedule, costs, and user satisfaction in order to decide to move on to the next activities of the process.

performance criterion, which is currently with the value 12.09, but it can vary from the values 11.62 to 13.02.

Figure 4 - The possibilities of weight change

Figure 3 – The overall result There are two main analyses that support the changes: analysis of options profiles (patterns profiles), which helps decision-makers to know which criteria most contributes for the appropriateness of each pattern; and sensitivity analysis, which identifies specific changes that can be made on the evaluation of the criteria in order to change the final result. Therefore, the analysis of options profiles is more general and lets the decision-makers free to make their changes. On the other hand, the sensitivity analysis is more direct by specifying which changes can be made to have a clear impact on the final result. First, the decision-makers can analyze the profiles of the options (patterns) by taking into consideration the criteria in order to identify how the changes in the option score can affect the final result. For instance, in our scenario for the “access applications” use case, the analysis of the profile of the ‘icon menu’ demonstrated that changes in the score of this option concerning three specific criteria (ease of learning, efficiency of use, and ease of remembering) have a greater impact on the overall result. So, it is useless to make changes in the score of the ‘icon menu’ for the maintenance or performance criteria, for instance. Second, the sensitivity analysis enables the verification of the change on the weight of a certain criterion necessary to swap the rank of two options in the overall result. For instance, in this scenario, to swap the rank of the options ‘icon menu’ and ‘fixed menu’ in the overall result, this analysis demonstrated that it was necessary to change the weight of the criterion performance to a specific value presented by the M-MACBETH software. The scores of the options and the weights of the criteria can be changed within a specific interval, which needs to be obeyed in order to maintain compatibility with the previous judgments. Figure 4 depicts, in red, the interval of changes of weight for the

All the options and decisions were documented by the facilitator and designers in the UI Definition Plan. This document starts by specifying the project, the problem to be solved and lists the participants of the workshops who influenced the final decision. The second section of this document lists the options of usability patterns for a specific task with name, description, and illustration. The third section lists the criteria considered and evaluated by users. The fourth section documents the selected pattern and explains the final decision. The last section is dedicated to formal acceptance of the decision with the signature of all participants. MACBETH can be applied throughout the software lifecycle whenever there is the need for decision making during Interaction design. We choose to illustrate its application in UPi, because it serves as a guide, providing useful steps and artifacts (among them, the UI Definition Plan) that can be tailored and customized when organizations intend to develop usable interactive systems.

5. APPLICATION OF MACBETH IN A USER INTERFACE DESIGN PROCESS We applied UPi in the national research project to define the Brazilian DTV System, in which our research group was responsible to develop the Access Portal, which is the main access to all the other interactive applications in the DTV. Since the DTV represents a new paradigm to interaction design, it became necessary to search for alternatives, but it was difficult to find existing material in the literature. The examples of UI for the DTV we found were adequate for a user profile different from the one we intended to focus on. Our target user profile is the Brazilian user, who has cultural peculiarities and social-economic constraints. Besides that, we also had to consider technical restrictions concerning hardware and transmission aspects because the new DTV hardware should be made available in Brazilian homes through an accessible price in order to be accepted by the

majority of the population, especially for people less privileged, who do not have access to computers, neither to the Internet. Therefore, in order to facilitate the selection of the most appropriate UI alternatives, UPi was applied in an iterative manner following the four phases: inception, elaboration, construction, and transition, based on the RUP [14]. Each phase is composed of the main disciplines, which are: requirements, analysis and design, implementation, deployment, and test. As the UI Definition Plan is built in the analysis and design discipline, the MACBETH approach is applied in this discipline through the Apply UI Definition Plan activity. Figure 5 illustrates the moment in which this activity is performed, and involves mainly the two first phases. The other two phases will not be described in this paper because they are more related to implementation and evaluation of product and designers have minor changes to work on. However, if the evaluations reflect on changes on the models, new iterations of the elaboration or even of the inception phases can be performed. We want to point out that it is out of the scope of this paper to describe the usability tests of the develop DTV applications. We have references of published works about the evaluation strategy that we applied with users.

5.1 Inception Phase In the inception phase, system analysts understand what users want the system to provide and present an initial version of the system to allow a clear conversation with users. At this stage, designers define which visual objects and which usability patterns can be part of the UI by performing the first step of the MACBETH. The second step of the MACBETH was performed when project participants selected the criteria by considering constraints informed by the Brazilian Government. The project manager (facilitator) prepared usability workshops and invited the participants from the institutions involved in the project. In these workshops, designers presented the possibilities of usability patterns/visual objects for each use case in order for them to decide which one was the most appropriate. In Figure 6, the designer used a TV to show these possibilities, in order to notice any existing distortions in size of letter, colors, etc.

In a nutshell, according to Figure 5, after the requirements are reviewed or use cases are detailed by describing their associate tasks, the designers need to define the appropriate usability pattern to execute the interactive tasks. They execute the activity Apply UI Definition Plan in order to define which usability patterns can be part of the UI according to the non-functional requirements defined in the previously activities. The development team that participated in the application of this process was the same that participated in the application of the MACBETH. The project participants were system analysts and designers and the facilitator took the role of project manager, responsible for assuring the quality of execution of each activity.

Figure 6 – Designer presented the possibilities of usability patterns/visual objects At this point, the third step of the MACBETH was performed with designers during the workshops, when they evaluated the patterns as depicted in Figure 1. Meanwhile, in workshops with users the fourth step of the MACBETH was performed in order to evaluate the criteria (Figure 2) that have a direct impact on the final result. As a result of these workshops, the most appropriate patterns were viewed in order of attractiveness, as depicted in Figure 3 and analyzed according to the fifth step of the MACBETH. Following, we describe the reasoning of designers during the workshops to select the most appropriate usability patterns for each use case.

Figure 5 – UPi workflow for interaction design This process is well aligned with the conceptual model design as proposed by Deborah Mayhew [16] and Constantine & Lockwood [5]. In the inception phase of UPi, the activity UI Prototyping can be performed using paper, image and executable prototypes. Among many techniques, paper prototyping can be done using the post-it technique as proposed by [5].

For the “access applications” use case, designers decided to navigate with the pattern “icon menu” because icons represent real-world objects and they make the UI friendlier, but designers had to be careful not to use too many options. On the other hand, even though the other two options of menu leave more space for the UI main content information, novice users may not know how to access the pattern “retractable menu” or different TV configurations may cut them off when they are too close to the edges. And the pattern “requested menu” requires a high level of expertise from users and novice users might never find out that there is a menu. This explanation exemplifies how the pattern “icon menu” was considered the most attractive for the criteria related to usability,

such as efficiency of use, ease of learning, ease of remembering, error rate, and subjective satisfaction. For the “visualize help” use case, designers decided to use the pattern “requested help” because it is simple and it allows user explicit control of the system. The automatic help can be unexpected, possibly making the user unsatisfied when it appears in inappropriate moments. For the “personalize the look and feel of the system” use case, designers decided to use the pattern “one choice at a time”, in which users select a color then select the font in any sequence but not simultaneously on the same screen, instead of the pattern “predefined templates with options of font colors and font size”, which lets users select both color and size on the same screen, or the pattern “free choice”, which lets users choose any combination of text and background color or font size. The use of templates provides more ease of use and accessibility to visually disabled users, even though it does not provide total freedom of choice. The use of free set of options increases flexibility of use, but it is very difficult for novice users.

Since users and designers were not satisfied with the prototype, they performed the sixth step of the MACBETH, in which they reevaluated the patterns and selected the pattern “pre-defined templates with options of font colors and font size”. Then, designers prepared the personalization prototype using this pattern. Figure 8 depicts the UI prototype for the personalization functionality with its objects: i) the personalization options; ii) the preview of the personalization with the icon menu representing application categories (communication, government, education) and TV categories (news, films, sports) and; iii) the quick access to other functions, such as TV, portal and help. This prototype uses the selected pattern “pre-defined templates with options of font colors and font size”, with which it is possible to offer to users a better preview of the selected option, thus, increasing the ease and efficiency of use because the user can complete the task in the same screen.

Not everybody participated in the usability events the facilitator organized and that led to a difficulty in maintaining a unique reasoning among all participants concerning the reasons for the design decisions. The events were incremental and if someone lost one event, they would not easily understand the results. As all the decisions were documented in the UI Definition Plan, this helped us to describe some reports required by the organization, called CPQD, responsible for the management of all consortiums involved in the for the Brazilian Digital TV System. Examples of these reports include Vision Document and UI Style Guide.

5.2 Elaboration Phase In the elaboration phase, designers design a stable version of the UI prototype. At this stage, designers make UI prototypes in drawings following the description specified in the task models and in the UI Definition Plan. Figure 7 depicts the personalization drawing prototype with the pattern “one choice at a time”, in which designers included two personalization options (font color and font size) on the top of the screen and the four options of font color on the bottom of the screen. The four options of font size can be seen when the user selects the option font on the top of the screen.

Figure 8 – The personalization prototype Since there was a change in the selected pattern, the UI Definition Plan was updated with the reason for the new decision and all the participants signed as a formal acceptance of the change. The new characteristics of the personalization prototype achieve some criteria by showing the changes in the portal as options of colors and font are selected, such as, ease of remembering, not requiring users to have to remember the selection made in a previous page; and also error rate, by avoiding users to select an undesired choice that requires them to return to the personalization page to make changes.

6. DISCUSSION AND FUTURE WORK This section talks about the participants’ behavior to learn and apply the MACBETH, organized in the following topics:

Figure 7 – The alternative for the personalization prototype

a) The application of this approach required: training for the facilitator on the approach and on the software; acquisition of the M-MACBETH software; and meeting room with data show. The facilitator organized the users and designers in a room in separate meetings, elicited their opinions and organized them using the software. Users and designers were not trained on the software because they were responsible for ranking the options and qualitatively evaluating the difference of attractiveness between them. The facilitator was the one responsible for using the software to summarize the opinions of the users and designers.

b) Users provided positive feedback concerning the meeting when they evaluated only the criteria and not the patterns, which is the responsibility of designers. In past projects, they usually gave opinions about which patterns to use, which is not always the best option. They were satisfied with the final results and believed that the technique is productive compared to their past project, well structured with pre-defined responsibilities and the meetings were objective and well organized. c) After applying this approach when performing the SDP, designers said it was better to use this approach than to rely only on guidelines, since most of them are not detailed enough for new situations that require expertise concerning recent technology issues that impact on the UI. Guidelines usually neglect the knowledge required for interaction design [9]. They found the UI Definition Plan very useful to document the results because they felt more secure to talk to users about changes with the previously signed paper in hands. The formal acceptance made users discuss more before making decisions and, consequently, avoided unnecessary changes and re-work. d) The methodology aimed to decrease the subjectivity of the opinions of all the people involved in the decision-making process. Users felt this was possible because they gave opinions about criteria and not patterns and also because they used predefined values for their opinions. The iterative, collaborative and constructive nature of the methodology is achieved through repetitions of the steps (iterative) in an environment where participants share their opinions (collaborate) until they create a final result (construct) that satisfies them. e) The definition of the groups is very important and should be done carefully and the meetings conducted by an experienced facilitator. Usually, most users want to participate in the meetings, but it is important to be selective because a full room can be disruptive and a room with few people may not bring the desired result. The group of designers was composed of four people and the group of users was composed of seven people. The meetings were fast and objective because of the small number of people. But, on the other hand, we did not have a complete acceptance of the result by the users who did not participate in the meetings. f) Integrating this methodology into an interaction design process is not difficult because most interaction design processes have specific steps for the selection of usability patterns to be included in the UI. Therefore, the decision to apply this methodology in a software organization depends if the professionals are willing to learn something new and managers are willing to spend money to attend the necessary requirements, as mentioned in topic a). The next paragraphs mention future works: From the application of the MACBETH, we have learned that it brings positive results, but its application is still slow, therefore, we intend to investigate and evaluate other decision-making processes and tools in order to be able to choose the approach that provides best results concerning precision and productivity. We will apply this approach in other projects both in the academia and in software organizations in order to acquire more feedback from the people involved in its application, and use this information to improve it. We also intend to include the reasoning for making changes and selecting patterns in the library of learned lessons of the Usability Lab where interaction design projects take place.

With additional refinements and validation, this work can be a significant contribution to usability practice, but we have already detected from its application that it is an efficient and reliable approach to select usability patterns during interaction design. It can also be extended to select UI prototype alternatives, which is a technique that is being widely applied in HCI. As we mentioned previously, the Brazilian Government sent a formal documentation listing constraints for the Brazilian DTV System as soon as some requirements were available. In addition, we had a restricted schedule which we needed to send reports and develop prototypes in a certain time. These restrictions did not allow us to apply more efficient strategies (such as ethnography) to know the users’ needs, their experience with cable TV, their expectations, and current practices in their own environment of use. This could have helped us to identify new criteria and new usability requirements. Therefore, we intend to integrate this proposal in ethno-methodologies in order to investigate how it could lead to the creation and representation of relevant scenarios of use of DTV applications.

7. STATE OF THE ART Rosson and Carroll proposed an approach based on user scenarios to decide which interaction features to include in the UI according to positive and negative ergonomic characteristics of each feature [18]. Borchers describes pattern languages to define interdisciplinary patterns and presents an approach for using patterns in usability engineering [4]. Each pattern is formally defined with the following semantics: name, context, reference to other related patterns, ranking its validity, illustration, problem that it addresses, examples of situations in which the problem is encountered, solution to the problem, and a graphical diagram to summarize its main idea. He also presents how patterns fit into each of the activities of Nielsen’s usability Engineering Lifecycle Model [17]. “Parallel design” is an activity in which designers develop alternative UI prototypes using HCI design patterns to attend usability goals. This is an activity in which our approach can be applied in order to facilitate the selection of a UI prototype among alternatives. In this paper, Borchers specifies and exemplifies how to use interdisciplinary patterns, but he does not specify how designers can select a UI prototype among alternatives. But, we identified some aspects in his research that can help us during the decisionmaking process: the ranking of patterns and the activities ‘iterative design’ and ‘collect feedback from field use’ can be useful for decision-makers to improve the corporate memory with information and lessons learned from the application of patterns that created successful solutions and those that did not. Corporate memory can be created using a tool, such as MetroWeb [15], which helps web designers create and access usability knowledge. Even though there are some researches on the definition of interaction patterns, Segerstahl and Jokela [19] believe that are some problems in the structure and organization of interaction patterns based on empirical studies. They conducted a case study in which they identified patterns for each problem case specified during user research and selected the most suitable patterns to develop an improved UI for each problem case. Since their work focuses on the usability of interaction patterns, they do not specify how they selected the most suitable patterns. The main identified

problem is that the lack of a standard way to organize, document, and group interaction patterns makes their efficient use difficult. Grouping patterns into task related problem areas (e.g. searching, navigation) is one of the proposed improvements on interaction patterns that is followed by our approach, which suggests the organization of patterns into usability tasks. In [9], the authors define a Pattern-Supported Approach (PSA), which focuses on the use of patterns throughout the entire interaction design process. The PSA patterns are: business domain, business proves, task and subtask, structure & navigation design, and GUI design patterns. These patterns are related to phases in the UI design process and they are double-linked among each other, for instance, task patterns point to structure & navigation design patterns and vice-versa. They specify that patterns are selected during discussions with stakeholders using patterns from the previous phase to guide the selection. But, they do not specify how these discussions are conducted or supported. [13] defines a systematic process for selecting UI elements based on tasks from usage scenarios. This process suggests the mapping of interaction tasks to abstract widgets (e.g. widget for selecting an action). This process is similar to our approach considering that concrete widgets are compatible to usability patterns, but concrete widgets are out of their scope. Two issues that we identified in this process are: there are various manipulation of tasks before actually mapping tasks to widgets; and this process assumes that usage scenarios of reasonable quality are available, which is not true in most software organizations.

In addition, we showed how to integrate three different areas (Software Engineering (SE), HCI and OR) describing a multidisciplinary approach for Interaction design: i) we brought to discussion a process which usability artifacts and activities are integrated with some SE artifacts and activities in a simple way; as it improves the communication among professionals from different areas, this can help them to work in a more productive way; ii) we made designers become aware of the need to consider other requirements, not only usability requirements, in their practices. This means to focus more on interaction design than on UI design and; iii) OR techniques can bring important solutions to HCI researchers interested in making qualitative analysis of the interaction, which leads to more objective results. When the technique is supported by software, it helps to receive mathematically significant results in a faster way.

9. ACKNOWLEDGMENTS The SBTVD project was sponsored by FINEP. The authors also thank all members of the AIMCOR-UFC Consortium for the great work in executing all activities of SBTVD project.

10. REFERENCES [1] Bana e Costa, Carlos A. Structuration, Construction et Exploitation Dún Modèle Multicritère D'aide à la Décision. Thèse de doctorat pour l'obtention du titre de Docteur em Ingénierie de Systèmes – Instituto Técnico Superior, Universidade Técnica de Lisboa, 1992.

In [6], they use a method to define the best sketching representations for each widget. This method can be analyzed in more detail for future comparison.

[2] Bana e Costa, Carlos A.; De Corte, Jean-Marie; Vansnick, Jean-Claude. MACBETH. London: Department of Operational Research – London Scholl of Economics, 2003.

Concerning tools, InFigura [21] incorporates user experience patterns and prototyping for websites. Depending on the page type (e.g. navigation page), the tool lists patterns that can be used in the web page, such as navigation links. But, it is still the responsibility of the information architect to decide whether to use text or image links, for instance.

[3] Baptista, Miguel Alberto Patinõ. Um Modelo Multicritério para Avaliar o Sistema de Qualidade de um Ambiente de Produção. Dissertação de mestrado em Engenharia de Produção. Universidade Federal de Santa Catarina, Florianópolis, 2000.

Accordingly, [8] suggests a list of patterns to be used in combination for designing websites, such as search pattern, site map pattern, go back to safe place pattern. But, the question still remains: how can we help designers in selecting the most appropriate usability pattern among various alternatives? Besides the support for selecting usability patterns, another advantage of our approach is the possibility to make changes after the analysis of the patterns and criteria. Therefore, this approach does not create an inapplicable solution; on the contrary, it analyzes preferences and presents solutions that can be easily adapted.

8. CONCLUSION We showed that conducting a project that is in sync with existing users’ needs as well as their socio-cultural and technological contexts requires a detailed analysis of criteria that affect each solution of interaction design. It is not a trivial task to do, mainly when each criterion has information that can affect other criteria. This scenario was made though the application of a technique which brings a solution to support decision-makers to qualitatively evaluate preferences of different points of view.

[4] Borchers, Jan. A Pattern Approach to Interaction Design. In Proceedings of the DIS 2000 International Conference on Designing Interactive Systems, ACM Press, New York, 2000. [5] Constantine, Larry and Lockwood, L. Software for Use: A Practical Guide to Models and Methods of Usage-Centered Design. Addison-Wesley, Reading, 1999. [6] Coyette, A., Vanderdonckt, J., A Sketching Tool for Designing Anyuser, Anyplatform, Anywhere User Interfaces, Proc. of 10th IFIP TC 13 Int. Conf. on Human-Computer Interaction Interact’2005, M.-F. Costabile, F. Paternò (eds.), Lecture Notes in Computer Science, Vol. 3585, SpringerVerlag, Berlin, 2005, pp. 550-564. [7] Furtado, Elizabeth, Sousa, Kenia; Vasconcelos, Patrícia; Carvalho, Fernando. Especificação da Aplicação Portal de Acesso – Aplicações em TV Digital. RFP No. 007/2004. Campinas, SP: CPqD, 2005c. [8] Gaffar, Ashraf; Seffah, Ahmed; Van der Poll, John A. HCI Pattern Semantics in XML: a Pragmatic Approach. Proc. Of Human and Social Factors of Software Engineering HSSE’2005, ACM, 2005.

[9] Granlund, Asa; Lafreniere, Daniel; Carr, David. A PatternSupported Approach to the User Interface Design Process. In Proceedings of HCI International 2001. 9th International Conference on Human-Computer Interaction, New Orleans, USA, 2001.

[16] Mayhew, Deborah. The Usability Engineering Lifecycle – A Practitioner’s Handbook for User Interface Design. Morgan Kaufmann Publishers, 1999.

[10] Gulliksen, J.; Goransson, B. Usability Design – Integrating user-centered systems design in the software development process. Tutorial at INTERACT 2003, Zurich, Switzerland, 2003.

[18] Rosson, M. B., Carroll, J. M. Scenarios, Objects, and Points of View in User Interface Design. Object Modeling and User Interface Design. Mark Van Harmelen Eds. Addison-Wesley, 2001.

[11] Jones M. and Marsden, G. Mobile Interaction Design. John Wiley & Sons Ltd. England, 2006.

[19] Segerstahl, Katarina; Jokela, Timo. Usability of Interaction Patterns. In: Work-in-Progress CHI 2006. April 22-27, 2006. Montréal, Quebec, Canada.

[12] Juristo, Natalia; Lopez, Marta; Moreno, Ana; Sánchez, Isabel. Improving Software Usability through Architectural Patterns. In: International Conference on Software Engineering (ICSE), 2003, Portland, Oregon. 2003, pp. 1219.

[17] Nielsen, Jakob. Usability Engineering. Morgan Kaufmann, San Francisco, 1993.

[20] Sousa, Kênia. UPi – A Software Development Process Aiming at Usability, Productivity and Integration. Dissertação de Mestrado em Informática Aplicada. Universidade de Fortaleza, Fortaleza, Dezembro de 2005.

[13] Kaindl, Hermann; Jezek, Rudolf. From Usage Scenarios to User Interface Elements in a Few Steps. Proc. of 4th International Conference of Computer-Aided Design of User Interfaces CADUI'2002, Kluwer Academics, Dordrecht, 2002.

[21] Tiedtke, Thomas; Krach, Thomas; Martin, Christian. InFigura, An Integrated Design Tool. Proc. of 5th International Conference of Computer-Aided Design of User Interfaces CADUI'2004, Kluwer Academics, Dordrecht, 2004.

[14] Kruchten, Philippe. The Rational Unified Process - An Introduction. 2 ed. New Jersey: Addison-Wesley, 2000.

[22] Winograd, T., CS147: Introduction to Human-Computer Interaction, 2006. Stanford, CA. http://cs147.stanford.edu.

[15] Mariage, Celine; Vanderdonckt, Jean. Creating Contextualised Usability Guides for Web Sites Design and Evaluation. Proc. of 5th International Conference of Computer-Aided Design of User Interfaces CADUI'2004, Kluwer Academics, Dordrecht, 2004.

[23] Welie. 2005. Available at: http://www.welie.com. Accessed in: September, 9th, 2005.