Journal of Interactive Learning Research. Volume 10, Number ¾. December 1999. (pp 401412); Special Issue on Intelligent Agents for Education and Training Systems, AACE, ISSN: 1093-023X.

Agent-Support for Problem Solving through ConceptMapping Svetoslav Stoyanov Piet Kommers University of Twente Faculty of Educational Science and Technology Division of Educational Instrumentation P.O. Box 217, 7500 AE Enschede, The Netherlands {stoyanov, kommers} @edte.utwente.nl

Abstract This article presents an experimental verification of a hypothetical construct explaining the basic mechanism behind the behavior of an intelligent agent implemented in the SMILE performance supported system. The SMILE agent, (called 'facilitator') supports a user in learning and applying a new concept mapping method for solving ill-structured problems. This article emphasizes the facilitator's master performer model of behavior. The model reflects upon SMILE Maker as a problemsolving tool and especially upon the SMILE concept mapping method. The SMILE concept mapping method is based upon the 4E hypothetical construct. It consists of four characteristics: Expressiveness, extension, externalization, and entireness. The facilitator as a master performer of the SMILE concept mapping method reacts to the user's behavior accordingly to the extent that these characteristics are accomplished.

Introduction The general characteristics of SMILE Maker as an agent-based system are further elaborated in the previous article of this special issue (Aroyo, L., Stoyanov, S. & Kommers, P., 1999). We will start with a short overview of the assumptions behind and the theoretical background underlying SMILE Maker in order to outline the context of the experimental validation of agent support for problem solving through the SMILE concept mapping method. SMILE Maker is a web-based knowledge support system promoting just-in-time, just-enough and justat-the-point-of-need intelligent support for dealing with open-ended problem situations. It has been targeted to everyone who tries to make sense of complex information in ill-structured problem situations, in order to generate alternative solutions, to select the most appropriate one among them and to implement it in practice. Initially, SMILE was designed to support the students from the Faculty of Educational Science and Technology, and the Faculty of Communication especially those enrolled in the courses of Linear & Hypermedia, Media in Communication, and some courses in the Master of Science Programme "Educational and Training System Design". SMILE Maker is a part of the project-

3/7/03

1

based learning environment proposed by those courses. It might be also a good tool for in-company training managers to cope with unpredictable problem situations in very competitive business environments. SMILE Maker capitalizes on the advantages of the systematic problem solving approaches and mapping techniques. It helps clients to escape from some syndromes like 'paralysis by analysis', and 'functional fixedness', and to avoid some negative effects such as perceptual defense, stereotyping, and expectancy when solving ill-structured problems. SMILE Maker proposes an intelligent user-centered learning environment for learning the SMILE concept mapping method and for applying it in effective problem. Conceptually, SMILE tries to combine the strong points of the dominant educational doctrines. It attempts to set up a balance between Instructionism and Constructionism as extreme educational paradigms, Content-Treatment-Interaction and Aptitude-Treatment-Interaction strategies, and the System-Locus-of-Control versus the User-Locus-of-Control in HCI (Human Computer Interaction). The 4-ID theoretical model behind the SMILE system reflects this challenge. It consists of four submodels: Content or SMILE Concept Mapping Method, User, Learning Events, and Facilitator. System helper

Advisor

Profiler

Information Collection

Navigator

Learning Styles

Facilitator

Idea Generation

CM Method

Idea Selection

Idea Implementation

SMILE Maker Model

Learning Events

Explanation

User

Locus of Control

Problem Solvig Styles

Prior Knowledge

Examples

Practice Procedures

Figure 1; Architecture of the Components, Goals and Processes in the SMILE System The sub-model of learning events includes four activities - explanation, example, procedure, and practice. Because SMILE is both a learning and a problem-solving tool, the user sub-model is divided into learner and problem solver sub-models. The learner sub-model is defined by four learning styles:

2

Activist, reflector, theorist and pragmatist (Honey & Mumford 1992). There is a close correspondence between learning styles and expected learning events. Theorists are likely to choose an explanation. Reflectors look for an example. Pragmatists start with a procedure, and activists should go directly to the more practical implications. The chosen problem solver sub-model describes four problem-solving styles: Seeker, diverger, converger, and practitioner. There is also a link between a particular problem solving style and the stages of problem solving. Seekers have preferences to map information collection, divergers feel comfortable with map idea generation, convergers are strong in idea selection and practitioners may go first to the implementation. A user may attribute her- or himself explicitly with either a problem solving- or a learning style. Recently two more characteristics of the user submodel were added: Locus of control (external or internal), and prior knowledge (low, medium and high). Four scenarios have been designed as operational instructional design solutions: Ready-made, tailormade, self-made and being in the atelier situation. They are built up from specific combinations between method, user and learning events (Stoyanov, Aroyo, Kommers, 1999). The facilitator is the one that makes the difference and enables the feasibility of the theoretical claims formulated above or not. The facilitator is the entity having four 'faces' that are complementary to each other - profiler, advisor, navigator, and system helper. As a profiler, the facilitator identifies, (explicitly or implicitly) users according to their learning- and problem solving styles, their locus of control and their prior knowledge. As a navigator, the facilitator explains how to navigate through the site. It also informs user about the point s(he) has arrived giving a help to save some time and to continue the work. As a system helper, the facilitator performs some routine functions on behalf of the system - reminding for saving, downloading procedures, etc. As an advisor, it gives some hints to user based upon two main principles: The completeness of the SMILE concept mapping stages and the completeness of the learning events cycle. The facilitator is expected not to adapt to a particular individual style, but to develop more versatile style. There are two models of the facilitator as and advisor: The master performer model and the user model. In the user model, facilitator is designed to have initially an abstract concept about a user. This abstract notion has four attributes: Learning style, problem solving style, locus of control and prior knowledge. The facilitator makes a judgement about learning style and problem solving style explicitly, and draws an inference about locus of control and prior knowledge implicitly. For example, a source of implicit inference judgement is the selection of the scenario. If a user chooses a ready-made scenario, it implies that s(he) is subservient to the external locus of control. The facilitator concretises his initial hypothesis on the user profiles on the basis of his concrete behavioural traces, making a kind of 'cognitive map'. Quite a diverse spectrum of user patterns may evolve. For instance, one can have the theorist learning style, have the divergent problem solving style, and at the same time have an internal locus of control and a low level of prior knowledge. The master performer model will be a matter of special and detailed consideration in the following sections. It reflects more closely SMILE Maker as problem solving tool and especially SMILE concept mapping method.

3

Concept Mapping Approaches Concept mapping has received a great recognition in the education and business professional activities. In education this technique has been used mostly as a graphical advance organizer, and assessment tool (Novak & Gowin, 1984; Pendley, Bretz, & Novak, 1994). In business it has been found valuable for problem solving and decision-making (Buzan, 1996; De Bono, 1994; Eden, Ackerman & Cropper, 1995). Concept mapping has been attributed a large range of definitions (Stoyanov, 1998). An extensive overview of them is not a deliberate purpose of this article but a short tour through it might be useful in order to get a general impression on the phenomena. Concept mapping is a technique of graphically representing concepts and their hierarchical interrelationship, using spatial configurations of nodes and links to communicate about the concepts in a given knowledge domain (Beyerbach, 1988; Lambiote, Dansereau, Cross & Reynolds, 1989; Novak & Gowin, 1984). Concept mapping has been defined also as a cognitive tool (Kommers, Jonassen, & Mayes, 1991), a kind of formalism for representing structural knowledge (Jonassen, Beissner & Yacci, 1993; Jonassen & Marra, 1998), an epistemic game (Sherry & Trigg, 1996), a schematic scaffolding device (Hammond, 1999), a research method (Trochin, 1999), an interactive interview technique (Zaff, McNeese & Snyder, 1993), and a navigational mechanism for the WWW (Kommers & Lanzing, 1997; Kremer & Gaines, 1999). Concept mapping has been associated with several psychological functions: perception (Kremer & Gaines, 1999; Novak & Gowin, 1984), memory (Ahlberg, 1993; Buzan, 1996; Kozma, 1987), understanding (Hale, 1997; Novak & Gowin, 1984), problem solving (Buzan, 1996;De Bono, 1994; Eden et al., 1995), meta cognition (Jonassen & Marra, 1998; McAleese, 1997), and attitudes (McCabe, 1997).

In this article we will take a closer look at concept mapping as a supportive technique in ill-structured problem situations. Concept mapping will be considered as a generic term for all problem solving mapping approaches. A few among them, but probably the most attractive ones are mind mapping, flowscaping, and cognitive mapping. They will be noticed as traditional approaches when comparing them with the new concept mapping method. ?

Mind mapping has been introduced as an explicit model of the associative way human mind organizes information. Mind mapping is based on natural free association and can be used as a brainstorming technique in problem solving (Buzan, 1996).

?

Flowscaping tries to grasp perception of a problem space at a particular moment. In order to know what to do when a problem occurs we have to understand our perception of the situation (De Bono, 1994). Thinking, according to De Bono, is a two-stage process - making and using maps.

?

Cognitive mapping has been described as a tool for reflective thinking and problem solving (Eden et al., 1995). It aims at helping people to understand complex ill-structured situations and to decide what to do and how to do it.

4

Most of the referred concept mapping approaches report on the theoretical models, they are built upon. The mind mapping is based upon radial thinking (Buzan, 1996). The rational of flowscaping is lateral thinking (De Bono, 1994). Kelly's personal construct theory is the theoretical framework of cognitive mapping (Eden et al., 1995). Several specific concept-mapping software packages have been designed in the framework of the discussed mapping approaches. Mind Manager is the most recent mind mapping software. Serious Creativity is a computer application supporting flowscaping. Decision Explorer is software similar to cognitive mapping. The existing concept mapping approaches for problem solving are representative for a widespread assumption within the concept mapping paradigm. They tend to describe concept mapping as a unique technique that naturally affords the problem solving activities of information collection and ideas generation, and automatically creates opportunities for brainstorming and free association. Within this framework at least three important points attract the attention: 1.

Concept mapping has been identified with only two of the stages of problem solving process representation of information and generation of ideas.

2.

Concept mapping as a problem solving technique has been associated mainly with the brainstorming procedure and the free association attitudes.

3.

Concept mapping effectiveness in problem solving was claimed to be available in its unique functional structure. Concept mapping problem solving effectiveness could be accomplished just by the fact of using this technique, following specific procedural rules. There is not a special term given to this function but for the purposes of the paper we call it 'concept mapping affordance'.

In this article we defend the thesis that a special effort is needed in order to derive deliberately the full latent potential of the concept mapping as a problem solving technique. A new concept mapping method, SMILE, was developed, applying a more elaborate and more structured systematic approach to problem solving. The new method is based upon the analysis of the unique nature of concept mapping. The effectiveness of concept mapping in problem solving depends on the extent to which all prominent characteristics of concept mapping technique have been activated. In the next section we are going to describe operationally concept-mapping problem solving effectiveness in the terms of its uniqueness as a special technique. Concept mapping is an information collection, idea generation, idea selection, and idea implementation cognitive and meta cognitive tool, capable to produce a general beneficial, compensation and a differential effect on clients. A hypothetical construct of concept mapping problem solving effectiveness based upon the 4E model will be presented. The 4E model could be used for predicting the effectiveness of users' behavior in problem solving. The four dimensions of the model are the main characteristics of the concept mapping that make it a unique technique.

Concept Mapping Problem Solving Effectiveness - A Hypothetical Model The hypothetical model of concept mapping for problem solving is based upon the 4E model. The model is presupposed on the assumption that concept mapping problem solving effectiveness is a

5

function of four unique characteristics of concept mapping: Expressiveness, Extension, Externalization and Entireness, hence 4E.

Expressiveness Concept mapping is one of the very few if not the single graphical techniques that models the way human mind organizes information in problem solving. A certain number of research have released the evidence that human mind organizes information in a spatial format. Concept mapping promotes a close correspondence between psychological constructs and its external mode of representations. Concept mapping applies a simple formal structural convention consisting of nodes, links, links' labels and a particular spatial configuration. The links could be one-directional, bi-directional, cross-links, with or without arrowheads. The spatial configuration might be a hierarchy, a network, a matrix, etc. This formal structure provides a general framework for representing the cognitive and the affective structure of a person. Concept mapping integrates two kinds of coding - verbal and visual. The technique capitalizes on the advantages of graphical representations without losing the flexibility and the power of natural language system. Concept mapping supports mental imagery. It can be seen as a flexible mode of visual representation, which allows a rapid anticipation of transformations in the problem situation. Visual imagery facilitates problem solving because of procedures that tap visual perception directly. One of the major functions of the imagery is to instantiate hypotheses, i.e. mentally anticipate on the potential issues of problem solving. The content structure of concept mapping is also important. Concept mapping captures our transient cognitive and affective experience during problem solving. It is tolerant to different kinds of problem solving representations - facts, hypotheses, metaphors, and even feelings. The technique also provides a good basis for the variety of contextual links - descriptive (is a), structural (part of), causal (lead to, because of), metaphorical (like), interrogative (why, or just a question mark), etc. The variety of problem solving representations and the variety of contextual links is important for a broad perception of a problem space, and for generating alternative solutions, as well. While concept mapping is a concise, compact, and parsimonious technique, it is at the same time very rich of information. We can say concept mapping is a special chunk of information because of its very strong integrative potential. It is a device capable to reproduce complex problem situations in an easy and an intuitive way because of opportunities to expresses the variety of problem solving representations and the variety of links between them, using a simple format.

Externalization Concept Mapping externalizes cognitive processes as problem solving occurs. A problem solver is able to reflect on her/his problem solving representations. Most of the people experience difficulties, while dealing with their internal thinking process. It is hard business to look upward and to communicate with yourself about your thinking processes during ill-structured problem solving. There are always two parallel processes - thinking on the problem itself and thinking on the thinking. This makes the

6

issue of problem solving much more sophisticated. The mental models of our own cognitive processes are rather difficult to deal with and this rises the need for a tool that can externalize these mental models. The attempt to make explicit what happens inside of our mind is a natural drive and concept mapping is a rather relevant means for that. Concept mapping, especially concept mapping software, allows problem-solving representations to be manipulated in its externalized state. Thus, working upon concept mapping we are building upon our cognitive structures. This is a mutual process: while improving the external model of cognitive structures we improve cognitive structure effectively. Concept mapping stimulates self-appraisal and self-reflection, giving a sense of distance and ownership. We need to distinguish ourselves from our internal world in order to understand it. The externalization of the cognitive processes enhances the internal locus of control of a problem solver on them. Concept mapping stimulates self-management (control and monitoring). It is a cognitive instrument for planning and guiding through the various problem-solving stages: problem formulation, information collection, idea generation, idea selection and idea implementation.

Extension Concept mapping is an external extension of the intellectual and affective structures of personality. It affords a physical distribution of cognition. Concept mapping enlarges the natural limited capacity of working memory. This leads to reducing the cognitive overload, improving the quality of problem solving and rising the speed of the ideational processes.

Entireness Concept mapping presents a whole picture of problem solving space and shows the relationships between its components. The complexity of the problem situation can be grasped at once. Concept mapping enables all psychological processes (attention, perception, memory, thinking, and language) to be involved and to be complementary to each other. Perception is a synecdoxial1 psychological construct containing all those cognitive functions. Perceiving problem solving space includes all other psychological processes. Thus, problem-solving effectiveness depends on how a perception for problem solving representations is built and how this perception is changed along time. Changing deliberately the concept map we are changing our perception of the problem. One of the most important meanings of the entireness is that concept mapping method itself includes the whole problem solving cycle - problem formulation, information collection, idea generation, idea selection and idea implementation. The 4E model is an ideal hypothetical construct that has to be verified empirically. We have formulated four assumptions about some shortcomings of traditional concept mapping approaches and some expectations to a new concept mapping approach when both were checked against the main features of the 4E model.

1

Part of speech when a part express the whole

7

Assumptions about Traditional Concept Mapping in Problem Solving 1.

Expressiveness. Traditional mapping approaches use nodes that are mainly facts and statistical data. They do not pay attention very much to some figural (metaphors) and some affective representations in problem solving. The traditional concept mapping approaches generate mostly ready-made solutions and sometimes make elaboration or modifications.

2.

Extension. Traditional concept mapping approaches use one map. It is restricted by this convention to arrive at possibly more great extension of cognitive structures.

3.

Externalization. Traditional concept mapping approaches realizes self-reflection but they are fallen short in self-management. They do not provide guidelines for reflection ni action in order to answer the questions where are we going and why?

4.

Entireness. One of the advantages of the traditional concept mapping methods to give a whole picture of the problem solving space and to show the relationships between problem solving representations might be shift into a disadvantage when a problem is more complex. Putting a lot of information in one map may cause a mess and personal embarrassing instead of clearness, and good orientation. Traditional concept mapping approaches are associated with only two of the problem solving stages - information collection and idea generation. It relies very much on spontaneous brainstorming and free association in idea generation phase.

The summarized conclusion from these statements is that the traditional concept mapping approaches has fallen short in deriving the full potential of concept mapping as an effective tool for problem solving. The more traditional concept mapping methods respond to all criteria of the concept mapping effectiveness given in the 4E model but in a limited extent. Concept mapping affordance is not just available. It has to be built upon in order to see the full potential of the technique. A new concept mapping method should cover this need. Before going to express some exp ectations to the SMILE concept mapping method in regard to problem solving effectiveness, a short description of the method is provided in the space below.

SMILE Concept Mapping Method - a Short Description SMILE method requires the creation of several maps: map-information collection, map-idea generation, map-idea selection, and map-idea implementation. Each map can be identified by purpose and particular components. Some creative problem solving techniques, specific for the type of map, are incorporated in.

Information-Collection Map The objective of this map is to scan all available information about the problem under exploration. The problem solving environment is explored in the terms of very broad scope of information items such as scientific facts, statistical data, personal experience, assumptions, metaphors and analogies, feelings, opinion, etc.

8

Idea-Generation Map The objective of this stage is to produce as many ideas as possible for getting a problem solution. Problem solving space is explored in the terms of ready-made solutions, elaboration, modification, and unusual or "crazy ideas".

Idea-Selection Map The objective of this stage in concept mapping problem solving is to select the most appropriate candidate for a problem solution among the number of ideas that have been produced in the idea generation phase.

Idea-Implementation Map The objective of this stage is to operationalize a problem solution in the terms of sequence of activities and events, and to present the needed steps in order to put solution into practice.

The SMILE Concept Mapping Approach - General Expectations 1.

Expressiveness. The new method provokes problem space exploration not only in the terms of facts, but also in the terms of personal experience, feelings, metaphors, analogies, and etc. It is more open for using structural, causal, metaphorical (remote associations), and interrogative type language on the concept mapping links.

2.

Extension. The new method uses several and conceptually different maps - map information collection, map idea generation, map idea selection and map idea implementation. An assumption built upon this fact states that the new method is superior than traditional ones regarding the external extension of the cognitive and the affective structures of personality, especially enlarging the capacity of the working memory.

3.

Externalization. The new method is not only a self-reflection tool, but it is also a self-management tool because it provides directives, perspectives and process support. Self-management functions of concept mapping are outlined by the requirements to follow a predefined order of different maps.

4.

Entireness. The new method covers the whole cycle of problem solving - collection of information, generation of ideas, choosing the best idea and idea implementation. It builds up an appropriate context for applying different and diverse problem solving techniques depending on the type of map.

Concept Mapping and Individual Differences The effectiveness of problem solving supported by concept mapping also depends on the individual characteristics in a learner. The general 4E model looks different among persons with different stylistic preferences in problem solving. The 4E model is being diffracted by the individual characteristics in problem solving. As a general rule, concept mapping should represent the individual profiles in problem solving. The profiles are nothing less than specific patterns of the individual problem solving

9

representations. On the criteria of individual differences, concept-mapping effectiveness might be operationalized as the extent to which the technique produces a differential effect, or a general beneficial effect or a compensation effect. The traditional concept mapping approach triggers the differential effect reproducing the individual preferences in problem solving. Students have different skills and aptitudes towards information collection, idea generation, idea selection or idea implementation. The traditional concept mapping approach reproduces a particular kind of cognitive schema marked by some syndromes of the individual differences in problem solving like 'analysis paralysis', 'functional fixedness', 'lack of insight', 'one idea', 'too many ideas', 'premature judgment', etc. (Wodtke, 1993). The new concept mapping method is supposed to lead towards an overall beneficial effect. Its effects propagate through the performance of various problem-solving activities - information collection, idea generation, idea selection and idea implementation, regardless of the personal style. The new concept mapping method might arrive at a compensation effect as well. For example, in the idea generation phase, concept mapping might support better the people with selective problem solving styles. As a conclusion, there are at least two general criteria for concept mapping effectiveness in problem solving: ?

The extent to which the concept mapping characteristics such as expensiveness, extension, externalisation and entireness are realized.

?

The extent to which concept mapping develops a general beneficial or (and) a compensation effects over a differential effect.

In order to check the validity of our assumptions and expectations, an experiment was designed aimed at comparing the classical concept mapping approaches and new concept mapping method against the ideal hypothetical 4E model. The specific projections of the individual differences on the model also were issues during the experimental investigation.

Experimental Method Experimental Design This is a factorial experimental design (2X2) with a post-test-only control group. The independent variables are the two-level problem solving mapping method (new concept mapping method and traditional concept mapping method) and the two-level learning style (doers and thinkers). The dependent variable is the concept mapping production numerically scored on different criteria. This experimental design was selected because a random assignment to the conditions was possible. The combination of random assignment and the establishment of a control group served to eliminate the majority of threats to both the external and internal validity of the study. Although the proportion of dropouts was reported as a potential threat to internal validity, not controlled for this type of design, it did not prove to be a problem in our case. The research was conducted in one-day session and the size of groups remained constant throughout the duration of the study.

10

Hypotheses The subjects in the experimental group using the new concept mapping method will score significantly higher than students in the control group applying classical concept mapping method, on the components of the map productions.

Subjects The sample for this study was selected from the total of fifty-two fourth grade students studying in the Faculty of Mathematics and Informatics of Sofia University. Thirty-two students were randomly selected and then randomly assigned to the experimental and the control group according to their learning styles.

Instruments The Learning Style Questionnaire of Honey and Mumford (1992) was used as one of the measuring instruments in this study. It consists of 80 items to identify four learning styles: activist, reflector, theorist, and pragmatist. Test-Retest reliability is reported to be high .89, according to the Pearson's product-moment coefficient of correlation. In order to ensure an equal representation of the learning styles, we merged the four learning styles into two - thinkers (theorist and reflector) and doers (activist and pragmatist). Honey and Mumford (1992) also suggested the possibility to reduce the four styles to two, if necessary. Traditionally, most of the classifications of cognitive and learning styles use a dichotomy: right brain - left brain, field dependent - field independent, holist - serialist, convergent divergent, etc. The concept mapping productions was used as a measurement instrument as well. The scoring scheme is based upon the criteria and indicators presented in the space below: 1.

Broad Perception

Fluency (Number of nodes; Number of links) Flexibility (Variety of nodes - Facts, Data, Personal Experience, Opinions, Feelings, Assumptions, Metaphors & Analogies; Variety of labels - Descriptive, Structural, Causal, Interrogative, Remote Associations; Variety of links - One-directional, Bi-directional, Cross-links) 2.

Divergency

Fluency (Number of ideas) Flexibility (Variety of Ideas - Ready-Made Solutions, Elaboration, Suggestions, Unusual Ideas) 3.

Convergency

Inverse Fluency (Reducing to one best idea, reducing to a few ideas arranged in consequence, reducing to a few ideas arranged by importance, reducing to a few ideas not arranged, no reduction, no idea selected at all)

11

4.

Planning

The extent to which the solution is presented in the terms of sequence of activities and events (Consequence, System, 'PERT' formats, No planning) 5.

General Fluency: Total number of nodes in all maps

6.

Problem Solving Cycle Completion: the extent to which a user passes through information collection, idea generation, idea selection and idea implementation phases.

The expressiveness of the concept mapping effectiveness has been operationalized mostly by the components of broad perception and divergency: fluency, flexibility. The extension is operationalized by the general fluency: total number of nodes in all maps. The externalization is operationalized by the components of convergency and planning. The entireness of the concept mapping is defined by the extent to which all problem-solving activities are involved: information collection, idea generation, idea selection and idea implementation. In the concept mapping scoring scheme broad perception stands for information collection, divergence stands for idea generation, convergence stands for idea selection, and planning stands for idea implementation.

Procedure The learning style questionnaire was distributed among the subjects and they were randomly assigned to the control and the experimental groups in order both learning styles to be equally represented. The students in the control group were introduced to classical concept mapping, mind mapping and flowscaping methods and they were asked to select one of the techniques or combine some of its components. Surprisingly, all of the students selected classical concept mapping method with a problem put in the middle of the map and levels of branches coming out of it. This kind of classical concept map is known as a spider map. The experimental group was introduced with the new concept mapping method. The control group got training in concept mapping. Because of the lack of time the experimental group was not able to receive any real training in the new method. The students were introduced to the procedure of applying the method in problem solving situation. A case to be solved was presented to the students in the control and experimental group and they were asked to use the concept mapping method. As a reinforcement mechanism in order to bring more motivation to the students several concept mapping software demo versions were installed for free to be used after the experiment.

Data Analysis The factorial analysis of variance was chosen as an appropriate statistical procedure for this study. Two-way ANOVA to raw data were applied. ANOVA has proved to be robust to the violations of the basic assumptions to apply a parametric statistical procedure. That means the level of significance is little affected by a violation of one or more of these assumptions. The probability of data analysis was established beforehand at the 5% level. SPSS 8.0 package was used for the data analysis.

12

Analysis of the Results The experimental group has proved to be significantly better than the control group on the nodes fluency of broad perception criteria (Sig. = .018). The subjects in the experimental group produce considerably more information items than subjects in the control group. The experimental group also demonstrates significantly higher results on the broad perception nodes flexibility (Sig. = .000) The students in the experimental group use much more statistical data and figures (Sig. = .001), personal experience (Sig. = .002), assumptions (Sig. = .001), feelings (Sig. = .000), and metaphors and analogies (Sig. = .008). The perception of the problem space in the control group is dominated mostly by facts (Sig. = .000) and opinions (Sig. = .000) (See Table 1 and Figure 2). Table 1. Broad Perception - Nodes Dependent

Independent variables

variables

Method F

Style Sig.

F

Method*Style Sig.

F

Sig.

1. Fluency (Number of nodes)

6.297

.018*

.017

.898

.820

.373

2. Flexi bility (Variety of nodes)

55.446

.000*

.554

.463

.139

.712

2.1. Facts

50.948

.000 **

.444

.511

.018

.895

2.2. Opinions

17.372

.000 **

.138

.713

.068

.713

2.3. Data

12.802

.001*

1.607

.215

1.607

.215

2.4. Personal Experience

11.510

.002*

1.364

.253

1.364

.253

2.5. Feelings

55.483

.000*

4.987

.034 ***

1.132

.296

2.6. Hypotheses

13.810

.001*

3.851

.060 ****

3.851

.060

2.7. Metaphors & Analogies

8.269

.008*

1.454

.238

.551

.464

Nodes that are:

* (In favor of the new concept mapping method) ** (In favor of the traditional concept mapping method)

*** (In favor of the 'doers') **** (In favor of the 'thinkers')

13

Figure 2. Broad Perception - Nodes

Broad Perception - Nodes 80 70 60 50 40 30 20 10 0 Facts

New Method

Opinions

Data

Personal Experience

Classic Method

Feelings Hypotheses Metaphors & Analogies

There is not a significant difference on the fluency of the links indicator. That means the students in the control group use relatively more links per node. The subjects working with the traditional concept mapping method score significantly higher than their fellows in the experimental group on bidirectional (Sig. = .004) and cross-links (Sig. = .033) indicators. The subjects in the experimental group use mostly one-directional links (Sig. = .000) (See Table 2 and Figure 3) Table2. Broad Perception - Links Dependent

Independent variables

variables

Method F

Style Sig.

F

Method*Style Sig.

F

Sig.

1. Fluency (Number of links)

.002

.965

.653

.426

1.813

.189

2. Flexibility (Variety of links)

10.469

.003 **

.062

.805

3.035

.092

2.1. Onedirectional

16.490

.000 *

.237

.630

.458

.504

2.2. Bi-directional

9.965

.004 **

2.550

.122

.088

.769

2.3. Cross-links

5.029

.033 **

1.095

.304

5.722

.024

* (In favor of the new concept mapping method) ** (In favor of the traditional concept mapping method)

14

Broad Perception - Links

100 90 80 70 60 50 40 30 20 10

New Method

0 Classic Method

One-directional links Bi-directional links Cross-links

Figure 3. Broad Perception - Links Subjects in the classical method are forced to use the whole repertoire of possible links because they have to represent everything on one sheet of paper. The subjects in the experimental group have more room to maneuver with four maps to be drawn. This particular feature of the new method gives the subjects in the experimental group more memory space, mapped into different sections - information collection, idea generation, idea selection, and idea implementation. While the traditional method puts all problem-solving activities in one picture, the new method creates a picture of the whole problem solving process sharing the cognitive load between the problem solving stages. The simplicity of the types of links frees up the memory processes to be more effective in problem solving. The complexity of the labels' structure provides a deeper perception on the problem solving representations. The variety of links' labels (Sig. = .025) is greater in experimental group map productions. While the classical concept mapping group use predominantly descriptive type (Sig. = .001) of links' labels, the new concept mapping method group uses more structural (Sig. = .007), causal (Sig. = .019), interrogative (Sig. = .028) and remote associations (Sig. = .001) links. (See the Table 3 and Figure 4). The simplicity of the links is determined by the opportunity to draw several maps and to distribute the cognitive overload over the whole process. The new method uses more complex verbal code combined with more simple one-directional visual links' structure.

15

Table 3. Variety of Labels (Language) Dependent

Independent variables

variables

Method F

1. Flexibility (Variety of Labels)

Style Sig.

F

Method*Style Sig.

F

Sig.

5.645

.025 *

2.032

.165

.226

.638

1.1. Descriptive

12.948

.001 **

.013

.909

.108

.745

1.2. Structural

8.483

.007 *

4.419

.045 ***

1.417

.244

1.3. Causal

6.192

.019 *

.040

.844

2.536

.122

1.4.Interrogative

5.358

.028 *

1.018

.322

.088

.769

1.5. Remote Associations

13.064

.001 *

.215

.647

.215

.647

* (In favor of the new concept mapping method) ** (In favor of the traditional concept mapping method) *** (In favor of the 'Thinkers')

Figure 4. Broad Perception - Labels

Broad Perception - Labels

70

60

50

40

30

20

10

0 New Method Discriptive Structural

Classic Method

Causal Interrogative Remote association

The experimental group is superior on the criteria of divergency as the scores on number (Sig. =.0001) and the variety (Sig. =.0001) of ideas are significantly higher than the same indicators of the control group. The subjects in the experimental group were significantly better on the convergency criteria when the best candidate (Sig. =.0001) among the alternatives should be selected. The students in the experimental group outperform the students in the control group on the planning criteria as well (Sig. =.0001). The experimental group is significantly better than the control group on the total number of nodes (Sig. = .0001) as a way of operationalizing the extension dimension of the 4 E model.

16

The experimental condition shows superiority on the criteria of completeness of the problem solving cycle (Sig. = .0001). All students in the experimental group have passed through the stages of information collection, idea generation, idea selection, and idea implementation.

No one of the

students in the control group has done so. Students in the control group were focused on the perception and representation of the problem space. They arrived at only the generation of few ideas. The analysis of the style variable shows that thinkers named significantly more structural types of links (Sig. =.045) than doers. They also formulated substantially more assumptions items than doers. (See the Table 1) The result is quite closed to the significant difference (Sig. =.060). Naturally, thinkers tend to present the information into classifications and clusters. They also have preferences to generate more hypotheses. A good prerequisite for that is a well-established structure. Doers express much more feelings in the perception of the problem solving space. (Sig. = .034) (See Table 1). This is, probably, because they are extraverts. With the new method the thinkers reduced considerably the number of cross-links (Sig. =.024). (See the Table 2). In the classical approach thinkers need more cross-links to express the structural complexity of the problem solving space. The new method gives them opportunities to distribute the structural complexity along several maps.

Discussion The experimental results support the hypothesis that the new concept mapping method is significantly better than the traditional concept mapping approach in problem solving. The new method proved to be more effective in information collection, but especially in idea generation, idea selection and idea implementation activities. The new concept mapping method enables broadened perception with more and diverse information items and more complex labels of the links. The broad perception is a good predictor for the number and the originality of ideas. The new method supports the evaluation of the ideas and the selection of the most appropriate one in order to be implemented into practice. The data support the expectation that the new concept mapping method brings a general beneficial effect over the different learning styles. It establishes a body of skills in all problem-solving activities information collection, idea generation, idea selection, and idea implementation. The new concept mapping method is a good problem-solving tool because it proves to be a reliable cognitive, affective, and meta cognitive tool. Concept mapping is a cognitive tool. The new concept mapping method represents a broader and more complex cognitive structure than traditional approaches with dominance of the structural, interrogative, causal and remote associative types of links over the descriptive types of links. The new method uses more complex verbal code combined with more simple one-directional visual links' structure. Cognitive mapping si an affective tool. The new concept mapping method gives more space for scanning a broad scope of not only cognitive but also affective problem solving representations - facts, assumptions, personal experience, and feelings. The psychological distance between type of information items on the scale of objectivity-subjectivity is larger in the experimental group. For example, data are very objective and feelings are very subjective. This is a prerequisite for braking the

17

fixedness of the existing patterns and stimulates real creative combinations in the idea generation phase. Concept mapping is a meta cognitive tool. Subjects in the experimental group knew that they would start with the map information collection and would end up with the map implementation. The externalization of the cognitive and affective structures by a sequence of maps involves very much perception. Perception itself takes over some of the mental tasks during the problem solving and thus reduces the memory overload. That makes the reasoning processes more easy and flexible. While the traditional method draws one picture trying to include all problem-solving activities, the new method creates the picture of the whole problem solving process sharing the cognitive load between the problem solving stages. The new method brings a perspective and a direction to the activities - from broad perception to planning.

It realizes not only a reflection on a particular map production

(information collection, idea generation, idea selection, or idea implementation), but also a reflection into (self-monitoring) concept mapping process itself. Concept mapping is a problem solving tool. The new concept mapping method brings the subjects in the experimental group to the full completion of the problem solving cycle - information collection, idea generation, idea selection and idea implementation.

Conclusions The experimental results have supported the validity of the master performer model of SMILE Maker agent and have given some insight for design solutions about functions of the Facilitator as a content advisor. Facilitator as a master performer of SMILE concept mapping method supports a user to score significantly on the expressiveness, extension, externalisation and entireness. Expressiveness Facilitator supports fluency and flexibility of nodes and links. For example, a pop-up message from the facilitator might appear when a client uses relatively small number of nodes, or predominantly 'hard' type of information items (facts and statistics) for an expense of 'soft' data like metaphors, and feelings. Facilitator could support establishment of a balance between the different types of links - descriptive, structural, causal, remote associative and etc. Some rules of free association and brainstorming, especially that of critics is rule out, also facilitate expressiveness. Extension The quota requirement for information items, alternative solutions, selection criteria and crucial factors for implementation broaden perception on what is the problem about and how it could be solved. Before a final version of a type of map can be delivered, several draft maps have to be made. They are an elaboration, modification and/or an extension of the first version. Externalisation There is a sequence of types of maps starting with map information collection and finishing with map idea implementation. Each type of map is based upon the results of the previous one. For example, during the first technique for idea generation the facilitator generates the components of the map information collection. Thus the facilitator supports the user's self-management. Entireness

18

Facilitator supports the completeness of SMILE concept mapping cycle consisting of map information collection, map idea generation, map idea selection and map idea implementation. The reminder from facilitator pops up if a user omits some of the stages.

On top of this, the facilitator helps a user to change her of his individual perceptual pattern, and to build more versatile individual problem solving style. Facilitator develops versatility in problem solving staying on the strong points of a particular style while minimising its weak points.

References 1.

Ahlberg, M (1993). Concept Maps, Vee Diagrams and Rhetorical Argumentation Analysis: Three Educational Theory-based Tools to Facilitate Meaningful Learning. The Proceedings of the Third International Seminar on Misconceptions and Educational Strategies in Science and Mathematics.

2.

Beyerbach, B. (1988). Developing a technical vocabulary on teacher planning: preservice teachers’ concept maps. Teaching and Teacher Education, Vol. 4, 1988.

3.

Buzan, T. (1996). The Mind Map. Book. N.Y.: Plume.

4.

De Bono, E. (1994). Water Logic. London: Penguin Books.

5.

Eden, C. L., Ackerman, F. & Cropper (1995). Getting Started with Cognitive Mapping. Supplied with Graphics COPE v2. Banxia Software, Glasgow.

6.

Hale, S. (1997). Concept Mapping. http://www.dc.peachnet.edu/~shale/humanities/composition/handouts/concept.html

7.

Hammond, L. (1999). Concept-mapping as directed reflection. http://xanadu.bournemouth.ac.uk/CD/HAMMOND/HAMMOND2.HTM

8.

Honey, P. & Mumford, A. (1992). The Manual of Learning Styles.

9.

Jonassen, D., Beissner, K. & Yacci, M. (1993). Structural knowledge. New Jersey: Lawrence Erlbaum Associates Publishers (LEA)

10. Jonassen, D. & Marra, R. (1998) Concept Mapping and Other Formalisms as Mindtools for Representing Knowledge. http://www.icbl.hw.ac.uk/~granum/class/altdocs/dav_alt.htm 11. Kommers, P., Jonassen, D. & Mayers, T. (1991). Mind Tools: Cognitive Technologies for Modelling Knowledge. Berlin: Springler Verlag. 12. Kommers, P. & Lanzing, J. (1987). Students' Concept Mapping for Hypermedia Design: Navigation Through World Wide Web (WWW) Space and Self-Assessment. Journal of Interactive Learning Research, 8 (3/4). 13. Kozma, R. (1987). The Implication of Cognitive Psychology for Computer-based Learning Tools. Educational Technology, 27 (11). 14. Kremer, R. & Gaines, B. R. (1999). Embedded Interactive Concept Maps in Web Documents. http://www.cpsc.ucalgary.ca/~kremer/webnet96/webnet_kremer.html

19

15. Lambiotte, J., Dansereau, D., Cross, P. & Reynolds, S. (1989). Multirelational Semantic Maps. Educational Psychology Review, 1 (4). 16. McAleese, R. (1997). A Theoretical View on Concept Mapping. http://www.icbl.hw.ac.uk/~granum/class/altdocs/ray_alt.htm 17. McCabe, D (1997). Concept Mapping Workshop. http://158.132.100.221/CMWkshp_folder/CncptMapp.Wkshop.html#CONTENTS 18. Honey, P.and A. Mumford. (1992). The Manual of Learning Styles. 19. Novak, J. & Gowin, D. (1984). Learning How to Learn. Cambridge: Cambridge University Press. 20. Pedley, B., Bretz, R. & Novak, J. (1994). Concept maps as a tool to assess learning in chemistry. Journal of Chemical Education, 71 (1). 21. Sherry, L., & Trigg, M. (1996). Epistemic forms and epistemic games. Educational Technology, 36 (3), 38-44. 22. Stoyanov, S. Aroyo, L., Kommers, P & Ku rtev, I (1999). SMILE Creator: A Web-based Tool for Problem Solving, Proceedings of WebNet'99 Conference, Hawaii. 23. Stoyanov, S. (1998) Ph.D. First Year Annual Report. TO-ISM. 24. Trochin, W. (1999). Research Methods Knowledge Base. http://trochim.human.cornell.edu/kb/conmap.htm 25. Wodtke, M von. (1993) Mind over Media. McGraw-Hill. 26. Zaff, B., McNeese, M. & Snyder, D. (1993). Capturing multiple perspectives: a user-centered approach to knowledge and design acquisition. Knowledge Acquisition, 5, 79-116.

20