A Study of End-User Programming: Challenges and Solutions prepared for

Penny J. Beebe Engineering Communications Program David Gries Associate Dean for Undergraduate Programs College of Engineering

by

Kenneth C. Arnold School of Electrical and Computer Engineering December 12, 2006

Table of Contents List of Figures......................................................................................................................... ii I.

Glossary................................................................................................................................... 1

II.

List of Abbreviations............................................................................................................... 3

III. Introduction............................................................................................................................. 4 IV. Sources.................................................................................................................................... 4 V.

Discussion................................................................................................................................5 A. Contexts............................................................................................................................5 B. Challenges........................................................................................................................ 6 1.

Cognitive Dimensions of Notation........................................................................... 6 a. General................................................................................................................. 6 b. Carrying out repetitive tasks................................................................................ 7 c. Codifying business processes...............................................................................7

2.

Industry resistance.....................................................................................................8

3.

Motivation................................................................................................................. 8

C. Solutions...........................................................................................................................9 1.

Integrate programming concepts into familiar tools................................................. 9

2.

Use more natural language......................................................................................10

3.

Demonstrate actions................................................................................................ 11

4.

Bring code to life.....................................................................................................13

5.

Add real-time structures..........................................................................................14

6.

Make programs concrete and visual........................................................................15

7.

Facilitate and encourage testing.............................................................................. 17

8.

Explain why things happened................................................................................. 19

VI. Conclusion............................................................................................................................. 19 VII. Works Cited...........................................................................................................................20 VIII. Appendix: Cognitive Dimensions of Notation...................................................................... 22

i

List of Figures Figure 1. Programming on a spreadsheet......................................................................................10 Figure 2. Testing a number-doubling robot.................................................................................. 12 Figure 3. A see-saw control program with dynamic variables..................................................... 13 Figure 4. Programming in Alice: Telling Kermit to hop over to Ladybug...................................16 Figure 5. Assertions on a spreadsheet...........................................................................................18

ii

I. †

Glossary Definitions for Cognitive Dimensions of Notation adapted from Green and Petre, 1996, p. 141;

full summary in the Appendix abstraction gradient†:

the degree to which program fragments can be grouped together for manipulation as a unit

assertion:

a concise statement of a condition that should be true

closeness of mapping†:

the degree of translation required between the user's typical way of thinking about a task and how the computer system requires that it be represented

Cognitive Dimensions of Notation: a common vocabulary used for discussing how a program is represented to the programmer consistency†:

the degree to which the existence, names, or properties of unfamiliar features can be predicted from knowledge of other features

continuous action:

an action that will run as soon as its trigger condition becomes true, regardless of control flow elsewhere in the program

dynamic variable:

a variable that constantly shows its current value

end-user programming:

programming done by ordinary, untrained users

explicit:

requiring teacher-like guidance from the user in the demonstration process (describing a PbE system)

guard:

synonym of assertion (in this paper)

implicit:

able to watch user actions and suggest automation without special guidance (describing a PbE system)

live:

always active, even during development

liveness:

a property of a programming environment where the elements of the program representation are live

loop:

a structure that executes a set of instructions repeatedly

natural language:

ordinary written or oral communication among humans

1

notation:

the computer presentation of a program, including both programming language and the interaction (browsing and manipulating) that the programming environment provides to the user

pokable:

the property of a programming environment where a fragment of code can be executed immediately (“poked”)

premature commitment†:

the necessity of deciding on a programming strategy or approach before having complete information about its applicability

program:

a description of a task at a level of detail appropriate for computer execution

programming:

the incremental task of developing a program, often within a computer application

programming by demonstration:

the process by which a computer semi-automatically automates a task by observing a user perform the task

programming by example (PbE):

synonym of programming by demonstration

real-time:

the property of a program or system that must process and respond to events as they occur

spreadsheet programming:

programming within a spreadsheet model of cells containing data and formulas

syntax error:

a cryptic error message that marks where the user's code deviates from the strict pattern required by the programming language

value:

the contents of a variable's storage location

variable:

a named storage location

viscosity†:

the difficulty of making changes to already-written parts of a program

visibility†:

the degree to which all relevant portions of a program can be made simultaneously visible

2

II. List of Abbreviations CD

one of the Cognitive Dimensions of Notation

PbD

programming by demonstration

PbE

programming by example

3

III. Introduction Classically, programmers have been professionals skilled in the art of abstract algorithmic thinking and well versed in the details of computer operation. These experts labor many years to craft applications, then package them for release to the public. End users learn to interact with the finished products in ways that have been predetermined by the experts. Such separation of programming from use works well when the user interaction is straightforward, such as creating a document to be printed. However, users increasingly want to produce more dynamic creations, such as interactive websites, dynamically updating spreadsheets, and games. Such computer use requires more sophisticated interaction: users must give instructions to the computer. That is, the end user is the programmer. My study focused on the contexts in which end users program, the challenges they face in doing so, and the solutions that have already been proposed. I focused my study on systems designed to empower end users, rather than to introduce them to concepts that they would later learn in studying popular programming languages. Justification for this focus is given in the Contexts section. IV. Sources The field of end-user programming is still in an exploratory stage, so most of my sources have been conference papers and journal articles. Most papers were published through the IEEE or the ACM. I also found useful information in a book chapter and a Ph.D. dissertation. The IEEE focuses a yearly symposium on this topic, the proceedings of which provide many useful papers. I have also used papers from various journals (by both IEEE and ACM). Finally, talking with Professor David Gries in the computer science department at Cornell helped me refine my focus.

4

V. Discussion A. Contexts A professional programmer is typically concerned with creating large, complete systems for customers. An end user typically approaches programming in much different contexts. This report will discuss challenges and solutions relevant to four general contexts. First, and least likely to be called “programming,” is carrying out repetitive tasks: upon noticing patterns in the user's actions, the computer system might guess an overall goal and offer to carry it out instead. The second context is codifying business processes such as accounting and payroll, which end users already do with spreadsheets like Microsoft Excel. Forms/3, another spreadsheet system, includes different structures and features, some described here, that make it suitable for end-user spreadsheet programming research (Burnet et al., 2001, p. 155). Making a spreadsheet may not look like programming, but, as discussed later, many challenges and solutions of more familiar domains of programming apply to spreadsheets as well. The third context is creating interactive art, including interactive stories, animations, and 3D worlds, for which the user must specify what should happen in response to various events. The final context is interacting with the outside world, such as programming a robot or running a laboratory experiment, which often necessitates detailed consideration of when events occur. Kelleher and Pausch (2005, pp. 84, 109) group programming systems for novices into two very broad (and somewhat overlapping) categories: (1) systems that introduce programming concepts to students who will go on to learn more widely used languages and (2) systems that focus on empowering the user to build new things. Designers of the second type of system do not limit themselves to developing skills that are transferable to popular programming languages; instead, they attempt to develop the best programming environment for the task at hand. However, the number of end users using programming as a means to an end is projected to grow much larger than those who describe themselves as regularly “do[ing] programming” (Scaffidi et al., 2005, pp. 1, 7). Since empowering systems (the second type) will thus affect a far greater number of people, I focused my research heavily on them. (In my opinion, ideas developed in empowering systems will eventually become integrated into widely used languages, making much less relevant systems whose only purpose is to teach the currently popular programming model.) 5

B. Challenges 1.

Cognitive Dimensions of Notation

Programming requires being able to think logically and communicate ideas precisely—fundamental abilities that can be difficult to develop. A programmer must also translate his/her ideas into the notations used by the computer programming environment. Here, notation is the computer presentation of a program, including both programming language and the interaction (browsing and manipulating) that the programming environment provides to the user. Good notations help the programmer develop and refine ideas; poor notations impede the programmer with unnatural formalisms. Yet good program notation has proven very difficult to develop. To help apply concepts from the fields of cognitive psychology and human-computer interaction to notation design, Green and Petre (1996, pp. 3-11) introduced the Cognitive Dimensions of Notation (each dimension is called a CD), summarized in the Appendix. The CDs identify qualitative properties that tend to make systems easier to use (marked by '+') or more difficult (marked by '-') and give very general metrics for determining to what degree each property describes the system. To illustrate these properties concretely, I summarize how they apply to conventional programming systems in the end-user contexts I identified. Afterward I discuss how various end-user programming systems address the identified shortcomings. a.

General

Some CD-identified shortcomings of conventional programming systems are apparent in all contexts. For example, they fare poorly in viscosity, which is a measure of the effort required to make a single change. After making any modification, a program almost always must be recompiled (translated into the computer's internal representation), often eliciting syntax errors (cryptic error messages that mark where the user's code deviates from the strict pattern required by the programming language). Once the system accepts the user's input, the program must usually be restarted from scratch. More fundamentally, conventional programs are usually sequences of instructions; when part of a sequence is changed, all instructions must usually be re-run. In contrast, many end-user programming systems I describe allow much more organic changes, instantly showing the results of any change. Such systems also help to structure the representation so that changes need to be made only in localized regions. 6

b.

Carrying out repetitive tasks

Users often need to do roughly the same thing to a large number of items. For example, a user may want to add a border to each figure in a document. Rather than repeating the same sequence of actions for each figure individually, the user may prefer to have the computer automate the process. To do so, he/she first must describe the intended action to the computer but may have trouble finding a programming-language description of the desired task (despite being able to perform the task readily by keyboard-and-mouse interaction). The “closeness of mapping” CD measures the degree of translation required between the user's typical way of thinking about a task and how the computer system requires that it be represented. For example, if the programming system does not offer a command “add border” or a list of “all figures,” then the user must translate these ideas into difficult technical concepts having confusing names like “createFramingObject” or “FloatingElementIterator.” On the other hand, if the user could just perform the action once, while the computer watches, practically no mapping would be needed. c.

Codifying business processes

Many users, especially in accounting positions, already create interactive spreadsheets and share them with co-workers. That is, they are already practicing end-user programming. However, they often find that to create reliable spreadsheet programs is difficult, since spreadsheets were not designed for programming. One of the worst failings is in the “abstraction gradient” CD: users cannot encapsulate formulas that they have created to use them elsewhere. After developing a working formula for a specific task in one place on the spreadsheet, the user must copy the formula and paste it everywhere it is needed, changing it to refer to the correct cells. Moreover, formulas may be correct for some cases but incorrect for others; without encapsulation, it is difficult for the user to explore different cases. If the user does find an error, fixing it in one copy of the formula does not change any of the copies already made. In CD terms, a spreadsheet system has high “viscosity,” since every copy of the formula must be manually changed if the user wishes to modify it.

7

2.

Industry resistance

Many in the industry contend that skill and experience are necessary prerequisites to developing software, so we should be cautious in giving ordinary users the power of programming. A common concern is security: self-taught coders tend to write insecure code. Warren Harrison, Editor in Chief for IEEE Software, worries about the breach of privacy and lost revenue that could result if an important website is hacked because a self-taught programmer wrote vulnerable code (2004, p. 7). Many have also been concerned about errors in important spreadsheets and databases designed by end users. This concern led directly to the work on debugging and testing that I summarize later, so I will not cover it further here. 3.

Motivation

Not all users will want to program in all cases. The average user prefers pressing the Record button on a VCR to programming it, even though programming would make recording a favorite show easier and more reliable. Blackwell explains this phenomenon by presenting the Attention Investment model, which analyzes programming behavior as decision theory. The user has certain “attentional units” to invest in either doing a task manually or programming a way to automate it (including the attention required to learn how to program). The decision of whether to program (or whether to attempt a specific programming strategy) is then a matter of balancing cost, investment, risk, and pay-off. In the VCR example, the user might choose to stay up late to start a recording manually because the pay-off of sleeping earlier is not worth the cost of learning how to program the VCR or the risk of the programming's not working (2002, pp. 6-8). The inventors of several systems that I studied used this model to help predict and quantify when users will attempt specific strategies or programming in general. The motivation for and approach to programming can differ by gender. One study found that females were reluctant to give feedback to a debugging system when they were not confident about their judgments. The study's authors addressed that flaw by creating a means to express judgments that are less confident. Females also tended to want more details before trying an action, so the authors modified their on-line help system to provide more targeted, detailed feedback instead of emphasizing learning-by-doing. By addressing gender differences in the way that end-

8

users approach programming tasks, designers of programming environments can encourage effective use by both genders (Beckworth et al., 2005). C. Solutions Having laid out the challenges that end users face when programming, I will now summarize a few basic approaches that some systems have taken to overcoming those challenges. This section is not intended to be an exhaustive coverage of end-user programming systems; nor are the identified approaches intended to be orthogonal. Instead, I highlight certain general ideas in the context of real-world systems. 1.

Integrate programming concepts into familiar tools

Many people use spreadsheets to codify business processes. I described in Section B, “Challenges,” how current spreadsheets fail in the CD of abstraction gradient: it is nearly impossible to capture and re-use a successful formula. To address this concern, Jones et al. (2003, pp. 165-176) propose to add reusable abstraction to the Excel spreadsheet, which would allow users to define new functions (like the built-in “sum” or “average” functions) using the familiar spreadsheet computation model. The CD framework described in II.A.1 was central to their design process. For example, instead of developing a separate programming language, they let the user define a new function by just making another spreadsheet. Leveraging the user's conceptual familiarity with creating and manipulating formulas in the spreadsheet model gives the new feature consistency with the rest of the environment—a Cognitive Dimension. Premature commitment was minimized by allowing the user to create functions from existing formulas, allowing the user to defer the choice of defining a function as long as possible. The authors also used the Attention Investment model to maximize the payoff of using the new feature while minimizing costs and risks. They paid particular attention to reducing risk when using the feature, identifying and avoiding the possibility that data could get “trapped” in their new matrix data type, that is, unable to be converted back to familiar blocks of cells. Figure 1 shows how a user might define a new function, called F2C, for converting temperatures from Fahrenheit to Centigrade. The top half shows the usual spreadsheet; the bottom half shows the definition of the F2C function. The model of cells containing formulas, used in the rest of the spread9

sheet, is also used in defining the function. Arrows clearly show bidirectional dependencies, avoiding hidden dependencies (another CD). Thus, within the limited domain of spreadsheet programming, the proposed addition to Excel offers what seems to be a viable solution to the challenges identified in the context of codifying business processes. 2.

Use more natural language

Many see programming in natural language—ordinary written or even oral communication—as the ultimate target for end users and even professionals. Most attempts in this direction have been troubled by the ambiguity and imprecision of unstructured human communication. One notable success, however, has come in the realm of interactive fiction. The Inform 7 system allows users to create text-based virtual worlds by describing them in a subset of English. The author can assert that “the kitchen is a room,” or trigger a certain action “if a policeman sees a gun that is carried by a criminal.” Since the limited domain eases the challenge of natural language understanding, the authors can concentrate on the system as a whole. Though the authors did not report considering CD principles, the resulting system is a good example of them. Inform 7 provides a very close mapping to the problem domain: En-

Figure 1. Programming on a spreadsheet (Jones et al., 2003, p. 168) in a subset of English. The mapping is also consistent, since the built-in models and default rules glish-language interactive stories are written

attempt to model how real-life objects typically behave in text adventure games. Changing most behaviors is easy, requiring modification of only a small amount of text, so (per the CD vocabulary) the representation has low viscosity. Visibility is also good, extending the familiar model of a book with browsing and indexing features. The system does not support arbitrary abstraction but does support various domain-specific abstractions, such as kinds (“an apple is a kind of food”) and rules (“can't reach inside closed containers”) with exceptions (“when the ring is worn, 10

ignore the can't reach inside closed containers rule”) (Nelson, 2006). Working in the system still has the “flavor” of programming, but the natural-language interface makes the process of programming more like the product: an interactive fiction story. While Inform 7 is limited to one domain, some progress has been made in more general natural language programming. No complete systems yet exist, but development progresses on critical supporting components. Mihalcea et al. (2006, pp. 319-330) present results for one of the most challenging problems: identifying the main steps and repetitive structures. Their system located these programmatic structures in a corpus of student programming assignments and created program skeletons—programs with structure but no actual instructions. The system correctly identified 75% of the steps and 71% of the loops that were identified by careful manual marking. 3.

Demonstrate actions

Performing a task is, in general, significantly easier than teaching someone else how to perform it. If the other person can learn just by watching, then explaining the process is unnecessary. Programming by demonstration, also called programming by example (PbE), applies this concept to instructing a computer. Instead of describing a task, the user simply performs it while the computer records the actions taken. Then, when the task is to be repeated, the computer follows the user's example. In the simplest implementation, the computer would parrot the user's actions exactly, regardless of suitability to the situation; if it saw the user change a '2' to a '4' when editing a document, it would replace any number (worse, any character) by '4'. In more advanced implementation, the computer might infer generalized actions and conditions; from observing user behavior in other cases, it might conclude that the user doubled a number if it was greater than zero. PbE systems fall into two broad categories: explicit and implicit. An explicit system must be specifically instructed, with the user playing the role of a teacher. The little-used macro recorder found in many office productivity applications is the simplest example of such a system; when activated, it records all of the user's actions for later playback. The recorded macro program is often editable, but only by those familiar with conventional programming languages. Much more advanced is ToonTalk, in which young children demonstrate actions to robots in a video game 11

environment. For example, in Figure 2, the user has just shown the robot how to double the number '1' and is now testing it. The thought bubble shows the number that the user has given the robot to work with. At this point, the robot is doing exactly what the user demonstrated: it made a copy of the number and has just summoned the mouse (who performs arithmetic) to “smash together” (add) the number and its copy. The robot shows that it knows how to double '1' to get '2', but if the user gives it a different number, such as '5', it will respond that it does not know what to do. The user should note that the number '1' in the robot's thought bubble is an extraneous detail; while the robot “thinks” that it can only work with that number, the procedure actually works for any number. The user removes the specific number from the box, so now the robot will take any number and double it (Kahn, 2004, pp. 253-256). The demonstration aspect in creating the program is very explicit, since the child must train the robot, then specify what details were relevant. The child must even make decisions explicit; even if the user demonstrates a different behavior when, say, a number is zero, the system will not automatically infer a comparison (Hancock, 2004, pp. 21-26). Instead of needing explicit training, an implicit PbE system passively watches the user's actions. Upon detecting that the user is repeating certain steps, it offers to automate the task. The immedi- Figure 2. Testing a number-doubling robot (Adapted from Kahn, 2004, p. 255) ate goal of implicit PbE differs from the rest of the approaches I have researched: it seeks not to enable the user to teach the computer to perform a general task but to reduce tedium in ordinary computer use (though the former may happen anyway as an unexpected benefit). For example, if the user adds a border to several figures in a document, the implicit PbE system will offer to add a border to the remaining figures. Or if the user always forwards messages with certain words in the subject to another person, it might offer to forward all such messages automatically. The main challenge for the computer is to identify repeated actions even when they are slightly different, done out of order, or interspersed with 12

non-repeated actions. Successful systems have used pattern-finding algorithms to identify repetition, but few can infer conditional operations. Moreover, upon identifying repetition, the system must decide if it should offer automation to the user. Offering too many suggestions annoys and distracts the user, but offering too few misses opportunities. Existing systems use machine learning techniques (well outside the scope of my research) to learn when to make suggestions (Ruvini, 2004, pp. 1-4). 4.

Bring code to life

Classical programming has a repeated cycle of design and execution. During design, the programmer constructs a program from lifeless symbols or graphics. When executed, the program comes to life, but the original representation is hidden or hard to change. Thus the programmer must imagine how each element will function while designing the code, then remember the design while the program is running—a difficult proposition for a beginner. In contrast, many enduser programming systems are live: the program representation is always active, even while under development. Liveness was crucial in the design of the Flogo II robotics programming system, described by Hancock (2003, pp. 72-83). Figure 3 shows an early developmental stage of a program written by two teenagers (aged 14 and 15) to control a robotic see-saw. The goal is to tilt the see-saw so that a ball rolls back and forth along its length without rolling off either end. The position of the ball, stored in a variable P, is given by a distance sensor (“white_distance()” in the program text). The number in braces (currently “{19}”) is always the current value of P. (A variable is a named storage location; its value is the contents of that location.) This single

P: {19} white_distance() SPEED: {11} delay(P,10) – P WHEN SPEED < -8 DRIGHT() WHEN SPEED > 8 DLEFT() Figure 3. A see-saw control program with dynamic variables (Hancock, 2003, p. 12)

line of code demonstrates two live aspects of the Flogo II environment. First, it has dynamic variables, like P, which always show their current value. Other programming systems require the user to stop the program or add display commands in order to see variable values. Second, actions in it are continuous: the value of P is always updated to the most current sensor reading. In an environment lacking continuous actions, P would get updated only when control flow reached 13

the assignment statement, an event that can be difficult to predict. The two live elements—dynamic variables and continuous actions—create what Hancock calls a “steady frame,” a stable representation of the state of the program, in which variables can be assigned a consistent and visible interpretation (Hancock, 2003, pp. 56-61). Other live programming environments include spreadsheets, where changes in data or formulas produce immediately visible results, and a prototype system called Subtext, developed by Edwards (2005, pp. 505-518). Although it does not explicitly target end-user programming, it has several interesting qualities. Like Flogo II, Subtext is live: the user can change any input value at any time and immediately see the results. But unlike Flogo II, it can show the hypothetical results of an action before executing it. For example, instead of running a program several times from the start to observe how it responds to what a user types, a Subtext programmer can investigate a program while it is waiting for input, exploring what would happen hypothetically if different inputs were given and changing the program in place if the outcome is incorrect. This capability would be especially useful in the contexts of codifying business processes or creating interactive art, cases where the user often has extensive non-programming experience and thus knows the correct result well but lacks the experience needed to program it correctly without aid. 5.

Add real-time structures

In addition to being live, most programs written in Flogo II have the property of being real-time, meaning that they process and respond to events as they occur, which is often not the most convenient time for the programmer. Real-time programs are possible in almost all languages, but their implementation in Flogo II is exemplary in its simplicity. The third through fifth lines of the students' see-saw control program (Figure 3) show real-time event handling. The first pair indicates that when (as long as) the ball's SPEED is less than -8, the DRIGHT task runs, which controls the motor to bring down the right side of the see-saw; the second pair implements the corresponding situation for positive speed. In a conventional programming environment, the programmer would have to either write nontrivial code to check the value repeatedly or add to the code that updates the SPEED variable1. Neither solution approaches the simplicity and readability of 1 Some languages provide event-handling structures like callback functions or signal-slot event passing (definitions outside the scope of this paper), but it can be very difficult to add to the set of events that the environment can fire. In Flogo II, however, using a user-defined event like SPEED > 8 is simple and compact.

14

these two-line Flogo II structures. Implementing even just the second line, which computes SPEED by subtracting the current position from the position one second prior (10 tenths of a second), requires advanced knowledge in many other languages. The bold format on DLEFT shows clearly that it is currently executing, unlike the unbolded DRIGHT. But even inactive fragments like DRIGHT can be “poked” (by clicking on a user-interface element not shown in the figure) to instantly try its behavior. The live, real-time, and pokable Flogo II system allowed children aged 10-15 to successfully complete a wide range of video game and robotics challenges (Hancock, 2003, pp. 72-83). An illustrative counterexample is the popular LEGO™ MINDSTORMS robots programming system, which lacks real-time structures even in its newest release. Since no papers have yet been published about the new system this information is based on screenshots and an online tutorial (Yocum, 2006). The programming environment is visual, but the paradigm is still a sequence of instructions; the robot executes visual “blocks” one after another. Instead of the “WHEN” statement of the Flogo II example, a MINDSTORMS program would require creating a loop in which the robot would read the speed, compare it to the two criteria (8 and -8), and make a threeway decision of how to control the motor (left, right, or stopped). Additionally, while MINDSTORMS does have a limited way of showing live sensor readings (though not values of variables), I have not encountered any means of “poking” the robot code to try out one individual action. Nor have I seen a way to group a set of blocks together to form a single reusable action. Thus, although I have not been able to try either system, I think that Flogo II is probably much easier to use than the MINDSTORMS system. 6.

Make programs concrete and visual

Alice (Cooper et al., 2003, pp. 191-195) is a graphical environment for programming object behavior in a 3D world. Although it was designed to introduce students to object-oriented programming concepts before switching to a conventional programming language, it is also applicable to end users. Its concrete visual structure eases the challenge of interaction and programming: the 3D world, continuously rendered on the screen in 3D, is pre-populated with some familiar objects (e.g., a frog and a ladybug) that respond to familiar commands ('move forward', 'point towards', etc.). Programs are also represented visually, as shown in Figure 4. In this example, the 15

user has programmed the frog (named Kermit) to hop over to the ladybug. The color-coded boxes show increasing levels of detail. Most generally, the task is composed of two subtasks done in order: turn Kermit to face the ladybug, then have Kermit hop until he reaches her. The second subtask requires a loop, specifying that something will be done continuously as long as some condition holds. In this case, the condition (called a “question” in Alice) is expressed in a natural form: “while kermit is at least 1 meter away from ladybug.” After creating the loop, the user probably entered only “kermit.littleHop,” telling the frog to perform its hop action (defined earlier, possibly by the user). However, upon trying the program, he/she noticed that the frog ran into a red flower and got stuck. So he/she added some logic to hop over the flower if it is in the way. We can make several observations from this example. First, using natural vocabulary such as “point at” and “1 meter away from” improves Alice's closeness of mapping to the familiar concepts of 3D worlds. (Turning a 3D object usually requires the concepts of matrix algebra; see Conway et al., 2000, p. 491 for a detailed discussion of vocabulary issues in Alice.) Second, the boxes and formatting make Alice's visual program representation less intimidating than the braces and parentheses in conventional programming languages, even though both express the same structure. (Alice can show programs in pure-text form on request.) Finally, object-oriented programming concepts, often difficult for beginners to grasp, are natural when objects are familiar (like a frog in a 3D world).

Figure 4. Programming in Alice: Telling Kermit to hop over to Ladybug (Cooper et al., 2003, p. 193) 16

Alice is not live, real-time, or pokable. To some extent, these properties are not applicable in its environment; there are no sensors or events out of the direct control of the programmer. But incorporating these ideas would make Alice more powerful and programming in it easier; for example, a “question” such as “kermit is at least 1 meter away from ladybug” could change color if it is currently true, or double-clicking an instruction could execute it immediately. 7.

Facilitate and encourage testing

Nearly all nontrivial programs contain flaws of some sort; almost everyone has experienced a computer crash. Unsurprisingly, end users also make programming mistakes. Lacking experience in identifying and correcting problems, they need guidance to use testing techniques. One simple and well known technique is the use of assertions, which are concise statements of conditions that should be true. An assertion is said to have failed if the condition ends up false when the program executes. Failures indicate either faulty code or faulty assertions, both of which clearly point out an error in the programmer's thinking. Despite the helpfulness of assertions, users often do not see the benefit of creating them. One reason is that an assertion is visibly helpful only if the program is visibly incorrect; many users consider their programs correct when in fact they have failed to consider some cases. The same pitfall applies to other reliability-enhancing techniques. To help end users design reliable programs, a system should seek to surprise the user by showing cases not considered. Seeing a potential bug piques curiosity, inspiring the user to search for an explanation for his/her creation's unexpected behavior. Locating and fixing the bug makes the program visibly more correct, a clear reward for the effort. Wilson et al. (2003, pp. 305-312) studied user response to the surprise-explain-reward strategy in the context of spreadsheet programming (a generalization of the grid-of-cells concept in, e.g., Microsoft Excel). Their system allows users to specify assertions about the values in each cell. Figure 5 shows, in simplified terms, how their system represents a pay calculation spreadsheet. The input data is a set of cells representing the number of hours worked every weekday. The user has asserted that the number of hours worked each day must be between 0 and 8; the assertion (also called a guard in this environment) passes for the actual values shown. To determine the total number of hours worked in the given week, the user has entered a formula in the “Weekday Hours” cell that references the named input cells. (Computed cells can also reference other com17

puted cells.) However, the user has made an error in the formula by including Tuesday twice. By trying various input values, the system has automatically determined that possible values for the Weekday Hours cell range from 0 to 48. This result surprises the user, who expects the maximum to be 40 and gives the corresponding user assertion. The red highlight represents that the user assertion does not match the actual range. The user then seeks an explanation for the mismatch, finds the mistake in the formula, and fixes it (not shown). Now the assertion passes, assuring the accuracy of the program and protecting against introducing errors in the future. The preceding example represents only the manual creation of assertions. The system that Wilson et al. describe goes a step further to automatically create assertions. A “Help Me Test” feature attempts to make a bad guess at an assertion, aiming to surprise the user by creating an obviously incorrect situation (such as a negative number of hours worked in a day). The user would then correct the assertion and apply it to the cell. The system includes explanations that guide the user to making the correction. The authors found that 94% of study participants followed the expected sequence of being surprised, seeking explanation, and taking corrective action, with some skipping the explanation step. Though the participants were never explicitly told about the assertions feature, 94% of them entered at least one assertion, showing that the “Help Me Test” feature and explanations helped them to discover the technique. They used assertions successfully, entering an average of 18 per participant, with an average accuracy of 95%. Input data Cell Name

Monday

Tuesday

Wednesday

Thursday

Friday

Assertion

0 to 8

0 to 8

0 to 8

0 to 8

0 to 8

Current Value

3

8

6

8

7

Computed data Cell Name

Weekday Hours

User Assertion

0 to 40

Actual Range

0 to 48

Formula

Monday + Tuesday + Tuesday + Wednesday + Thursday + Friday ( = 40) Figure 5. Assertions on a spreadsheet (Adapted from Wilson et al., 2003, p. 307) 18

8.

Explain why things happened

Any programmer will eventually run into a perplexing situation when testing a program; sometimes a puzzling behavior is caused by a mistake in typing, other times by a mistake in thinking. Either way, the first question that programmers ask is often “why did” or “why didn't” something happen. Ko and Myers (2004, p. 151) developed an Interrogative Debugging system called the Whyline, prototyped within the Alice environment. By tracing the reason that the program performed or did not perform certain possible actions, the Whyline helps users hypothesize what caused errors. In a user study, the prototype tool reduced debugging time by a factor of 8. VI. Conclusion End users face many challenges when approaching programming. Some challenges are fundamental, but others are merely the result of poor design of programming environments. Prototype systems have taken various strategies to overcome these challenges, some quite successfully. Combining techniques and ideas from different systems and situations while consistently applying the principles of human-computer interaction embodied in the Cognitive Dimensions vocabulary promises to bring continued advances in end-user programming. An untrained user may never be able to write a word processor or web browser, but with these advances he/she will be able to automate repetitive tasks, codify business processes, create interactive art, and create robots that interact intelligently with the world. Finally, as new techniques are developed and perfected for end-user programming, they will eventually become applicable to professional programming also, boosting productivity and reducing software errors.

19

VII. Works Cited Beckworth, L., S. Sorte, M. Burnett, S. Wiedenbeck, T. Chintakovid, and C. Cook. (2005) Designing Features for Both Genders in End-User Software Engineering Environments. Proceedings of the IEEE 2005 Symposium on Visual Languages and Human-Centric Computing. ftp://ftp.cs.orst.edu/pub/burnett/. Accessed on October 29, 2006. Blackwell, A.F. (2002) First steps in programming: a rationale for attention investment models. In: Proceedings of the IEEE 2002 Symposia on Human Centric Computing Languages and Environments. http://ieeexplore.ieee.org/. Accessed on October 29, 2006. Burnett, M., J. Atwood, R. Djang, H. Gottfried, J. Reichwein, S. Yang. (2001) Forms/3: A FirstOrder Visual Language to Explore the Boundaries of the Spreadsheet Paradigm. Journal of Functional Programming, v. 11, no. 2: pp. 155-206. Conway, M., S. Audia, T. Burnette, D. Cosgrove, K. Christiansen. (2000) Alice: lessons learned from building a 3D system for novices. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York, NY: ACM Press. http://portal.acm.org/. Accessed October 20, 2006. Cooper, S., W. Dann, and R. Pausch. (2003) Teaching objects-first in introductory computer science. ACM SIGCSE Bulletin, v. 35, no. 1: pp. 191-195. http://portal.acm.org/. Accessed November 20, 2006. Edwards, J. (2005) Subtext: uncovering the simplicity of programming. In: Proceedings of the 20th annual ACM SIGPLAN conference on Object oriented programming, systems, languages, and applications, eds. Johnson and Gabriel. New York, NY: ACM Press. http://portal.acm.org/. Accessed on October 1, 2006. Green, T. R. G. and M. Petre. (1996) Usability Analysis of Visual Programming Environments: A 'Cognitive Dimensions' Framework. Journal of Visual Languages and Computing. v. 7, no. 2: pp. 131-174. Hancock, C. M. (2003) Real-Time Programming and the Big Ideas of Computational Literacy. Doctoral dissertation. School of Architecture and Planning, Massachusetts Institute of Technology, Cambridge, MA. http://llk.media.mit.edu. Accessed on October 15, 2006. Harrison, W. (2004) From the Editor: The Dangers of End-User Programming. IEEE Software. v. 21, no. 4: pp: 5-7. Jones, S. P., A. Blackwell, M. Burnett. (2003) A User-Centered Approach to Functions in Excel. ACM SIGPLAN Notices, v. 38, no. 9: pp. 165-176. http://portal.acm.org/. Accessed November 19, 2006. Kahn, K. (2004) ToonTalk – Steps Towards Ideal Computer-Based Learning Environments. In: A Learning Zone of One's Own: Sharing Representations and Flow in Collaborative Learning Environments, eds. Tokoro and Steels. Amsterdam: IOS Press. 20

Kelleher, C. and R. Pausch. (2005) Lowering the Barriers to Programming: A Taxonomy of Programming Environments and Languages for Novice Programmers. ACM Computing Surveys. v. 37, no. 2: pp. 83-137. Mihalcea, R., H. Liu, and H. Lieberman. (2006). NLP (Natural Language Processing) for NLP (Natural Language Programming). In: Proceedings of the 7th International Conference on Computational Linguistics and Intelligent Text Processing. Berlin: Springer Berlin / Heidelberg. http://www.gelbukh.com/cicling/2006/. Accessed November 20, 2006. Nelson, G. (2006) Natural Language, Semantic Analysis, and Interactive Fiction. Unpublished white paper. http://www.inform-fiction.org/. Accessed October 29, 2006. Ruvini, J. (2004). The Challenges of Implicit Programming by Example. In: Proceedings of the 2004 International Conference on Intelligent User Interfaces. http://www.research.ibm.com/iui-workshop/. Accessed on November 9, 2006. Scaffidi, C., M. Shaw, and B. Myers. (2005) Estimating the Numbers of End Users and End User Programmers. In: Proceedings of the IEEE 2005 Symposia on Visual Languages and Human-Centric Computing. http://ieeexplore.ieee.org/. Accessed on November 19, 2006. Wilson A., M. Burnett, L. Beckwith, O. Granatir, L. Casburn, C. Cook, M. Durham, G. Rothermel. (2003) Harnessing curiosity to increase correctness in end-user programming. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York, NY: ACM Press. http://portal.acm.org/. Accessed on November 9, 2006. Yocum, D. (2006) NXT Tutorial. http://www.ortop.org/NXT_Tutorial/. Accessed November 19, 2006.

21

VIII.Appendix: Cognitive Dimensions of Notation (adapted from Green and Petre, 1996, p. 141) +

Abstraction gradient

What are the highest and lowest levels of abstraction? Can fragments be encapsulated?

+

Closeness of mapping

How closely do the building blocks and concepts in the programming environment map to the way the user specifies the problem? (What “programming games” need to be learned?)

+

Consistency

When some of the language has been learned, how much of the rest can be inferred?

-

Diffuseness

How many symbols or graphic entities are required to express a meaning?

-

Error-proneness Does the design of the notation induce careless mistakes?

-

Hard mental operations

Are there places where the user needs to make a better representation for him/herself (e.g. penciling-in annotation) to keep track of what's happening?

-

Hidden dependencies

Are there some dependencies that are not indicated in both directions? Are dependency indications merely symbolic, or also perceptual?

-

Premature commitment

Do programmers have to make decisions before they have the information they need?

+

Progressive evaluation

Can a partially complete program be executed to obtain feedback on “How am I doing”?

+

Roleexpressiveness

Can the reader see how each component of a program relates to the whole?

+

Secondary notation

Can programmers use layout, color, or other cues to convey extra meaning, above and beyond the 'official' semantics of the language?

-

Viscosity

How much effort is required to perform a single change?

Visibility

Is every part of the code simultaneously visible (assuming a largeenough display), or is it at least possible to juxtapose any two parts sideby-side at will? If the code is dispersed, is it at least possible to know in what order to read it?

+

22