Agile Software Development Methodology Mandar A.Mulherkar North Carolina State University [email protected]

Abstract The rapid change and competition in the software market demands that the time required for any software product to reach the market be reduced for the product to survive competition from other developers. In this regard, we look at issues faced by the developers and try to understand the need for more flexible software development methods, for faster product development. Here flexibility means the ability to adapt to the changing requirements of the project over the period of time the software is being developed and maintained. ‘Agile software development methods’ are a new breed of flexible software development practices and principles which cater to the growing needs of the software industry by creating software rapidly using methodologies that favor the people working on the project. [1]. The agile software development methodology is characterized by iterative and incremental software delivery, more interaction with customers, clients and team-members, a simple approach, rapid software delivery, a people-centric approach and an adaptive behavior which allows for making last minute changes to the software requirements. [3, 15] In this paper, after looking at what constitutes an agile development method, we will look at ways to determine the suitability of the agile methods to various project situations. The decision to implement the agile methods will depend on the state of the communication between people and the work culture that surrounds them. [3,15]. We will then look at a prominent agile software development method, Extreme Programming (XP). We will look at the goals and values surrounding XP, and how the principles of “Agile” are applied to the various stages of software development. We look at the applications and suitability of XP to various types of development projects.

1. Introduction:

The Agile software development processes are trying to bring about a change in the way software is developed. Agile methods give more importance to people, face-to-face interaction with team members and clients and having more emphasis on creating working software rather than on documentation [1]. Many new methods that conform to the agile principles have been introduced, and XP (extreme programming) is one of these.

2. What is Agile Software development methodology? Pekka et al [3, 15] describe Agile Software Development Methodology as the “quality of being Agile; readiness for motion, nimbleness, activity, dexterity in motion”. All over the years, the main problems faced by the industry have been a race with time to deliver products. There are various process models available, which help in building software products in an organized manner, but we have realized, (and I can say even from my personal experience) that only organized efforts do not mean faster delivery. The various issues that cause deadlines to be extended and late bug prone products to be shipped are not yet completely addressed. The main factor of any software development are the people working on the project, and they should be given due importance. A “people-centric approach”, which is the philosophy of Agile development should be taken seriously [3, 15]. From my personal experience in the industry, currently the most important factor in any project development is getting the work done. This is according to the requirements specification as decided initially, and the Software Process followed. This leaves us with no “agility” to adapt to changing situations. Risk management might not always be proactive about some unforeseen changes, nor does it address the issues of how developers should “feel” [16]. Agile Software Development starts with the thought of “change is permanent” and accordingly readies itself for change. [2]

The agile software development methodology is characterized by iterative and incremental software delivery, more interaction with customers, clients and team-members, a simple approach, rapid software delivery, a people-centric approach and an adaptive behavior, which allows for making last minute changes to the software requirements. [3, 15] According to Pekka et al, [3, 15], we have the following definition:

2.1 Incremental Incremental means having an iterative approach with small but continuous software releases which are a result of rapid software development cycles.

2.2 Cooperative This means the customer (as defined and understood during project initiation, could be an internal or external customer) will be in close contact throughout the software development cycle, and work with the developers and maintain good communication. Good communication means being in constant touch with the developers and attending meetings to discuss project progress and direction.

2.3 Straightforward Straightforward means the method or the development process will be intuitive, easy to use and easy to learn.

2.4 Adaptive Adaptive is the ability to make changes to the product even at critical phases of delivery. This is the essence of Agile development in addition to a peoplecentric approach. [3, 15]

3. Definition and the birth of the Agile Development methods. The business community always needs to deliver products faster and cheaper to stay in the competition. They require light weight software development processes and fewer overheads. The Agile software development came into existence in the software development world with the Agile Software development manifesto. The Manifesto came up with the following ideas (which were described in section 2 brief). Agile Philosophy is based on the following core ideas:

3.1 Individuals and Interactions should be given priority over processes and tools used. Many a times developers are expected to use tools that are available, are maybe inexpensive, or which have “always been used” [7]. Also all developers are expected to stick to the defined development cycle, and to come out with a test plan and code as given in the requirements specifications. The Agile philosophy means that the already existing processes, tools and techniques are not considered final, and tools and processes are determined according to the particular needs of the people (developers, customers, testers involved) and are flexible enough to allow for the developers involved to focus on the current status of the development process. This helps to adjust to changes in a better way. [3]

3.2 Agile processes emphasize a relationship and communality of software developers In the agile process, the above philosophy is manifest as good team relationships, a close working environment and close physical arrangements (not in separate cubicles), and other procedures which can work to boost team spirit. It is a well known fact that boosting team morale will and does increase productivity, and having a close knit team helps strengthen the bonds and the trust among team members, essential to making the Agile software development processes work in the way it is envisioned. An example of agile development would be the Google development teams, which work in teams of two-five developers, and the developers get to decide how they want to work on a problem, as well as which team they want to join. (http://www.google.com/jobs/working.html)

3.3 Working Software over comprehensive documentation: The main objective of a software team, which also helps build a good morale and bond among the team members, is the feeling that the team as a whole is performing well. As programmers, common knowledge tells us that this can be achieved when we develop a working system, when a working product is made available, which is appreciated by the client. So, in Agile development processes, working software is continuously turned out, at frequent intervals. New software is released at frequent intervals, sometimes hourly or daily basis, but more often bi-monthly or monthly. [10] Also, the developers keep the code technically as advanced as

possible, but simple and straightforward. This allows for less amount of documentation required to be turned out. Also it is expected that the code itself has documentation in the form of comments, so less time is spent on separate documentation [2]

practices they use, but their recognition of people as the primary drivers of project success, coupled with an intense focus on effectiveness and maneuverability. This yields a new combination of values and principles that define an agile view”. [17]

3.4 Customer collaboration over contract negotiation

The Agile process is both light and sufficient. Lightness means being able to change the set rules, as and when required, but adhering to the Agile principles. Sufficiency is when the process makes the development team stay in the competition, by providing competitive methodologies that get quality work done. Cockburn has suggested some rules, which helps in successfully completing projects. These are: - There should be few people in a room, which enhances communication and gives a community feeling. - There should be a presence of onsite experts, who can provide short and continuous feedback cycles. - The project can be delivered in short increments, which allows for quick testing and repairing. - Fully automated regression tests which allow for unit and functional tests that help stabilize the code and allows for continuous improvement. [17]

Customer collaboration is given more importance than in traditional processes. The relationship and cooperation between clients and developers are given preference over strict contracts, but also, the importance of well drafted contracts is seen as necessary as the project size grows. The negotiation process should be used to maintain a good relationship among the clients and developers. The Agile development focuses on delivering software with business value as quickly as possible so as to not risk any non-fulfillment regarding the set contract. Also, close customer interaction will help build long term good-will and help acquire more customer business. At the same time, too much involvement from the customers’ side may harm the company image if indiscipline is exposed to him. The Customer may also be responsible to help in the project completion, if the project is going off course. [13]

3.5 Responding to change over following a plan The customer representatives as well as the developers should be well informed at all times so as to take competent decisions regarding the changes and the project situation. The team members should be authorized to take these decisions, which again is the central idea of Agile development, and which helps in reacting rapidly to change, and not wait for management decisions for problems which could be dealt with at the programmer level. The developers and the clients should be involved and be well informed, competent enough and authorized to adjust to changing requirements during the development life cycle. Also, the contracts should be flexible enough to allow for such changes, in permission with the developers and the customer representatives. [17] Also, a team will be more effective in responding to change if the time to seeing the consequence of the decision from the time the decision was made can be reduced. This should happen in addition to the cost of communication being low [17] According to Highsmith and Cockburn (2001, p. 122), “what is new about agile methods is not the

Miller (2001) gives the following characteristics to agile software processes which allow fast delivery and short software development life cycle. - There should be modularity on development process level. Modularity allows for cleaner code and better maintenance. - The process should be iterative with short cycles, enabling fast verifications and corrections. Iterative development helps in reconsidering newer changes in requirements. - The process should be time bound with iteration cycles from one to six weeks. - Parsimony, or showing extreme care spending money in the development process removes all unnecessary activities. This is important for faster delivery of those functionalities which are of the utmost importance in that stage. - Having an adaptive approach with possible emergent new risks, which can be dealt with proactively. - An incremental process approach that allows application building in small steps. This will again facilitate better and exhaustive testing and maintenance, in addition to delivering important functionalities in steps. - A Convergent and incremental approach minimizes risks by not handling the entire project in one go. An incremental approach helps the project to be manageable.

- People oriented, that is, Agile processes favor people over technology and processes. People here mean the clients as wel ass the developers. - Collaborative and communicative working style. This includes collaboration with customers and creating a good rapport with the clients. (Miller, 2001) [18] Favaro (2002) discusses the ways of handling processes with changing and unclear, variable requirements. He proposes the iterative development paradigm as the common factor of all agile processes. Requirements may be added, changed or removed in successive iterations. He believes that the need for requirements is not to satisfy the customer, or minimize risk, or control damage, but to add business value [19]. Highsmith and Cockburn (2001) [17] report that the software business keeps changing due to competition and changing market needs, as well the need of the businesses to bring about change in the market. This in turn affects the software development life cycle. The development teams and management look more at satisfying the customer at the time of delivery rather than when the project was started. This requires that processes be developed that do not handle the situations where early changes are taken care of, but processes where inevitable changes can be handled throughout the life cycle. It is further claimed that Agile methods are designed to: - Produce first delivery in weeks for an early good impression and quick feedback. This helps build customer relations, as well as boosts developer morale. This, also helps serve as a platform for further development, and helps set the pace of development. - Come up with simple implementations so that changing these will be easy in the later stages of development, if any (and obviously) changes occur in the requirements. Once simple implementations are made, and at the same time advanced tools and coding techniques are used, changing some functionalities or adding to it later on becomes easy. This should be supported with enough documentation, as much is feasible and as much is required. No separate documentation phase should be required. -That the design quality be improved continuously, so that the next implementation is less costly. Once this is done, every iteration will churn out a production capable product, thus maintaining quality throughout. -And the product should be tested constantly for early and less expensive defects. The testing should be maximum for required functionality and minimal

for trivial features. “Trivial” features are those which should be taken care of while coding itself, by following strict standards. Time should not be wasted in testing if object assignment is made properly. The overall functionality should be tested thoroughly. Ambler (2002b) suggests some approaches which are a result of the Agile philosophy. These are: - Giving importance to a good working code so that a product is ready and successful - Having a small team which stays together with goodwill, - And a great focus on teamwork due to the importance of a people centric approach. The people centric approach mean that : - People in a team matter and should be given enough importance. These are the driving force of any company and project, and personnel power cannot be replicated [9, 15] -That a product with less documentation (and more quality and working code) is possible. This product is also easy to maintain if it has been “Pair Programmed” and enough comments are inserted at al the critical places in the code. -Communication is to be identified as a critical issue and time an effort should be spent on improving it. Without communication, a “shrink-wrapped” product is made available to the customer, whereby risking failure [9]. -Modeling tools may not be as useful as they may be thought to be. A complete design is not required in the initial phase itself. [8, 14]

3.6 Requirements gathering in Agile methodology Requirements in any software development effort are hardly known beforehand. If we do not know the requirements even for well defined problems, we should assume that not knowing requirements is normal. And once we accept that requirements can never be fully known at the initial stage, it can be handled in a more practical way by using agile methodologies. [16] In Agile methodologies, it is expected that we gather more and more requirements as and when they are known, and not label the late requirements as defects. [12, 16] Tomayko [16] talks about Agile methods appealing to customers who find “IKIWISI” (I’ll know it when I see it) intuitive. Agile methods have the concept of a “spike”. A spike is a throwaway program that is used to solve a particular problem, and answers a single question about the requirements, and hence

avoids unnecessary details about the other requirements. That is, Agile methods concentrate on single functionalities, and requirements may be discovered one at a time. [16] Favaro believes that having a robust requirements structure is a good way to mange risks and problems early in the life cycle, and to avoid unnecessary misunderstandings, but an actively managed requirements process gives us much more than robust behavior. It gives us the ability to be handle “change”, the most dreaded aspect of requirements engineering. [19] Tomayko believes that there are no wrong requirements, just those waiting to be discovered. Once they are discovered, an Agile process iteration can accept these new requirements and functionalities in the development cycle. [16]

3.7 Difficulties due to Agile method of gathering requirements The above attitude of spikes, “IKIWISI” and postponed requirements gathering makes estimation and software architecture more difficult. But, at the same time, verification is much easier due to each requirement being separately implemented in consultation with the client. Estimation is difficult due to unknown market demands. For Agile methods, because estimating may be difficult, each leg of the cycle may be separately funded, which is same as funding an entire project using traditional methods. (As each component of the Agile Methodology iteration may be an adaptation of a waterfall model). Still various methods of estimating project cost may be implemented, which are beyond the scope of this paper. Also significant refactoring will be expected due to the variable nature of real projects. And Refactoring could mean rework to some people, and some might accept it as part of agile process, as a natural iteration [7,10]. Confidence in the requirements translates to confidence in the architecture. [16] Apart from refactoring and effective prototyping, agile methods have other advantages for a situaion in which requirements are unstable. Test first proramming, a principle of XP, is a way of early detection of almost all minor errors. This also provides for more certain detection of defects during integration, and early “thinking-through” of tests for a Graphical User Interface. This is an advantage to any system. Also is a requirement can be tested, it can immediately be integrated into a system [16].

4. Implementation of Agile Methodology

The Agile software development processes are trying to bring about a change in the way software is developed. Agile methods give more importance to people, face-to-face interaction with team members and clients and having more emphasis on creating working software rather than on documentation (Mikio Aoyama, 1998). eXtreme programming, or XP, is one such popular method. The decision to implement the agile methods will depend on the state of the communication between people and the work culture that surrounds them (Pekka Abrahamsson et al) [3,15]. The Following XP methodology can be implemented by knowing the state of communication among the group as well ass the customers, and the level and expertise of the developers involved.

4.1 We will now look at the prominent agile software development method, eXtreme Programming (XP) XP has evolved due to the problems caused by the long development cycles of traditional development models. XP could be looked at something that came into existence as a method to “get the project done” (Haungs 2001) by using the practices that were found to be effective earlier (Anderson et al). The XP methodology came into existence and was formalized on the important practices and principles used. The individual practices in XP are not new, but the individual components have been used and a flow has been set in such a way that these form a new methodology all together, and functions in a way to maximize productivity. (forming a new methodology for software development). The term extreme is due to these common sense principles and practices being implemented in an extreme manner. [3, 15]

Figure 2, from Pekka et al [3,15].

4.2 Life cycle of XP The life cycle of XP consists of five phases: According to Beck’s description (1999b), namely, Exploration, Planning, Iterations to Release, Productionizing, Maintenance and Death.

4.2.1 Exploration Phase This involves regular updates from clients and from one developer to another. The clients write out “story cards” which describe the features to be added to the program in the first release. In parallel, the project team familiarizes itself with tools, technology and practices that they will be using throughout the project. The technology to be used is tested and the architecture possibilities for the system are explored. This is made possible by building a prototype of the required system. The exploration phase may take from a few days to a few months depending on how familiar the technology is to the programmers. [8, 9]

4.2.2 Planning Phase: Stories for next iteration. (priorities and effort estimates) This phase sets the priority order for the stories submitted in the previous phase and an agreement is reached on which contents to be included in the first “small” release. The programmers first estimate how much effort each story requires and a schedule is then agreed upon. The first release is normally completed within two months. The planning phase itself takes a couple of days [9, 15].

4.2.3 Iterations to release phase

This includes continuous review, pair programming, Analysis, Design, plan to test and finally testing. This includes much iteration before the first release. The schedule set in the previous phase is followed for coding and testing. The schedule set in the previous planning phase is broken down into parts and each part is completed in iterations, which takes about one to four weeks to complete. Those stories are selected which will help in the building of the entire framework, or a skeleton for the whole system. This facilitates the building of an entire system, in parts, but a major chunk can be initially delivered, on which various embellishments may be added on later. [15]. This is brought about by selecting those functionalities or stories, which will build the framework. This selection, for each iteration, is done by the customer himself. The customer also creates functional tests which are used against the system, and when the tests succeed, the system can be put into production. [7, 8]

4.2.4 Productionizing This is the production phase. This phase requires that a lot of testing be done before it can be declared fit to be given to the customer (put in production). During this phase, new functionalities may be thought to be required to be added to the product. These may be decided to be added then itself, or they may be documented and added in the next release. The new missing requirements are not considered defects, as they were previously unknown, but can always be included with the framework developed in the previous phases. During this phase, the time each iteration requires is reduced for faster delivery. The ideas not implemented could be taken care of in the maintenance phase. After the first delivery, the XP team should bring out more iteration at the same time maintaining the earlier system in production. The maintenance phase should see efforts put in for production support. The customer must be provided support for the software in production. The overall development speed of the project may usually decrease after the product is in production. The maintenance phase can see new people joining the team and the team may change at this stage. Also the product in production may be declared usable and further development may be stopped, or the product may be scrapped altogether [9, 15].

4.2.5 Death phase, final release

The death phase nears when there are no more “stories” left to be implemented. This will not only mean that all required stories are implemented, but also that various non functional requirements like performance and reliability are met. (Pekka et al) [15]. At this point, documentation is prepared, as any more change is not expected to the design, architecture or the code. The death phase may also be due to the system not delivering the desired outcomes or the system becomes too expensive to handle its further development.

Metrics like effort estimates made by the development team are tracked and a feedback is provided about how accurate it is so as to bring about improvements in the future estimations. Also during each ongoing iteration, the tracker traces and estimates if the given goals are reachable with the available time and resources, and if any improvement is needed to make it possible. Acontinuous and fast feedback is expected from the tracker.

4. 3 “Roles” in XP

The responsibility of the entire process is taken up by the coach. The Coach should himself or herself understand the process properly so as to guide the team. The coach is the point of contact if the process is not moving well as expected. More precisely, the coach teaches the team the “Agile” practices.

In an XP project, there are “Roles” defined for the participating people. According to (Beck 1999), these are:

4.3.1 Programmer The programmer will write code and test cases, and as a part of the Agile philosophy, it is expected that the programmer will coordinate and communicate effectively with other team members and programmers. The programmer should keep the code simple and precise, and participate in pair programming. The programmer is expected to follow standards and write documentation (comments) on a regular basis.

4.3.2 Customer The customer writes “stories” and functional tests for the product. It is the customer who decides when a requirement can be declared to be satisfied. The customer also decides which requirement has to be completed and included first. The customer should also be responsible in communicating with the team and not get too much involved in the decision making process. The decision making f process of implementation belongs to the programmer.

4. 3. 3 Tester The Testers help write functional tests and run them regularly. The tester is also responsible with broadcasting the test results and maintaining the testing tools. This is done with the help of the customer.

4.3.4 Tracker Feedback is given by the tracker in XP. It is the tracker’s responsibility to “track” the progress of the overall system and the project implementation.

4.3.5 Coach

4.3.6 Consultant A consultant is a person outside the team, who will guide the team members on technical matters. The consultant will solve specific problems faced by the team. The consultant will be an experienced person, in terms of implementing agile methodologies

4.3.7 Manager The Manager takes decision regarding the process and the development effort. The Manager should communicate effectively with the project team and be aware of the current situation and the current problems faced by the team. The manager should be able to resolve problems faced by the team, on a personnel and on a team level.

4.4 XP Practices According to (Beck 1999), XP is a collection of ideas and practices that are a new arrangement from already existing best practices. (Alexander 1979), Discusses that the business-decisions making procedure be made by the customers, and the programmers have a say in the technical aspects of the project. The rapid evolution of XP is an idea that has come up from another implementation of Agile development methodology “Scrum” (Takeuchi and Nonaka 1986) and the pattern language (a pattern is a proven solution to a recurring problem in a certain context (Cunningham 1996). A pattern language is a collection of patterns which work in unison to solve difficult problems according to a predefined goal) (Cunningham 1996). The XP idea of scheduling projects based on the stories submitted by customers

is similar to the use-case approach of Jacobsen (1994), (use-cases are the modeling “diagrams” which capture the high level user required functionalities) and the evolutionary delivery is adopted from Gilb (1988). [15]

implementing the code and running it continuously. Software development depends on the feedbacks from these tests, and is dependent on it.

XP aims at completing such projects successfully, which are done in small to medium sized teams, and are handling a project with unclear requirements or which are constantly undergoing change. The main characteristics of XP according to Pekka [15] are short iterations with short small releases and rapid feedback, customer participation, communication and coordination, continuous integration and testing, collective ownership of code, limited documentation and pair programming.

Refactoring is the process of improving the code and optimizing it by removing duplicate code parts. It improves communication between modules, simplifies it and makes it adaptable. An adaptable code is important in handling changes.

4. 5 The Practices of XP are as follows [15] 4.5.1 Planning Game

4.5.6 Refactoring

4.5.7 Pair programming Pair programming is two people writing code at one computer. The following is a picture from [6], showing a single computer allowing two people to write code simultaneously. Pair programming is done only at times, and not continuously. Enough time should be kept aside for pair programming, individual efforts should also be put in.

This involves close interaction between the customer and the programmers. The programmers estimate the effort needed for the implementation of the customer stories and the customer decides about the scope and timing of releases. In this, equilibrium is expected to be reached. [15]

4.5.2 Small/short releases: In this, a simple system is made available to be put in production at least once in two-three months. New versions can then be released monthly or even on a daily basis. [15]

4.5.3 Metaphor

4.5.8 Collective ownership

The system is defined by a metaphor/set of metaphors between the customer and the programmers. This “shared story” guides all development by describing how the system works.

This means that anyone can change any part of the code at anytime. This is made possible by making developers aware of the responsibility of the parts of the code, and which code belongs to which programmer. Irresponsible changes are avoided and collective ownership also means collective responsibility.

4.5.4 Simple design The team works hard to implement the simplest possible solution, which can be implemented in that phase. Extra code and unnecessary complexity is removed with priority. This simple design is the key to adapting to future changes in requirements.

4.5.5 Testing The customers and the testers write the functional tests, and the developers are responsible for

4.5.9 Continuous integration Whenever a new piece of code is written, it can be integrated with the system after it passes a rigorous testing phase. The code may be integrated and built many times a day, then accepted in the system after it passes the tests.

4.5.10 40-hour week

A maximum of 40-hour working week is to be worked. No two overtime weeks are allowed in a row, and if that happens, it is considered to be a problem which has to be solved. This helps maintain a good work culture and atmosphere, and overtime should happen only when the developer wishes so. (http://google.com/jobs)

4.5.11 On site customer The customer has to be present on site and available always for the team. The customer could be internal or external, depending on availability.

4.5.12 Coding Standards Various coding rules and standards exist and are to be followed for standardized coding, and documentation purposes. Communication through comments in code is encouraged. Without coding standards, the other practices of pair programming and collective ownership do not old ground

4.5.13 Open Workspace The physical environment too, is defined for XP practices to take ground. A large room with small cubicles is preferred. Pair programmers should be placed in the center of the space. Communication should be guaranteed between all developers and the clients.

4.5.14 “Just Rules” The team is allowed to have its own rules and that are to be followed, but can also be changed at any time. The changes have to be agreed upon and their impact has to be assessed. The rules that a team sets should be in tune with philosophy of Agile programming.

4.6 Adopting XP Beck (1999a, p.77) suggests that XP should be adopted gradually. “If you want to try XP, for goodness sake do no swallow it all at once. Pick the worst problem in your current process and try solving it in the XP way”. The fundamental idea of XP is that there is no process that can suit all projects, but the best practices should be custom fitted according to the needs of the various projects and situations (Beck 1999). This allows to have a flexible and agile development environment, and makes it people centric. It is always helpful and more successful

when XP is fitted to the problem, and no the other way round of fitting the problem to XP [15]. Generally, only a few of the XP practices are found to be adopted on many of the projects that were studied by Pekka et al. [9, 15] The practicalities for adopting XP have been documented in Extreme Programming Installed (Jeffries et al. 2001) This book has a collection of techniques about XP practices. The theme of the book is about the difficulty of estimating the difficulty and duration of the tasks to be performed. (Pekka) It is suggested that “spikes” be used. “Spikes” as discussed is coding a throwaway program to see how a particular problem may be solved and implemented. The book has concrete examples about how to adopt XP for projects. [13, 14, 15] XP cannot be applied to all project situations and all its limits have not been documented. XP can be applied to project situations with a small team, roughly three to twenty members. The physical environment too is important, and there should be continuous communication and coordination be facilitated between team members. XP expects that team members be even present on the same floor, and in communicable distance. At the same time, geographically distributed teams can follow XP, under the case of “two teams working on related projects with limited interaction” (Beck 1999,

p.158) [1, 2, 15] Among the factors that may go against XP as a development methodology would be resistance from management, clients or team member themselves being responsible for failed processes. Also technologies that have a long feedback time are not suitable for XP processes. (Pekka) [15]. But as discussed ahead, long feedback time may be divided into chunks of smaller goals and XP functionality be applied to the project development.

4.7 Suitability of XP to various situations The paper [9] discusses various university projects which were completed using XP practices. \The university projects, though a little different than real life projects, still allowed a variety of similarities and analogies to be derived, and helped in reaching conclusions about various project situations. The following are the observations. In this situation, the customer was the University Professor supervising the project. A member of staff who is not involved in the teaching of the Software Engineering course can be called the customer. As such, the course lecturers were not involved as customers, but were available as “agile consultants”,

who were called upon from time to time to help guide the process. There was limited access to the customer, as would be in real life situations. XP encourages continuous customer interaction and testing. However, customers were members of academic staff with busy schedules, thus limiting their availability for dealing with XP teams. They were thus unable to sit in on every XP session with the students. Different teams handled this in different ways. There were three prevalent ways. The various problems were handled in the following ways: Weekly meetings were carried out because the customer was not available throughout the development of the project. A team member was elected as a proxy customer, who contacted the customer at a convenient time. Some teams took the decisions by consensus and did not elect one single member as a proxy. The project timescale was short due to academic constraints. Therefore the product was not released in stages but iteration was considered as a release. The iteration length varied from group to group, from one iteration per week to one iteration every three weeks. [9] Sudents had assigned their own working hours, and tried to stick to the schedule, by assigning appropriate number of hours for pair programming etc. Students devoted a certain number of hours per week to the project. The project had no XP manager/champion/coach to ensure adherence to practices. The students were their own coaches, or a team member would assume the responsibility. All decisions were then made by this member

completely involved when the other member is coding. Another lesson learnt was due to the team spending too much time “spiking” [9] and thus losing valuable development time. This led to abandoning pair programming, and other problems of falling behind followed. Pair programming can sometimes lead to a large reduction in project velocity. This is due to a lack of understanding of its purpose. Both members need to program actively, or one of them will lose focus. It should be a mutual effort. Additionally, it was observed that in a pairing of an inexperienced and an expert programmer, the valuable opinions of the “weaker” person were often left out. The members should thus, actively contribute to the development efforts, and no distinction be made as such. Also, when a “weaker” member is paired with a stronger coder, it should be taken as an opportunity for exchange of ideas [9] Also, the fact that user stories are implemented in a simple manner does not mean that non-functional requirements like scalability be ignored. This could be taken care of by building the applications on available platforms, which provide security, scalability and fault tolerance. Over testing of the product should be avoided. The main functionalities should be tested, and trivial tests should be taken care of in coding responsibly, in pairs. Also when no testing tools are available, time would be spent on writing tests manually. This could be avoided by having a testing tool ready in the initial phases.

4.7.2 Planning is important All team members took equal responsibility for the process. Team members had varying skills and backgrounds. These could be fresh graduates to highly experienced professionals.

4.7.1 XP Pair Programming encountered, solutions found) [9]

(problems

In pair programming, when inexperienced coders were paired with experienced ones, the results could be varying. In this case, the inexperienced coder refused to code when paired with an experienced coder, claiming inexperience. Also, the weaker member did not interrupt and no knowledge exchange was made possible. Also, the coding responsibility was sometimes given to the other member of the pair and, and the other member may take a break from coding, thus defeating the purpose of pair programming. Thus one member will not be

Long term plans should be made, like considering the need for scalability and other non-functional requirements.

4.7.3 Set working hours Sticking to set working hours helps in maintaining a good rhythm for the project. Adhering to the set hours and completing planned things on schedule helps to maintain a good environment in the project team. The extra work load that will be generated if all practices are not properly applied, will lead to the newer iterations being slower and out of sync with the XP philosophy. [9]

4.7.4 Collective code ownership

This requires that everyone knows who is working on what part of the code, so as to avoid conflict, and accidental changes to someone else’s code. [9]

4.7.5 Standards In XP, as in any software development, it is important to stick to standards. Everyone should use the same nomenclature and style of coding, so as to have collective ownership, and anyone can access and change code as required. Also the benefits will be visible when the code grows in size. These standards should be agreed upon early. [9]

4.7.6 Communication Members should be in constant communication with each other for the duration of the project. Also the customer should be involved in the loop of communication. It

4.7.7 XP Champion It is always useful to have an XP champion to guide the process in the early stages of development. A single person may be responsible for influencing the team, but at the same time, all should embrace the XP ethos, for successful project completions.

4.8 XP for Scientific Research [6] The above paper [6] talks about XP being applied to a projects requiring scientific research, namely, after a successful completion of a product for the NASA

Langley Research Center. The problems with applying XP to a research environment were as follows. The first barrier was that a complete upfront design is usually requested rather than just starting with a project and deciding along the way. The proponents of the XP practices should be able to convince the client that an upfront design does not guarantee success, The second Factor is the requirement of specifications, which are not necessarily turned out at the beginning of an XP project. Also highly experienced professionals do not wish to involve in a fuzzy set of requirements and assuming that they will keep changing. They prefer less communication than an everyday communication with new requirements and understanding of the problem gradually over the days by experimenting. Also peer review and the reward structure depends on

a person’s stature in the field, where most have a doctoral degree. Hence it becomes difficult to apply peer reviews, e.g., in pair programming practices. Also Pair programming requires a rotating set of partners, and a mediator is required during confrontation, if at all it arises. This calls for a constant availability of resources and quick feedback [6]. In research environments, a long feedback cycle may be in place, negating the effect of rapid development and fast feedback and iterations. This could be taken care of by dividing long term goals into short term tangible increments, using XP’s simple design methodology. [6] An on-site customer in a research environment will be required, but given the nature of development and uncertainty, a customer to be available full-time may not be feasible. In this case, one of the people involved in the project with the most knowledge could take the form of a customer. He or she could then intermittently have updates from a real customer. But an internal customer of this type will have to come up with stories for implementation, at the same time think about cost estimation. [6]

4.8.1 Pair programming This does not provide any known difficulties in XP implementation in a research type environment. This can, in fact help develop better algorithms as found out by the authors William A. Wood and William L. Kleb. [6] Productivity can be achieved using pair programming. This can actually produce simple and cleaner code. Though not completely related to XP in a research environment, pair programming does help in speeding up the development process. [6]

4.8.2 Test-driven development This would perhaps be the most important part of the XP cycle. For a scientific product, testing will be important for reliability. Testing should be done at a unit level, and the granularity for the tests should be decided. [6] Full stress tests, will include acceptance tests, performance monitoring, distributed processing, and numerical proofs of the algorithms for properties such as positivity and the order of accuracy. [6]

4.8.3 Continuous integration

This could be different than the traditional approach of implementing algorithms in large chunks. A dedicated integration machine was used in the case of the authors of the paper quoted, and testing was automated. Planning and a simple design helped in continuous integration. In the above case, the authors found that the XP approach was approximately twice as productive as traditional approach. Also, enough testing ensured that critical tests were carried out, essential for the research effort’s scientific validity. Also the code was less than traditional approaches and it was much more intelligible. [6]

4.8.4 Simple design Performance will be an issue for numerical analysis, and refactoring should be considered for code optimization. Performance measures can be included in the process at regular intervals in the acceptance tests. Another potential problem will be the requirement for simple designs. As performance is always an issue for numerical analysis and the authors [13, 15] experience with procedurally implemented and speed-optimized algorithms has verified that the cost of changing the fundamental design of elaborate codes is very high. Premature optimization for the developer, will be tempting in such cases, particularly in the absence of a business-value-oriented customer. This practice is accepted with skepticism, because poorly understood numerical analysis algorithms are costly to run. The approach suggested is to include performance measures in the acceptance tests to know excessive execution times and then to go ahead with the simplest design until the performance limits are exceeded. Once a performance issue is encountered, a a profiler could be used to target refactorings that would speed the algorithms enough to pass the performance criteria. The speed bottlenecks are not always intuitive, and it became evident for the authors that premature optimization would have wasted effort on areas that were not bottlenecks, or the “Rate Determining Steps” (RDS, a chemistry term) while still missing the real problem of optimization, and missing out on the real slow steps in the algorithm. [6]

5. Suitability of Agile Methods distributed Software Development

to

(Web Based Agile Software Development). XP methodology relies on the fact that the team developing the system can be at the same location geographically and can work together. XP actually expects the team to be small and be present in the

same room on the same floor. This is not the case in real life scenarios. With onshore-offsite methodology of outsourcing becoming the industry standard, teams have expanded and moved out of the realm of sharing common geographic location and they can be in varied places. [2] In such a scenario, communication, which is simple when the team members are present face to face, takes up a different meaning. Time zones will be different, for an onshore and an offshore team, and communication may mostly be through the previous day’s emails. Communication is a simple matter of talking to a fellow developer, but when the developers are separated over geographical distances, suddenly becomes a huge problem and an overhead, and a potential threat for miscommunication. Difference in time zones and unavailability of common infrastructure makes the problem worse. Moreover unavailability of resources when required makes the problem worse. This coupled with teams trying to set up common times to talk, or communicate, can only add to more confusion at times. Also, with different teams working on different aspects of the problem, but on the same overall system, integration becomes the centre of all activities. [2] To cater to all these requirements in a diverse, distributed environment, Mikio Aoyama came up with the “Web based Agile methodology” and suggests the use of “time based process enactment” and “just in time process management”. [2] In his famous paper, “Web based Agile Software Development”, he encourages the use of web-based information sharing environment to overcome the communication gap. To make this web-based system adaptable to change he proposes the use of “macroprocess” (huge chunks of high level processes) architecture as shown in Figure2 below (figure from [1]). He further proposes the division of Agile Software Processes (ASP) in to an Upper Stream; covering analysis to implementation phase and the lower stream; covering integration and system testing. The teams follow incremental, iterative approach to go through the Upper and lower stream processes and each increment incorporates the development of a new unit, which Mikio calls enhancements. [1]

Figure 2.

6. Comparison of Agile with Waterfall We will now compare and comment on the existing software development methodologies (the waterfall model) Agile processes not only expect quick delivery of software products but also quick adaptation to changing requirements. To be agile, the process must be flexible enough to adapt smoothly to changes in requirements and delivery schedule. Conventional software process models, such as the waterfall model, are monolithic and slow, focusing as they do on a single long cycle time [1]. The Agile software processes are an extension of the waterfall model. The Agile processes are an iterative version of the Waterfall model. At the unit level, the Agile process consists of analysis, design, code and test, iterated over, in macro phases. The Waterfall model is extended by applying the “people-centric” approach and rapid developments. Common knowledge tells us that Waterfall models are not used in the industry and their adaptations are used instead. Waterfall Models differ from Agile processes in that they do not allow for iterations, nor are they people centric. They do not assume a continuous communication cycle with the customer, and stick to a requirements document throughout. Changes late in the process are costly, and deadlines are usually tight. Whereas in Agile methods it is expected that a product can be delivered at any time. This can be done without testing, as testing is an umbrella activity for the project. [3,15]

7. Conclusion Studies show that traditional practices are not used practically, and are usually considered as a framework for software development. These methods do not always apply to real life project situations, and most projects steer away from the decided course. This has resulted in extended deadlines and high project costs. Because of this, newer software

processes are not readily accepted, and the existing ones are not completely understood. Some solutions may be formed for particular project situations and may be difficult to understand and implement. The Agile software methods officially started off with the publication of the agile manifesto, which attempted to bring about a paradigm shift in the way projects are handled. For long, the importance of people in the project was understood, but the importance of people (developers, customers) in the process was not taken seriously. It was due to a lot of importance only to the customer, or to the tools and to the set rules of the process which was being implemented. Agile methods claim more importance to people, communication, working software, customer collaboration, and change, rather than on processes, tools, contracts and plans. Many Agile methodologies conforming to these rules have been introduced. This paper served the purpose of understanding what Agile methodologies are, and an Agile methodology of XP programming was looked at. The suitability of Agile practices to various project situations, particularly of XP was studied. We also tried to understand that a project is not to be suited to XP (or any process), but the XP process should be tailored to fit the project and the problem, and at the same time should conform to the Philosophy of Agile methodology. Agile methodologies are defined as incremental (iterative and rapid), cooperative (good communication with the customer on a very frequent basis), straightforward (easy to learn) and adaptive (ability to make last minute changes to project requirements). After looking at applying XP to various situations, we looked at how XP practices could be misunderstood or not applied properly. These could include keeping the design simple according to XP, and in the process neglect non-functional requirements. Also, pair programming may be implemented, but the people involved in pair programming may not take active interest, when the other person is coding, thus negating the effect of exchange of ideas. [15] Anecdotal evidence suggests that Agile methods can be applied to many project situations, from research environments to distributed programming, between geographically separated teams. [15] We then also compared Agile methods with the traditional approach of the Waterfall Model. In this we realized that the agile methodologies, at the unit level are waterfall models. These units are what constitute the iterative model of the Agile methods.

In conclusion, Agile methods are the same “best practices” arranged in a different manner with a different philosophy, and a people centric approach. Implementing these in the right way, and fitting these processes to the problems will and has definitely helped solve many of the prevalent “people oriented” problems.

8. References [1] Mikio Aoyama, “Web Based Agile software development”, IEEE Software, Volume 15, Issue 6, Nov.-Dec. 1998 pp. 56 - 65. [2] Mikio Aoyama, “Agile Software Process and Its Experience”, Software Engineering, 1998. Proceedings of the 1998 (20th) International Conference on, 19-25 April 1998 pp.3 – 12. [3] Pekka Abrahamsson, Outi Salo, Jussi Ronkainen, and Juhani Warsta, “Agile software development methods, review and analysis”, ESPOO, VTT Publications 478, 2002, pp 18-27. [4] Lucas Layman, Laurie Williams, Lynn Cunningham, “Exploring Extreme Programming in Context: An Industrial Case Study”, Agile Development Conference, 2004, 22-26 June 2004 pp. 32 - 41 [5] Abrahamsson, P.; Warsta, J.; Siponen, M.T.; Ronkainen, J., “New Directions on Agile Methods: A Comparative Analysis”, Software Engineering, 2003. Proceedings. 25th International Conference on, 3-10 May 2003 pp. 244 – 254 [6] Wood, W.A.; Kleb, W.L., “Exploring XP for Scientific Research”, IEEE Software, Volume 20, Issue 3, May-Jun 2003 pp. 30 – 36 [7] Joseph Bergin, Fred Grossman, “Extreme Construction: Making Agile Accessible” Extreme construction: making agile accessible”, Agile Conference, 2006, 23-28 July 2006 Page(s):6 pp. [8] Mark Striebeck, “Ongoing Quality Improvement, or: How We All Learned To Trust XP”, Agile Conference, 2005. Proceedings, 24-29 July 2005 pp. 267 – 271 [9] Andrew Jackson, Shiu Lun Tsang, Alan Gray, Cormac Driver, Siobhán Clarke, “Behind the Rules: XP Experiences”, Agile Development Conference 2004, 22-26 June 2004 pp. 87 - 94

[10] Kelly Weyrauch, “What are we arguing about? A framework for defining agile in our organization”, Agile Conference, 2006, 23-28 July 2006 pp. 8 [11] Barrett Koster, “Agile Methods Fix Software Engineering Co”, Consortium for Computing Sciences in Colleges Conference, 2006,pp 131-137

[12] Peter Manhart, Kurt Schneider, “Breaking the Ice for Agile Development of Embedded Software: An Industry Experience Report”, Proceedings of the 26th IEEE International Conference on Software Engineering (ICSE’04).

[13] Paul S Grisham, Dewayne E. Perry, “Customer Relationships and Extreme Programming”, Human and Social Factors of Software Engineering (HSSE) ACM, May 16, 2005, pp 1-6 [14] Helen Sharp, Robert Biddle, Lynn Miller, Jeff Patton, Phil Gray; “Agile Development: Opportunity or Fad?” ACM, CHI, April 22–27,2006, pp 32-35

[15] Pekka Abrahamsson, Juhani Warsta, Mikko T. Siponen and Jussi Ronkainen; “New Directions on Agile Methods: A Comparative Analysis”; IEEE, 2003 [16] James E. Tomayko “Engineering of Unstable Requirements Using Agile Methods”, Carnegie Mellon University.

[17] Highsmith, J.; Cockburn, A. “Agile software development: the business of innovation”, Computer, Volume 34, Issue 9, Sept. 2001 Page(s):120 - 127 [18] Miller, G.G., “The characteristics of agile software processes”, Proceedings of the Conference on Technology of Object-Oriented Languages and Systems, TOOLS, n TOOL, 2001, p 385-387

[19] Favaro, J.; “Managing requirements for business value” Software, IEEE, Volume 19, Issue 2, March-April 2002 Page(s):15 - 17