Published online ahead of print November 11, 2011

Information Systems Research

Copyright: INFORMS holds copyright to this Articles in Advance version, which is made available to subscribers. The file may not be posted on any other website, including the author’s site. Please send any questions regarding this policy to [email protected].

Articles in Advance, pp. 1–17 issn 1047-7047 — eissn 1526-5536

http://dx.doi.org/10.1287/isre.1110.0392 © 2011 INFORMS

In Search of Efficient Flexibility: Effects of Software Component Granularity on Development Effort, Defects, and Customization Effort Ramanath Subramanyam Department of Business Administration, University of Illinois at Urbana–Champaign, Champaign, Illinois 61820, [email protected]

Narayan Ramasubbu School of Information Systems, Singapore Management University, Singapore 178902, [email protected]

M. S. Krishnan Business Information Technology Department, Ross School of Business, Ann Arbor, Michigan 48109, [email protected]

S

imultaneously achieving efficiency and flexibility in enterprise software production has been a considerable challenge for firms. Newer software development paradigms such as component-based and model-driven development attempt to overcome this challenge by emphasizing modular design of complex systems. However, there is a paucity of rigorous empirical research on the use of such software methodologies and the associated extent to which trade-offs between efficiency and flexibility can be influenced. Addressing this gap, we investigate the performance outcomes of a model-driven, component-based software development methodology using data collected from an enterprise software development firm that deployed such a methodology for its product development processes. Examining the design, development, and implementation of 92 business software components of the firm’s enterprise resource planning product, we discuss how the design of software components, specifically component granularity, affects development efficiency (development effort and defects) and flexibility (customization effort). Our results suggest that (a) components that are coarse grained are associated with higher flexibility (lower customization effort) but are also associated with lower development efficiency (more development effort and defects), and (b) defect proneness of a component plays a mediating role on the relationship between component granularity and flexibility. These findings present strong evidence for the existence of trade-offs between efficiency and flexibility in mass-customized software product life cycles. They establish component granularity as a key design dimension that needs to be managed judiciously to enable potential trade-off shifting mechanisms through the use of software methodologies that emphasize modular design approaches. Key words: modular design; model-driven development; component-based software development; efficiency; flexibility; complexity; component granularity; software engineering; project performance History: Sandra Slaughter, Senior and Associate Editor. This paper was received on June 2, 2007, and was with the authors 2 years and 8 months for 6 revisions. Published online in Articles in Advance.

1.

Introduction

customized to certain groups of customers—a strategy that product vendors use to leverage scope advantages (cf. Duray et al. 2000, Tu et al. 2004). When business process workflows and business rules are affected by changing business conditions, modular design of mass-customized software products can be expected to enable quick modification and redeployment of affected modules, thereby positively contributing to achieving enterprise agility (Byrd and Turner 2000, Sambamurthy et al. 2003). However, it is not clear if achieving flexibility through modular software design comes at a price. For instance, enterprise software vendors pursuing a mass-customized product development strategy are likely to be concerned with potential losses in

Modular design approaches partition complex systems into several modules of manageable complexity and limit the scope of interactions between these modules through well-defined interfaces and design rules (Parnas 1972, 1979; Simon 1996; Baldwin and Clark 2000). A modular design is expected to facilitate a shortened production cycle time by enabling concurrent work on different modules of a partitioned complex system and thereby helps firms leverage economies of scale through mass production (cf. Baldwin and Clark 2000). Modular design approaches also support mass customization by partitioning products into modules that are common to all customers and modules that are specifically 1

Copyright: INFORMS holds copyright to this Articles in Advance version, which is made available to subscribers. The file may not be posted on any other website, including the author’s site. Please send any questions regarding this policy to [email protected].

2

Subramanyam, Ramasubbu, and Krishnan: In Search of Efficient Flexibility Information Systems Research, Articles in Advance, pp. 1–17, © 2011 INFORMS

development efficiency, i.e., the ability to maximize outputs (high quality functional source code) for a given level of inputs (development effort). Baldwin and Clark (2000, p. 91) noted, The independent blocks in a modular task structure can all be worked on simultaneously. Thus, as long as the “design rules” and the “integration-and-testing” stages do not eat up too much time (a crucial assumption!), a modular division of tasks will shorten the endto-end time needed to complete a given process.

It is not fully clear if the crucial assumption noted in the Baldwin and Clark synthesis related to design rules, integration effort, and testing effort of modular components will hold true in the context of the mass-customized enterprise software lifecycle. Lending support to such a suspicion is evidence from automotive and manufacturing industry sectors, which shows that mass customization facilitated by modular design approaches is associated with a rapid proliferation of product varieties and a significant increase in both direct and indirect production costs (Blecker et al. 2004, Blecker and Abdelkafi 2006). Because software products are distinctly more malleable than physical goods, and testing and verification of software code is highly resource intensive (cf. Hamlet 2010, pp. 23–28), there is a need to verify if methodologies emphasizing modular design approaches would indeed reduce end-to-end development and deployment costs of mass-customized enterprise software products. In other words, there is a need to reexamine the dynamics between flexibility and efficiency in the mass-customized enterprise software life cycle. In this paper, we develop and empirically test a conceptualization of the relationship between software component design dimensions and software development outcomes in the context of model-driven, component-based software development (MDCD). This relatively recent software development methodology emphasizes a modular design approach and is geared toward the development of flexible software applications (Szyperski 2002, Bast et al. 2003, Hailpern and Tarr 2006, Schmidt 2006). Specifically, we focus on component granularity, a critical design aspect that captures the extent of modularity (Brereton and Budgen 2000; Hamlet 2010, pp. 29–34) and its impact on mass-customized software product development efficiency and flexibility. MDCD emphasizes modular design through “componentization,” i.e., the decomposition of a large system into a set of distinct components that communicate with each other using well-defined interfaces (Szyperski 2002, Hailpern and Tarr 2006). MDCD also facilitates “separation of concerns,” i.e., the decoupling of business-specific aspects of software applications from their implementationspecific aspects (Kulkarni and Reddy 2003). Moreover, MDCD enables automatic or semiautomatic

transformation of variations in business logic (often specified in the form of machine-readable business process models) to implementation-specific software code facilitating easier customization, i.e., the creation of customer-specific product variants. Thus, MDCD supports a modular design approach—an excellent context for examining the relationship between flexibility and efficiency in the development and deployment of mass-customized enterprise software products. The data for our empirical investigation come from a field study of MDCD at Ramco Systems, a leading packaged enterprise software product vendor with more than 800 clients (firms) worldwide. As the company deployed its mass-customized enterprise software products for a growing clientele, it was faced with significant recurring rework and customization costs that hindered its rapid business expansion goals. To address these problems, the firm adopted MDCD for its enterprise software product development and deployment. We observed the MDCD process of the firm over repeated intervals and augmented it with archival data collection—an approach consistent with the empirical research tradition in the information systems development literature (e.g., Banker et al. 1998, Banker and Slaughter 2000, Krishnan et al. 2000). We operationalize granularity of software components developed with MDCD using measurable design subdimensions and empirically establish the relationship among component granularity, development efficiency, and flexibility. We utilize the results of our empirical models to discuss the implications for software designers facing the challenge of simultaneously achieving both efficiency and flexibility in the mass-customized enterprise software development life cycle. The rest of the paper is organized as follows. We define and operationalize component granularity in §2 and present our research model and hypotheses relating component granularity, development efficiency, and flexibility in §3. We describe our data collection process in §4 and present our empirical models in §5. We enumerate hypotheses testing procedures and results in §6 and discuss contributions and implications of our findings in §7. Finally, we acknowledge the limitations of this study in §8, along with our suggestions for future research; we provide concluding remarks in §9.

2.

Component Granularity, Efficiency, and Flexibility

As noted before, in MDCD a large system is decomposed into a set of subparts or components with well-defined interfaces. Deciding on the granularity of components, i.e., the extent of subdivision in a large system, is a key design decision in MDCD

Subramanyam, Ramasubbu, and Krishnan: In Search of Efficient Flexibility

3

Copyright: INFORMS holds copyright to this Articles in Advance version, which is made available to subscribers. The file may not be posted on any other website, including the author’s site. Please send any questions regarding this policy to [email protected].

Information Systems Research, Articles in Advance, pp. 1–17, © 2011 INFORMS

(Brereton and Budgen 2000; Messerschmitt 2007; Hamlet 2010, pp. 29–34). Fine-grained components are typically smaller in scope (narrow and focused business functionality), highly amenable to specialization, and often involve only one implementation technology (e.g., one programming or scripting language). In contrast, coarse-grained components are typically larger in scope, often involving a mix of two or more implementation technologies, and they are typically characterized as loosely coupled modules of language-independent code (Whitehead 2002, p. 21; Messerschmitt 2007). Prior research highlighting the significance of component granularity design decision notes that component granularity affects (a) the choice of development tools and languages, (b) system development and production time scales, and (c) the effort required for system testing (e.g., Brereton and Budgen 2000, Messerschmitt 2007, Hamlet 2010). For exposition, we illustrate component granularity with two contrasting examples from our data set. A component in our data set called “employee information processing” is considered an important part of many mass-customized human resource enterprise systems. As one can imagine, human resource laws, employee recruiting, compensation, benefits, and overall human resource management processes and policies vary considerably across industry sectors and countries. System designers at our research site addressed this business process heterogeneity by incrementally incorporating business logic variations into the “employee information processing” component over time. Thus, the “employee information processing” component was realized as a coarsegrained component, which accommodated several variations of human resource business processes without subdividing these variations into different finerlevel components. As a coarse-grained component, it exchanged “document-oriented” data, such as entire files containing data about an employee’s benefits selections, with other components in the system. This was in contrast to components that exchanged minor fragments of data such as employee birth dates and names. As a case in point, another component in our data set, called “exchange rate computation,” was used in a variety of business processes including payroll, tax computation, and employee reimbursements business processes. Despite its use in a variety of business processes spanning different industry sectors and countries, the functionality of the component was highly specialized, and it exchanged fine fragments of data (currency exchange rate) with other interacting components. Thus, the “exchange rate computation” component is a finer-grained component relative to the “employee information processing” component.1

A system can be built with all coarse-grained components, fine-grained components, or a combination of the two. Deciding on the desirable level of component granularity is often challenging because coarsegrained and fine-grained components have opposing design properties that typically impose trade-offs between efficiency of mass-customized production and flexibility of deployment across heterogeneous environments. Because coarse-grained business components are broader in scope, they are usually associated with lower levels of customization effort, i.e., the effort necessary to integrate the component with a specific existing enterprise information technology (IT) application infrastructure. In other words, coarsegrained components offer higher flexibility for a vendor of a mass-customized product that deploys the component in heterogeneous client environments. At the same time, because of the larger scope of a coarsegrained component, a more complex programming logic and associated higher development effort is typically involved in its production. That is, a vendor typically incurs higher production costs to enjoy the flexibility benefits offered by coarse-grained components. In contrast, fine-grained components are narrow in scope and can be developed with less effort, but they need to be extensively modified to be integrated with diverse IT application infrastructures of clients (i.e., higher customization effort or lower flexibility). Further, the verification costs of a system composed of a large number of fine-grained components is typically high because of the large number of interfaces between the fine-grained components that need to be tested and verified to ensure system quality (Blecker and Abdelkafi 2006, Beisiegel et al. 2007). In other words, fine-grained components economize on the internal development efficiency but sacrifice downstream flexibility because of the higher customization effort needed to integrate these components in a client’s IT environment (and vice versa for coarse-grained components). Thus, our main postulation is that the component granularity design decision plays an important role in the relationship between realized development efficiency and flexibility in mass-customized enterprise software product development and deployment. We develop specific hypotheses relating component granularity and measures of development efficiency and flexibility in the next section.

1

(coarse) to high granularity (fine) depending on a variety of subdimensions, which we describe in detail in §4.2.

It is important to note that granularity of a system is not a binary measure, but it spans a spectrum that ranges from low granularity

3.

Research Model and Hypotheses

The conceptual research model for this study is shown in Figure 1. Our unit of analysis is a masscustomized business software component developed

Subramanyam, Ramasubbu, and Krishnan: In Search of Efficient Flexibility

4

Information Systems Research, Articles in Advance, pp. 1–17, © 2011 INFORMS

Copyright: INFORMS holds copyright to this Articles in Advance version, which is made available to subscribers. The file may not be posted on any other website, including the author’s site. Please send any questions regarding this policy to [email protected].

Figure 1

Conceptual Research Model

Flexibility Control Component size H3(–)

Customization effort H4B(+)

Component granularity

H2(+)

H1(+)

Controls Component size Construction effort Testing effort

In-process defects

interrelationship concerns of a component that a system integrator has to be aware of when integrating the component with a collection of other components in an enterprise IT application (cf. Sullivan and Notkin 1990). Coarser granularity of components developed and deployed using MDCD is associated with larger code volume, greater functionality, and more independence requirements (Bachmann et al. 2000, Whitehead 2002, Beiseigel et al. 2007, Messerschmitt 2007). 3.1.

H4A(+)

Development effort

Development efficiency

and deployed using MDCD. The key constructs in our model are flexibility, measured as customization effort, and development efficiency, measured in terms of in-process defects and development effort. As mentioned before, customization effort is the effort required to integrate a mass-customized software component with a client’s existing IT application environment. In-process defects capture the defect proneness of a component and the associated rework necessary to fix the errors, and development effort represents the total effort incurred in the analysis, design, construction, and verification of the component. In developing our hypotheses relating component granularity with the performance outcome measures of efficiency and flexibility, we build on the wellestablished observation in software engineering that system designs that lead to higher structural complexity of software impose a cognitive burden on development personnel, which has a detrimental effect on eventual project performance outcomes (cf. Basili and Perricone 1984, Card and Glass 1990, Banker and Slaughter 2000, Subramanyam and Krishnan 2003, Darcy et al. 2005). To conceptualize component granularity design in terms of the established structural complexity measures, we follow Messerschmitt (2007) and characterize component granularity along three design subdimensions: code volume, functionality, and independence. “Code volume” design subdimension captures a component’s overall quantity of source code written in a specific programming language. “Functionality” design subdimension represents the range of distinct business process operations that can be accomplished by a component. The “independence” design subdimension measures the extent of

Component Granularity and Development Effort Mass-customized components that are coarse grained encapsulate numerous logical paths within program code to accommodate the different variations in business logic found in heterogeneous business process environments. Coarse-grained components typically incorporate a larger scope of business functionality and are strongly connected to the underlying data repository, often executing several queries to the data repository (concurrently or in sequence) to process document-oriented data (e.g., an employee benefits report). Furthermore, in the model-driven engineering design followed by the MDCD methodology, coarse-grained components tend to be designed independently from the underlying technological platforms to render them suitable for a modular task design, and they accommodate future changes to the multitude of business functionality that they support (Sullivan and Notkin 1990, Schmidt 2006). This independence requirement places an additional burden on development personnel to test the component behavior in a variety of technological platforms on which the component might be deployed. Thus, because of the multitude of logical pathways present in a coarsegrained component and its relatively higher independence from the underlying technology platform, the structural complexity that developers encounter in the construction and verification of a coarse-grained component is high. Overcoming higher levels of structural complexity is resource intensive (Darcy et al. 2005); therefore, we expect that coarse-grained components would require more development effort than fine-grained components. Hypothesis 1 (H1). All else being equal, components that are coarse grained would be associated with greater development effort (lower development efficiency). 3.2.

Component Granularity and In-Process Defects As mentioned before, coarse granularity of components leads to larger code volumes, a multitude of logical pathways, and platform independent data repository interactions—all of which increase the likelihood of unpredictable system behavior unless

Subramanyam, Ramasubbu, and Krishnan: In Search of Efficient Flexibility

Copyright: INFORMS holds copyright to this Articles in Advance version, which is made available to subscribers. The file may not be posted on any other website, including the author’s site. Please send any questions regarding this policy to [email protected].

Information Systems Research, Articles in Advance, pp. 1–17, © 2011 INFORMS

testing and verification efforts are comprehensive. Often, development teams are resource constrained, and achieving 100% code coverage during testing and verification is not possible in reality because of aggressive project schedules. Further, it might not be possible to anticipate and test mass-customized components internally in the development lab for all the scenarios encountered in heterogeneous business process production environments. If developers fail to comprehensively test the range of functionality encapsulated in the component (lower code coverage), these components can be expected to be more defect prone. Also, debugging errors pertaining to data access of a component are known to be cognitively demanding and time consuming (von Mayrhauser and Vans 1996, Chan and Lakhotia 1998). Finally, redesign and rework activities associated with coarse-grained components to address such errors can themselves become sources of further instability due to potential ripple effects from changes, which can be difficult to comprehend fully (Mookerjee and Chiang 2002, Darcy et al. 2005). Thus, we expect coarsegrained components to be more error prone than finegrained components. Hypothesis 2 (H2). All else being equal, components that are coarse grained would be associated with more inprocess defects (lower development efficiency). 3.3.

Component Granularity and Customization Effort Customization effort is the amount of effort required to integrate a mass-customized component with a customer’s specific IT application environment during deployment. It measures a component’s flexibility to adapt to heterogeneous business process environments. Mass-customized software components do not often readily and completely fit with the IT application environments at client sites and need to be modified (customized) to be integrated with the diverse target environments. Modification of software components can be done either through black-box customization or white-box customization.2 In black-box customization, changes to software code are minimal, and modifications are usually related to changes in configuration settings of a component (e.g., switching flags on and off). In contrast, white-box customization involves examining a component’s software code and making extensive code changes to integrate a component with a target implementation environment. Because blackbox customization does not involve detailed analysis 2

Similar arguments were also presented in prior literature in the context of component reuse by Ravichandran and Rothenberger (2003).

5

or rewriting of software code (and the associated program comprehension effort), for a given component size, black-box customization effort is typically lower than white-box customization effort. Coarse-grained components are usually designed to be self-contained,3 encapsulating a broad set of business functionalities. Coarse-grained components also typically use generic and programming languageindependent component data services that act as encapsulating wrappers over technology-dependent data retrieval mechanisms that interact with a target implementation environment. Leveraging the availability of a variety of business process rules and logic encapsulated in a coarse-grained component, designers can expose configuration options pertaining to the choice of different business process rules and logic for system integrators and application administrators. An application administrator can influence the runtime behavior of the component by switching on or off the different exposed configuration flags without the need for extensive code changes, i.e., perform black-box customization. Changes made to client-specific business rules using black-box customization of a component are less resource intensive, easier to implement, standardized across different application instances, and involve minimal code-level changes. In contrast, fine-grained components often have a specific functional focus and typically use technologydependent data retrieval mechanisms that are resource efficient and responsive; this limits the extent of configuration options that designers can expose for the use of system integrators and application administrators. Thus, fine-grained components cannot be easily adapted to diverse target environments by simply switching certain configuration settings on and off, but detailed analysis of the software code is necessary to understand the data retrieval mechanisms and technological platform of a component and to match them with the specific configurations of a client environment (white-box customization). Overall, these arguments suggest that coarse-grained components are more amenable to white-box customization and hence are more flexible than fine-grained components during downstream implementation of the component at a client site. Thus, we hypothesize that Hypothesis 3 (H3). All else being equal, components that are coarse grained would be associated with lower customization effort ( greater flexibility). It is important to note that the consequences of component granularity design decisions are different in the development and implementation environment in the context of an enterprise system assembled 3

This design aspect is called “container independence”; container refers to the component’s hosting interface in a managed environment (Beisiegel et al. 2007).

Copyright: INFORMS holds copyright to this Articles in Advance version, which is made available to subscribers. The file may not be posted on any other website, including the author’s site. Please send any questions regarding this policy to [email protected].

6

Subramanyam, Ramasubbu, and Krishnan: In Search of Efficient Flexibility

using a collection of mass-customized components. The implications of component granularity design decisions on efficiency in the development environment (development effort and in-process defects) are presented in H1 and H2, and the consequences of the component granularity design decision on flexibility (customization effort)—a key performance outcome in the implementation environment—are portrayed in H3. Collectively, H1, H2, and H3 present an end-toend scenario depicting a trade-off between efficiency and flexibility induced by the component granularity design decision. Although coarse-grained components tend to increase structural complexity and place additional cognitive burdens on programmers involved in upstream development activities, there are potential advantages that could be reaped by system integrators downstream in the implementation environment. 3.4.

In-Process Defects, Development Effort, and Customization Effort A positive association between in-process software defects and project development effort has been reported in prior research (Banker and Slaughter 2000, Harter et al. 2000, Krishnan et al. 2000). Defect-prone software components require more development and verification effort as programmers encounter repeated ripples of redesign and rework activities when they localize and fix the identified defects. In-process defects identified during the upstream development stage can also be a leading indicator of the quality of the code delivered downstream to the system implementers in the field. Defect-prone software components, when integrated with a client’s existing IT application infrastructure, might destabilize the entire infrastructure, causing unexpected behaviors because of unknown root-causes. To ensure reliability of components that have a large number of in-process defects, a higher level of code coverage and testing effort is necessary during both the upstream development and downstream implementation stages, which increases both the development and customization efforts associated with a component. Thus, our final set of hypotheses is as follows. Hypothesis 4A (H4A). All else being equal, components with a higher level of in-process defects would be associated with higher development effort (i.e., lower development efficiency). Hypothesis 4B (H4B). All else being equal, components with a higher level of in-process defects would be associated with higher customization effort (i.e., lower flexibility). Given the hypothesized relationships between component granularity and in-process defects (H2) and inprocess defects and effort measures (H4A and H4B),

Information Systems Research, Articles in Advance, pp. 1–17, © 2011 INFORMS

it is possible that in-process defects play a mediating role in the relationship between component granularity design decisions and the development effort and customization effort measures.4 We empirically test the potential mediation role of in-process defects in impacting the trade-off between efficiency and flexibility along with the other hypothesis tests and report the results in §6.

4.

Research Site and Variable Descriptions

4.1. Research Site The research site for this study was Ramco Systems Limited,5 a global provider of enterprise software products and services since 1989 and a division of the diversified $2 billion Ramco Business Group headquartered in Chennai, India. The research site provides an excellent context for our study because the firm employed the MDCD methodology for developing and deploying enterprise resource planning software products for a broad clientele. Moreover, the software development processes employed by the firm conform to the ISO 9001 quality standards and were assessed at SEI-CMMI® level 5 (optimized processes with highest maturity6 )—providing a suitable environment to collect and analyze highquality and reliable data. The firm’s MDCD methodology was based on a commercial framework called VirtualWorks® , which was developed in house with a research and development effort of more than 300 person years.7 The characteristics of the MDCD methodology followed at our research site are similar to the component-based software development process described by Ravichandran and Rothenberger (2003). During the data collection process, the authors had access to several key product development managers, developers, and the source code repository of the research site, which enabled data collection specific to each business component of a complex enterprise resource planning product.8 4

We thank the review team for raising this issue.

5

Our field observations were supplemented with information gathered from shareholder reports, press releases, and the corporate website of our research site (http://www.ramco.com). 6

CMMI is a process improvement approach developed by the Software Engineering Institute at Carnegie Mellon University; see http://www.sei.cmu.edu/cmmi/. 7

A comparison of the Ramco’s approach to MDCD with the recommendations on model-driven development by the Object Management Group and other published component-based development frameworks is presented in §1 of the online appendix. An electronic companion to this paper is available as part of the online version that can be found at http://isr.journal.informs.org/. 8

An illustration of how the boundaries of business components are assessed is provided in §2 of the online appendix for interested readers.

Subramanyam, Ramasubbu, and Krishnan: In Search of Efficient Flexibility

7

Information Systems Research, Articles in Advance, pp. 1–17, © 2011 INFORMS

Copyright: INFORMS holds copyright to this Articles in Advance version, which is made available to subscribers. The file may not be posted on any other website, including the author’s site. Please send any questions regarding this policy to [email protected].

4.2.

Variable Definitions and Measurement

4.2.1. Dependent Variables. Consistent with Thomke (1997), we capture flexibility in terms of customization effort, i.e., the effort expended in customizing a generic mass-customized software component to the specific business needs and technical environment of a client. To this end, we collected and analyzed data on the time expended by system implementers for making changes to the software components of an enterprise human resource system, which they were implementing at a client site—a municipal government in the Republic of South Africa. Thus, our customization effort measure is captured from a client site, outside the context of the component development organization. Our next two performance outcomes are in-process defects and development effort. Defect proneness of a component and the likelihood of rework are assessed by gathering the number of in-process defects that occur during the construction of each component. In-process defects are measured as a count of the unique defects identified during system testing and integration testing of the component at the development organization prior to its delivery and deployment at a customer site. Development effort is measured as the cumulative person days spent on all activities related to the requirements gathering, analysis, design, construction, and verification of a component. A key advantage of our data collection procedure is that we were able to gather the individual time expended by system implementors on the customization activity for each of the masscustomized components that were assembled together (integrated) to form a complex enterprise resource planning system. Unlike project-level data typically used for empirical investigations in information systems development studies, component-level data, as considered in this study, do not cause loss of important information because of aggregation at a project Figure 2

level; rather, they facilitate more detailed analysis by simultaneously taking into account upstream (development) and downstream (implementation) lifecycle stages of an individual business component. Because the development efficiency measures were collected from the development organization prior to delivery and deployment, and customization effort data were directly obtained from a client site outside the development organization during deployment, our study does not suffer from potential commonmethods bias problems. 4.2.2. Independent Variables. As noted earlier, we assess the granularity of a software component along three subdimensions of structural design of a system: code volume, functionality, and independence (Whitehead 2002, Messerschmitt 2007). Figure 2 summarizes our operationalization of component granularity along these dimensions. We treat the loadings of the individual measures (subdimensions) on component granularity as a linear combination and hence use joint tests (Wald, chi-square) of statistical significance of the three subdimension measures for our hypothesis tests. A higher volume of code in a component is generally considered to be associated with “coarsegrainedness” (Bachmann et al. 2000, Messerschmitt 2007). Similarly, a component with a greater level of embedded functionality is considered a coarsegrained component (Messerschmitt 2007). Further, a component that is loosely coupled to other components and exhibits higher levels of independence from a specific component implementation framework is considered a coarse-grained component (Sullivan and Notkin 1990, Beiseigel et al. 2007). Corresponding to three subdimensions of granularity design, we adopt three measures of component granularity for this study: number of data elements, data layer interfaces, and internal interfaces. Data elements are inputs, outputs, internal logical files, and user interface elements of a component that

Operationalization of Component Granularity

Component granularity

Subdimensions

Measures

Code volume

Data elements

Functionality

Data layer interfaces

Independence

Internal interfaces

Copyright: INFORMS holds copyright to this Articles in Advance version, which is made available to subscribers. The file may not be posted on any other website, including the author’s site. Please send any questions regarding this policy to [email protected].

8

Subramanyam, Ramasubbu, and Krishnan: In Search of Efficient Flexibility

distinctly represent the code volume subdimension of system structure that is different from code size measures such as function-points or lines of code. Prior research posits that software modules with a large number of data elements are usually more complex and are likely to be associated with poor maintainability (Halstead 1977, Card and Glass 1990, Banker et al. 1998, Banker and Slaughter 2000). However, a greater number of data elements in a component also provides the adequate basis for accommodating a variety of business rules and thus enables easier deployment of the component in a wide variety of target IT environments. Data layer interfaces are dependency mechanisms of a component through which it interacts with underlying data repositories (e.g., relational databases). A component reads from and writes to the data repositories using the data layer interfaces in order to enact the business functionality that it encapsulates. Components with large numbers of data layer interfaces typically span a wider range of business functionality, covering a variety of business processes and associated data flows. Data layer interface specifications9 expose data to the components through encapsulated and standardized interfaces. Such standardized and decoupled data layer interface design makes it easier to adapt a component to a target customer’s specific application environment during deployment. In special cases, components with large but decoupled data layer interfaces can also be designed to be fully independent of the underlying database schema or to accept data from multiple data repositories. Internal interfaces of a component characterizes the independence subdimension of granularity design and denote the dependencies between a given component and its underlying component implementation framework. Components with relatively fewer internal interfaces (increased levels of component independence) are expected to be easier to isolate from changes that occur in the external environment. Proliferation of internal interfaces (decreased levels of component independence) is likely to increase the difficulty of automatically transforming the technology-independent design elements (e.g., business process models) into technologyspecific software code (e.g., Java code). In the MDCD environment, transformations from business logic to software code can be automated or semiautomated as long as the business functionality of the components is reasonably independent from the technological implementation framework. When internal interfaces

Information Systems Research, Articles in Advance, pp. 1–17, © 2011 INFORMS

increase, the component independence decreases, and black-box customization becomes difficult. Extensive white-box customization efforts and middleware investments might become necessary to integrate a component with a large number of internal interfaces with a target client IT environment. 4.2.3. Control Variables. Code size has been shown to be strongly associated with certain project performance outcomes in prior literature (e.g., Banker et al. 1998, Banker and Slaughter 2000, Krishnan et al. 2000) and is controlled for in our empirical models. We developed a redefined form of function points (Albrecht 1979) suitable for MDCD to assess component size by treating the component boundary as the application boundary of the original function point metric.10 Next, function point entities such as external inputs, outputs and queries are mapped to components by associating activities with tasks/services performed by each component. Because the anchors from the original function points definition of “value adjustment factor” (Albrecht 1979) reflect an application deployment environment and not a software component, we retained the unadjusted count of function points as the component size measure. In MDCD, considerable effort is often needed during the construction and testing stages of the software development life cycle (e.g., Fraternali and Paolini 2000, Hamlet 2010). At the same time, prior research (Krishnan et al. 2000) notes that greater investments in upfront analysis and design activities (in comparison to a more equal phasewise effort distribution) are likely to be associated with fewer software defects. To account for variances observed in the way effort is spread across the different phases of component development, we used two control variables that are associated with development efficiency: (i) component construction effort ratio and (ii) component testing effort ratio. The first ratio represents the proportion of total development resources that were deployed in the construction stages of building a component, after planning and high-level design activities were completed. This measure is defined as the ratio of person days expended during construction to the total person days expended on the component development. The component testing effort ratio measures the extent to which a development team expended project resources on verification and quality control activities. We define this measure as the proportion of total development person days expended on testing and verification. One could argue that our performance measures could be inherently confounded by factors such as the capability of developers. At our research site, for

9

MDCD implementations often include the XML metadata interchange specifications and metaobject facility specifications that enable clear laying out of the necessary data specifications for a model-driven system (OMG 2001).

10

The computation procedure for calculating component size based on the modified function point measure is explained in detail in §3 of the online appendix.

Subramanyam, Ramasubbu, and Krishnan: In Search of Efficient Flexibility

9

Information Systems Research, Articles in Advance, pp. 1–17, © 2011 INFORMS

Copyright: INFORMS holds copyright to this Articles in Advance version, which is made available to subscribers. The file may not be posted on any other website, including the author’s site. Please send any questions regarding this policy to [email protected].

Table 1

The summary statistics for the variables in our data set are shown in Table 1, and Pearson correlations between the variables are reported in Table 2.

Summary of Component Data

Variable Customization effort (person days) Efficiency: Development effort (person days) Efficiency: Total number of in-process defects Number of data elements Number of data layer interfaces Number of internal interfaces Component construction effort ratio Component testing effort ratio Component size

Mean

Std. dev.

Min

Max

N

14044

18053

005

128

92

86001

71026

1

398

91

29025 190087

32061 182098

0 1

147 11013

92 92

13012 12023

21035 9006

0 1

135 60

92 90

0026

0029

001

0.53

90

0018 463048

0011 414099

0008 58

0.50 11805

91 92

∗

5.1.

Empirical Model Specification

Efficiency Models ln4In-process defects5 = ‚0 + ‚1 ∗ ln4Data elements5 + ‚2 ∗ ln4Data layer interfaces5 + ‚3 ∗ ln4Internal interfaces5

Variable Correlations (1)

(1) (2) (3) (4) (5) (6) (7) (8) (9)

Empirical Models

5.1.1. Linear vs. Nonlinear Specification. The relationships between software project measures such as effort and size have been reported to be nonlinear because of the presence of economies and diseconomies of scale in software development (Boehm 1981, pp. 187–195; Banker and Kemerer 1989; Krishnan et al. 2000). Similarly, we expect a nonlinear relationship between our count-based granularity measures and the continuous scale measures of customization effort and development effort. Hence, consistent with Boehm (1981, pp. 347–367), we use a multiplicative specification for our empirical models, applying a logarithmic transformation to variables on both sides of our regression equations (cf. Kmenta 1986, p. 521). Prior literature points to software size as an important determinant of performance outcomes such as effort and quality (e.g., Banker and Slaughter 2000, Harter et al. 2000). However, in our data set, component size was highly correlated with the number of data elements (correlation of 0.95; p < 0005 as reported in Table 2), which is a potential concern for multicollinearity effects on the parameter estimates. Hence, we normalized all the variables in the regression to make them scale independent, which sacrifices the typical ease of interpreting regression estimation coefficients from log linear models. Overall, we analyzed the following empirical equations.

the specific components that are examined in this study, we found that approximately three developers (median of 3; mean of 3.29) were involved in the development of each component; the development team of a component had an average of 22 months of experience in MDCD. All personnel underwent the same training program on MDCD for about six months prior to engaging in actual development and client site deployment. Our research site also implemented standardized procedures for personnel evaluation and other human resource processes based on the CMMI level-5 key process areas throughout all of its development centers in the world. Hence, we expect that personnel-specific factors do not play an influential role in impacting performance outcomes in the context of this study, especially at the componentlevel unit of analysis. We were able to gather detailed personnel-specific data such as overall career experience and within-firm experience for 25% of the components in our sample. None of the development outcomes had a statistically significant correlation with the experience variables, further strengthening our belief that personnel-specific data are not crucial for our analysis. Table 2

5.

Customization effort Efficiency: Dev. effort Efficiency: In-process defects No. of data elements No. of data layer interfaces No. of internal interfaces Component construction effort ratio Component testing effort ratio Component size Indicates 5% level of significance.

1000 0069∗ 0042∗ 0004 0041∗ 0009 −0026∗ 0045∗ 0004

(2)

(3)

1000 0045∗ 1000 0023∗ 0024∗ 0063∗ 0046∗ 0008 0015 0002 −0020∗ 0034∗ 0018 0025∗ 0025∗

(4)

(5)

(6)

(7)

1000 0011 1000 0060∗ −0007 1000 0003 0015 −0009 1000 0010 0040∗ 0007 0004 0095∗ 0077∗ 0060∗ −0004

(8)

1000 0002

Subramanyam, Ramasubbu, and Krishnan: In Search of Efficient Flexibility

10

Information Systems Research, Articles in Advance, pp. 1–17, © 2011 INFORMS

Copyright: INFORMS holds copyright to this Articles in Advance version, which is made available to subscribers. The file may not be posted on any other website, including the author’s site. Please send any questions regarding this policy to [email protected].

Table 3

Regression Results

Dimension >

Flexibility

Dependent variable >

Efficiency

ln(Customization effort)

Variables

3SLS

SURE

ln(In-proc. defects) 3SLS

SURE

ln(Dev. effort)

ln(Data elements)

1

−00955∗∗ 4004895

−00982∗∗ 4005365

‚1

00389∗ 400245

00387∗ 400245

Œ1

ln(Data layer interfaces)

2

−00497∗∗ 4002275

−00529∗∗∗ 4001765

‚2

00347∗∗∗ 4000825

00348∗∗∗ 4000825

Œ2

ln(Internal interfaces)

3

−00296 4001965

−00311 4002045

‚3

00138 4000975

00139 4000975

Œ3

ln(In-proc. defects)

4

20229∗∗∗ 4004655

ln(Component construction effort ratio) ln(Comp. testing effort ratio) ln(Component size)

20318∗∗∗ 4000785

−00228∗∗∗ 4000595

‚4

— — 5

Model fit

−00329 4004635 • 2 = 41037∗∗∗ Adj. R2 = 0.50

Œ4

— −00219∗∗∗ 4000285

−00326 4005305 • 2 = 101000∗∗∗ Pseudo R2 = 0.53

‚5

00126 4002525 • 2 = 72010∗∗∗ Adj. R2 = 0.42

00122∗∗ 4002515 • 2 = 120075∗∗∗ Pseudo R2 = 0.50

SURE

−00344 4002605

−00345 4002475

00253∗∗ 400135

00249∗∗∗ 4000905

−00101 4001075

−00102 4000955

00627∗∗ 4003225

00627∗∗∗ 4000945 —

Œ5

—

3SLS

Œ6

00114 4000725 00486∗∗ 4002395 • 2 = 87094∗∗∗ Adj. R2 = 0.49

00123∗ 4000695 00487∗∗ 4002435 • 2 = 126083∗∗∗ Pseudo R2 = 0.50

Notes. Normalized variables are employed for analysis; standard errors of coefficients are given in parentheses. SURE, seemingly unrelated regression estimation. ∗∗∗ Indicates less than 1% level of significance; ∗∗ indicates less than 5% level of significance; ∗ indicates less than 10% level of significance.

+ ‚4 ∗ ln4Component construction effort ratio5 + ‚5 ∗ ln4Component size5 + e1 0

(1)

ln4Development effort5 = Œ0 + Œ1 ∗ ln4Data elements5 + Œ2 ∗ ln4Data layer interfaces5 + Œ3 ∗ ln4Internal interfaces5 + Œ4 ∗ ln4In-process defects5 + Œ5 ∗ ln4Component testing effort ratio5 + Œ6 ∗ ln4Component size5 + e2 0

(2)

Customization Effort Model ln4Customization effort5 = 0 + 1 ∗ ln4Data elements5 + 2 ∗ ln4Data layer interfaces5 + 3 ∗ ln4Internal interfaces5 + 4 ∗ ln4In-process defects5 + 5 ∗ ln4Component size5 + e3 0

(3SLS) regressions for our system of equations.12 We checked if the models were identified properly using the Sargan statistic and the Stock–Yogo (Stock and Yogo 2005) weak identification tests and did not notice any problems. In addition to the 3SLS regression, seemingly unrelated regression (SUR) estimations were also performed to examine consistency of results (Zellner 1962). The SUR parameters were estimated using a feasible generalized least squares procedure allowing for correlation of disturbances across the equations (Greene 1997). Estimates from the 3SLS and SUR estimation were consistent with each other and are presented in Table 3. From our original sample, three components could not be used in multivariate analyses because we could not obtain data about either the efficiency or one of the granularity subdimensions, and listwise deletion resulted in an analyzable set of 89 components. We tested for the presence of multicollinearity and observed a maximum variance inflation factor of 4.90 and a maximum condition index of 5.98, which are well within acceptable limits (Belsley et al. 1980). We performed outlier analysis by

(3)

5.2. Estimation Procedure and Diagnostics To account for sources of possible simultaneity among certain measures (e.g., in-process defects and effort measures11 ), we employed three-stage least squares

component, if the inherent quality of a component is poor, it is likely that the component may require additional verification effort when it is integrated with a client’s environment. Endogeneity between defects and effort have been discussed in other software studies as well (e.g., Harter et al. 2000, Krishnan et al. 2000). 12

11

Despite the fact that defects discovered during the development process are patched and resolved prior to delivery of the

3SLS has an advantage over instrument variable regression using two-stage least squares, as it is more efficient (Greene 1997). We thank an anonymous reviewer for this suggestion.

Subramanyam, Ramasubbu, and Krishnan: In Search of Efficient Flexibility

Copyright: INFORMS holds copyright to this Articles in Advance version, which is made available to subscribers. The file may not be posted on any other website, including the author’s site. Please send any questions regarding this policy to [email protected].

Information Systems Research, Articles in Advance, pp. 1–17, © 2011 INFORMS

computing Cook’s distance measure after our estimations (Cook and Weisberg 1982). The highest Cook’s distance was 0.148, and none of the observations was found to significantly influence the results. The results of our estimations are presented in Table 3. As noted earlier, our tests of hypotheses pertaining to granularity are based on joint tests of the parameters of the three granularity measures (see Figure 2). Specifically, for testing H1–H3, we conduct joint hypothesis tests (Wald tests) on the coefficients for our component granularity measures—data elements, data layer interfaces, and internal interfaces in our regressions. As we explain in §6, our tests indicate strong statistical evidence in support of the hypothesized relationships between the component granularity measures and outcome measures, namely, development effort {• 2 435 = 11021, p < 0001}, in-process defects {• 2 435 = 25076, p < 0001}, and customization effort {• 2 435 = 17048, p < 0001}.

6.

Discussion of Results

In this section, we discuss the results from our hypothesis tests as well as the parameter estimates for our empirical models (presented in Table 3). As noted earlier, the results from 3SLS and SUR regression are consistent with each other. For convenience, we refer to the 3SLS estimates while discussing the results for individual variables. 6.1. Development Efficiency To test H1, which predicted that coarse-grained components would be associated with greater development effort, we performed a joint hypothesis test including the parameters for the three component granularity subdimensions (refer to Œ1 , Œ2 , and Œ3 in Table 3). The results provides evidence of a statistically significant relationship between component granularity and development effort supporting H1 {• 2 435 = 11021, p < 0001}. Observing the individual parameter estimates, we find the number of data layer interfaces measure, which represents the functionality subdimension of component granularity, to be the most influential factor that explains development effort. For example, our results suggest that an increase in data layer interfaces by 1% is associated with about a 0.25% increase in development effort (Œ2 = 00253, p < 0005 from Table 3). We do not find a statistically significant direct effect of data elements and internal interfaces on development effort, but they are jointly significant with data layer interfaces, as indicated by our joint hypothesis test. A joint hypothesis test of the three subdimensions of component granularity design (refer to ‚1 , ‚2 , and ‚3 in Table 3) indicates evidence of a statistically significant relationship between component granularity and in-process defects, thereby supporting H2

11

{• 2 435 = 25076, p < 0001}. Observing the effects of the subdimensions individually, our results suggest that data layer interfaces was an influential component granularity design dimension that significantly affects in-process defects (‚2 = 00347, p < 0001 from Table 3). We also find that the internal interfaces dimension had no significant direct effect on in-process defects, but components with more data elements were positively associated with in-process defects (‚1 = 00389, p < 0010 from Table 3). Supporting H4A, our results show that more inprocess defects are significantly associated with an increase in the development effort of a component (Œ4 = 00627, p < 0001 from Table 3). The result portrays the cost of quality expended on the components, i.e., the additional effort spent on bug-fixing and rework activities to rectify critical errors in the development stage before delivery of the component for implementation at the client site. In summary, from our results, we infer that among the three component granularity design subdimensions, data layer interfaces and data elements are highly critical and any increase along these dimensions from a coarse-grained design can be detrimental to development efficiency. These findings corroborate the notion that increased complexity stemming from complex code as well as increased business functionality embedded in a software component places increased demands on development resources. 6.2. Flexibility Joint tests of the three component granularity design measures (refer 1 , 2 , and 3 in Table 3) support H3, which predicted a statistically significant negative relationship between component granularity and customization effort {• 2 435 = 17048, p < 0001}. At an individual design subdimension level, we find that components with larger numbers of data elements are associated with lower customization effort (1 = −00955, p < 0005), and an increase in the number of data layer interfaces is associated in a statistically significantly manner with a lower customization effort (2 = −00497, p < 0005). However, we do not observe a statistically significant direct relationship between the number of internal interfaces and customization effort. Overall, we find evidence for the flexibility benefits of more coarse-grained design in terms of reduced customization effort during the implementation stage. Supporting H4B, our results show that higher levels of in-process defects of a component significantly increase the customization effort necessary to implement the component in a target IT environment (4 = 20229, p < 0001 from Table 3). The result portrays the cost of quality expended on the components, i.e., the extra effort spent on bug-fixing and rework

Copyright: INFORMS holds copyright to this Articles in Advance version, which is made available to subscribers. The file may not be posted on any other website, including the author’s site. Please send any questions regarding this policy to [email protected].

12

Subramanyam, Ramasubbu, and Krishnan: In Search of Efficient Flexibility

activities to rectify critical errors during the implementation stage when the mass-customized component is integrated with a target IT environment. 6.3.

Structural Tests for Mediation and Interaction13 Our results for H4A and H4B presented evidence for the significant direct effects of in-process defects of a component in influencing performance of upstream development activities (development effort) and downstream implementation activities (customization effort) in MDCD. It is possible that the in-process defects measure mediates the relationships between component granularity design and effort expended during development and implementation of a component. To test the possibility of a mediated relationship among component granularity, in-process defects, and the effort measures, we adopted a widely used procedure from Baron and Kenny (1986). The mediation analysis found that in-process defects partially mediate the relationship between component granularity design measures and effort measures.14 Partial mediation was further confirmed using the Sobel test (1982). Prior research in the context of object-oriented design indicates that structural complexity subdimensions such as coupling and cohesion are likely to interact with each other (Darcy et al. 2005) in impacting performance outcomes. Extrapolating these effects to the component granularity subdimensions, one might expect interaction effects among the three component granularity subdimensions that we conceptualized in this study. Accordingly, we tested for the possibility of interactions among the data elements, data layer interfaces, and internal interfaces design measures of component granularity in impacting efficiency and flexibility outcomes. F -tests of the empirical models specifying the interaction effects indicated no statistical support for the presence of interaction effects among our component granularity measures. However, we acknowledge that this result could also be attributed to the particular sample of components in our data set and needs to be verified in other diverse component repositories in the future. 6.4. Analysis of Flexibility–Efficiency Trade-offs Summarizing our hypotheses testing results pertaining to component granularity design, we find that a more coarse-grained design of mass-customized software components is likely to introduce a higher 13 We thank the review team for the suggestion to include the additional mediation-related structural tests in our analysis. 14

Please note that we are not inferring a causal relationship from the mediation tests. Rather, we only posit associations among our dependent and independent variables. We thank an anonymous reviewer for advocating this caution.

Information Systems Research, Articles in Advance, pp. 1–17, © 2011 INFORMS

number of defects and necessitates greater development effort but could facilitate greater downstream flexibility, i.e., easier integration of mass-customized software components with target client IT environments. The increased downstream flexibility enabled by a more coarse-grained component design could be attributed to (a) the use of less resource-intensive black-box customization of components and (b) the ability of MDCD methodology to elegantly separate business logic from implementation-specific aspects. Our results also indicate that there is a strong positive association between in-process defects and the upstream and downstream effort outcomes as hypothesized in H4A and H4B, respectively (4 = 20229, p < 0001; Œ4 = 00627, p < 0001). Although coarse-grained components (especially the greater number of data elements and data layer interfaces) are generally associated with a lower customization effort, they are prone to a greater number of inprocess defects, which in turn causes an increase in both customization effort and development effort. This interlinked relationship among component granularity, in-process defects, customization effort, and development effort presents strong evidence for the presence of trade-offs between development efficiency and flexibility in MDCD. The MDCD approach presents designers with tools for decoupling implementation—and business logicspecific details, which enables them to rapidly adapt enterprise systems to changing business requirements and to counter environmental volatility. However, our results indicate that the negative effects of defects introduced during the decoupling process, primarily arising from component granularity design choices, might sometimes outweigh the positive effects of the decoupling mechanisms enabled by the MDCD approach. In other words, the effort spent with the goal of improving flexibility through MDCD necessitates additional defect correction and rework effort, which causes a significant loss of overall development efficiency. Our results also establish that the extent of in-process bugs introduced in a MDCD life cycle and the resulting strength of the trade-offs between flexibility and efficiency in MDCD is significantly impacted by component granularity. To illustrate the interlinked trade-offs observed in MDCD, we analyzed the marginal effect of the two statistically significant component granularity measures on in-process defects and the resulting simultaneous impact of in-process defects on customization effort using the parameter estimates (reported in Table 3) of our regression equations (Equations (1)–(3) presented in §5.1). The results of this analysis are presented in Figures 3(a)–3(d). While performing the marginal analysis, we held all other variables of the simultaneous equations at their mean values.

Subramanyam, Ramasubbu, and Krishnan: In Search of Efficient Flexibility

13

Information Systems Research, Articles in Advance, pp. 1–17, © 2011 INFORMS

Effect of Data Elements on Customization Effort

Figure 3(c)

Customization effort–Data elements

Effect of Data Layer Interfaces on Customization Effort

Customization effort–Data layer interfaces

50

40

30

Person days →

Person days →

40

20

10

30

20

10 Data elements = 154

0 –3

–2

–1

0

1

2

Data layer interfaces = 4 0 –3

3

–2

Figure 3(b)

Figure 3(d)

Effect of Data Elements on In-Process Defects

12

In-process defects–Data elements

12

1

2

3

Effect of Data Layer Interfaces on In-Process Defects

In-process defects–Data layer interfaces

8

Count →

8 6

6 4

4

0 –3

0

10

10

2

–1

Normalized In(Data layer interfaces)

Normalized In(Data elements)

Count→

Copyright: INFORMS holds copyright to this Articles in Advance version, which is made available to subscribers. The file may not be posted on any other website, including the author’s site. Please send any questions regarding this policy to [email protected].

Figure 3(a)

Data layer interfaces = 4

–2

2

Data elements = 154

Data elements = 1

–1

0

1

2

0 –3

3

–2

–1

0

1

2

3

Normalized In(Data layer interfaces)

Normalized In(Data elements)

As we can see from Figures 3(a)–3(d), there exists a threshold component granularity at which the benefits of lower customization effort outweigh the loss of efficiency (increased defects). From the cutoff points and threshold values indicated in Figures 3(a)–3(d), we can infer the following. First, for an incremental increase in data elements from 1 to 154, we observe a sharp reduction in customization effort (Figure 3(a)) but a gradual increase in in-process defects (Figure 3(b)). After this threshold, the customization effort as well as in-process defects increase sharply. Second, for an incremental increase in data layer interfaces from 0 to 4, there is a sharp reduction in customization effort (Figure 3(c)), and it is accompanied by a very gradual increase in-process defects (Figure 3(d)). After this threshold, an increase in data layer interfaces is accompanied by increases in both customization effort and in-process defects. These results highlight an important implication for software architects from this study—the importance of managing the trade-off between flexibility and effi-

Data layer interfaces = 0

ciency through the choice of component granularity. For example, to achieve simultaneous benefits of both flexibility and efficiency through MDCD, a desirable range of component granularity values for the components examined in this study is 1–154 data elements and 0–4 data layer interfaces.15 Taken together, our findings imply that a coarsegrained component design, which emphasizes encapsulation of several logical paths, user inputs, outputs, and interactions with the underlying data repository within a component, facilitates increased flexibility during the implementation phase. However, such a component design could lead to an increase in inprocess defects and expenditure of greater development effort. 15

Although the lowest point of the customization effort curve was observed around the point {data layers = 4}, there was a range of values {until data elements ≤ 30}, as seen in Figures 3(a) and 3(b), until which the customization effort did not change beyond two person days. Similarly, we observed that in the range of data layers between 0 and 13, defect counts were less than 3 and customization effort did not increase by more than three person days.

Subramanyam, Ramasubbu, and Krishnan: In Search of Efficient Flexibility

14

Information Systems Research, Articles in Advance, pp. 1–17, © 2011 INFORMS

Copyright: INFORMS holds copyright to this Articles in Advance version, which is made available to subscribers. The file may not be posted on any other website, including the author’s site. Please send any questions regarding this policy to [email protected].

Table 4

Component Granularity and Postimplementation Defects

Measure Data elements Data layer interfaces Internal interfaces ∗∗

Pearson correlations 00375∗∗ 00667∗∗ 00329∗∗

p < 0005.

Fixing these in-process defects takes a toll on development effort (Œ4 = 00627, p < 0005), but fault localization and debugging activities could also give opportunities for refactoring the overall component design and thereby making a component more selfcontained with minimal dependencies on other modules of the system. Further, it is also possible that the one-time loss in development efficiency could be offset by flexibility-related gains realized from multiple deployments across a broad client base. 6.5. Postdeployment Concerns Over time, even well-designed software components can become unmanageably complex, unpredictable, and unusable through several (and sometimes conflicting) adaptations. To understand the long-term effects of component granularity on the integrity of a component, we collected data on the defects detected per component after installation (and customization) at the customer site for a subset of components in our data set (24 business components). Given the relatively small sample size and limited degrees of freedom available, we do not perform multivariate statistical analysis using these data. Instead, we use the Pearson correlations (presented in Table 4) between the component granularity measures and the count of postrelease defects. While these bivariate correlations possibly overstate the ability of our granularity measures to predict postrelease defects, they nevertheless corroborate the argument that these measures could be useful in understanding long-term implications of granularity choices. The observed positive correlation between in-process defects and postimplementation defects (albeit for a fraction of the components) suggests that these postrelease quality problems are likely to originate from design instability observed during component development. The correlations shown in Table 4 suggest that all the component granularity measures are positively associated with postrelease defects. Thus, if the granularity design subdimensions are not actively managed, there is a possibility that the integrity of an enterprise system made of coarse-grained components could deteriorate over time.

7.

Contributions and Implications

The effort involved in incorporating flexibility to a system is often nontrivial, and it is a considerable

design challenge to build flexible enterprise software systems in an efficient way. Prior software engineering research has posited that the ability of a complex software system to change is a function of its design structure (cf. Parnas 1979, Simon 1996). Several of the established software structural complexity measures, such as coupling, cohesion, information hiding, and depth of inheritance (e.g., Yourdon and Constantine 1979, Chidamber and Kemerer 1994, Darcy et al. 2005), are not directly applicable for MDCD and do not lend themselves to analyses at a software component level. At the same time, we do know that modular and loosely coupled designs provide the ability to accommodate change. Taking inspiration from the modularity and loose-coupling conceptualizations in organization theory (e.g., Weick 1976, Sanchez and Mahoney 1996, Simon 1996, Adler et al. 1999, Baldwin and Clark 2000) and simultaneously applying the measurable properties of software design, we portray component granularity design as the lever that drives the relationship between efficiency and flexibility in enterprise software development and deployment. We specifically provide three measures of component granularity design that match well with the generic structural complexity dimensions of code volume, functionality, and independence, which can be used to assess flexibility– efficiency trade-offs in software production. Our empirical results establish the importance of component granularity design decision, illustrating the management of trade-offs between upstream efficiency and downstream flexibility in mass-customized software product life cycle through appropriate decomposition of complex enterprise systems into loosely coupled, modular, self-contained components. To achieve higher development efficiency, architects and design teams should limit the data linkages for each component and limit the number of logical paths within a component. However, carefully planned increases in the number of data elements and data layer interfaces of a component can offer downstream benefits during client deployments in terms of lower customization effort. Our finding that there exists a threshold component granularity, around which the benefits of lower customization effort might not be neutralized by the loss of efficiency (increased defects and development effort), should be encouraging for architects making scrupulous component design choices and aiming for the elusive goal of simultaneously achieving flexibility and efficiency. To realize the flexibility benefits offered by modular design approaches such as MDCD, it is also necessary for software managers to invest in complementary activities such as refactoring to maintain optimal balance across the different granularity subdimensions.

Subramanyam, Ramasubbu, and Krishnan: In Search of Efficient Flexibility

15

Copyright: INFORMS holds copyright to this Articles in Advance version, which is made available to subscribers. The file may not be posted on any other website, including the author’s site. Please send any questions regarding this policy to [email protected].

Information Systems Research, Articles in Advance, pp. 1–17, © 2011 INFORMS

Because component granularity design is a specific instance of a design decision regarding module boundaries, we believe that the results of this study can be extended to broader mass-customized product development settings beyond software development. We have shown that investments to achieve flexibility in a modular design structure are often nontrivial and may not yield corresponding returns, especially when a firm pursues a mass-customization product strategy, unless the firm makes complimentary efficiency-improving investments. For example, flexibility-related costs can be recouped through potential efficiency gains stemming from improvements in testing and verification procedures, such as automated testing and process quality improvements that pay careful attention to measurable and leading indicators of product quality early in the product development life cycle. More research is clearly needed to confirm such broader generalization of our findings; this study lays a foundation to rigorously analyze efficiency–flexibility trade-offs in masscustomized products by drawing careful attention to the design of module boundaries and the effects of modularity design on both upstream product development and downstream customization activities.

8.

Limitations

There are certain limitations to this study that could be addressed by future research. First, we have been able to analyze a sample of 92 business components from one modern development organization. Future research is needed, preferably with a larger sample size of diverse components drawn from several organizations, to further generalize our findings. Second, we only examined the application of MDCD in a business context. It is necessary to verify our results in other scenarios that place emphases on different performance outcomes of systems development, such as response time, ease of use, security, and privacy. Third, although we considered the separation of concerns facilitated by the MDCD approach, we did not specifically focus on the impact of changes that arise from lower-level model abstractions of the model-driven architecture. As Hailpern and Tarr (2006) point out, incorporating changes originating from the lower-level abstractions (e.g., the platformspecific source code) into the higher-level abstractions (such as the business process model) is often cumbersome and would involve human intervention. Future research could explore such transformation processes and study the flexibility–efficiency tradeoffs accounting for the effort expended in model transformations at the different layers of a system’s architecture. Fourth, the component library at our research site was designed for mass customization,

specifically with the intent to enable reuse across different client environments. Hence, our findings might need to be interpreted with caution in alternate settings such as libraries created for specific organizational boundaries and different business settings. We acknowledge that the generalizability of our findings would be limited to organizations that follow a masscustomized product development strategy and create libraries of software components with the intent to reuse the components in diverse target client settings. A related concern is that the influences of architectural choices for component construction need to be considered when generalizing our findings. For example, unlike the component repository examined in this study, an alternate component framework might primarily use technology-specific libraries of application-programming interfaces (APIs), such as open database connectivity APIs, for accessing relational data. In contrast, the component repository examined in this study used generic component services to implement data retrieval and persistence mechanisms. Studying the effects of such architectural choices with respect to higher-level project performance outcomes is an interesting avenue for future research on MDCD. Finally, we did not fully explore the role played by the model-driven architecture in acting as the common ground and the communication link between the different stakeholders involved in the software development life cycle—end users, designers, business analysts, managers, and implementation experts. Future research examining the flexibility–efficiency trade-off in software development could specifically account for the value created by modular design approaches such as MDCD in facilitating a common information repository and communication medium for different stakeholders.

9.

Conclusion

This paper analyzes the trade-offs between flexibility and development efficiency when a firm employs modular design approaches such as MDCD for building mass-customized enterprise systems. We gathered detailed design, code, and project management data from the commercial implementation of 92 software components developed using the MDCD approach. Taking inspiration from the modularity and loosecoupling conceptualizations in organization theory, and simultaneously applying the measurable properties of software design, we used component granularity as the lever that drives the relationship between efficiency and flexibility in enterprise software development. We operationalized component granularity in terms of three relatively easy-to-collect (at the design stage) measures that match well with

Subramanyam, Ramasubbu, and Krishnan: In Search of Efficient Flexibility

Copyright: INFORMS holds copyright to this Articles in Advance version, which is made available to subscribers. The file may not be posted on any other website, including the author’s site. Please send any questions regarding this policy to [email protected].

16

Information Systems Research, Articles in Advance, pp. 1–17, © 2011 INFORMS

the established structural complexity dimensions of code volume, functionality, and independence, which can be used to assess flexibility–efficiency tradeoffs in software. We then empirically established the relationship between the conceptualized component granularity measures and the outcomes of flexibility and development efficiency. We find that there exists a threshold component granularity around which the flexibility benefits such as lower customization effort are not neutralized by the loss of efficiency (increased defects and development effort). Thus, the key implication from our study for software managers and designers looking to adopt modular design approaches is that active and judicious management of component granularity resulting from the decomposition of complex enterprise systems is necessary to simultaneously achieve flexibility and efficiency in software development. We believe that this study provides a good foundational framework for examining and understanding the business value of emerging assembly-oriented software development methodologies such as MDCD. Although the flexibility–efficiency trade-offs typically observed in procedural and object-oriented software development continue to exist even in the MDCD regime, the modular design approach and the automatic model transformation features offered by MDCD show promise in their ability to shift the flexibility–efficiency trade-offs in mass-customized software product development. We believe that examining the flexibility–efficiency trade-off shifting mechanisms enabled by assembly-oriented information systems development using software components in diverse business contexts and organizational settings is a rich avenue for future research.

10.

Electronic Companion

An electronic companion to this paper is available as part of the online version that can be found at http:// isr.journal.informs.org/. Acknowledgments The authors are grateful to P. R. Venketrama Raja, D. Raghuram, M. Krishnamurthy, and other employees at Ramco Systems for their support and assistance in the data collection for this project. The authors also thank Tridas Mukhopadhyay, Jason Woodard, and the review team for valuable feedback on earlier versions of this paper. Financial assistance was provided in part by the University of Illinois, Singapore Management University, and the Joseph Handelman fellowship at the University of Michigan.

References Adler, P. S., B. Goldoftas, D. I. Levine. 1999. Flexibility versus efficiency? A case study of model changeovers in the Toyota production system. Organ. Sci. 10(1) 43–68.

Albrecht, A. J. 1979. Measuring application development productivity. Proc. Joint SHARE, GUIDE IBM Application Development Sympos., IBM Corporation, Monterey, CA, 83–92. Bachmann, F., L. Bass, C. Buhman, S. Comella-Dorda, F. Long, J. Robert, R. Seacord, K. Wallnau. 2000. Technical concepts of component-based software engineering. CMU/SEI Technical Report ESC-TR-2000–007, Carnegie Mellon University, Pittsburgh. Baldwin, C. Y., K. B. Clark. 2000. Design Rules—The Power of Modularity. MIT Press, Cambridge, MA. Banker, R. D., C. F. Kemerer. 1989. Scale of economies in new software development. IEEE Trans. Software Engrg. 15(10) 1199–1205. Banker, R. D., S. A. Slaughter. 2000. The moderating effects of structure on volatility and complexity in software enhancement. Inform. Systems Res. 11(3) 219–240. Banker, R. D., G. Davis, S. A. Slaughter. 1998. Software development practices, software complexity and software maintenance performance: A field study. Management Sci. 44(4) 433–450. Baron, R. M., D. A. Kenny. 1986. The moderator-mediator variable distinction in social psychological research: Conceptual, strategic and statistical considerations. J. Personality Soc. Psych. 51(6) 1173–1182. Basili, V. R., B. R. Perricone. 1984. Software errors and complexity. Comm. ACM 27(1) 42–52. Bast, W., A. G. Kleppe, J. B. Warmer. 2003. MDA Explained: The Model Driven Architecture: Practice and Promise. Addison-Wesley, New York. Beisiegel, M., D. Booz, M. Edwards, E. Herness, S. Kinder. 2007. Software components: Coarse-grained vs. fine-grained. Report (December 6), IBM developerWorks Technical Library, IBM, Armonk, NY. Accessed June 9, 2011, http://www.ibm .com/developerworks/webservices/library/ws-soa-granularity/. Belsley, D., E. Kuh, R. Welsch. 1980. Regression Diagnostics: Identifying Influential Data and Sources of Collinearity. Wiley, New York. Blecker, T., N. Abdelkafi. 2006. Complexity and variety in mass customization systems: Analysis and recommendations. Management Decision 44(7) 908–929. Blecker, T., G. Friedrich, B. Kaluza, N. Abdelkafi, G. Kreutler. 2004. Information and Management Systems for Product Customization. Springer, New York. Boehm, B. W. 1981. Software Engineering Economics. Prentice-Hall, Englewood Cliffs, NJ. Brereton, P., D. Budgen. 2000. Component-based systems: A classification of issues. IEEE Comput. 33(11) 54–62. Byrd, T. A., D. E. Turner. 2000. Measuring the flexibility of information technology infrastructure: Exploratory analysis of a construct. J. Management Inform. Systems 17(1) 167–208. Card, D., R. Glass. 1990. Measuring Software Design Quality. PrenticeHall, Englewood Cliffs, NJ. Chan, T. W., A. Lakhotia. 1998. Debugging program failure exhibited by voluminous data. J. Software Maintenance 10(2) 111–150. Chidamber, S. R., C. F. Kemerer. 1994. A metrics suite for object-oriented design. IEEE Trans. Software Engrg. 20(6) 476–493. Cook, R. D., S. Weisberg. 1982. Residuals and Influence in Regression. Chapman & Hall, London. Darcy, D. P., C. F. Kemerer, S. A. Slaughter, J. E. Tomayko. 2005. The structural complexity of software: an experimental test. IEEE Trans. Software Engrg. 31(11) 982–995. Duray, R., P. T. Ward, G. W. Milligan, W. L. Berry. 2000. Approaches to mass customization: Configurations and empirical validation. J. Oper. Management 18(6) 605–625.

Subramanyam, Ramasubbu, and Krishnan: In Search of Efficient Flexibility

Copyright: INFORMS holds copyright to this Articles in Advance version, which is made available to subscribers. The file may not be posted on any other website, including the author’s site. Please send any questions regarding this policy to [email protected].

Information Systems Research, Articles in Advance, pp. 1–17, © 2011 INFORMS

Fraternali, P., P. Paolini. 2000. Model-driven development of Web applications: The autoweb system. ACM Trans. Inform. Systems 18(4) 323–382. Greene, W. H. 1997. Econometric Analysis, 3rd ed. Prentice-Hall, Englewood Cliffs, NJ. Hailpern, B., P. Tarr. 2006. Model-driven development: The good, the bad, and the ugly. IBM Systems J. 45(3) 451–461. Halstead, M. 1977. Elements of Software Science. Elsevier, New York. Hamlet, D. 2010. Composing Software Components: A Software Testing Perspective. Springer, New York. Harter, D. E., M. S. Krishnan, S. A. Slaughter. 2000. Effects of process maturity on quality, cycle time and effort in software product development. Management Sci. 46(4) 451–466. Kmenta, J. 1986. Elements of Econometrics, 2nd ed. Macmillan, New York. Krishnan, M. S., C. H. Kriebel, S. Kekre, T. Mukhopadhyay. 2000. An empirical analysis of productivity and quality in software products. Management Sci. 46(6) 745–759. Kulkarni, V., S. Reddy. 2003. Separation of concerns in modeldriven development. IEEE Software 20(5) 64–69. Messerschmitt, D. G. 2007. Rethinking components: From hardware and software to systems. Proc. IEEE 95(7) 1473–1496. Mookerjee, V., I. R. Chiang. 2002. A dynamic coordination policy for software system construction. IEEE Trans. Software Engrg. 28(7) 684–694. OMG. 2001. Model driven architecture: A technical perspective. Report, Architecture Board MDA Drafting Team, OMG, Needham, MA. Accessed May 29, 2007, ftp://ftp.omg.org/ pub/docs/ab/01–02-04.pdf. Parnas, D. L. 1972. On the criteria to be used in decomposing systems into modules. Comm. ACM 15(12) 1053–1058. Parnas, D. L. 1979. Designing software for ease of extension and contraction. IEEE Trans. Software Engrg. 5(2) 128–138. Ravichandran, T., M. A. Rothenberger. 2003. Software reuse strategies and component markets. Comm. ACM 46(8) 109–114. Sambamurthy, V., A. Bharadwaj, V. Grover. 2003. Shaping agility through digital options: Reconceptualizing the role of information technology in contemporary firms. MIS Quart. 27(2) 237–263. Sanchez, R., J. T. Mahoney. 1996. Modularity, flexibility, and knowledge management in product and organization design. Strategic Management J. 17(Winter special issue) 63–76.

17

Schmidt, D. C. 2006. Model-driven engineering. IEEE Comput. 39(2) 25–31. Simon, H. A. 1996. The Sciences of the Artificial, 3rd ed. MIT Press, Cambridge, MA. Sobel, M. E. 1982. Asymptotic confidence intervals for indirect effects in structural equation models. S. Leinhardt, ed. Sociological Methodology. American Sociological Association, Washington, DC, 290–312. Stock, J. H., M. Yogo. 2005. Testing for weak instruments in linear iv regression. D. W. K. Andrews, J. H. Stock, eds. Identification and Inference for Econometric Models: Essays in Honor of Thomas Rothenberg. Cambridge University Press, Cambridge, UK, 80–108. Subramanyam, R., M. S. Krishnan. 2003. Empirical analysis of CK metrics for object-oriented design complexity: Implications for software defects. IEEE Trans. Software Engrg. 29(4) 297–310. Sullivan, K., D. Notkin. 1990. Reconciling environment integration and component independence. Proc. 4th ACM SIGSOFT Sympos. Software Development Environments, ACM, New York, 22–33. Szyperski, C. 2002. Component Software: Beyond Object Oriented Programming. Addison-Wesley, Reading, MA. Thomke, S. H. 1997. The role of flexibility in the development of new products: An empirical study. Res. Policy 26(1) 105–119. Tu, Q., M. A. Vonderembse, T. S. Ragu-Nathan, B. RaguNathan. 2004. Measuring modularity-based manufacturing practices and their impact on mass customization capability: A customer-driven perspective. Decision Sci. 35(2) 147–168. von Mayrhauser, A., A. M. Vans. 1996. Identification of dynamic comprehension processes during large scale maintenance. IEEE Trans. Software Engrg. 22(6) 424–437. Weick, K. E. 1976. Educational organizations as loosely coupled systems. Admin. Sci. Quart. 21(1) 1–19. Whitehead, K. 2002. Component-Based Development: Principles and Planning for Business Systems. Addison-Wesley, Reading, MA. Yourdon, E., L. L. Constantine. 1979. Structured Design: Fundamentals of a Discipline of Computer Program and Systems Design. Prentice-Hall, Englewood Cliffs, NJ. Zellner, A. 1962. An efficient method of estimating seemingly unrelated regression and tests for a regression bias. J. Amer. Stat. Assoc. 57(June) 348–368.