A Systematic Review on Formal Testing Approaches for Web Services Andr´e Takeshi Endo, Adenilso da Silva Sim˜ao 1

Instituto de Ciˆencias Matem´aticas e de Computac¸a˜ o, Universidade de S˜ao Paulo Cx. Postal 668 – 13560-970 – S˜ao Carlos – SP – Brasil {aendo,adenilso}@icmc.usp.br

Abstract. Service-oriented architectures and web services have been used as important technologies to foster the development of loosely coupled and distributed applications. Web services pose new challenges for the testing activity, mainly because mission-critical and complex business process systems are implemented with them. In this context, formal testing approaches are necessary to guarantee the service quality. In this paper, we identify formal approaches to test service-oriented architectures and web services, analyzing the application context, authorship, and references, as well as classifying the used models. A systematic review was planned and executed to select the studies considered in this paper.

1. Introduction Service-Oriented Architecture (SOA) is a promissing architectural style that aims at creating loosely coupled software. SOA is based on standards and provides a distributed and protocol-independent computing paradigm. Software resources are encapsulated as services which are well-defined and self-contained modules with business functionalities, and being independent of state and context of other services [Papazoglou and Heuvel, 2007]. Web services represent the most widespread technology that implements SOA. Web services are applications that provide operations invoked by other applications via Web, using open XML standards, such as SOAP, WSDL, and UDDI. In a SOA environment, a set of services can be used to create new and more complex services that solve business process objectives. This process, called web service composition, can be developed either as orchestration or as choreography. In service orchestration, there is a main entity that is responsible for coordinating the partner services. The partner service is not aware of the composition. Currently, the main language to implement an orchestration is the Web Service Business Process Execution Language (BPEL) [OASIS, 2007]. In service choreography, there is no control entity and all partner services work cooperatively. All services must be aware of the composition. There are several languages used to describe a service choreography, e.g. Web Services Choreography Description Language (WS-CDL) [W3C, 2005]. As new software technologies, SOA and web services bring challenges for testing activity, such as dynamic binding, lack of observability, and service composition. In the last years, testing of web services and SOAs has been an intensively investigated topic. Formal approaches have been applied to test services, as the importance of SOA-based software development demands that rigorous testing approaches are employed. Formal approaches use a mathematical model or specification, e.g. finite state machines, labeled

transition systems, and control flow graphs, to support the test activity. According to Hierons et al. [2009], the presence of formal models or specifications can lead to more efficient and effective testing. In this context, there is an interest in answering questions such as “which are the formal approaches to test SOAs and web services?” and “which models support these approaches?”. Some surveys, e.g. [Bozkurt et al., 2010; Canfora and Di Penta, 2009], attempt to answer more generic questions about service testing. Although these surveys provide an overview of the service testing area, they were accomplished without a systematic process and do not handle the formal testing of web services in-depth. Seeking a methodology for performing rigorous review of the current research, the systematic review, originating from the medicine, has been proposed for the software engineering community [Kitchenham and Charters, 2007]. The systematic review follows a well-defined and strict sequence of methodological steps described in a previously established protocol [Biolchini et al., 2005]. It is conceived to be an unbiased and (to a certain degree) repeatable process, summarizing the existing evidence related to a specific technology and providing a background to place new studies [Kitchenham and Charters, 2007]. In this paper, we present a systematic review that shows studies in applying formal testing to SOAs and web services. The proposed approaches that use formal testing were selected, analyzing quantitatively the application context, models, authorship, and references. We identified 37 adequate studies, showing the current interest in this topic. However, it was noted that more effort to cooperation among research groups is necessary. We believe that the results presented in this paper are relevant for researchers seeking for an analysis of the state-of-art of the formal testing of web services. The remainder of this paper is organized as follows. Section 2 briefly summarizes the planning of the systematic review. Section 3 shows the obtained results and the analysis of the selected studies. Finally, in Section 4, we present the conclusions and discuss future work.

2. Systematic Review Planning The systematic review technique has been proposed for the software engineering area to provide a more rigorous process to gather and evaluate evidence in a specific topic. The evidence is collected from primary studies which are individual studies that contribute to a systematic review [Kitchenham and Charters, 2007]. Systematic review is a secondary study since it reviews and synthesizes primary studies related to a research question. The objective of this systematic review is to identify formal approaches to test SOAs and web services. Moreover, we also aim at identifying the application context, the adopted models, and the relation among research groups. Based on these goals, we formulate the following questions: Main Research Question: Which approaches have been proposed to test SOAs and web services using formal models or specifications? Secondary Questions: 1. Which are the application contexts, e.g. single service and service composition? 2. What formal models or specifications are used?

3. What are the main research groups working in this topic? 4. What are the most referenced studies? The primary study selection was carried out by searching in a set of the selected repositories. Our repositories include ACM, IEEE, SpringerLink, Scopus, and Compendex. We also constructed a list of potential relevant studies based on manual searches. The adopted search string is presented as follows. (’web service’ OR ’web services’ OR soa OR ’service oriented’ OR ’service based’ OR ’service centric’ OR ’service driven’) AND (testing OR test) AND (formal OR model OR specification OR mathematical)

Due to space limitations, we do not provide a complete description of the planning and conduction. Further details can be found elsewhere [Endo and Sim˜ao, 2010].

3. Analysis of Results We analyzed the 37 selected primary studies considering the topics: application context, models, authorship, and references connection. The topics were defined to answer the presented research questions. Figure 1 summarizes the considered topics in a graph, modeling each primary study as a node (the node label is its reference number in Table 1). Node forms, edges, and colors represent the different topics considered. More explanation about Figure 1 and these topics are presented in the following sections.

Figure 1. Relationships among the selected primary studies.

3.1. Application Context During the development of service-oriented applications, the stakeholders involved in testing activity must handle different levels or application contexts. As in traditional software testing, it is common that different strategies are carried out for unit and integration testing. Analyzing the selected studies, there are two clearly distinct contexts: (1) single service testing and (2) service composition testing. Single service testing is basically interested in assuring the reliability of an isolated service. Service composition testing focuses on the integration and execution of a set of services contained in a composition. Figure 1 represents single service studies as circles and service composition studies as squared boxes. We also present in this section the references for the 37 primary studies grouped in each application context. Table 1 presents the distribution of primary studies for each context. The references are also presented. Table 1. Distribution of studies considering application context and number reference for Figure 1. Type

References

# Studies

Testing

(01) Tsai et al. [2005b]; (02) Tsai et al. [2005c]; (03) Tsai et al. [2005a]; (04) Chan et al. [2007]; (05) Ma et al. [2008]; (06) Bertolino et al. [2004]; (07) Bertolino et al. [2008]; (08) Frantzen et al. [2008]; (09) Li et al. [2008a]; (10) Belli and Linschulte [2008]; (11) Keum et al. [2006]; (12) Dranidis et al. [2007]; (13) Ramollari et al. [2009]; (14) Paradkar et al. [2007]; (15) Heckel and Mariani [2005]; (16) Lohmann et al. [2007]; (17) Park et al. [2009]; (18) Chakrabarti and Rodriquez [2010]; (19) Kourtesis et al. [2010].

19

Service Composition Testing (Ref# 20–37)

(20) Benharref et al. [2006]; (21) Lallali et al. [2008]; (22) Yan et al. [2006]; (23) Zheng et al. [2007]; (24) Corradini et al. [2008]; (25) Karam et al. [2007]; (26) Liu et al. [2008]; (27) Endo et al. [2008]; (28) Li et al. [2008b]; (29) Mei et al. [2008]; (30) Mei et al. [2009c]; (31) Mei et al. [2009b]; (32) Baldoni et al. [2005]; (33) Mei et al. [2009a]; (34) Ruth et al. [2007]; (35) Wieczorek et al. [2009]; (36) Hou et al. [2009]; (37) [Bentakouk et al., 2009]

18

Single Service (Ref# 01–19)

There is research interest in both application contexts with an almost equal distribution of studies (19 single versus 18 composition). We also note that there is no approach that fits for both single services and service composition since the studies focus on a specific context. It is clear that the current approaches can be combined to provide a more complete testing strategy. However, we could not identify any research effort in this context. Single Service Testing. In the single service testing, the proposed approaches can be classified based on service state, i.e., whether a service keeps internal state or not. We consider a service stateless when it is not necessary a context (previous operation invocations) to invoke a service operation. Using a stateless service, only the input for the service operation is necessary. On the other hand, a stateful service keeps state information and the current state can change the output of a service operation. We could also identify approaches that are applicable for both types. Figure 1 presents different circles for each context of single services. In stateless services, the studies focus on test case generation. The work of Tsai et al. [2005a,b,c] is based on the Swiss Cheese model derived from semantic specifications in Ontology Web Language for Service (OWL-S) [W3C, 2004]. Ma et al. [2008] use available WSDL and XML Schema specifications to support the tests. Using a different approach, Chan et al. [2007] use metamorphic relations to deal with the oracle problem and generate new test cases.

In stateful services, conformance testing is the main goal using test case generation to support it [Belli and Linschulte, 2008; Bertolino et al., 2004; Dranidis et al., 2007; Frantzen et al., 2008; Keum et al., 2006; Kourtesis et al., 2010; Paradkar et al., 2007; Ramollari et al., 2009]. Bertolino et al. [2008] focus on stub generation using a state model and QoS properties. Li et al. [2008a] propose the usage of control flow graphs to model the valid sequence of operation invocations. Chakrabarti and Rodriquez [2010] present a formal approach to test RESTful web services. In stateless and stateful services, all studies are based on Graph Transformation rules, enabling the applicability for stateless and stateful services [Heckel and Mariani, 2005; Lohmann et al., 2007; Park et al., 2009]. Besides supporting test case generation, the process of service discovery is also supported. We note more interest in stateful services (11 studies) than stateless services (five studies). A group of studies focuses on automating the process of test case generation mainly as consequence of the dynamic nature of web services. Some studies consider that the syntactic specification (WSDL) was augmented with semantic descriptions (OWL-S, RDF) [Ramollari et al., 2009; Tsai et al., 2005b]. Although the approaches can be applied for any type of service, only one study considers the RESTful web services [Chakrabarti and Rodriquez, 2010]. There are few approaches that can be applied to stateless and stateful services (only three studies). Service Composition Testing. Service composition testing is divided based on the used paradigms: orchestration and choreography. Figure 1 presents different squared boxes for each context of service composition. In orchestration testing, BPEL is the main language used to describe the composition (12 studies). There is research focused on test case generation handling different aspects, such as BPEL specific structures [Bentakouk et al., 2009; Hou et al., 2009; Liu et al., 2008; Zheng et al., 2007], concurrency [Li et al., 2008b; Yan et al., 2006], timing properties [Lallali et al., 2008], and run-time composition [Corradini et al., 2008]. There are also studies related to coverage criteria that: use workflow modeling [Karam et al., 2007], handle communication among processes [Endo et al., 2008], consider XPath artifacts [Mei et al., 2008], and evaluate traditional data flow criteria [Mei et al., 2009b]. Regression testing is also approached considering minimization [Li et al., 2008b] and prioritization [Mei et al., 2009c]. Moreover, Benharref et al. [2006] propose an observer architecture to support passive testing. In choreography testing, we identified three studies that handle conformance testing [Baldoni et al., 2005], test generation [Wieczorek et al., 2009], and coverage criteria [Mei et al., 2009a]. Baldoni et al. [2005] propose a framework inspired in multi-agent systems for conformance testing between the peer (single service) behavior and the global behavior (choreography). Wieczorek et al. [2009] apply model checking to generate modelbased integration tests for choreography models. Mei et al. [2009a] propose some test adequacy criteria for service choreography specified in WS-CDL that manipulates XPath queries. We identified only one study that handles composition in a generic way, i.e., there is no distinction between orchestration and choreography. Ruth et al. [2007] propose an approach for safe regression testing through the usage of control flow graphs to identify

the changes. There is a large difference between the number of studies based on orchestration (14 studies) and on choreography (three studies). The composition languages cited in the analyzed work are BPEL (orchestration) and WS-CDL (choreography). A high number of studies in orchestration testing can be consequence of the maturity level of the composition languages. BPEL has been accepted as a standard language for orchestration and business process. In contrast, there is no consensus on a choreography language. Currently, WS-CDL is the most cited one. Only the work of Ruth et al. [2007] addresses the service composition in a generic way. 3.2. Models In this section, we discuss the main models used to support the formal testing of web services. Figure 1 presents different colors for each group of models. State-based Models: In this group, the studies use state-based models to support the test activity, ranging from simple to complex models. These models are Finite State Machine (FSM) and extensions [Benharref et al., 2006; Keum et al., 2006], Labeled Transition System (LTS) and extensions [Bentakouk et al., 2009; Bertolino et al., 2008, 2004; Frantzen et al., 2008; Mei et al., 2009a; Wieczorek et al., 2009], Stream X-Machine (SXM) [Dranidis et al., 2007; Kourtesis et al., 2010; Ramollari et al., 2009], finite state automaton [Baldoni et al., 2005], timed automaton [Lallali et al., 2008], and Web Service Automaton (WSA) [Zheng et al., 2007]. The models are used to describe the control flow, data flow, and timing properties. While most of the studies reuse or adapt established models, Zheng et al. [2007] propose the WSA to represent the semantic operation of the BPEL language. Control Flow Graph and Extensions: Control Flow Graph (CFG) and wellknown extensions like Def-Use Graph [Rapps and Weyuker, 1985] have constantly been used for supporting structural testing. In the selected set, CFGs and extensions are used mainly to test BPEL-based service composition [Endo et al., 2008; Karam et al., 2007; Li et al., 2008b; Liu et al., 2008; Mei et al., 2008, 2009b; Yan et al., 2006]. However, some studies do not use CFGs to support structural testing. In [Li et al., 2008a], CFGs model the sequences of operations for testing stateful services. Ruth et al. [2007] and Mei et al. [2009c] apply CFGs and coverage criteria to support regression testing. Graph Transformation Rules: Graph Transformation (GT) rules are used to augment the service specification with pre-conditions, effects, and a notion of states. A GT rule refines the service operation, adding information about the parameters and internal data. The service state can be recorded as a graph attribute, though GT rules is not a statebased model. As a consequence, GT rules based approaches are adequate for stateless and stateful services, supporting discovery, monitoring, automatic testing, and regression testing [Heckel and Mariani, 2005; Lohmann et al., 2007; Park et al., 2009]. Swiss Cheese Model: Swiss cheese model is the basis for the work of Tsai et al. [2005a,b,c]. It is a model based on boolean expressions, extracted from semantic specifications (OWL-S), that are represented by Karnaugh maps and finally a Swiss Cheese map. The model is used to generate positive and negative test cases. Other Models: We identified seven primary studies which use models that cannot

be classified into the previous groups. In this context, we found metamorphic relations [Chan et al., 2007], input element type model [Ma et al., 2008], Event Sequence Graph [Belli and Linschulte, 2008], IOPE [Paradkar et al., 2007], BIR model [Corradini et al., 2008], POST Class graph [Chakrabarti and Rodriquez, 2010], and Message Sequence Graphs [Hou et al., 2009]. Remarks: Each primary study addresses the model construction in a specific form. There are two common approaches: (1) to suppose that there exists a formal model and (2) to generate a model based on service specifications. The first approach is commoner in single service testing due to the fact that only syntactic specifications (WSDL, XML Schema) are available. The second approach is most used in composition testing, since supposing that there are BPEL or WS-CDL specifications in this context is reasonable. Semantic specifications are also considered in this way, though they are not frequently available. 3.3. Authorship In Figure 1, dotted edges connect studies that have at least one author in common. Based on this information, we identified two main research groups working on formal testing for web services. The first group (five studies) works with service composition testing (orchestration and choreography). The second group (four studies) concentrates their effort on testing of stateful services. We also note other small groups (three or two studies) and a large number of isolated studies. Note in Figure 1 that there is no connection among the groups which implies that no cooperation have been carried out in this topic. This fact can justify the lack of approaches or strategies that handle testing of both single services and compositions. 3.4. References Connection In Figure 1, the references were represented as directed edges. An directed edge from node A to node B means that the primary study A makes a reference to the study B. The two most-referenced studies are coincidentally one in single service testing (Heckel and Mariani [2005] - nine references) and one in service composition testing (Yan et al. [2006] - six references). Another important fact is the low connection among the studies. The primary study that makes more references to other studies is connected with five studies (Kourtesis et al. [2010]). Moreover, there are many primary studies with zero or one connection. The low reference connection suggests the presence of problems to find related studies and the lack of secondary studies that summarize the topic.

4. Conclusion and Future Work In this paper, we presented a systematic review on formal testing for SOAs and web services. A set of 37 primary studies was selected based on searches in the main repositories for the computer science area. We noted an increasing interest in formal testing for web services. The selected studies focus on testing aspects for single services and service compositions. We grouped and discussed about the formal models used to support the tests. The authorship and references connection related to the studies were also discussed. We conclude that there is a considerable amount of research on formal testing for web services, though some points deserve more attention. Based on the analysis of

authorship and references, we noted a low level of interaction among the research groups. We identified many types of adopted models, providing a good range of options for the service developers. However, this range of options cannot be found in choreography testing with only three selected studies. As future work, we plan to replicate the systematic review to update the selected work, evolving the planning and analysis and reducing the threats to validity. We also intend to use this systematic review and published surveys as a starting point to produce a complete systematic mapping of the service testing area.

Acknowledgments The authors are thankful to Fabiano C. Ferrari, Marcelo M. Eler, Marco A. G. Silva, and the anonymous reviewers for providing useful suggestions on this paper. This work was financially supported by FAPESP/Brazil (grant 2009/01486-9) and CNPq/Brazil (grant 305236/2009-1).

References Baldoni, M., Baroglio, C., Martelli, A., Patti, V., and Schifanella, C. (2005). Verifying the conformance of web services to global interaction protocols: A first step. In International Workshop on Web Services and Formal Methods (WS-FM), pages 257–271. Springer. Belli, F. and Linschulte, M. (2008). Event-driven modeling and testing of web services. In IEEE International Computer Software and Applications Conference (COMPSAC), pages 1168– 1173. Benharref, A., Dssouli, R., Glitho, R., and Serhani, M. A. (2006). Towards the testing of composed web services in 3rd generation networks. In IFIP International Conference on Testing of Communicating Systems (TESTCOM), pages 118–133. Bentakouk, L., Poizat, P., and Za¨ıdi, F. (2009). A formal framework for service orchestration testing based on symbolic transition systems. In International Conference on Testing of Software and Communication Systems (TESTCOM), pages 16–32, Berlin, Heidelberg. Springer-Verlag. Bertolino, A., Angelis, G. D., Frantzen, L., and Polini, A. (2008). Model-based generation of testbeds for web services. In IFIP International Conference on Testing of Communicating Systems (TESTCOM), pages 266–282. Bertolino, A., Frantzen, L., Polini, A., and Tretmans, J. (2004). Audition of web services for testing conformance to open specified protocols. In Architecting Systems with Trustworthy Components International Seminar, pages 1–25. Biolchini, J., Mian, P. G., Natali, A. C. C., and Travassos, G. H. (2005). Systematic review in software engineering. Technical report, Systems Engineering and Computer Science Department COPPE/UFRJ. Bozkurt, M., Harman, M., and Hassoun, Y. (2010). Testing web services: A survey. Technical Report TR-10-01, Department of Computer Science, King’s College London. Canfora, G. and Di Penta, M. (2009). Service-oriented architectures testing: A survey. In Software Engineering: International Summer Schools (ISSSE), pages 78–105. Chakrabarti, S. K. and Rodriquez, R. (2010). Connectedness testing of restful web-services. In India software engineering conference (ISEC), pages 143–152, New York, NY, USA. ACM. Chan, W., Cheung, S., and Leung, K. (2007). A metamorphic testing approach for online testing of service-oriented software applications. International Journal of Web Service Research, 4(2):61– 81.

Corradini, F., Angelis, F., Polini, A., and Polzonetti, A. (2008). Improving trust in composite eservices via run-time participants testing. In International conference on Electronic Government (EGOV), pages 279–290, Berlin, Heidelberg. Springer-Verlag. Dranidis, D., Kourtesis, D., and Ramollari, E. (2007). Formal verification of web service behavioural conformance through testing. Annals of Mathematics, Computing & Teleinformatics, 1(5):36–43. Endo, A. T., Sim˜ao, A. S., Souza, S. R. S., and Souza, P. S. L. (2008). Web services composition testing: a strategy based on structural testing of parallel programs. In Testing: Academic and Industrial Conference Practice and Research Techniques (TAIC PART), pages 3–12. Endo, A. T. and Sim˜ao, A. S. (2010). Formal testing approaches for service-oriented architectures and web services: a systematic review. Technical Report 348, Universidade de S˜ao Paulo (USP), S˜ao Carlos, SP, Brazil. Frantzen, L., Las Nieves Huerta, M., Kiss, Z. G., and Wallet, T. (2008). On-the-fly model-based testing of web services with jambition. In International Workshop on Web Services and Formal Methods (WS-FM), pages 143–157, Berlin, Heidelberg. Springer-Verlag. Heckel, R. and Mariani, L. (2005). Automatic conformance testing of web services. In International Conference on Fundamental Approaches to Software Engineering (FASE), Lecture Notes in Computer Science, pages 34–48. Hierons, R. M., Bogdanov, K., Bowen, J. P., Cleaveland, R., Derrick, J., Dick, J., Gheorghe, M., Harman, M., Kapoor, K., Krause, P., L¨uttgen, G., Simons, A. J. H., Vilkomir, S., Woodward, M. R., and Zedan, H. (2009). Using formal specifications to support testing. ACM Computing Surveys (CSUR), 41(2):1–76. Hou, S.-S., Zhang, L., Lan, Q., Mei, H., and Sun, J.-S. (2009). Generating effective test sequences for BPEL testing. In International Conference on Quality Software (QSIC), pages 331–340. Karam, M., Safa, H., and Artail, H. (2007). An abstract workflow-based framework for testing composed web services. In IEEE/ACS International Conference on Computer Systems and Applications (AICCSA), pages 901–908. Keum, C., Kang, S., Ko, I.-Y., Baik, J., and Choi, Y.-I. (2006). Generating test cases for web services using extended finite state machine. In IFIP International Conference on Testing of Communicating Systems (TESTCOM), pages 103–117. Kitchenham, B. and Charters, S. (2007). Guidelines for performing systematic literature reviews in software engineering. Technical Report EBSE 2007-001, Keele University and Durham University Joint Report. Kourtesis, D., Ramollari, E., Dranidis, D., and Paraskakis, I. (2010). Increased reliability in soa environments through registry-based conformance testing of web services. Production Planning & Control: The Management of Operations, 21(2):130–144. Lallali, M., Zaidi, F., Cavalli, A., and Hwang, I. (2008). Automatic timed test case generation for web services composition. In European Conference on Web Services (ECOWS), pages 53–62. Li, L., Chou, W., and Guo, W. (2008a). Control flow analysis and coverage driven testing for web services. In IEEE International Conference on Web Services (ICWS), pages 473–480. Li, Z. J., Tan, H. F., Liu, H. H., Zhu, J., and Mitsumori, N. M. (2008b). Business-process-driven gray-box soa testing. IBM Systems Journal, 47(3):457–472. Liu, C.-H., Chen, S.-L., and Li, X.-Y. (2008). A ws-bpel based structural testing approach for web service compositions. In IEEE International Symposium on Service-Oriented System Engineering (SOSE), pages 135–141. Lohmann, M., Mariani, L., and Heckel, R. (2007). A model-driven approach to discovery, testing and monitoring of web services. In Test and Analysis of Web Services, pages 173–204. Ma, C., Du, C., Zhang, T., Hu, F., and Cai, X. (2008). WSDL-based automated test data generation for web service. In International Conference on Computer Science and Software Engineering (CSSE), volume 2, pages 731–737.

Mei, L., Chan, W., and Tse, T. (2008). Data flow testing of service-oriented workflow applications. In International Conference on Software Engineering (ICSE), pages 371–380. Mei, L., Chan, W. K., and Tse, T. H. (2009a). Data flow testing of service choreography. In Symposium on the Foundations of Software Engineering (FSE), pages 151–160. Mei, L., Chan, W. K., Tse, T. H., and Kuo, F.-C. (2009b). An empirical study of the use of franklweyuker data flow testing criteria to test bpel web services. In IEEE International Computer Software and Applications Conference (COMPSAC), volume 1, pages 81–88. Mei, L., Zhang, Z., Chan, W. K., and Tse, T. H. (2009c). Test case prioritization for regression testing of service-oriented business applications. In International Conference on World Wide Web (WWW), pages 901–910, New York, NY, USA. ACM. OASIS (2007). Web services business process execution language (WSBPEL) v2.0. Papazoglou, M. P. and Heuvel, W.-J. (2007). Service oriented architectures: approaches, technologies and research issues. The International Journal on Very Large Databases (VLDB), 16(3):389–415. Paradkar, A., Sinha, A., Williams, C., Johnson, R., Outterson, S., Shriver, C., and Liang, C. (2007). Automated functional conformance test generation for semantic web services. In IEEE International Conference on Web Services (ICWS), pages 110–117. Park, Y., Jung, W., Lee, B., and Wu, C. (2009). Automatic discovery of web services based on dynamic black-box testing. In IEEE International Computer Software and Applications Conference (COMPSAC), volume 1, pages 107–114. Ramollari, E., Kourtesis, D., Dranidis, D., and Simons, A. (2009). Leveraging semantic web service descriptions for validation by automated functional testing. In European Semantic Web Conference (ESWC), pages 593–607. Rapps, S. and Weyuker, E. J. (1985). Selecting software test data using data flow information. IEEE Transactions on Software Engineering, 11(4):367–375. Ruth, M., Oh, S., Loup, A., Horton, B., Gallet, O., Mata, M., and Tu, S. (2007). Towards automatic regression test selection for web services. In IEEE International Computer Software and Applications Conference (COMPSAC), volume 2, pages 729–736. Tsai, W., Wei, X., Chen, Y., and Paul, R. (2005a). A robust testing framework for verifying web services by completeness and consistency analysis. In IEEE International Symposium on Service-Oriented System Engineering (SOSE), pages 151–158. Tsai, W., Wei, X., Chen, Y., Xiao, B., Paul, R., and Huang, H. (2005b). Developing and assuring trustworthy web services. In International Symposium on Autonomous Decentralized Systems (ISADS), pages 43–50. Tsai, W.-T., Wei, X., Chen, Y., Paul, R., and Xiao, B. (2005c). Swiss cheese test case generation for web services testing. Transactions on Information and Systems, E88-D(12):2691–2698. W3C (2004). OWL-S: Semantic markup for web services. W3C (2005). Web services choreography description language version (WS-CDL) 1.0. Wieczorek, S., Kozyura, V., Roth, A., Leuschel, M., Bendisposto, J., Plagge, D., and Schieferdecker, I. (2009). Applying model checking to generate model-based integration tests from choreography models. In International Conference on Testing of Software and Communication Systems (TESTCOM), pages 179–194, Berlin, Heidelberg. Springer-Verlag. Yan, J., Li, Z., Yuan, Y., Sun, W., and Zhang, J. (2006). BPEL4WS unit testing: Test case generation using a concurrent path analysis approach. In International Symposium on Software Reliability Engineering (ISSRE), pages 75–84. Zheng, Y., Zhou, J., and Krause, P. (2007). An automatic test case generation framework for web services. Journal of Software, 2(3):64–77.