Introducing a Test Suite Similarity Metric for Event Sequence-Based Test Cases
Written by Penelope A. Brooks and Atif M. Memon Presented by Il-Chul Yoon
Motivation
2 < 2>
GUI is as much as 60% of overall code GUIs are one type of Event-Driven Systems (EDS) State-based Large number of event combinations
3
T ti Testing Radio R di Button B tt GUI Consider context Square: Circle:
4
Existing g Metrics Test Suite Reduction
5
Previous Work in Similarity y In Natural Language Processing Determining g document similarity y [Aslam [ and Frost]]
In Software Engineering Detecting test case redundancy [Li et al.] Detecting code changes [Antonio et al., Karniket al.]
6
Contributions The first context-based similarity metric for sequenceq based test cases Application of the metric to reduced test suites Empirical E i i l study t d d demonstrating t ti effectiveness ff ti
7
Measuring Test Suite Similarity Parameterized metric CONTeSSi(n) (CONtext Test Suite Similarity) CONTeSSi is a method of counting how many times each event appears in the test suite The parameter, n, allows the metric to consider event sequences rather than single events Value of CONTeSSi is between 0 and 1
8
Computing Similarity Count frequency of each event
9
Computing Similarity (2) Count frequency of event pairs (n (n=1) 1)
10
Computing Similarity (3) Count frequency of event triples (n (n=2) 2)
11
Computing CONTeSSi
where A and B are the vectors corresponding to the two test suites, A · B is the dot product of the two vectors, i.e., , where j is the number of terms in the vector, and
12
Computing Similarity (4) Compute CONTeSSi(n)
13
Empirical Study Goal: Evaluate a metric that measures the similarity of two test suites and to determine the quality of this metric
14
Research Questions RQ1:Is CONTeSSi(n) a better indicator of similarity for larger g values of n? Metrics: CONTeSSi(n) value for n = 0..5
RQ2:Does CONTeSSi(n) agree with existing metrics in determining the similarity between suites, specifically relating to fault detection effectiveness? Metrics: Line coverage, Method coverage, Pair coverage
15
Subject Applications CrosswordSage C dS 035 0.3.5 Creates and solves professional-looking crossword puzzles All-time All time activity: 76.87% 76 87%
FreeMind 0.8.0 Popular p mind-mapping pp g application pp All-time activity: 99.96%
GanttProject 2.0.1 P Project j t scheduling h d li application li ti All-time activity: 99.85%
Test Suites Original suite Model Model-generated generated test cases (Torig i ) 58k for FreeMind Reduced suites Line coverage (Tline) Method coverage (Tmethod) Event pair coverage (Tpair)
Procedure
17
Fault Matrices Parameterized Test Case Generator
Test Case Executor
T_orig
Coverage Matrices
TS3 TS2 TS1
Method-Reduced
TS3 TS2 TS1
TS3 TS2 TS1
Line-Reduced
Event Pair-Reduced
Compute CONTeSSi(n)
18
Results RQ1: CONTeSSi Metric
Results – RQ1: Code Coverage
19
20
RQ1 Faults -- CrosswordSage g
21
RQ1 Faults -- FreeMind
22
RQ1 Faults -- GanttProject j
23
RQ1 Faults -- JMSN
24
Comparing p g Test Suites
where T is the suite being compared to the original suite m is i one off th the existing i ti metrics t i on which hi h suites it are compared, d such as line, branch or event pair coverage, em(suite) is a function returning the set of elements for metric m covered by suite, and N is a function returning the number in the set given
25
Results – RQ2: Computing f(Torig, T, m)
26
Conclusions CONTeSSi(n) is the first context-based similarity metric CONTeSSi(n) is a better indicator of similarity between suites than existing, code-based metrics
27
Future Research Extend empirical study to include more subject applications Further investigate the relationship b t between ttestt case length l th and d choosing h i a value of n
Introducing a Test Suite Similarity Metric for Event ...
test suite. â« The parameter, n, allows the metric to consider event sequences ... Test Case. Executor. Matrices. Coverage. Matrices. TS3. TS2. TS1. TS3. TS2. TS1.
The quality of this matching would be im- proved if it also relied on knowledge of structural semantics. This knowledge would be used to help capture and represent more precisely the meaning and information that are actually conveyed by the texts and
Aug 14, 2013 - call each Wi = ãIi, Oiã a workset of D. Ii, Oi are the inner set and outer set of Wi ..... Figure 4 illustrates the inefficiency by showing a 4-way partitioned dataset ...... In WSDM Conference, pages 203â212, 2013. [2] D. A. Arb
With G Suite, you can easily schedule a video-conference meeting in Google Calendar and invite anyone, even if they don't have a Google account. Create a Hangouts Meet event from your mobile device the same way. When you create a meeting in Calendar
Resumen: En este artıculo se presenta ParTes, el primer test suite en espaËnol y catalán para la evaluación cualitativa de analizadores sintácticos automáticos. Este recurso es una jerarquıa de los fenómenos representativos acerca de la estru
Abstract: Software Testing is an important activity in Software Development Life Cycle. (SDLC). Software testing is a process of executing a program or application with the intent of finding the bugs. Testing is expensive and prone to mistakes and ov
Development of an indigenous Computer Package .... A concept of degrees of freedom is introduced in these tests in the form of blocks or classes. For such ...
best solution for developers with regard to COLLADA. The WG agreed ... COLLADA Conformance Test Suite (CTS) should support OpenCOLLADA plugins for. MAX and ... MAYA plugins based on Feeling software FCOLLADA API. The core ...
Mar 27, 2016 - redirects stdout to a string buffer. After. sizeOfBuffer bytes printed, the test is aborted with a FATAL failure. int _printFault(const char* limit,.
Other company and product names may be trademarks of the respective ... MAYA plugins based on Feeling software FCOLLADA API. .... Budget: $15K (US) ... framework, and of any test development you have previously performed in this.
knowledge. It seems to be a simple fact that Kristin and I disagree over when .... vocal critic of notions of concept similarity, it seems only fair to give his theory an.
Enterprises integration has recently gained great attentions, as never before. The paper deals with an essential activity enabling seam- less enterprises integration, that is, a similarity-based schema matching. To this end, we present a supervised a
popular business applications like Adobe Acrobat, Salesforce, IBM, Slack and more. âBox and G Suite together allow us to leverage Docs for working files, like.
http://citeseer.ist.psu.edu/mostcited.html ... Using D Ã D matrix A = {ai,j}, we ... ai,j(x m i â xn i. )(x m j â xn j. ) â. â . 1. 2 . The necessary and sufficient condition for ...
13 Aug 2012 - i,j ı{rij. 2 + sij ⤠θij} â â i,j ı{pi â pj â rij = 0} â. â i,j ı{sij â qi = 0}... . (3). We will derive the primal of this expression using the following variant of Fenchel duality, min x f(Ax) = max y:AT y=0. âfâ
free energy of this inference process, i.e., the discrepancy between .... such that their encoding bit rates are all higher than 100 ..... approach in the DCT domain.
interval type-2 fuzzy sets (IT2 FSs), the CWW engine's output can also be an IT2 FS, eA, which .... similarity, inclusion, proximity, and the degree of matching.''.
soil surveys are a major source of soil spatial information. There are ...... paid to sample the small draws when the 64 sites were distributed over the study area.
Dec 1, 2012 - DROID 6.1 was used alongside Signature File v65. As of 25 October 2012, ... fmt/436: Digital Negative Format (DNG) 1.0. Gave explicit .... forward adding manually created skeleton-files for each new DROID signature created.
