Improving Code Recommender Systems using Boolean Factor Analysis and Graphical Models Marcel Bruch

Mira Mezini

Software Technology Group Technische Universität Darmstadt

Software Technology Group Technische Universität Darmstadt

[email protected] 1.

INTRODUCTION

The demands of modern software engineering to develop highquality software in the least possible amount of time has become a crucial challenge in software engineering - and reuse of code has become an effective method to satisfy these demands. One approach to code reuse are application frameworks which provide a generic infrastructure that can be reused and customized by developers to build new applications with less efforts. However, before using a framework efficiently a developer has to learn the underlying framework concepts and (framework) documentation becomes a vital source of information. Unfortunately, the available documentation frequently does not provide all required information and other resources like mailing lists, code snippets or source code of already existing (and working) framework instantiations are used to learn possible usages of the framework [2]. Motivated by manual code reading techniques several research tools have been developed each aiming to support novice developers on different framework understanding problems. Closely related to our proposal are code recommender systems like FrUiT [3], CodeWeb [5], and Mapo [6]. These tools examine code of existing framework instantiations to find framework elements that co-occur frequently together in code. From these observations FrUiT and CodeWeb derive API usage rules like “In 95% of the cases where you extend class Wizard, you also override its method Wizard.addPage.” or “In 80% of the cases you called Button.1 , you also call Button.setText”. Mapo instead uses the information about frequently co-occurring method calls to infer typical method call sequences that might help developers to understand how to use particular framework objects correctly. Although these tools have proven to find valuable patterns in code we argue that many of these rules and sequences are still hard to comprehend and mislead novice developers. In this position paper we propose a novel way of mining and modeling relations between framework elements (basically object usage patterns consisting of sets of method invocations) based on Boolean factor analysis [1], a data mining technique that examines the observed 1

represents construtor calls in Java bytecode.

[email protected]

data to find latent (i.e., unobservable) factors that explain parts of the observations and can be combined to form new compounded patterns. In this context, a factor should be considered to describe a group of methods frequently called together to accomplish some specific behavior - for instance object creation patterns etc. In addition to pure factor identification, we transform the obtained factors into a graphical model (namely a bayesian network) that expresses possible factor combinations and the likelihood of their appearance. Since pure statistical constraints are not suitable to eliminate incomprehensible and misleading factors combinations (cf. Sec. 2) we propose to further refine the resulting graphical model by taking into account specific properties of the programming language like control- and data-flow information to further reduce the possible factor combinations to a meaningful and comprehensible set. Note, that our approach is complementary to existing approaches and is not considered to supersede existing mining approaches like the ones of Mapo or FrUiT. In fact, we reuse parts of FrUiT’s mining capabilities to extract frequent relations between framework elements and to identify code constrains caused by the programming language. We also consider Mapo to be a powerful extension to our flexible factor-based model that could be build on top of our approach.

2.

ISSUES WITH EXISTING APPROACHES

Rules don’t express complex call patterns. When designing an API, framework developers typically have some expectations how the API should be used by application developers. Typically these expected usages comprise groups of methods that should be invoked together as well as several constraints when to apply them. Application developers need a good understanding of these expectations because unless their API usage follows these expectations their code is unlikely to function properly. Rules of the form call:A.m→call:A.n try to catch these expectations and make them intelligible for novice developers. However, we argue that rules are not appropriate to express groups of method calls and their constraints. For instance, a typical object initialization of an SWT Button consists of up to six different method calls - some are mandatory, others are optional - and expressing one such pattern typically results in dozens of rules (potentially the cross product of each method call). We argue that such rule-sets do not allow novice developers to grasp the underlying usage pattern (i.e., the group of methods to invoke) and also do not allow a developer to understand which method calls are mandatory and which are not. Furthermore, we found that some methods can be called in completely different contexts, thus, belong to different and sometimes mutually exclusive usage patterns. Hence, current

pattern-unaware tools mix several mutually exclusive patterns and recommend invalid method combinations that result in erroneous code. Instead of searching for relations between single method calls, we propose to identify cohesive groups of strongly correlated method calls (patterns or factors) and recommend method calls based on these patterns only. Furthermore, we propose that code recommender tools (when used interactively during development) should recommend only those patterns that are applicable in a given (method) context and should narrow down the set of possible patterns as development progresses.

Incomprehensible patterns. When mining for API usage rules, typically no constraints are made which framework elements can be used to form the antecedent and consequent of an association rule.2 The rationale for this is, that every rule which satisfies minimum support, confidence, and significance thresholds, is considered to represent a significant and meaningful knowledge that might be helpful for inexperienced developers. However, based on our experience with FrUiT we found that even if strict thresholds are used, still a large number of incomprehensible rules were found which do not provide any useful information for novice developers. Consider for example the rule “If you call Composite.setLayoutData you also call Label.setText” found by FrUiT. Clearly, no direct relation between a Composite’s method setLayoutData and a Label’s method setText can be found. However, since these two elements occurred frequently together in code this rule passed all significance thresholds and thus has been considered to be interesting by the system. So far, existing tools tried to incorporate as little as possible prior knowledge about code when searching for framework concepts. But as illustrated above, statistical tests are not satisfactory to eliminate misleading patterns. However, we argue that taking into account data-flow and control-flow properties of the code and explicitly modeling this knowledge in the recommendation model helps to eliminate misleading patterns and leads to more comprehensible models with a small loss of accuracy.

Why sequences standalone are not enough. At a first glance, sequence mining seems to be a natural way to find and express method call patterns. However, due to our experiments with FrUiT-SEQ, a sequence mining version of FrUiT, we found that using sequence mining on code suffers from the problem of proliferation of sequences. To be effective, sequence mining requires ordering constraints between method calls to exist. We found that only few such ordering constraints in code exist - mostly in usage patterns that involve interactions between several different objects but rarely in (sub-)patterns that involve single objects only. Consequently, since no clear ordering between method calls exists a large number of different sequences are found each expressing the same pattern. Furthermore, sequences try to cover compounded patterns completely which complicates to identify optional extensions or combinations of basic patterns. However, we believe that expressing call patterns as sequences in general is a great opportunity to support novice developers on learning complex call patterns. But instead of trying to find sequences for compounded usage patterns we propose to find one characteristic sequence for each subpattern (or factor) and some projection rules how sequences of several factors can be combined 2 We assume that self-containing rules like “If you extend class A you also extend superclass(A)” and similar ones are filtered.

Button Usage Patterns none query_state create

call:Button.(Comp...

0% 0% 100%

call:Button.setText(Stri...

call:Button.setLayoutD...

True 100%

True 99%

True 60%

False

False 1%

False 40%

0%

call:Button.setFont(Font)

call:Button.addSelectio...

call:Button.getSelectio...

True 24%

True 69%

True

False 76%

False 31%

False 100%

0%

Figure 1: Sample Usage Patterns for SWT Button

to create correct compounded sequences for compounded usage patterns.

3.

PROPOSED SOLUTION

In the following we give (i) a brief introduction to binary factor analysis, (ii) briefly describe how we currently identify factors (or patterns) from data, and (iii) how we use these factors to create context-specific, tailored graphical models for recommending usage patterns and method calls.

Brief introduction to binary factor analysis. In literature, the known (binary) data is often thought of as a Boolean matrix Y ∈ Bn×m where Y [i, j] expresses whether or not attribute j occurred in observation i. The goal of binary factor analysis is to find Boolean matrices F ∈ Bn×d and M ∈ Bd×m such that X = F ⊕M is close to Y . The sign ⊕ marks disjunction, i.e. logical sum. In other words, the goal of binary factor analysis is to find a new set of d (latent) factors and their corresponding factor loadings (stored in M ) that explain large (not all) parts of the observed data [1]. Several approaches to identify Boolean factors exist ranging from brute force methods over binary clustering approaches up to the use of closed itemsets as candidates for binary factors. For an overview of existing approaches we refer to [1]. In this position paper we present our preliminary approach based on a bayesian clustering approach for factor identification.

Identifying object usage patterns and their relations. Throughout this paper, we consider the term usage pattern to be equivalent to the term factor. Each factor has a factor loading that states which methods belong to this factor and gives the likelihood that this factor causes the invocation of these methods. Note, that we not assume that the likelihood of each factor needs to be either 0 or 1. We accept any arbitrary probability value which allows us to significantly decrease the number of factors in our model. The mining task itself is straight-forward. For each framework type we extract all its usages from the code samples and given them to the clustering algorithm. For clustering we use AutoClass [4], a bayesian clustering approach that allows (i) a fuzzy clustering of binary attributes and observations (similar to the ExpectationMaximization algorithm) (ii) to incorporate mining constraints like mutually exclusive constructor calls etc., and (iii) returns a set of factors and their corresponding factor loadings. Based on these factors, for each framework type a graphical model is created. Fig. 1 depicts such a (simplified) graphical model for the SWT

Method Context IDialogPage_createControl 100% others

Composite "parent"

0%

Composite "container"

Use Button

Use Text

Label "label"

True 100%

True 100%

True 40%

True 24%

True 100%

False

False

False 60%

False 76%

False

0%

none

0%

0%

0%

Patterns none

call:Composite.(...

Patterns

0%

create 100%

create 97%

other

other

0%

call:Composite.setLay... True 95%

True

False

False 5%

False 95%

0%

call:Composite.setLay...

call:Label.(Comp...

call:Composite.getFont()

True 100%

3%

5%

call:Composite.setFont...

call:Label.setLayoutDa...

True 100%

True 52%

False

False 48%

0%

call:Label.setText(Stri...

call:Label.setFont(Font)

True 65%

True 20%

True 86%

True 22%

False 35%

False 80%

False 14%

False 78%

Figure 2: Proposed Recommendation Network for DemoDialogPage.createControl class Button. The graphical model contains a pattern node (the uppermost node in Fig. 1) and one node for each method that belongs to at least one usage pattern (all nodes whose names start with “call:Button.”). In this example two usage patterns have been found and the factor loadings for the “create”-pattern are depicted. One can see that calling and setText are mandatory for the “create”-pattern whereas the other method calls are optional but likely. More sophisticated models that model factors as individual nodes and use an additional node to express factor combinations are in consideration but are part of the near future work. Further, for each observed framework usage its used factors and the framework-defined method this usage has been observed in (if one) is stored. This information is used later during the recommendation process to recommend only those patterns that are applicable in the given framework-context (if available) and allows to recommend other usages of framework elements based on this context as we will see in the example scenario presented in the next paragraph.

How inference of code recommendations works. To illustrate how the code recommendation process works consider the code fragment depicted in Listing 1. Assume that a novice developer has started to implement the framework method IDialogPage.createControl, already created a new instance of Composite and Label and now queries the recommender system for new proposals. 1 2 3 4 5 6 7 8 9

public class DemoDialogPage implements IDialogPage { ... @Override protected Control createControl(Composite parent){ Composite container = new Container(...); Label label = new Label(...); } }

Listing 1: Example Framework Extension Fig. 2 shows the graphical model the recommender engine builds from the information available in the currently edited method. The root of the model consists of a method-context node. The second level contains two different types of nodes: One node for each existing variable (parent, container, and label) and one placeholder for each framework type that have been observed fre-

quently within this framework-method and thus might be interesting for a developer (Text and Button). Each placeholder node expresses the likelihood that the developer will use such an instance but does not provide further recommendations until the developer decided to create a variable declaration of the corresponding type. For the variable nodes container and label the usage pattern model is inserted into the model. One can see that for both variables the “create”-pattern is selected (bold and underlined states in the node) due to the observed constructor calls. The relevant recommendations can be simply read from this model without further modifications or presented to the developer for example as tool tips, quick fixes or simply in tabular form.

4.

FUTURE WORK

An IDE integration of this approach is work in progress; Alternatives to mine and model relations between binary factors will be examined.

5.

REFERENCES

[1] R. Belohlavek and V. Vychodil. Formal concepts as optimal factors in boolean factor analysis: implications and experiments. In Fifth International Conference on Concept Lattices and Their Applications, 2007. [2] J. Bosch, P. Molin, M. Mattsson, and P. Bengtsson. Object-oriented framework-based software development: Problems and experiences. ACM Computing Surveys, 32(1):3–8, 2000. [3] M. Bruch, T. Schäfer, and M. Mezini. FrUiT: IDE support for framework understanding. In OOPSLA Workshop Eclipse Technology Exchange, pages 55–59. ACM, 2006. [4] R. Hanson, J. Stutz, and P. Cheeseman. Bayesian classification with correlation and inheritance. In In Proceedings of the 12th International Joint Conference on Artificial Intelligence, pages 692–698. Morgan Kaufmann, 1991. [5] A. Michail. Data mining library reuse patterns using generalized association rules. In ICSE, pages 167–176. ACM, 2000. [6] T. Xie and J. Pei. MAPO: mining api usages from open source repositories. In MSR, pages 54–57. ACM, 2006.