Dynamic Resource Allocation in Hybrid Optical-Electrical Datacenter Networks Dinil Mon Divakaran∗, Soumya Hegde, Raksha Srinivas, Mohan Gurusamy Department of Electrical and Computer Engineering, National University of Singapore, Singapore 117583

Abstract A promising development in the design of datacenters is the hybrid network architecture consisting of both optical and electrical elements, in which end-toend traffic can be routed through either an electrical path or an optical path. The core optical switch is used to dynamically create optical paths between pairs of electrical edge-switches in such a datacenter network. In this context, the joint problem of bandwidth allocation and VM-placement poses new and different challenges not addressed yet in hybrid datacenter. In particular, we foresee two issues: (i) the number of edge-switches that can be simultaneously reached using optical paths from an edge-switch is limited by the size of the optical switch, (ii) the dynamic creation of optical paths can potentially establish a constrained optical network topology leading to poor performance. In this work, we abstract the requests of tenants as virtual networks, and study the problem of embedding virtual networks on a hybrid datacenter. We formulate the problem as a nonlinear optimization problem and analyze its complexity. We develop and analyse three algorithms for embedding dynamically arriving virtual network demands on a hybrid optical-electrical datacenter. Through simulations, we demonstrate the effectiveness of not only exploiting the already established optical paths, but also of using electrical network in embedding requests of virtual networks. Keywords: datacenter, bandwidth, optical, embedding, virtual network I This

article is an extended version of the paper published in IEEE ICCCN 2014 [1]. author Email addresses: [email protected] (Dinil Mon Divakaran), [email protected] (Soumya Hegde), [email protected] (Raksha Srinivas), [email protected] (Mohan Gurusamy) ∗ Corresponding

Preprint submitted to Elsevier

June 14, 2015

1. Introduction Datacenters today are hosting increasing number of services and applications, which in turn generate tremendous amounts of traffic. The global datacenter traffic growth rate is estimated to be approximately 25% per year till 2017; and the annual global datacenter IP traffic is estimated to reach 7.7 zettabytes (1021 bytes) by the end of 2017 [2]. The annual traffic growth rate between datacenters as well as within datacenters are also predicted to be ≈ 30%. To meet such traffic growth trends, optical switching based on WDM (wavelength division multiplexing) technology has recently been proposed as a promising approach to connect a datacenter [3, 4, 5]. Optical networks provide not only huge bandwidth, but also reduce the cabling complexity and power consumption in comparison to electrical networks. An important feature of optical networks is its ability to dynamically reconfigure optical paths between any pair of (electrical) switches connected using an optical switch. We can leverage on this capability to solve one important challenge in datacenters—VM (virtual machine) placement problem. As optical paths between edge-switches (top-of-rack switches, to which server machines are connected) can be created on-demand, there is more flexibility in placing VMs of a request, than in an electrical datacenter network. However, building an alloptical datacenter that provides simultaneous connectivity between every pair of edge-switches is expensive and impractical for large datacenters hosting tens of thousands of servers. This, along with the fact that electrical network is better suited for multiplexing short and bursty traffic, makes a hybrid optical-electrical network architecture the right choice for future datacenters [3, 4, 6]. A hybrid datacenter gives flexibility in connecting edge-switches with high communication demands dynamically using optical network, while maintaining connections between edge-switches with bursty traffic using electrical network. Fig. 1 illustrates a hybrid optical-electrical datacenter network (similar to Helios [3]) considered in this work. In the hybrid architecture, both electrical and optical networks coexist; traffic from an edge-switch to another can be routed either through the electrical switch(es) or the optical switch. The

2

Core

Edge 11 00 00 11 00 11 00 11 00 11 00 11

11 00 00 11 00 11 00 11 00 11 00 11

Server Mux

1111 0000

11 00 00 11 00 11 00 11 00 11 00 11

11 00 00 11 00 11 00 11 00 11 00 11

WDM link

Electrical switch

Electrical link

Optical switch

Figure 1: Illustration of a hybrid optical-electrical datacenter network

optical switch in the core connects the electrical edge-switches using optical fibers. The number of optical fibers from an edge-switch is limited by design, and defines the reachability factor k. Traffic from a fixed number of ports (on different wavelengths) of an edge-switch are multiplexed into an optical fiber, and switched through the optical switch to any other edge-switch, where they are demultiplexed. For a given number of ports at an edge-switch, the value of k determines a trade-off between the size of the optical switch (in number of ports) and the number of edge-switches that can be simultaneously reached. The higher the value of k, the larger the size of the optical switch and more the number of simultaneous optical paths from an edge-switch. In Fig. 1, k is set to two; hence the maximum number of edge-switches that can be reached from any edge-switch using (one hop) optical paths at a given time is limited to two. This is a cost-effective simple optical network which can connect large number of edge-switches using one or a few optical switches. For example, for k equal to four, an optical switch with 400 ports can connect 100 edge-switches. Although an edge-switch can reach only k other edge-switches simultaneously using the optical paths, due to the dynamic reconfiguration capability of the optical switch it can reach different sets of edge-switches at different times. As the architecture is hybrid, the edge-switches can also reach other edge-switches through the electrical core switches (as shown in the figure).

3

In a datacenter supporting multi-tenancy, guaranteeing bandwidth is important to deliver predictable performance to applications running on the VMs [7]. Recent research works addressed this problem in all-electrical datacenter networks; the corresponding problem in all-optical network is also getting attention [8]. But there are new challenges posed by a hybrid optical-electrical datacenter architecture not yet addressed. While the dynamic creation of optical paths gives flexibility in placing VMs, the reachability factor limits the number of edge-switches that can be reached simultaneously from one edge-switch (using optical paths). Besides, the topology that gets established dynamically may also pose constraints. We abstract a request from a tenant in the form of a virtual network, where a node of a virtual network corresponds to a set of VMs (VM-cluster), and the weight of an edge connecting two nodes gives the bandwidth required between the two corresponding VM-clusters of a tenant. This is a natural abstraction for most applications in datacenters, such as mapreduce, communicating tasks (where each task is carried out by a set of VMs), etc. [9]. We focus on the problem of embedding of virtual networks on hybrid datacenter networks, which translates to the joint problem of bandwidth allocation and placement of VMclusters such that topology constraints of the virtual network are satisfied. An intuitive way to solve this problem is to create a new optical path for each edge of the virtual network (until no more can be created), and then explore the existing optical paths and electrical network for embedding the remaining edges. But such an approach can potentially create a constrained optical network topology, which may not suit well for future virtual network demands. To investigate this, we develop an algorithm based on the above approach called NLFE (NewLink-First Embedding). In addition, we develop another embedding algorithm called ELFE (Existing-Link-First Embedding) that embeds edges on the existing optical paths and electrical network, and only in the worst case will create new optical paths. We also develop a greedy algorithm, called GLE (Greedy Link Embedding), that is faster than the above two algorithms in selecting the edges of an input virtual network to be mapped on to the hybrid datacenter network. We define a control parameter for flexible control of the proportion of edges that can be embedded on the electrical network of a datacenter. Using 4

simulations, we evaluate these algorithms with different topologies for dynamically arriving virtual networks. Our results provide interesting insights. One, ELFE and GLE algorithms consistently outperform NLFE, demonstrating the effectiveness of exploiting existing optical paths. Two, partial (but limited) embedding of edges on electrical links can decrease the rejection ratio further while still increasing the utilization of optical network capacity. Finally, the greedy algorithm GLE performs as good as ELFE, reinforcing that the restricting factor in embedding virtual networks is the topology constraints and not the capacity. We carried out preliminary work and developed the NLFE and ELFE heuristics in [1]. In this article we extend this work further and make new contributions. We explicitly formulate the problem of embedding a virtual network on a hybrid datacenter network, satisfying the constraints in both the input and the datacenter, as an optimization problem, and present its complexity. We develop a new algorithm based on greedy approach called Greedy Link Embedding (GLE). Algorithm GLE is computationally faster than NLFE and ELFE algorithms and differ in the way edges in an input virtual network are selected for mapping to the hybrid optical-electrical network. After discussing the related works in Section 2, we define and formulate the problem in Section 3. The operations involved in solving the problem of embedding a virtual network are described in Section 4, where we also define the three embedding algorithms—NLFE, ELFE and GLE. Performance studies are carried out in Section 5. 2. Related works 2.1. Bandwidth allocation in all-electrical datacenter networks The importance of allocating bandwidth for requests in all-electrical datacenter was recently highlighted by the research community. In this direction, Seawall enforces link-bandwidth allocation to competing VMs based on the weights assigned to VMs [10]; whereas Gatekeeper achieves bandwidth-sharing among competing tenants [11]. Other works consider advance reservation of bandwidth, given input demands. SecondNet [12] is a solution for datacenter network virtualization. It assumes a matrix specifying bandwidth demands between every VM pair as part of the input. The proposed embedding of a tenant request

5

proceeds by first locating a cluster based on the server resource requirement, and then solving the matching problem of a bipartite graph formed of VMs and servers based on both the server resource and bandwidth requirements. Oktopus [7], a similar work that allocates both server and network resources, abstracted bandwidth demands between VMs of a tenant into two topologies. While one topology was a single cluster of VMs requiring a star communication pattern, another topology abstracted the communication demand of clusters of VMs (or VM-clusters) as a tree. Our previous work [13, 14] proposes a solution for the joint problem of bandwidth allocation and VM-placement in all-electrical network by considering a matrix of bandwidth demands between every VM pairs as input. But assuming the bottleneck to be in the core network connecting the switches of a datacenter, the results showed that, for efficient allocation VMs can be grouped to form small number of VM-clusters (often four, and not more than six), and bandwidth allocation on these core links is dictated only by the traffic demands between VM-clusters [14]. In the past, we have also proposed and evaluated flexible models for allocating bandwidths for requests in Cloud and datacenter networks [15, 16]. In [15], we proposed a model that allows users to specify two bandwidth values between a VM pair, and provides bandwidth guarantees to traffic flows. While the minimum bandwidth is guaranteed throughout the duration of request, the peak bandwidth is provided only for a fraction of a duration (the range of which is defined by the user) [15]. The model in [16] aims to guarantee both bandwidth and time (deadline) to different kinds of requests in a datacenter, by giving incentives for flexible demands, thereby generating higher revenue while reducing rejection ratio. Works in [17, 18, 9] address the joint problem of VM-placement and reduction of bandwidth between VM-clusters in all-electrical datacenters, with [9] explicitly focussing on abstracting input as a graph called Tenant Application Graph. This is similar to the abstraction a network of VM-clusters that we define below in our work here. A TAG represents the bandwidth requirements between applications component. The problem of mapping TAGs on to a datacenter network such that the requested bandwidth is guaranteed is computationally hard (an N P-hard problem); in [19] a heuristic is developed to solve this as well as 6

to evaluate the TAG model. The works mentioned in this section attempt to solve the problems of bandwidth allocation and VM placement (sometimes as a joint problem) in allelectrical datacenter, and do not solve the problem in hybrid optical-electrical datacenter (for example, the number of optical paths is limited, and the topology of the dynamically created optical network is not known ahead and is potentially constrained). 2.2. Hybrid optical-electrical datacenter networks Chen et al. proposed an optical switch architecture for an optical datacenter network [5]. In the absence of packet-switched network, and as optical paths cannot be created between every pair of edge-switches, the architecture uses multi-hop communication, routing traffic through a sequence of optical paths with electrical-optical-electrical (o-e-o) conversions. Such o-e-o conversions are not only undesirable for applications, but also add load to the edge-switches. Another possibility is to dynamically and frequently reconfigure the optical network based on traffic demands at the ToR switches. This is studied in [20], where the authors propose end-to-end scheduling algorithms between ToR switches. However, reconfiguration of optical switches causes delay due to the hardware switching time. While active research is being pursued to bring down the reconfiguration time, it is still early to be applied to all traffic types, for example latency sensitive applications [21]. Besides, without optical buffers, achieving packet-level switching is a hard problem in itself. This is where a hybrid datacenter consisting of both optical and electrical networks finds advantage. Hybrid optical-electrical network architectures were proposed recently [4, 3]. At any instant of time, only one or a few optical paths can be provisioned from an edge-switch, limiting the number of simultaneously reachable edgeswitches. Therefore the above works monitor and estimate traffic between racks, and create or reconfigure optical paths between racks with high communication traffic, so as to meet changing demands. But they assume VMs to be already placed, and hence do not take into consideration the impact of VM-placement on the network traffic. Also, the reconfigurations of optical paths are for the entire datacenter. Our work, on the other hand, solves the VM-placement problem

7

along with bandwidth allocation. Besides, the solutions we propose in this article creates only a few optical paths, if required, for dynamically arriving virtual networks. We highlight that none of these works mentioned above can be directly applied to solve the similar yet different problem (see Section 3) in hybrid opticalelectrical datacenter network. In a recent work [22], we studied the joint problem of VM-placement and bandwidth allocation to requests in a hybrid optical-electrical datacenter, but for a different optical network architecture that had dynamic wavelength grouping (DWG) capability. Unlike the architecture considered in the work presented in this paper where the fibers carry fixed and same set of wavelengths, the architecture with DWG capability can be reconfigured to decide the set of wavelengths that can be carried in each group. While the DWG architecture is more flexible, it introduces wavelength-group continuity constraint which requires that the optical path should carry the same set of wavelengths in all fibers along the path. Hence the work in [22] studied bandwidth-allocation solutions specifically suited for the DWG architecture. Different from the algorithms presented in this paper, the embedding algorithms proposed in [22] exploit the ability of the DWG architecture to partition wavelengths dynamically into groups, considering the wavelength-group continuity constraints. 3. Problem definition and formulation Bandwidth demands between VMs of a tenant can be specified in multiple ways. At the finest granularity, traffic demand between every VM pair can be specified by a tenant, thereby giving a bandwidth matrix as input for each request. Another possibility is to consider a virtual network as an input, where the nodes of a virtual network are VM-clusters, and the weight between two connected nodes gives the bandwidth demand between the corresponding VM-clusters. Providing such an interface to tenants is useful to abstract the communication demands of the VMs, removing the difficulty of tenants in specifying traffic demand between every VM pair. Besides, it localizes the traffic between VMs of a cluster [14]. Observe that such an abstraction can be used to represent many different workflows in a datacenter network [23]. Fig. 3 gives a simple example of a virtual network. The input request has 13 VMs (v1 through 8

v13 ). A, B and C are the nodes (VM-clusters) in the virtual network, with b1 being the bandwidth demand between A and B, and b2 being the bandwidth demand between A and C. We assume that a node of a virtual network can be mapped on an edgeswitch such that the bandwidth requirements of the individual VMs (from VM to edge-switch) of the corresponding VM-clusters can be met using the links connecting the servers to the edge-switch. We also assume, not more than one node of a virtual network should be mapped to an edge-switch; otherwise two such nodes can be merged to form a bigger node in the virtual network. Taking cue from [7, 9], we consider three topologies for input virtual networks—star, tree, and random graph (Section 5.1)—which together cover various relevant application scenarios in a datacenter. Placement of a VM-cluster on an edge-switch implies placement of the VMs of the cluster on the servers connected to the edge-switch. Multiple VMs can be placed on a server depending on the capacity of the server, in terms of both computing resource capacities as well as server-to-switch link capacity. Similarly, placement of a virtual network means, to find an edge-switch for each VM-cluster such that the bandwidth requirement between two VM-clusters is satisfied by the link connecting the corresponding edge-switches. 3.1. Problem definition Definition 3.1. Problem: Given an input request in the form of a virtual network, the problem is to embed the virtual network on to the hybrid datacenter network. This translates to the joint problem of bandwidth allocation and placement of VM-clusters, while satisfying the topology constraint of the input virtual network. The objective while embedding a virtual network is to maximize the bandwidth allocated on the optical network. The problem is to embed an arriving request on the optical-electrical datacenter (refer Fig. 1). While both optical and electrical networks will be used to embed an input virtual network, we aim is to maximize the optical network utilization. Observe that the placement of VM-clusters (nodes of a given virtual network) is closely tied to the embedding of the corresponding edges of the

9

request on the physical network. Embedding of a VM-cluster on an edge-switch results in the placement of the corresponding VMs of the VM-cluster on the servers connected to the edge-switch. However, being a scarce resource, bandwidth is more likely to become a bottleneck much before computing resources; and hence we do not consider computing resources for allocation here. 3.2. Problem formulation Let r denote a request in the form of a virtual network, with ϑr denoting the set of nodes (VM-clusters) and ξ r denoting the set of edges connecting the nodes. r Bu,v is the bandwidth demand between two VM-clusters u and v of the request r,

where u, v ∈ ϑr . For the hybrid datacenter network, we denote by H the set of edge-switches in the network. The connection is between edge-switches of a datacenter; and hence H does not include the core electrical and optical switches (see Fig. 1). The path between two edge-switches can be seen as a logical link, whether it is an electrical path or optical path. The (residual) bandwidth on an optical path between two edge-switches i and j, i, j ∈ H, is denoted as θi,j ; and the bandwidth on an electrical path between the corresponding edge-switches is denoted by λi,j . Obviously, θi,i = 0 and λi,i = 0, for every edge-switch i in H. For each edge-switch i in the hybrid network, li represents the number of optical paths already established from that edge-switch. Two edge-switches i and j, each of which has less than k established optical paths, can establish a new optical path between them. The bandwidth θi,j available between these edge-switches is then equal to the fiber capacity. To formulate the problem, we also define π : ϑr → H, a mapping from the nodes in the input virtual network to the edge-switches in the datacenter network. For the optimization objective, we maximize the total bandwidth demands satisfied using optical paths. We formulate the problem of embedding an input request in the form of virtual network on a hybrid datacenter as a nonlinear programming problem:

10

maximize

X

r xu,π(u) xv,π(v) Bu,v yπ(u),π(v)

u,v∈ξ r

subject to: r xu,π(u) xv,π(v) Bu,v ≤ bπ(u),π(v) ,

∀(u, v) ∈ ξ r

(1a)

bi,j = yi,j θi,j + yˆi,j λi,j ,

∀i, ∀j ∈ H

(1b)

yi,j + yˆi,j = 1, X xu,i = 1,

∀i, ∀j ∈ H

(1c)

∀i ∈ H

(1d)

∀u ∈ ϑr

(1e)

∀i ∈ H

(1f)

∀u ∈ ϑr , ∀i ∈ H

(1g)

∀i, ∀j ∈ H.

(1h)

u∈ϑr

X

xu,i = 1,

i∈H

li +

X

yi,j ≤ k,

j∈H,i6=j

xu,i ∈ {0, 1}, yi,j , yˆi,j ∈ {0, 1}, yˆi,j ∈ {0, 1},

The first constraint checks for a link in the datacenter network with sufficient bandwidth to satisfy the demand of an edge in the request graph. If the link is established using optical path, the link bandwidth bi,j is the residual optical capacity θi,j in the logical path; otherwise, if the link is an electrical path, bi,j is the residual capacity in the electrical path λi,j . The second and third constraints ensure that only one of the paths, either the optical or the electrical, is selected. An edge-switch should not host multiple VM-clusters (Constraint (1d)); also a VM-cluster should be mapped on to only one edge-switch (Constraint (1e)). In Constraint (1f), li is the number of logical links already established using optical paths. Therefore, with Constraint (1f), the reachability factor of every edge-switch is respected. x’s, y’s and yˆ’s are binary variables. As mentioned earlier, the nonlinear objective function maximizes the bandwidth demands of an input virtual network satisfied by optical paths. Problem complexity: The above problem that we formulated belongs to the class of well-known combinatorial optimization problem called Quadratic Assignment Problem (QAP), and is N P-hard in nature [24]. 11

4. Virtual network embedding: operations and algorithms A hybrid datacenter offers multiple choices to embed a virtual network. A straightforward approach is to create as many optical paths as the number of edges in the virtual network, giving lesser priority to exploring existing optical paths. But such an approach can potentially create a constrained optical topology that rejects more requests than it would have if it gave least preference to creating new optical paths. In Section 4.3, we develop three algorithms to investigate this. In the next two sections, we break the problem into two subproblems and describe the operations involved in the algorithms. For simplicity, we assume bandwidth demands are symmetric, and so are link capacities. 4.1. Degree constrained maximum weight subgraph As no edge-switch can simultaneously connect to more than k other edgeswitches using optical paths, an input virtual network with maximum degree greater than k cannot be embedded on the optical network. Hence, the first problem is to extract a subgraph of the given input virtual network, such that maximum degree of any node in the subgraph is k. The remaining edges (not part of the subgraph, but of the input virtual network) have to be embedded on the electrical network. To maximize bandwidth allocated on optical network, we set the objective as to maximize the sum of the weights (bandwidth demands) of the selected edges. As defined earlier, ϑr denotes the set of nodes in an input r , which virtual network r, and ξ r the corresponding set of connecting edges. Bu,v

is also referred to as the weight of the edge (u, v), is the bandwidth demand of the edge (u, v) ∈ ξ r . The subproblem can be formulated as: maximize

X

r xu,v Bu,v

(u,v)∈ξ r

subject to

X

xu,v ≤ k;

∀u ∈ ϑr

v∈ϑr ,v6=u

where x’s are binary variables selecting the edges of the virtual network. This becomes the well-studied problem of finding a degree constrained maximum weight connected subgraph of a given graph, and is N P-hard [25]. However, as the induced subgraph need not be connected, the problem can be solved in

12

polynomial time (without the connectivity constraint) [25, 26]. This degreeconstrained maximum weight subgraph (DCMWS) problem is also the maximum weight b-matching problem. If the number of edges in a virtual network is e, and the number of nodes is n, the fastest known algorithm due to Gabow runs in O(e2 log n) [27] time. We use a linear programming (LP) solver to obtain the degree constrained maximum weight subgraph of a given virtual network. 4.2. Subgraph embedding Consider the optical network as a graph G, where the nodes correspond to the edge-switches, and a link connecting two nodes is the optical path connecting the corresponding edge-switches. The edge-switches of G can be partitioned into two sets, L and F, such that the subgraph formed of nodes in L is a logical topology established by optical paths. F forms a free set of nodes which can later be used to create optical paths, in which case the corresponding nodes with incident links move to L. Any node in the logical topology would have at least one link incident on it; whereas there is no link incident on any node in F. Observe, logical topology is dynamic and may change (by adding new optical paths) to accept a request. For example, in Fig. 2(b), L = {1, 3, 4}, and F = {2}, and the logical topology is the graph formed of nodes in L. In the illustrations, numbers denote the edge-switches of the datacenter network and letters denote the nodes in an input virtual network. Once we obtain the subgraph of the input virtual network after solving the first subproblem, the next task is to embed this subgraph on to the hybrid datacenter. The link on which an edge (of the subgraph) should be embedded, can either be an existing link or a new link created by establishing a new optical path. The various possible operations are: Operation 1: Create a new optical path using the free set of nodes F, and embed the edge on this link. Operation 2: Find a link in the logical topology L that can satisfy the bandwidth (weight) of the selected edge. Operation 3: Find an appropriate node in the logical topology L with degree less than k, from which a new optical path can be created. The other node can be either a node in the logical topology L, or one in the free set of nodes F.

13

Operation 4: Find an electrical link with sufficient bandwidth to satisfy the weight of the edge. All along each of the four operations, it is important that the constraints due the previously embedded edges (topology constraints of the virtual network) are not violated. By embedding an edge on a physical link, we are also doing the placement of the corresponding VM-clusters on the end-points of the link (edge-switches). Below we describe solution for each operation. 4.2.1. Operation 1 Given an edge to embed, Operation 1 is trivial; the only action required is to randomly select two nodes from F and establish an optical path. These two nodes are then moved to the set L as they are now part of the logical topology. The capacity constraint is also checked. 4.2.2. Operation 2 Algorithm 1 finds and embeds an edge of the subgraph, say r, on the logical topology in this operation. Let B r and B l be the bandwidth matrices of the subgraph r and the logical topology, respectively (with −∞ denoting bandwidth on unestablished optical links). Let Er be the set of edges of subgraph r previously embedded, and Vr be the corresponding set of nodes of r that have been mapped to edge-switches (in implementation, they are ordered lists for easy access to the corresponding mapped element). We denote by El and Vl the set of links and edge-switches on the logical network, on which the sets of Er and Vr , respectively, are embedded. Also, for each of the above set, say X, we denote the complement by X. For example, consider the subgraph of an input virtual network given in Fig. 2(a) for embedding. The graph of the optical network, with edge (a, b) embedded on link (1, 3), is shown in Fig. 2(b). At this time, Er = {(a, b)}, Vr = {a, b}, El = {(1, 3)}, Vl = {1, 3}, Er = {(b, c), (b, d)}, Vr = {c, d}, Vl = {2, 4}. Consider using Operation 2, to embed the next edge. Algorithm 1 finds the edge to be embedded (u, v) and an existing link on the logical topology, (s, t) for embedding. The mapping between an edge of the subgraph and a link on the physical network is stored in the list M. The first seven lines in Algorithm 1 check if there is an unmapped edge (u, v) such that both the end-points u and v are already mapped; in such a case this link between the 14

1,a

a 3,b b 4 c

d

2

(a) Subgraph of an input virtual

(b) Graph of optical network, af-

network r

ter embedding edge (a, b) of r on link (1, 3) 1,a

1,a 3,b

3,b

4,d

2,c

4,d

2

(c) Graph of optical network, af-

(d) Logical topology after creat-

ter embedding two links of the

ing link (3, 2) and embedding the

subgraph of request r

edge (b, c) on it

Figure 2: Example illustrating embedding of a subgraph

corresponding edge-switches of u and v is chosen for embedding the edge. In the case where there is no mapped node in the subgraph, we select a least degree node u and a corresponding edge-switch s (lines 10-12); and then select the other node and edge-switch in lines 17-18. Lines 13-15 are for the case where there is only one of the two end-points of an unmapped edge in the subgraph is mapped. For our example, the two if conditions (line no. 1 and line no. 10) fail, and b is selected in line no. 14. The corresponding edge-switch in Vl is 3. Assuming edge (b, d) and link (3, 4) satisfy the constraints in line numbers 17 and 19, the edge (b, d) gets embedded on the optical link (3, 4), as depicted in Fig. 2(c). 4.2.3. Operation 3 This operation comes only after at least one edge of the subgraph r is embedded; i.e., when Er 6= {}. For the example, from Fig. 2(c) (after Operation 2), Er = {(a, b), (b, d)}, Vr = {a, b, d} and Vl = {1, 3, 4}. Algorithm 2 finds an 15

Algorithm 1 ExistingOpticalLinkEmbedding(r) 1: if ∃(u, v) ∈ Er , 3 (u ∈ Vr ∧ v ∈ Vr ) then 2:

Find s, t ∈ Vl corresponding to u, v respectively

3:

l ) ≥ (B r ) then if (Bs,t u,v

4:

M.append([(u, v), (s, t)])

5:

return

6:

else

7:

return FAIL

8:

end if

9: end if 10: if @(u, v) ∈ Er , 3 (u ∈ Vr ∨ v ∈ Vr ) then 11:

Pick least degree node u ∈ Vr

12:

Pick least degree node s ∈ Vl , 3 degree(s) ≥degree(u)

13: else 14:

Let u be the node 3 (u, v) ∈ E r ∧ (u ∈ Vr ∧ v ∈ / Vr )

15:

Find s ∈ Vl corresponding to u

16: end if P

17: if 18: 19: 20: 21: 22: 23:

l )≥ (Bs,i

P

r ) then (Bu,j

∀j∈neigbours(u) ∀i∈neigbours(s) l ) ; v = max(B r ) t = max(Bs,i u,j j i l ) ≥ (B r ) then if (Bs,t u,v

M.append([(u, v), (s, t)]) else return FAIL end if

24: end if

unmapped edge with the maximum weight, such that at least one incident node is already mapped, and finds the appropriate edge-switch to which a new optical path should be created. The algorithm will embed the edge (b, c) on the newly created optical link (3, 2), as depicted in Fig. 2(d). 4.2.4. Operation 4 This operation is invoked only after one or more edges of the subgraph have been mapped on to the optical network. This, as in Operation 3, introduces constraints for selecting the edge-switches, based on the previously mapped edges and nodes. We proceed exactly as in Operation 3, except that, instead of establishing an optical path, here we check if the aggregate electrical paths between the two edge-switches have bandwidth greater than or equal to the weight of the 16

Algorithm 2 NewOpticalLinkEmbedding(r) 1: Find maximum weighted edge (u, v) ∈ Er , 3 u ∈ Vr 2: Let s ∈ Vl be the node on which u is mapped 3: if degree(s) ≥ k then 4:

return FAIL

5: end if 6: if v ∈ Vr then 7:

Let t ∈ Vl be the node on which v is mapped

8: else 9: 10: 11: 12: 13:

if ∃t ∈ L, 3 degree(t) < k then Let t be the other edge-switch else Choose a node randomly from F as t end if

14: end if 15: Create a new optical path between s and t; Update all sets 16: M.append([(u, v), (s, t)])

edge; and if so, this set of paths, identified as a logical link (assuming multipath routing will take care of load balancing traffic among the electrical paths), is selected for embedding the edge. To ensure high optical utilization, the number of edges of a given subgraph that can be mapped to electrical network is restricted by a control parameter m. We define m as a decreasing step function of the current (or in the words, ongoing) acceptance ratio of the input requests. This is to allow more edges to be embedded on electrical network when the current acceptance ratio is decreasing. The number of edges of a subgraph (of a newly arriving request) that can be embedded onto the electrical network will be evaluated after processing (and deciding on) every request, as each request affects the current acceptance ratio. This control parameter assists in attaining the objective of the problem we formulated in Section 3—maximize the bandwidth allocated on optical network. The exact definition (specifically, the values the function would take) is provided in Section 5. 4.3. Embedding algorithms Below we define three embedding algorithms that use the defined Operations. Definition 4.1. Existing-Link-First Embedding (ELFE) algorithm: After obtaining the subgraph by solving the first subproblem, ELFE algorithm first ex17

plores the logical topology for an appropriate link (Operation 2), and then explores the electrical network (Operation 4). If ELFE cannot find an appropriate link in the electrical network, it tries to establish a new optical path using a node in the logical topology (Operation 3). If all these options fail, the ELFA algorithm forgoes all previously selected links for embedding, and embeds the entire subgraph using the free set of nodes (F) by creating as many optical links as the number of edges in the subgraph. The ELFE gives least priority to creating a new optical path. The motivation for this is to exploit the already established optical paths. On the other hand, the next algorithm (defined below) first attempts to create a new optical path. Definition 4.2. New-Link-First Embedding (NLFE) algorithm: The NLFE algorithm (after obtaining the subgraph) first explores Operation 1 (create new optical link, both nodes from F), followed by Operation 3 (create new link, one node from L), and then Operation 2 (use existing optical link), for embedding an edge of the subgraph; and only if all these operations fail, will it explore the electrical network for an appropriate link (Operation 4). The NLFE algorithm iterates over each edge in the subgraph, performing the operations in the order specified to embed the entire subgraph on to the hybrid network. Both ELFE and NLFE solve the DCMWS problem (Section 4.1) to obtain a maximum weight subgraph with no node having degree greater than k, the reachability factor. These algorithms always pick the maximum weight unmapped edge of the subgraph for embedding. They differ only in the order in which they explore the hybrid network to embed the subgraph of an input virtual network. Besides, in both algorithms, the edges of a virtual network not selected by the DCMWS solution for the subgraph (due to the degree constraint of k) are all mapped to the electrical network (after the subgraph is embedded) using Operation 4 described previously, but without being constrained by m. This is naturally the only way to embed the remaining links, as they cannot be embedded on the optical network. Also recall, no two nodes of a given virtual network can be placed on the same edge-switch.

18

Definition 4.3. Greedy Link Embedding (GLE) algorithm: The GLE algorithm forms a subgraph by first sorting the edges of an input virtual network. It then picks edges one after the other in decreasing order of weights to form the subgraph, such that no node in the subgraph has degree greater than k (the reachability factor). Any edge that increases the maximum degree of the subgraph beyond k is discarded, and not considered for forming the subgraph. Once the subgraph is formed, the embedding algorithm proceeds with and explores the Operations in the same order as in ELFE—Operation 2, followed by Operation 4, and finally Operation 3. As in ELFE, if an edge can not be mapped using any of the three operations, all previous mappings are discarded, and the algorithm tries to map the entire input request graph on to the hybrid network. Both ELFE and NLFE algorithms solve the DCMWS problem, after which the edges of the subgraph are sorted and selected one after the other in decreasing weights to be mapped to the hybrid network. But the GLE algorithm skips the phase that solves the DCMWS problem, and simply sorts the edges in the decreasing order of the weights and picks them one after the other (in decreasing order), respecting the degree constraint of k, to form the subgraph for embedding. This is a faster approach than the other two algorithms (observe, all three algorithms sort the edges in decreasing order of weights, just that ELFE and NLFE do it only for the edges of the subgraph, while GLE does it for the entire graph). Therefore, the complexity of GLE in forming the subgraph of a input virtual network is O(e log e), whereas ELFE and NLFE require O(e2 log n) steps to select the edges (recall, e is the number of edges in a virtual network, and n is the number of nodes). Besides, the greedy approach though might choose the locally optimal solution during an iteration, it may not perform worse (in embedding an input request) than the other two that find the optimal solution—the constraints of the optical topology may not allow all optimal solutions to be mapped on to the optical network. When an edge of the request graph can not be mapped on to the hybrid network, the only option left is to map using the free set of nodes. But then, the previously made mappings can not be used. Hence, in such a situation, GLE (like ELFE) forgoes all mappings previously decided, and tries to map the 19

entire request graph in the following way. The GLE selects a node, say u, of the input graph, and maps it on to one of the free set of nodes. If the number of edges of this node is greater than k (the reachability factor) it maps the most weighted k edges incident on u on to the optical network, and the remaining (least weighted) edges on to the electrical network. 5. Performance analysis We study the performance of the three algorithms for a hybrid datacenter architecture illustrated in Fig. 1. We simulated a hybrid datacenter network with 100 edge-switches, each with 32 ports connected to the electrical network and another 32 ports connected to the optical switch (using k fibers). The bandwidth per port was 1 Gbps. The reachability factor, k, was set to four; i.e., an optical link carries eight wavelengths, thus delivering a total optical capacity of 8 Gbps between two edge-switches connected using the optical switch. The step function for the control parameter m used here takes value seven for acceptance ratio below or equal to 0.75, and decreases by 1 at steps of 0.05, reaching a lower limit of 2 for acceptance ratio between 0.95 and one. Fig. 4 shows the step function for m. When the acceptance ratio increases, lesser number of edges gets mapped to electrical network.

8 7 6 m

5 4 3 2 1 0.7

0.75

0.8 0.85 0.9 Acceptance ratio

0.95

1

Figure 4: Step function defining the control Figure 3: Example of an input virtual network

parameter m. For acceptance ratio ≤ 0.75, m = 7.

5.1. Input scenarios The input to the algorithms, as mentioned previously, are a set of virtual networks. We consider three scenarios, each having one particular topology

20

for the set of input virtual networks. In each virtual network, irrespective of the topology, the number of vertices is chosen randomly and uniformly to be between five and ten. Recall, a node in a virtual network is a clusters of VMs, hence the virtual network is small in size. Previous works have either considered small number of VM-clusters as input [7, 9], or found grouping VMs into small number of VM-clusters (given bandwidth demands between every VM pair) suffices for efficient allocation [14]. The bandwidth on the edge connecting any two nodes (two VM-clusters) is drawn from the Exponential distribution with a mean of 200 Mbps. The following are the three topologies: 1) Star: This topology abstracts communication pattern of one master and multiple slaves (all connected only to master). Applications that have data aggregation and data distribution workflows [23]. 2) Tree: In this scenario, we have three-level trees as input virtual networks. The number of nodes in level two (with root being at level one) is fixed as three. Hence, as the number of nodes is in the range [5 − 10], the minimum number of nodes in level three is one. We allow over-subscription; the bandwidth on an edge connecting the nodes of levels one and two (say, n1 and n2 , respectively) is half the total bandwidth on the links connecting the node in level-two (n2 ) to its children. 3) Random: Virtual networks are randomly generated connected graphs, with the number of edges limited to twice the number of vertices. To generate a virtual network with one of the above topologies, a random number for the number of vertices (in the range [5−10]) is given as input. Given the number of vertices, the number of edges is automatically limited for star and tree topology. For the random graph topology, the number of edges is a random number between [10 − 20]. 5.2. Results We compare the algorithms using the following metrics: • Rejection percentage: This is the percentage of input virtual networks rejected by an embedding algorithm. • Utilization: This is the utilization of the optical network. For this metric, we define the total capacity as the sum of the capacities of the optical

21

ELFE NLFE GLE

Utilization percentage

Rejection percentage

40 30 20 10 0 25

50

75

100 125 150 175 200

100 90 80 70 60 50 40 30 20 10 0

ELFE NLFE GLE

25

No. of input virtual networks

50

75

100 125 150 175 200

No. of input virtual networks

(a) Comparing rejection percentage

(b) Comparing optical utilization

Figure 5: Scenario 1: Virtual networks with star topology

links that are established. Utilization is the percentage of this capacity that is allocated for requests. For an experiment, the embedding algorithm processes a set of dynamically and incrementally arriving virtual networks. The number of virtual networks at input is varied (for different experiments) from 25 to 200 at steps of 25. We plot the rejection percentage and utilization for each scenario. Each point on the graphs is the mean value from 15 runs. We discuss results for each scenario. 5.2.1. Scenario 1 - Star topology Here the input virtual networks have the star topology. Fig. 5(a) plots the percentage of rejected requests, for varying number of input virtual networks. ELFE and GLE algorithms outperform the NLFE algorithm. As NLFE first attempts to embed the input subgraph by establishing new optical paths, it leads to the formation of a constrained logical topology. This eventually results in the rejection of future requests, even while bandwidth is available in the optical network. From the utilization plotted in Fig. 5(b), we see that the optical network is not fully utilized under any load, and is least utilized under the NLFE algorithm. There is negligible difference between the performances of ELFE and GLE algorithms. This is not surprising, as the greedy subgraph formation in GLE will pick the same optimal set of edges selected by ELFE for a star topology. Both ELFE and GLE bring down the rejection ratio of NLFE by ≈ 28% on average. By accepting higher number of requests, they also increase the utilization by ≈ 20% on average, as seen in Fig. 5(b).

22

ELFE NLFE GLE

Utilization percentage

Rejection percentage

20 15 10 5 0 25

50

75

100 125 150 175 200

100 90 80 70 60 50 40 30 20 10 0

ELFE NLFE GLE

25

No. of input virtual networks

50

75

100 125 150 175 200

No. of input virtual networks

(a) Comparing rejection percentage

(b) Comparing optical utilization

Figure 6: Scenario 2: Virtual networks with tree topology

Observe that in a virtual network with star topology and having n edges, the master node has degree n. Hence, accepting a virtual network with many edges, such that some edges are mapped on to electrical network will leave the logical topology less constrained. On the other hand, NLFE always embeds an edge on a newly created optical path between two nodes in the free set of nodes (F), until there is none available. This leads to a more constrained optical topology. 5.2.2. Scenario 2 - Tree topology The input virtual networks have a three-level tree topology in this scenario. The rejection percentage, as seen in Fig. 6(a), is lesser for all the embedding algorithms with tree topology than with the star topology. The maximum degree of an input virtual network in this scenario is less than that of a virtual network with star topology. Comparing plots in figures 6(a) and 6(b), both ELFE and GLE algorithms again perform better than NLFE. In comparison to NLFE, the ELFE and GLE algorithms bring down the rejection by a minimum of ≈ 30%. Again, note that the greedy algorithm does not lead to higher rejection ratio in comparison to the ELFE algorithm. On an average, both ELFE and GLE bring down the rejection ratio of NLFE by ≈ 70%. From Fig. 6(b), we note that this improvement in performance is achieved by a considerable increase in the utilization of optical network capacity. 5.2.3. Scenario 3 - Random graph topology The input virtual networks are formed as random graphs. As this scenario abstracts the most complex worflows, the rejection ratios are high for all the three embedding algorithms; see Fig. 7(a). Also observe in Fig. 7(b) that the 23

ELFE NLFE GLE

Utilization percentage

Rejection percentage

40 30 20 10 0 25

50

75

100 125 150 175 200

100 90 80 70 60 50 40 30 20 10 0

ELFE NLFE GLE

25

No. of input virtual networks

50

75

100 125 150 175 200

No. of input virtual networks

(a) Comparing rejection percentage

(b) Comparing optical utilization

Figure 7: Scenario 3: Virtual networks with random graph topology

optical capacity utilization is worst with the random graph topology. Therefore the increase in rejection ratio is not due to the lack of bandwidth, but due to the constraints of the logical topology. The differences between the three algorithms have reduced in this scenario. However, the NLFE algorithm leads to rejection of high number of requests even when the number of input virtual networks is low. Overall, the ELFE and GLE algorithms perform better. Recall, the mean number of edges on a virtual network is 15 in this scenario, and the maximum value of m is seven for the embedding algorithms. This scenario reveals the importance of electrical network in embedding virtual networks. An all optical datacenter with limited reachability may perform worse with high rejections. 5.2.4. Impact of m To further study the impact of the control parameter m, we also increased the maximum value of m from seven to ten. Correspondingly, to maintain the same number of steps in m (refer 4), the minimum value was increased from two to five. We refer to the ELFE modified this way as ELFEmmax =10 , and the corresponding GLE as GLEmmax =10 . We compare these against the standard ELFE and GLE (referred to as ELFEmmax =7 and GLFmmax =7 , respectively) for all the three scenarios. (However, recall that rejection rates for star and random topologies were higher than that for tree topology.) Fig. 8 plots the results. Increasing the maximum value of m (from 7 to 10) reduced the overall average rejection by ≈ 85% for virtual networks with star topology (see Fig. 8(a)) and ≈ 50% for virtual networks with random graph topology (see Fig. 8(c)). Even for the tree

24

ELFEm

25

max=7

max=10

GLEm

20

Rejection percentage

Rejection percentage

ELFEm

30

GLEm

max=7

max=10

15 10 5 0 25

50

75

100 125 150 175 200

10 ELFEm ELFEm

7.5

GLEm GLEm

2.5 0 50

75 100 125 150 175 200

No. of input virtual networks

(a) Star topology Rejection percentage

max=7

max=10

5

25

No. of input virtual networks

max=7

max=10

(b) Tree topology ELFEm

30

ELFEm

25

max=7

max=10

GLEm GLEm

20

max=7

max=10

15 10 5 0 25

50

75

100 125 150 175 200

No. of input virtual networks

(c) Random graph topology Figure 8: Comparing rejection percentage by varying m

topology, a reduction in rejection is observed (see Fig. 8(b)), with the maximum rejection percentage now being just 3%. We observed, the overall average utilization improved by more than ≈ 50% and ≈ 17%, respectively, for star and random topologies. For the tree topology, the utilization increased by ≈ 10%. This set of experiments reveals an important aspect of hybrid optical-electrical network. Exploiting the electrical network not only reduces the rejection but also increases utilization of optical network as more requests are accepted. 6. Conclusions In this paper, we addressed the problem of embedding dynamically arriving virtual network demands of tenants on a hybrid optical-electrical datacenter network. We developed three algorithms, NLFE, ELFE and GLE, and evaluated them for different virtual network topologies. Overall, ELFE and GLE algorithms outperformed NLFE algorithm, bringing down the rejection percentage. This highlights the importance of exploiting existing optical paths leading to the formation of a less constrained logical topology. Along with this, embedding

25

part of a request on the electrical network also helps in reducing rejection ratio of requests while maintaining high utilization of optical network capacity. This was achieved using the control parameter (m) which dynamically decides the number of edges of a request that should be embedded on the electrical network of a datacenter. The formation of constrained logical topology leads to higher rejection of requests, as was observed with the case of NLFE algorithm. This insight was reinforced with the performance of GLE algorithm. The GLE algorithm greedily selects the maximum weighted edges to form a subgraph of an input virtual network, and yet performs as good as the ELFE algorithm, though the latter selects the optimal set of edges. This happens as the rejection of requests are mostly governed by the topology rather than the bandwidth demands of the requests. For the same reason, the utilization of the optical network capacity was always below 80%.

7. Acknowledgement This work was supported by Singapore Ministry of Education Academic Research Fund Tier 2 Grant No. MOE2013-T2-2-135, NUS WBS No. R-263000-B11-112.

References [1] S. Hegde, R. Srinivas, D. M. Divakaran, M. Gurusamy, Dynamic Embedding of Virtual Networks in Hybrid Optical-Electrical Datacenters, in: 23rd International Conference on Computer Communications and Networks, ICCCN 2014, 2014, pp. 1–8. [2] Cisco

Global

Cloud

Index:

Forecast

and

Methodology

(20132018),

http:

//www.cisco.com/en/US/solutions/collateral/ns341/ns525/ns537/ns705/ns1175/ Cloud_Index_White_Paper.html (2014). [3] N. Farrington, G. Porter, S. Radhakrishnan, H. H. Bazzaz, V. Subramanya, Y. Fainman, G. Papen, A. Vahdat, Helios: a hybrid electrical/optical switch architecture for modular data centers, in: Proc. ACM SIGCOMM 2010, pp. 339–350. [4] G. Wang, D. G. Andersen, M. Kaminsky, K. Papagiannaki, T. E. Ng, M. Kozuch, M. Ryan, c-Through: part-time optics in data centers, in: Proc. ACM SIGCOMM 2010, pp. 327–338.

26

[5] K. Chen, A. Singla, A. Singh, K. Ramachandran, L. Xu, Y. Zhang, X. Wen, Y. Chen, OSA: An Optical Switching Architecture for Data Center Networks With Unprecedented Flexibility, IEEE/ACM Transactions on Networking 22 (2) (2014) 498–511. [6] K. Christodoulopoulos, D. Lugones, K. Katrinis, M. Ruffini, D. O’Mahony, Performance evaluation of a hybrid optical/electrical interconnect, IEEE/OSA Journal of Optical Communications and Networking 7 (3) (2015) 193–204. [7] H. Ballani, P. Costa, T. Karagiannis, A. Rowstron, Towards predictable datacenter networks, in: Proc. ACM SIGCOMM ’11, pp. 242–253. [8] S. Peng, R. Nejabati, B. Guo, Y. Shu, G. Zervas, S. Spadaro, A. Pages, D. Simeonidou, Enabling multi-tenancy in hybrid optical packet/circuit switched data center networks, in: European Conference on Optical Communication (ECOC), 2014, pp. 1–3. [9] J. Lee, M. Lee, L. Popa, Y. Turner, S. Banerjee, P. Sharma, B. Stephenson, CloudMirror: Application-Aware Bandwidth Reservations in the Cloud, in: 5th USENIX Workshop on Hot Topics in Cloud Computing, HotCloud’13, 2013. [10] A. Shieh, S. Kandula, A. Greenberg, C. Kim, B. Saha, Sharing the Data Center Network, in: Proc. 8th USENIX Conference on Networked Systems Design and Implementation, NSDI’11, 2011, pp. 309–322. [11] H. Rodrigues, J. R. Santos, Y. Turner, P. Soares, D. Guedes, Gatekeeper: supporting bandwidth guarantees for multi-tenant datacenter networks, in: Proc. of the 3rd conference on I/O virtualization, WIOV ’11, 2011. [12] C. Guo, G. Lu, H. J. Wang, S. Yang, C. Kong, P. Sun, W. Wu, Y. Zhang, SecondNet: a data center network virtualization architecture with bandwidth guarantees, in: Proc. ACM Co-NEXT ’10, pp. 15:1–15:12. [13] M. Gurusamy, T. Le, D. M. Divakaran, An integrated resource allocation scheme for Multi-Tenant data-center, in: 37th Annual IEEE Conf. on Local Comp. Networks (LCN 2012), 2012, pp. 496–504. [14] D. M. Divakaran, T. Le, M. Gurusamy, An online integrated resource allocator for guaranteed performance in data centers, IEEE Transactions on Parallel and Distributed Systems (TPDS) 25 (6) (2014) 1382–1392. [15] D. M. Divakaran, M. Gurusamy, M. Sellamuthu, Bandwidth Allocation with Differential Pricing for Flexible Demands in Data Center Networks, Computer Networks 73 (2014) 84–97. [16] D. M. Divakaran, M. Gurusamy, Towards Flexible Guarantees in Clouds: Adaptive Bandwidth Allocation and Pricing, IEEE Transactions on Parallel and Distributed Systems (TPDS) 26 (6) (2015) 1754–1764.

27

[17] X. Meng, V. Pappas, L. Zhang, Improving the scalability of data center networks with traffic-aware virtual machine placement, in: Proc. IEEE INFOCOM, 2010, pp. 1–9. [18] J. W. Jiang, T. Lan, S. Ha, M. Chen, M. Chiang, Joint VM placement and routing for data center traffic engineering, in: Proc. IEEE INFOCOM, 2012, pp. 2876–2880. [19] J. Lee, Y. Turner, M. Lee, L. Popa, S. Banerjee, J.-M. Kang, P. Sharma, Applicationdriven Bandwidth Guarantees in Datacenters, in: Proc. ACM SIGCOMM, 2014, pp. 467–478. [20] C.-H. Wang, T. Javidi, G. Porter, End-to-End Scheduling for All-Optical Data Centers, in: Proc. IEEE INFOCOM, 2015. [21] H. Rodrigues, R. Strong, A. Akyurek, T. Rosing, Dynamic optical switching for latency sensitive applications, in: ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS), 2015, pp. 75–86. [22] R. Srinivas, S. Hegde, D. M. Divakaran, M. Gurusamy, Virtual Network Embedding in Hybrid Datacenters with Dynamic Wavelength Grouping, in: IEEE 6th International Conference on Cloud Computing Technology and Science, CloudCom 2014, pp. 905–910. [23] S. Bharathi, A. Chervenak, E. Deelman, G. Mehta, M.-H. Su, K. Vahi, Characterization of scientific workflows, in: Third Workshop on Workflows in Support of Large-Scale Science, WORKS 2008, pp. 1–10. [24] E. Cela, The Quadratic Assignment Problem: Theory and Algorithms, Kluwer Academic Publishers, 1998. [25] M. R. Garey, D. S. Johnson, Computers and Intractability; A Guide to the Theory of NP-Completeness, W. H. Freeman, 1979. [26] W. J. Cook, W. H. Cunningham, W. R. Pulleyblank, A. Schrijver, Combinatorial Optimization, John Wiley & Sons, Inc., NY, USA, 1998. [27] H. N. Gabow, An Efficient Reduction Technique for Degree-constrained Subgraph and Bidirected Network Flow Problems, in: Proc. ACM STOC ’83, pp. 448–456.

28