DISTRIBUTED RESOURCE ALLOCATION IN COGNITIVE RADIO SYSTEMS BASED ON SOCIAL FORAGING SWARMS Paolo Di Lorenzo, Sergio Barbarossa “Sapienza” Univ. of Rome, INFOCOM Dept., Via Eudossiana 18, 00184 Rome, Italy E-mail: {dilorenzo, sergio}@infocom.uniroma1.it ABSTRACT The goal of this paper is to propose a distributed resource allocation strategy for the access of opportunistic users in cognitive networks. The opportunistic, or secondary, users are modeled as a set of agents looking for the most appropriate slots to use, in the time-frequency domain, with the goal of avoiding conflicts among themselves and to yield possibly no interference to the primary users. The problem is how to coordinate the access from the secondary users in a totally decentralized fashion, requiring a minimal coordination among them. We propose a solution to this problem based on a social forage swarming model, where the search for the most appropriate slots is modeled as the motion of a swarm of agents in the resource domain (typically the time-frequency plane), looking for “forage”, representing a function inversely proportional to the interference level. The swarm tends to move in the time-frequency region where there is less interference, while satisfying one basic requirement: minimize the spread in the resource domain while avoiding collisions between the allocations of different users. These properties are enforced through the decentralized minimization of a potential function that incorporates a long-range attraction between the resources occupied by secondary users (to minimize spreading) and a short-range repulsion (to avoid conflicts)1 . I. INTRODUCTION The opportunistic access of the radio spectrum by unlicensed users is a problem that is attracting a large interest in the research community as well as in the industry sector, as a way to improve the efficiency of the conventional rigid spectrum access [1]. The idea is to design cognitive networks, where unlicensed users, also known as secondary users (SU’s), are allowed to use temporally unoccupied communication resources, like frequency bands, time slots or user codes, under the constraint of not interfering (or producing a tolerable interference) towards licensed, or primary, users. A key step in cognitive networks is the ability of the opportunistic users to sense the spectrum accurately and then use the unoccupied resources until possible and release them as soon as some primary users access the network. As shown in a series of papers [2], [3], [4] the sensing and access strategies should be designed jointly to optimize system performance. Most works concentrate on the access from a single secondary user into a system partially occupied by primary users. Relatively fewer papers addressed the problem of the uncoordinated access of multiple secondary users in a cognitive network. A game-theoretic approach was proposed in [5], assuming perfect knowledge of the aggregated primary users’ interference power spectral density. The joint optimization of detection thresholds and access in a multiple secondary user scenario was recently addressed in [6]. In this work, we assume the presence of potentially many secondary users trying to access the available radio resources, e.g. time and/or frequency slots, with minimum coordination, avoiding potential conflicts with each other, but still trying to minimize the spread in the time-frequency 1 This

occupancy. We model the problem as the search for forage by a swarm of agents, where the spatial distribution of food corresponds to the inverse of the interference power in the time-frequency domain. As a consequence, the swarm tends to move towards the areas where there is more food, i.e. less interference. Each agent is looking for an appropriate time/frequency slot, where there is low interference power, trying to avoid the slots that get dynamically occupied by other secondary users, while avoiding an excessive spread of the domain occupied by the SU’s. Furthemore, to limit the energy wasted for coordination, each agent is supposed to listen to nearby nodes only, in a narrowband interval, over consecutive time slots. These requirements translate into a cohesiveness requirement for the swarm, yet avoiding collisions. The contribution of this paper is twofold: we generalize the swarm model of [7] to the case of local (rather than full) interactions and we apply this model to the distributed resource allocation in cognitive radios. Furthermore, we check how the swarm behaves in the presence of packet drops, assuming realistic inter-agent propagation channels. II. SWARM MODEL We consider a set of M secondary users aimed at allocating power in an n-dimensional Euclidean space. A typical setting is the one where the resource space is the time-frequency domain (i.e., n = 2) and every secondary user is trying to access time and/or frequency slots where there is small interfering power. To keep the notation as general as possible, the single resource selected by agent i is described by a vector xi ∈ Rn denoting, for example, a frequency subchannel and a time slot. In the presence of many SU’s, the problem is how to avoid conflicts between them, without requiring a centralized coordination node. To avoid conflicts, we propose an iterative algorithm where, at each iteration, every node broadcasts to its nearest neighbors the vector that it is planning to occupy. The interaction between the SU nodes can be modeled as an undirected graph G = (V, E), where V = 1, 2, ..., M denotes the set of nodes and E ⊆ V × V is the edge set. We assume that there is a link (edge) between two nodes if the distance between them is less than a prescribed value (the coverage radius), dictated by the node’s transmit power. We denote by A := {aij } the adjacency matrix of graph G, composed of nonnegative entries aij ≥ 0, and by Ni the set of neighbors of agent i, defined as Ni = {j ∈ V : aij %= 0}.

We also assume channel reciprocity, so that the coefficients aij satisfy the symmetry condition aij = aji . We formulate the resource allocation problem as the distributed minimization of a global potential function defined as follows: J(x) =

M X i=1

work has been funded by FREEDOM Project, Nr. ICT-248891.

(1)

σ(xi )+

M M 1 XX aij [Ja (&xj −xi &)−Jr (&xj −xi &)] 2 i=1 j=1 (2)

where x := (x1 , . . . , xM )T and σ : Rn → R represents the interference power over the optimization domain (e.g., the timefrequency plane). Neglecting for a moment the second term on the right-hand side (RHS) of (2), the minimization of (2) leads every node to find a position xi such that the overall interference power is minimum. However, such a solution would not prevent two different nodes to choose the same position, thus conflicting with each other. Actually, if the interference function σ(x) had a single minimum, every node would tend to occupy the same value, thus inducing an overall conflict among SU’s. To avoid this situation, thePsecond term on the RHS of (2) contains a repulsion function M PM 1 i=1 j=1 aij Jr (&x j −xi &) that is maximum when two nodes 2 occupy the same position (i.e., xj = xi ). Hence, the purpose of the repulsion term is to avoid collisions. However, the repulsion alone might lead to an extreme dispersion of the slots occupied by the whole set of nodes, in the resource domain. This is also undesirable, because it could lead to the occupation of an unjustified large region in the spectral domain or of distant time slots in the time domain, thus running into non-stationarity problems. To avoid excessive we introduce a long range attraction function PM Pdispersion, M 1 i=1 j=1 aij Ja (&x j −xi &) that is minimum when the vectors 2 xi are all close to each other. Hence, in summary, the attraction and repulsion terms in (2) are chosen so that the overall system tends to remain as cohesive as possible, but without creating conflicts. In particular, there is a unique equilibrium distance at which the attraction and repulsion forces balance each other. The swarm model in (2) is conceptually equivalent to the one introduced in [7], [8] to describe social foraging swarms, with one important exception: In [7], [8], each node interacts with every other node, i.e. aij > 0, for all i, j ∈ {1, . . . , N }; in our case, aij > 0 only if nodes i and j are within the coverage radius of each other, otherwise aij = 0. This is particularly useful in our application because it only requires short-range communications among the nodes. This feature is actually relevant also in biological swarms, where every individual tends to keep track not of the whole swarm, but only of a small set of neighbors. We are interested in the distributed minimization of (2). A possibly way to achieve the solution in decentralized form is to simply use a gradient based optimization, so that every node starts with an initial guess, let us say x0 , and it updates its own resource allocation vector xi in time according to the following dynamical system: x˙ i = −∇xi J(x) = −∇xi σ(xi ) +

M X j=1

aij g(xj − xi ),

(3)

i = 1, . . . , M , with x(0) = x0 and g(·) denoting a vector function defined as g(y) = [ga (&y&) − gr (&y&)] y, (4) where ga (r) and gr (r) are the derivatives of Ja (r) and Jr (r) with respect to r, respectively. The parameters of the functions ga (·) and gr (·) are chosen so that the attraction term dominates at large distances, the repulsion dominates at short distances, and there exists a unique distance where attraction and repulsion balance. In our setting, this equilibrium distance is chosen proportional to the bandwidth of the frequency slot, in the frequency domain, or to the duration of the elementary time slot. It is important to remark, about the updating rule (3), that each individual in the swarm only has to estimate local parameters: the gradient of the interference level, evaluated only on its intended running position xi , and the balance of attraction and repulsion forces with its immediate neighbors. III. STABILITY AND COHESION ANALYSIS To analyze the behavior of the swarm, it is useful to observe the evolution of the time derivative of the global potential function (2)

along the system trajectory (3) ˙ J(x)

=

[∇x J(x)]T x˙ =

M M X X [∇xi J(x)]T x˙ i = [−x˙ i ]T x˙ i i=1

=

−

M X i=1

i=1

&x˙ i &2 ≤ 0

∀t.

(5)

This means that, moving along the trajectory given by (3), the potential function J(x) is always nonincreasing and it stops decreasing (i.e., J˙(x) = 0) only if x˙ i = 0, ∀i = 1, . . . , M . To get more insight into the collective behavior of the swarm modeled asPin (3), let us consider first the center of the swarm: ¯ = 1/M M x i=1 xi . The trajectory of the center is given by x ¯˙

=

−

=

−

M M M 1 X 1 XX ∇xi σ(xi ) + aij g(xj − xi ) M i=1 M i=1 j=1 M 1 X ∇xi σ(xi ), M i=1

(6)

where the equality of the second term of the RHS of (6) to zero follows from the symmetry condition aij = aji and from the fact that g(·) is an odd function. The above equation means that the center of the swarm moves until the agents reach a position where the average gradient is zero. The fact that is the average gradient to determine the motion of the swarm center, rather than the individual gradients, is appealing in our context because it reduces the effect of undesired zero-mean fluctuations due to estimation errors or observation noise of the gradient estimated by each node. This property will be verified numerically in Section IV. In this paper, we consider attraction/repulsion functions having a linear attraction term, i.e., ga (&xj − xi &) = cA

cA > 0,

(7)

and unbounded repulsion, i.e gr (&xj − xi &)&xj − xi &2 = cR

cR > 0,

(8)

∀ &xj − xi &. Unbounded repulsion prevents collisions among neighbor agents. We will now analyze the cohesiveness of the swarm under a general condition satisfied by almost any realistic profile. To this end, we define the distance between the position ¯. xi of individual i and the center of the swarm as ∆i = xi − x Deriving a bound on the magnitude of ∆i is useful to quantify the size of the swarm and then, ultimately, the spread in the resource allocation a cumulative Lyapunov function as P domain. Defining 2 1 V = M i=1 Vi , with Vi = 2 &∆i & , and taking its time derivative along the system trajectory (3), after a few algebraic manipulations, we can write V˙

=

M X

V˙ i =

i=1

=

− +

−

M X i=1

M X

˙ Ti ∆i = ∆

(9)

i=1

[∇xi σ(xi ) −

M 1 X ∇xj σ(xj )]T ∆i + M j=1

M M 1 XX aij ga (&xj − xi &)&xj − xi &2 + 2 i=1 j=1

M M 1 XX aij gr (&xj − xi &)&xj − xi &2 . 2 i=1 j=1

(10)

Exploiting the features of linear attraction and unbounded repulsion of the coupling function g(·) in (7) and (8), we obtain V˙

=

−

−

M X i=1

[∇xi σ(xi ) −

1 M

M X j=1

∇xj σ(xj )]T ∆i + (11)

where D is the degree diagonal matrix with diagonal entries that are the row sums of the adjacency matrix A. The graph Laplacian L is an M ×M matrix associated with graph G, defined as L = D −A. The matrix L always has, by construction, a right eigenvector 1M composed of all ones, associated to the null eigenvalue λ1 = 0. Properties: Let G = (V, E) be an undirected graph of order M with a symmetric non-negative adjacency matrix A = AT . Then, the following statements hold true: 1) L is a positive semidefinite matrix that satisfies the following sum-of-squares (SOS) property M M 1 XX aij (xj − xi )2 , 2 i=1 j=1

x ∈ RM ;

(12)

2) The graph has c≥1 connected components iff rank(L) = M − c. Particularly, G is connected iff rank(L) = M − 1; 3) Let G be a connected graph, then xT Lx λ2 (L) = min >0 x⊥1M &x&2

ˆ = L ⊗ In L

where ⊗ denotes the Kronecker product. This multidimensional Laplacian satisfies the following SOS property: ˆ = xT Lx

M M 1 XX aij &x j − xi &2 , 2 i=1 j=1

x ∈ RnM .

(15)

where x := (x1 , . . . , xM )T and xi ∈ Rn . If the symmetric graph modeling the interaction among the nodes is connected, the multiplicity of the null eigenvalue of the Laplacian is one. The state vector x can always be decomposed into the motion of the center of the swarm plus the displacement vector ∆, according to the following equation: ¯ +∆ x = 1M ⊗ x

(16)

where the disagreement vector ∆ = (∆1 , . . . , ∆M )T ∈ RnM satisfies ∆ ⊥ 1nM . The vector ∆ belongs to an (nM − 1)ˆ that is dimensional subspace (the disagreement eigenspace of L) ˆ As a consequence of (15) and orthogonal to the nullspace of L. (16), the expression (11) can be recast as V˙

M X

[∇xi σ(xi ) −

=

−

−

ˆ + cA ∆T L∆

i=1

M 1 X ∇xj σ(xj )]T ∆i + M j=1

cR T r(D). 2

(17)

−

M

[∇xi σ(xi ) −

∀y.

(19)

This assumption is quite general and only requires the gradient of the profile to be bounded. This hypothesis is indeed very reasonable in the context of interest. Under such an assumption, ‚ ‚ M X ‚ ‚ 2¯ σ (M − 1) ‚∇x σ(xi ) − 1 (20) ∇xj σ(xj )‚ ‚≤ ‚ i M j=1 M and, denoting by dii the diagonal entries of D and setting yi = &∆i &, the expression (18) assumes the form

V˙

≤ = =

(14)

≤

¯, &∇y σ(y)& ≤ σ

(13)

The quantity λ2 (L) is known as the algebraic connectivity of the graph and is related to the speed of convergence of the dynamical system (3). In particular, we consider M-dimensional graph Laplacians defined by

M X

1 X ∇xj σ(xj )]T ∆i + M i=1 j=1 cR T r(D). (18) − cA λ2 (L)&∆&2 + 2 Now, if we can show that there exists a constant # such that, for all &∆i & > #, we have V˙ i < 0, then we can guarantee that in that region &∆i & is decreasing and eventually &∆i & ≤ # will be achieved. Assumption: There exists a constant σ ¯ > 0 such that V˙

M M cR cA X X aij &x j − xi &2 + T r(D) 2 i=1 j=1 2

xT Lx =

ˆ = λ2 (L), we obtain Then, applying (13) and considering λ2 (L)

M M X 2¯ σ (M − 1) cR X dii &∆i & + M 2 i=1 i=1 i=1 – M » X 2¯ σ (M − 1) cR dii 2 &∆i & − −cA λ2 (L) &∆i & − M cA λ2 (L) 2cA λ2 (L) i=1

−cA λ2 (L)

−c1

M X i=1

M X

&∆i &2 +

[yi2 − c2 yi − c3 ]

(21)

with c2 , c3 > 0. Therefore, as long as yi2 − c2 yi − c3 > 0, V˙ < 0 and the system evolves in order to reduce V and then yi . Denoting by #l and #u the lower and upper roots of the polynomial yi2 − c2 yi − c3 , we have V˙ < 0 for &∆i & > #u (&∆i & < #l < 0 is not feasible). A first condition guaranteeing the convergence of the system is then &∆i & > #u . Conversely, deriving a lower bound on V , we proved that the condition guaranteeing convergence of the swarm is &∆i & < βl , where βl < #u depends on the maximum eigenvalue of L [9]. In summary, at convergence, βl < &∆i & < #u , i.e., the distance between the individuals of the swarm and its center is both lower and upper bounded. This proves swarm cohesiveness and prevents the individuals to collapse on the swarm center. IV. NUMERICAL RESULTS In this section we provide some numerical results validating the proposed model. Example 1: Swarming in time-frequency plane The first example shows the evolution of the swarm in the time-frequency plane. In this case, the vector xi referred to node i has two entries representing the position of the frequency subchannel and the time slot that node i intends to occupy. Before occupying the slot, each secondary node interacts with its neighbors exchanging information on the intended slot. Every node then updates its intended position according to (3), implemented in discrete-time. In the application at hand, there is an intrinsic quantization of the time-frequency resources given by the subchannel bandwidth and by the duration of the time slot. In our implementation, we let the system evolve according to (3) until successive differences in allocation become smaller than the discretization step in the time-frequency domain. At that point, the evolution stops and every SU is allowed to transmit over the

selected resource, i.e. a pair of frequency subchannel/time slot. The four boxes in Fig. 1 represent the power allocation of primary users, in the time-frequency domain. These are the regions that do not have to be occupied by the SU’s. The initial guesses of the SU’s are represented as blue dots, scattered randomly across the time-frequency plane. The evolution of the resource allocation is depicted by the red curves and the final allocations are given by the green dots. It is evident how the secondary users avoid

One of the most critical issues in cognitive radios is the accuracy with which the SU’s estimate the channel. Estimation is critical as it may induce undue channel occupancy, in case of miss detection of primary signals, or efficiency losses, in case of false detection causing missed access. We show next how the swarm behavior can help to reduce the effect of local inaccuracies in the final resource allocation. We consider, for simplicity, allocation in the frequency domain only. An example of the true spectrum, occupied by the primary transmitters is reported in Fig. 3 (solid line), together with its estimation with a finite number of samples (dashed line). The spectrum decay is dictated by the usual raised cosine transmit filters. The estimation of the interference power 16

14

Interference Profile

12

10

8

6

4

2

0

−2

0

5

10

15

20

25

30

35

40

45

50

Frequency axis

Fig. 1. Example of 2D allocation Fig. 3. Interference power spectral density: true spectrum (solid line) and its estimate (dashed). over each sub-carrier is carried out as an energy detector. The choice of the number of samples used to estimate the interfering power is critical. In fact, using a large number of samples improves the estimation accuracy but, at the same time, increases the risk of averaging over a nonstationary process. It is then of interest to look at the performance of the proposed distributed resource allocation method, as a function of the number of samples. To measure the effectiveness of the distributed resource allocation strategy, in Fig. 4 we report the interference level averaged over the frequency slots occupied by the SU’s, after convergence. 5.5

Fig. 2. Example of 2D allocation the positions occupied by the primary users, tend to keep the spread in the time-frequency plane as small as possible and, at the same time, they avoid collisions with each other. A further example is shown in Fig. 2, where the primary users are spread more randomly. We can see that, also in this case, even though the agents cannot end up into a cohesive group, they still tend to be as close as possible, still avoiding collisions and interference with the PU’s. Example 2: Swarming in frequency domain in the presence of estimation errors

Average Interference Level perceived by the swarm

CA=0.1 5

CA=0.2 CA=0.3

4.5

4

3.5

3

2.5

2

1.5

1

0.5

0

5

10

15

20

25

30

35

40

45

50

Number of samples of the Energy Detector

Fig. 4. Average interference perceived by the swarm versus the number of samples of the energy detector.

Example 3: Swarming in frequency domain in the presence of link failures In a realistic communication scenario, some packets may be received with errors because of channel fading or noise. It is then of interest to look at what happens if the erroneous packets are simply dropped, without requiring retransmissions, to avoid excessive complexity of the swarm radio access. Random packet drops induce a time-varying, or switched, network topology, depending on the link failures. One of the most important features of consensus protocols is that consensus is reached even if the network is disconnected most of the time, provided that the expected graph is connected. This result applies also to the swarm behavior decribed here. As an example, assuming the interference profile of Fig. 3, we consider the allocation of 12 resources that, due to the foraging and cohesion strengths, at convergence fill the low interference band in the middle of the spectrum. The resources are initially scattered randomly across the frequency spectrum in order to constitute a connected network. We consider the case in which a communication link among two neighbors has a certain probability p to be established. To show the convergence rate and the robustness of the algorithm to topology switching, in Fig. 5 we report the average behavior of the system potential function, e.g., (2), averaged over 500 independent realizations, vs. the time index, considering three values of the probability p of establishing a link. From Fig. 5 we can notice that, after a sufficient time, in all the cases the network reaches the same minimal energy configuration that coincides with a swarm cohesion in the low interference region of the spectrum. As expected, reducing the probability to establish a communication link, the network requires a longer time to reach the final equilibrium state. V. CONCLUSION In this paper we have proposed a distributed resource allocation strategy for the access of opportunistic users in cognitive networks based on a swarm model mimicking the foraging behaviors of a swarm of animals. We have shown how, choosing the proper balance between attraction and repulsion forces can help, in the

80 P = 0.6 P = 0.8

70

P=1 60

System potential function

The result is also averaged over 100 independent realizations, versus the number of samples Ks used by the energy detector for estimation. We considered three different values of the attraction constant CA that determines the swarm cohesion strength. The gradient of the interference profile is locally computed by each SU through a linear regression using a number of samples Ns that can be properly chosen to reduce the effect of the estimation noise. Ideally, the average interference level should be as small as possible, compatibly with the existence of spectrum holes sufficient large to allocate the SU’s. In our example, considering the profile of Fig. 3, we consider the allocation of 10 resources that, in the absence of estimation noise, should be able to fill the low interference band in the middle of the spectrum. From Fig. 4, we notice that, at low values of Ks , the estimation is quite poor and many SU’s end up allocating power in the region occupied by the primary users (this explains the high average interference power perceived by the swarm). As Ks increases, the estimation accuracy improves and each SU tends to move towards the interference-free region, thus making the overall swarm experience a smaller total interference. But what is most interesting to notice from Fig. 4 is that, for any given Ks , the system performance may be improved by simply increasing the cohesion strength, acting on the attraction parameter CA . In fact, increasing CA , the agents allocating over the low interference band tend to form a cohesive block that exerts an attraction towards the agents trapped by mistake over the subchannels occupied by the primary users, because of local estimation errors. This shows that, under a proper choice of the system parameters, the swarm cohesion strength can induce a positive intrinsic robustness against individual errors.

50

40

30

20

10

0

0

50

100

150

200

250

Time index

Fig. 5. System Potential function value versus the time iteration index.

application at hand, to identify suitable resource slots, while avoiding conflicts among SU’s, and to counteract the effect of estimation errors, which can cause power allocation in undue regions. In the proposed model, each agent is supposed to be able to listen only to nearby nodes, in a narrow band spectral interval, over consecutive time slots. The method is also robust against packet drops, whose effect is only to slow down the convergence process. VI. REFERENCES [1] S. Haykin, “Cognitive radio: Brain-empowered wireless communications,” IEEE J. Select. Areas Commun., vol. 23, no. 2, pp. 201-220, February 2005. [2] Q. Zhao, L. Tong, A. Swami, and Y. Chen, “Decentralized cognitive MAC for opportunistic spectrum access in ad hoc networks: A POMDP framework,” IEEE J. Select. Areas Commun., vol. 25, no. 3, pp. 589-600, Apr. 2007. [3] Z. Quan, S. Cui, H. V. Poor and A. H. Sayed, “Optimal multiband joint detection for spectrum sensing in cognitive radio networks,” IEEE Trans. on Signal Processing, vol. 57, pp. 1128–1140, March 2009. [4] S. Barbarossa, S. Sardellitti, G. Scutari, “Joint optimization of detection thresholds and power allocation for opportunistic access in multicarrier cognitive radio networks,” Proc. of CAMSAP 2009, Aruba, Dec 12-14, 2009. [5] G. Scutari, D. P. Palomar and S. Barbarossa, “Cognitive MIMO Radio,” IEEE Signal Processing Magazine, vol. 25, pp. 46–59, Nov. 2008. [6] S. Barbarossa, S. Sardellitti, G. Scutari, “Joint Optimization of Detection Thresholds and Power Allocation in Multiuser Wideband Cognitive Radios”, Proc. of COGIS 2009, Paris, Nov. 16-18, 2009. [7] V. Gazi, K. M. Passino, “A class of attractions/repulsion functions for stable swarm aggregations,” Int. Journal on Control, 2004, vol. 77, no. 18, pp. 1567-1579, 2004. [8] V. Gazi, K. M. Passino, “Stability Analysis of Social Foraging Swarms,” IEEE Transactions On Systems, Man, And Cybernetics - Part B: Cybernetics, vol. 34, No. 1, pp. 539-557, February 2004. [9] P. Di Lorenzo, S. Barbarossa, “Distributed Resource Allocation in Cognitive Radio Systems Based on Swarming,” submitted to IEEE Transactions on Signal Processing.