Host Measurement of Network Traffic DongJin Lee and Nevil Brownlee Department of Computer Science The University of Auckland [email protected], [email protected]

Abstract—In modern passive network measurement, flow-based analyses have been widely used. In this paper, we regard ‘hosts’ as aggregations of flows, and investigate host rather than flowbased measurements. We show that analyzing host behaviors contribute significantly to better understanding of network behavior, thus it is a feasible study that should be of interest to both researchers and network operators. By empirically exploring two university edge networks, our findings suggest that up to 40% of the unique hosts re-enters the network, and many of those tend to do so within three hours. Furthermore, at least 20% of all hosts are active for between 0.1s and 10s, and only a quarter of the hosts stay in the network for more than 10s. Additionally, many hosts are identified as abnormal, producing many flows despite being short-lived. Lastly, we examine the correlations between numerous host attributes to find relationships; for instance, host size and size-rate are highly positive-correlated. Keywords - passive measurement, host behaviors, lifetime, reentry, correlations

I.

INTRODUCTION AND BACKGROUND

Internet measurement is still a discipline in its infancy, but it holds a vital key to the health of the current and future Internet [4]. In a typical passive measurement analysis, a network traffic meter is often used to monitor the packets, where packets can either be inbound (to the edge network) or outbound (to the edge network’s service provider). Many network measurements work by capturing each packet in ‘trace’ files for later processing, for example accumulating packets into flows so as to monitor network usage. Although the actual monitoring process could differ vastly depending on the purpose of the analysis, packets are usually grouped according to common characteristics. Early work done in [7] used packet-train models to group packets with a specific number of inter-arrival times (e.g., 500ms); better parameterized models were investigated in [6], and these groups are what we now call flows. Many flow definitions exist, but the most commonly used definition is the flow representation of 5tuple packets: ‘a series of packets that have the same source IP address, destination IP address, source port, destination port and protocol’ is regarded as a single flow. If one of these five attribute values is different, then the packet belongs to a different flow. Additionally, flows are assumed to expire when packets for those flows are no longer seen (e.g., after 64 seconds) [6]. In a typical measurement, per-flow analysis has various advantages over per-packet analysis since a single flow (often regarded as a ‘connection’) represents a group of the same 5-tuple packets, and holds abstract information such as the flow duration and sizes. Recently, there have been broad 1-4244-1557-8/07/$25.00 ©2007 IEEE

research into analyzing flow behaviors and patterns, for example, flow lifetimes [3, 5], rates [13], ranks [12] and anomalies [2, 9]. Additionally, in an edge network setup, a traffic meter will observe two types of flows: one-way (packets travelling in only one direction) and two-way (packets travelling both ways) flows [10]. We find that often more than 90% of the total traffic is carried in two-way TCP flows, transferring packets in both directions. Thus, one-way flows are usually identified as malicious flows because a recipient host rejecting their request packets. These malicious flows could also change the flow lifetime distributions significantly. Such flows are unwanted and could degrade network performance. In our initial analysis, we observe that a single host can produce multiple flows (e.g., outbound proxy server), but many hosts may produce only a few flows (e.g., inbound DNS server). This is not surprising since analyzing flows alone may not give sufficient detail about host relationships. For instance, malicious one-way flows are often produced by a few (dominant) hosts, and servers that produce numerous two-way flows are usually busy hosts such as web, DNS or proxy servers. In our review of existing research, we find that previous studies are either based on per-packet or per-flow measurements. So far, host-based analysis has not been studied and we feel that host-based measurements are valuable for understanding host behaviors, and the way that they contribute to traffic within a network. For instance, network operators are often interested to know typical host characteristics such as durations, volumes and connections, so as to help monitor their customers’ service quality. Therefore, instead of packet or flow analysis, we are interested in grouping flows by hosts and analyzing those hosts. Since the hosts often produce more than one flow, we find that understanding network traffic only at a flow level unfortunately does not produce helpful summaries of host behaviors. Furthermore, we gain several advantages through using host-based analyses. Although per-flow analysis reduces information by grouping packets with common characteristics, multiple flows could originate from a single host; these flows could thus be grouped together since they have a common end-point. Analyzing network traffic per-host could narrow the focus towards a user perspective as the flows are grouped by an even more common characteristic, thereby removing the details of packets and flows. This allows network operators to work on higher abstract levels, as well as producing simpler observations with a diverse host behavior analysis. In this paper, we investigate various aspects of host beha-

(b)

(a)

Flow timeout expires (garbage collection)

- First packet - First flow

Host timeout expires (garbage collection)

Host-table

Building Host-entities Hosts

Flow1 (active)

(1) or (2)

Inactive (64s)

IP Flow lifetime

Flow-table

Flow2 (active)

Inactive (64s)

Inactive (64s)

Flows Host exits

Host enters 5-tuple

Flow3 (active)

Inactive (64s)

Packets Host lifetime - Last packet - Last flow

Figure 1. (a) three-tier network traffic model, (b) an illustration of host and flow lifetimes

viors, and attempt to observe generic and yet appealing notions that have not been considered by researchers or network operators before. We measured two university campus edge networks, so as to compare different host behaviors. We will first provide the definition of a host, and observe host behaviors in the network. In particular, we suggest that host behaviors such as host re-entry and host lifetimes are dynamic, and we will also investigate the correlations between various host attributes. The rest of the paper is structured as follows. In section II, we show our definitions of a host and describe our network traces. In section III, we observe host entry/exit/re-entry and lifetimes. Section IV conducts further host measurements to examine their sizes, rates and lifetimes to discover their relationships. Section V summarizes the paper and suggests further work. II.

MEASUREMENT PREPARATIONS

Host Definitions: We first clarify and explain the perceptions of a host in our study. Like the aforementioned definition of a flow, the definition of a host is as follows: ‘a series of flows that have either the same source or destination’. In other words, hosts appear simply as groups of flows. Thus, two host entities could be produced for each two-way flow, and each entity would also hold information such as duration, number of flows and their sizes. This is better explained by the illustrations shown in Figure 1. The lowest level of Figure 1(a) is concerned with the packet; the meter observes every packet on the link it monitors. At every collection interval of 60s, the observed packets are grouped (by 5-tuple) into flows, then these are again grouped (by IP address) into hosts thereby producing a flow- and a host-table. The host-table stores each IP host together with pointers to its flows, and the flow-table stores individual flows from the observed packets. Here, we say that a host enters the network when it is first observed, i.e., a host appears in the network that was not already in the current host-table. We then say that a host exits the network when packets or flows associated with it are no longer observed after an expiry timeout period, i.e., a host is inactive and assumed expired, to be cleared from the host-table. Thus, host-table size is basically the number of active hosts. To be more precise and consistent in our definition, we say that flows

TABLE I. Traces (24h) UoA 2006 [Auck06] 27-Jul-2006 UoW 2006 [Wits06] 30-Oct-2006

BRIEF STATISTICS OF TWO NETWORKS

# Packets (M)

# Flows (M)

# Hosts (M)

# Unique hosts (M)

486.9

27.3

4.3

1.3

173.1

26.3

2.6

0.6

and hosts are active as long as packets for them are observed. We call a host’s active duration its lifetime; the time interval between the first and last flow packets. A host’s lifetime is only computed for analysis when it is expired (thus, garbage collected from the memory). Expiry Timeout: For each interval, the meter checks on the flow expiry timeout so that if the flows do not see any new packets (e.g., flows are inactive) for more than the timeout, then they are assumed to be expired. Early work in [6] showed that shorter flow expiry timeouts tend to split longer flows into several short ones, thus smaller timeouts yield a larger number of flows, and a greater proportion of smaller flow lifetimes. Since observations using shorter or longer timeout did not affect the overall flow lifetime distributions or behaviors, our study uses a fixed interval of 64 seconds as a timeout value to expire flows. Here, host expiry timeout needs to be defined. We suggest two generic approaches: (1) expire a host immediately when all flows are expired, or (2) wait for an additional host inactive time (i.e., 64s) to finally expire a host. The first approach is to expire a host whenever it contains no flows, where a host can be inactive (i.e., no packets are seen from it) for as long as 64 seconds. The second approach is similar to that of flow expiry timeout where a host is to be expired when it contains no new flows for more than the timeout, i.e., a host can be inactive for as long as 128 seconds. We consider further studies of host expiry times could raise interesting observations, such as host-table memory tradeoffs. However, we find that both are valid approaches that do not affect the overall host behaviors in our study. Here, we use the second approach. Network Traces: we observe two one-day (24hr) traces from New Zealand campus networks: the University of Auckland captured using tcpdump (Auck06) and the University of Waikato (Wits06 [11]) captured using a DAG-based [1] monitor. A brief summary of the two traces is shown in Table 1. For Auck06, about 4.3 million total hosts are observed, where 1.3 million of them are unique hosts. This means that over half of

100

1.00

10‐4 hosts (top) flows (mid) packet (bottom)

10‐6 10‐8 0

A. Host arrival rates The packet inter-arrival times (IAT) distributions are well understood in local and wide area traffic [7, 8] where the consecutive (independent) packet arrivals follow a Poisson distribution. This implies that the IATs are exponentially distributed, which means that they appear as a straight line on a log-linear plot. Since flow IATs derive from the packets and host IATs derive from the flows, we expect the flow and host IATs to be the Poisson process as well. Figure 2(a) shows one-day log Complementary Cumulative Distribution Function (CCDF) plots for Wits06 of packet, flow and host IATs; we observe that all are approximately linear, i.e., they all have high proportions of small IATs and a decreasing number of long IATs. Similar to [8], we observe back-to-back IATs clustered (Figure 2(b)) around 5-6ߤs, exhibiting train-like behaviors. Figure 2(c) is a one-day time series plot of Auck06 showing the number of active hosts; we find that the high spike rises are caused by attack hosts from inbound networks (e.g., DDoS). These large numbers of active hosts are observed throughout the day and can reach as high as 70,000. Other than the high spikes generally, we observe a little less than 10,000 active hosts over time, and the plot shows that the number of active hosts increases during busy hours (e.g., between 7am and 1pm) and decreases during non-busy hours (e.g., between 1am to 5am). We also observe that almost all the entering hosts seemed to exit the network after short intervals as shown in Figure 2(d). Here, the most likely reason is that many hosts are short-lived and only a few are long-lived, generally causing the new hosts to be expired and thus dominating the plot. B. Host and flow lifetimes We measure the overall host lifetimes in network, and we considered two main questions. First, are the host lifetime distributions similar to that of the flows? Second, do the host lifetimes depend on some particular behavior? To address the first question, Figure 3 is a Cumulative Distribution Function (CDF) plot showing flow and host lifetimes. A lifetime less than 1ms shows that those hosts or flows lasted virtually zero seconds. The causes were studied in [10] and are mostly due to malicious single-packets flows, where a single-flow host contains only a single packet. In other words, a flow or host lifetime cannot be found since the traffic monitor requires at least two packets to compute duration. Such behaviors include portscans enumerating various hosts inside the campus network. Comparing flow lifetimes between our two traces, about 23% of flows are single-packet flows, Auck06 has 10% more

0.88 0.84

(b)

0.80

200 300 inter‐arrival time (ms)

400

500

0

10 20 30 40 inter‐arrival time (us)

50

(c)

45000 hosts

In this section, we observe various aspects of hosts entering the network. Specifically, we are interested to understand the arrival rates of (new) hosts, host lifetimes, and their re-entry process behaviors.

100

0.92

60000

30000 15000 0 1:00 PM

5:00 PM

9:00 PM 1:00 AM 5:00 AM Local Time (NZDT, UTC +1300) [24h]

9:00 AM

entry exit

7000 hosts

HOST ENTRY, EXIT AND RE-ENTRY

(d)

5000 3000 1000 13:00

13:15

13:30 Local Time (NZDT, UTC +1300) [1h]

13:45

14:00

Figure 2. (a,b) Wits06: inter-arrival times for packets (bottom), flows (mid) and hosts (top) showing approximate Poisson distribution, (c) Auck06: oneday active host counts, (d) Auck06: one-hour host entry/exit counts 1 Auck06 (hosts) Auck06 (flows) Wits06 (hosts) Wits06 (flows)

0.75

CDF

III.

0.96

(a)

10‐2

CCDF

CCDF (log)

the hosts only entered (or appeared) once. In general, we observe about 30 times more inbound hosts (i.e., hosts originating from outside the campus network) than outbound hosts. Also, TCP dominates the volumes of the network, and there was very little traffic other than TCP and UDP.

0.5

0.25

0 0.001

0.1

lifetime (s)

10

1000

Figure 3. CDF of flows and hosts vs their lifetimes

flows lasting 10s onwards (presumably indicating that Auck06 has more file-transfers). For host lifetimes, Auck06 has a higher proportion (51%) of single-packet hosts than Wits06 (40%). When compared to flows, this indicates that anomaly hosts contribute to a fairly large proportion of the total observed hosts, and many hosts seem to produce more than one flow. The rest of the host lifetime proportions are similar for both of the traces. For instance, both have about 24% of their hosts that lasted more than 10s, and about 0.5% of their hosts lasted more than 1000s. Additionally, a greater proportion of longer-lived hosts are observed than flows, presumably because there are several hosts producing more than one flow thereby lasting longer. Also, increasing the expiry timeout (for flows and hosts) to 256s (not shown) slightly increased the appearance of longer-lived hosts. Overall, we observe that flow and host lifetime distributions are similar for both networks. C. Host re-entry While hosts enter and exit, we find that not all hosts that exit expire permanently: some of the expired hosts regularly reenter the network. That is, hosts that were once inactive (e.g., 128s) and assumed to have expired, often reappear in the network. These host re-entry behaviors are not unusual, prompting

40%

1 Auck06 Wits06

(a)

Outbound Inbound

40%

(b)

(a)

20%

10%

percentages

0.75 CDF

percentages

30%

0.5

0.25

30%

20%

10%

outbound inbound 0%

0 1

2

4

8 16 32 64 128 256 512 number of re‐entries

0% 0

6

12 18 host idle time (h)

24

0%

Figure 4. (a) Percentages of re-entering hosts, (b) Auck06: CDF of idle time for re-entering hosts

1

D. Host connection and repetition ratio We now consider host connections in more detail. In particular, we speculate whether re-entering hosts communicate with previously-connected hosts. For example, if a host reappears in the network, does it connect to the previous hosts or to different hosts? Clearly, keeping all of the previous host records for each host over a whole day is infeasible since any host might connect to the previous destination IP addresses. Thus, we only keep track of each host’s previously connected IP addresses; when this host reappears we then match its destination IP address. In this, we find average ‘repetition ratios’ by matching the consecutive host’s reappearance. For instance, assuming that a host was communicating with one host and exited the network, and it now re-enters by connecting to the same host as previously, then the repetition ratio would be 100%, however if it was also connecting to two additional hosts, then the ratio would be 33%. In other words, 0% repetition ratio would mean that a host had not connected to any of the hosts that were previously connected, and 100% repetition ratio implies that a host connects to only to hosts that were connected to previously.

CDF

40% 60% repetition ratio

80%

[1..3] [4..15] [16..63] [64..255] [256..]

0.75

us to investigate host re-entry in more detail. Of all the unique hosts observed, about 20% and 40% of hosts reappeared in Auck06 and Wits06 respectively, and the rest of the hosts appeared only once. Figure 4(a) is a histogram plot (excluding the hosts that did not re-enter) showing percentages of hosts vs the number of times they reappeared. Both of the networks have similar contributions of re-entering hosts, e.g., more than 36% of hosts re-entered once, 23% of hosts re-entered twice and so on. Surprisingly, we also observe a tiny fraction of hosts that re-entered more than 512 times. Furthermore, a host that reenters often tends to last longer than one that re-enter less. We next measure the idle times of those hosts that re-enter the network. That is, we find the average idle times for each host by computing the consecutive time gaps between the last seen and re-entry times. Here, Figure 4(b) shows a CDF of hosts that re-entered the network vs their average idle time. Generally the shapes are similar for both outbound and inbound traffic. We find that more than 50% of hosts have reappeared within one hour, but we observe slightly asymmetric idle times between outbound and inbound hosts. That is, inbound hosts tend to reappear with rather longer idle times. For instance, 80% of outbound hosts reappeared within three hours but only 70% of inbound reappeared within three hours.

20%

100%

(b)

0.5

0.25

0 0

0.25

0.5 repetition ratio

0.75

1

Figure 5. Auck06: (a) outbound and inbound host repetition ratios, (b) CDF of host repetition ratios, separated by their re-entry counts

Our measured repetition ratio plots are shown in Figure 5(a). Several host behaviors are observed; two immediate observations are that about 24% of all hosts have a 0% repetition ratio, and 24% (outbound) and 43% (inbound) of all hosts have 100% repetition ratios. The repetition ratios other than 0% and 100% are seldom observed. Also, we observe that outbound hosts were much less likely than inbound hosts to have a 100% repetition ratio. That is due to the fact that many inbound hosts are DNS servers that frequently only connect to our dedicated outbound servers. Despite this, Figure 5(a) may be misleading, because many hosts only re-enter once or twice (as shown in Figure 4(a)), which could easily bias the overall distribution of repetition ratios. Figure 5(b) shows CDF repetition ratio plots for various host re-entry counts, exhibiting two effects. First, hosts that reentered between one and three times dominate the contributions and they resemble Figure 5(a). The rest of the hosts behave differently; hosts with a 0% repetition ratio make a less significant contribution. Second, at least 20% of all hosts that re-entered have 100% repetition ratios. For instance, some of the hosts that re-entered more than 64 times still connect to the same hosts as previously. These behaviors mostly suggest that if hosts re-enter the network a few times, then they are likely to return with either 0% or 100% repetition ratios. If they re-enter many times, then still about 20% of them would return 100% repetition ratio. To summarize, we observe that many short-lived hosts enter and exit the network immediately, and many of those contain only a single flow, and a single-packet. Furthermore, between 20% and 40% of unique hosts re-enter after they are assumed to have left the network. Some of those reappear as many as 512 times or more. Of those re-entry hosts, more than 50% re-enter within one hour and up to 80% re-enter within three hours. Also, many of these reappearing hosts either connect to the same hosts as previously, or to completely different hosts.

1

1

(a)

0.8

(c)

0.8

0.1 0.6

0.6

0.01

0.4 0.001

0.2

Auck06 Wits06 0.0001

CDF

1..3 4..15 16..63 64..255 256..1023 1024..

CDF

number of hosts (fractions)

1

(b)

0.4

0.2

0 1..3

4..15

16..63 64..255 per‐host flows

256..1023

1024..

0 0.001

0.1

10 lifetime (s)

1000

100000

0.001

0.1

10 lifetime (s)

1000

100000

Figure 6. (a) Distribution of per-host flows, (b) Auck06: Host lifetime distributions of per-host flows, (c) Wits06: As for Figure 6(b), but with anomaly hosts excluded. Note that when plotted using per-host sizes and per-host host-connections, the distributions are similar to per-host flows.

IV.

HOST OBSERVATIONS AND CORRELATIONS

In this section, we separate groups of hosts into several range sets; flow counts, sizes (i.e., bytes transferred) and host connections, according to their lifetime distributions. We then attempt to find various correlations for host attributes. A. Host distributions: flow counts, sizes, host connections Figure 6(a) shows percentages of flows produced by the hosts; we observe that more than 90% of hosts produced between one and three flows and then expired. Overall, host lifetime distributions for both traces are similar, including the outbound and inbound hosts. In particular, hosts that produced one flow with only a single packet were most common. Also, there are very few hosts that produced many flows. For instance, less than 0.1% produced between 4 and 15 flows and less than 0.001% of hosts produced more than 256 flows. Similarly, Figure 6(b,c) show six different sets of flows vs their lifetimes, ranging from hosts that produced from one flow to 1024 flows or more. Here, Figure 6(b) is a separation of host lifetimes by their flows from Figure 6(a) of Auck06 (distributions for Wits06 were very similar and are thus not shown). As mentioned previously, about 51% of all hosts lasted zero seconds and contained only a single-packet flow. The rest of the hosts producing between 4 and 63 flows seemed to follow a lognormal distribution, e.g., there are a few hosts that had short or long lifetimes, and we clearly observe that hosts with higher flow counts last longer. That is, the number of flows per host seems to be correlated with their lifetimes. However, hosts producing more than 256 flows (and 1024 flows) do not follow a similar distribution; instead, lifetimes up to 0.5s contributed more than 10%. Such behaviors were not expected since it is abnormal for hosts to produce many flows in such a short time; this behavior was apparent not only in Auck06, but also in Wits06. To further investigate the varieties of the hosts so as to remove such anomalies, we first inspect and eliminate hosts with single-packet flow (lasting zero seconds), then also eliminate hosts from the distribution if they contained more than 90% one-way flows. We believe that if the hosts produce legitimate flows, then they should produce two-way flows instead of single-packet flows. Figure 6(c) shows the Wits06 trace with the anomalies eliminated; now all the host lifetime distributions appear to be approximately lognormal distributions. In general the top and bottom 5% of their

plots suggest that there are small proportions of hosts that last shorter (or longer) than the majority of a given set of flows. Further, when we separate hosts by their sizes and by their number of host connections vs their lifetimes, we find similar observations to that of Figure 6 (those plots are not shown). For example, we observe that a host with small size tends to be short-lived, and a host that connects to many other hosts tends to be long-lived. B. Host attribute correlations In a flow analysis, the authors in [13] considered three flow attributes (size, rate, duration), and found that flow size and rates are strongly correlated. Again, since the rate is not defined for single-packet flows, we ignore hosts that last less than 5s, with size less than 1kB. Due to the large range of attribute values, we compute their log correlation. More precisely, we compare hosts with seven attributes. Thus, each host has: (1) size transmitted in bytes, (2) number of flows produced, (3) number of hosts connected, (4) size-rate (bytes per second), (5) flow-rate (flows per second), (6) host-rate (hosts per second), and (7) lifetime (duration). Table II shows correlation matrices for these attributes in our two networks. All hosts are first listed in the row field with their seven variables listed in the column field, to compute the correlation coefficients for every combination. Thus, each field in the table represents coefficient values for two attributes, showing Auck06 (left) and Wits06 (right). In general, both networks are found to be similar. Note that a correlation coefficient assumes two variables’ linearity, and whether these are strongly or weakly correlated are not dependent on the actual number (e.g., 1, 0, -1), but in comparison with other coefficients. Here, we consider that attribute pairs are highly correlated if they are close to or more than 0.5. Although, it is appealing to extract individual hosts to find out exact causes and reasons, we discuss only a few general observations of host correlations. Positively correlated: We find that numerous host attribute pairs are positive correlated. In particular, ‘size vs size-rate’ is highly correlated, exhibiting close results as in [13], i.e., a host transmitting a large size also tends to do so at a fast rate. Also, ‘flow vs host’ shows that a host producing multiple flows is likely to connect to multiple hosts. Another highly correlated attribute pair is ‘flow vs lifetime’ which shows that longerlived hosts produce more flows, whereas a ‘host vs lifetime’ is

TABLE II. Log-Scale size (1) flow (2) host (3) size-rate (4) flow-rate (5) host-rate (6) lifetime (7)

SUMMARY OF CORRELATION COEFFICIENT FOR [AUCK06], [WITS06]

PH: Highly-Positive Correlated, PC: Positive Correlated, NH: Highly-Negative Correlated, NC: Negative Correlated, UC: Uncorrelated size (1) flow (2) host (3) size-rate (4) flow-rate (5) host-rate (6) lifetime (7) --------------PC [0.41], [0.27] ------------UC [0.03], [-0.10] PH [0.53], [0.57] ----------PH [0.72], [0.70] UC [0.06], [-0.19] NC [-0.17], [-0.36] --------UC [0.11], [0.08] PC [0.43], [0.38] PC [0.25], [0.20] PC [0.35], [0.29] ------UC [-0.10], [-0.09] UC [0.03], [-0.08] PC [0.39], [0.27] PC [0.19], [0.28] PH [0.62], [0.56] ----PC [0.32], [0.16] PH [0.45], [0.58] PC [0.28], [0.39] NH [-0.43], [-0.58] NH [-0.43], [-0.58] NH [-0.40], [-0.51] ---

only weakly correlated. Such behaviors are presumably because often users browse a few websites (host connections), but individual web-objects generate new flows during their lifetime. Also, ‘flow-rate vs host-rate’ is highly correlated, showing that the rate that a host produces flows also increases with the rate of host connections. This is because many short-lived hosts ‘burst out’ connections. Negatively correlated: We observe that all size-, flow- and host-rates are highly (negative) correlated with their lifetimes. This shows that a host transmitting files at a high rate is likely to last a shorter time, or similarly, a host with a high flow/host-rate tends to last a shorter time. Also, we observe that ‘host vs size-rate’ are weakly correlated. For instance, a host transmitting files at a high rate tends to connect fewer hosts (e.g., a user decides to wait for transmissions and does not attempt newer host connections). We observe that ‘size vs host’, ‘size vs flow-rate’ and ‘size vs host-rate’ are uncorrelated. Presumably, a user’s file sizes may or may not affect the number of host connections, or the rate of flows. Similarly, ‘flow vs size-rate’ and ‘flow vs hostrate’ are also uncorrelated. We repeated the correlation test again with the anomaly hosts removed (Section IV-A) and observe several differences. Although overall coefficients are not greatly changed, we observe three pairs of attributes that changed significantly. First, ‘flow vs host-rate’ changed from uncorrelated to negative correlated. Second, ‘host vs host-rate’ changed from positive correlated to uncorrelated. Lastly, ‘hostrate vs lifetime’ became even more highly-negative correlated to about -0.75. These changes demonstrate the various behaviors that anomalies could cause. To summarize, a host producing various flows, sizes and host connections seems to be related with their lifetimes, for example, a long-lived host tends to produce many flows, transmit large sizes and connects to multiple hosts. Also, we observe a few exceptions such as anomaly host behaviors producing numerous flows that are short-lived. Further, our correlation matrix shows host dynamics in various ways, for instance, ‘size vs size-rate’ is highly-positive correlated, all the rates are negatively correlated with their lifetimes, and there are a few differences between anomaly hosts inclusion and exclusion. V.

CONCLUSIONS AND FURTHER WORK

In this paper, we present host-based analysis and compare it to the more typical flow-based measurements. In particular, we find that that hosts are highly dynamic, and need to be better understood. We examined various aspects of host behaviors

such as host re-entry and lifetimes. For instance, many hosts are short-lived, at least 20% to 40% of hosts re-enters the network, with many hosts reappearing within a few hours, and those hosts seemed to either connect to the same previous hosts or to completely different hosts. Furthermore, our host attribute correlations showed various relationships, for example, host size and size-rate are highly positive-correlated. Further work is necessary as we think that our host-based measurement is an ongoing project leading to further analysis of host behaviors. In particular, we think that grouping the hosts by associating host-to-host relationships allows a more comprehensive analysis. Also, our work has not conducted host traffic modeling which could better help us to understand host behaviors. ACKNOWLEDGEMENTS The authors are thankful to Perry Lorier from the University of Waikato [11] for providing Wits06 traces and helpful information regarding them.

REFERENCES [1] [2]

[3] [4] [5]

[6]

[7]

[8]

[9]

[10] [11] [12]

[13]

"Endace," http://www.endace.com. P. Barford and D. Plonka, "Characteristics of network traffic flow anomalies," in Proceedings of the 1st ACM SIGCOMM Workshop on Internet Measurement San Francisco, California, USA: ACM Press, 2001. N. Brownlee, "Some Observations of Internet Stream Lifetimes," in Passive and Active Measurement Conference (PAM) Boston, MA, USA, 2005, pp. 265-277. N. Brownlee and K. C. Claffy, "Internet Measurement," IEEE Internet Computing, vol. 8, pp. 30-33, 2004. N. Brownlee and K. C. Claffy, "Understanding Internet traffic streams: dragonflies and tortoises," Communications Magazine, IEEE, vol. 40, pp. 110-117, 2002. K. C. Claffy, H. W. Braun, and G. C. Polyzos, "A parameterizable methodology for Internet traffic flow profiling," Selected Areas in Communications, IEEE Journal on, vol. 13, pp. 1481-1494, 1995. R. Jain and S. Routhier, "Packet Trains--Measurements and a New Model for Computer Network Traffic," Selected Areas in Communications, IEEE Journal on, vol. 4, pp. 986-995, 1986. T. Karagiannis, M. Molle, and M. Faloutsos, "Long-Range Dependence: Ten Years of Internet Traffic Modeling," IEEE Internet Computing, vol. 8, pp. 57-64, 2004. A. Lakhina, M. Crovella, and C. Diot, "Characterization of network-wide anomalies in traffic flows," in Proceedings of the 4th ACM SIGCOMM conference on Internet measurement Taormina, Sicily, Italy: ACM Press, 2004. D. Lee and N. Brownlee, "Passive measurement of one-way and two-way flow lifetimes," SIGCOMM Comput. Commun. Rev., vol. 37, pp. 17-28, 2007. R. Nelson, D. Lawson, and P. Lorier, "Analysis of long duration traces," SIGCOMM Comput. Commun. Rev., vol. 35, pp. 45-52, 2005. J. Wallerich, H. Dreger, A. Feldmann, B. Krishnamurthy, and W. Willinger, "A methodology for studying persistency aspects of internet flows," SIGCOMM Comput. Commun. Rev., vol. 35, pp. 23-36, 2005. Y. Zhang, L. Breslau, V. Paxson, and S. Shenker, "On the characteristics and origins of internet flow rates," in Proceedings of the 2002 conference on Applications, technologies, architectures, and protocols for computer communications Pittsburgh, Pennsylvania, USA: ACM Press, 2002.

Host Measurement of Network Traffic

Host Measurement of Network Traffic. DongJin Lee and Nevil Brownlee. Department of Computer Science. The University of Auckland.

277KB Sizes 1 Downloads 263 Views

Recommend Documents

Splitting and merging of packet traffic: Measurement ...
Aug 15, 2005 - 1. Router diagram illustrating the multiplexing of input streams contributing to the output link C2-out. ...... He spent 1 year at the CNET of France Telecom in ... She received her master of science degree in computer science ...

Quality Monitoring of Network Traffic Forecasts using ...
Abstract— The literature of network traffic analysis has investigated different forecasting methods applied to traffic engineering in computer and communication networks. Several sophisticated models have been proposed, evaluated and compared, show

InBrowser Network Performance Measurement - World Wide Web ...
Oct 29, 2012 - provides application developers a best practice for understanding their ..... We run the experiment 10 times for each browser for a total of 1,000 ...... use a machine running Ubuntu 12.04 to host a custom HTTP server written in.

InBrowser Network Performance Measurement - World Wide Web ...
Oct 29, 2012 - standalone measurement software, on the other hand, presents portability as well as ... The popularity of iOS with its lack of Flash support (Android is also not ..... download from servers that are relatively close to endusers.

Using Trustworthy Host-Based Information in the Network
18 Nov 2009 - Once authorized by a verifier, the code module can insert cryptographically- secured information into outbound traffic. This information is checked and ...... ±990 (+0.2%). 6631. ±721 ( +12.8%). Up L. 706.5 ±61.4. 777.4 ±153 (+10.0%

Using Trustworthy Host-Based Information in the Network
Nov 18, 2009 - Once authorized by a verifier, the code module can insert cryptographically- secured information into outbound traffic. This information is checked and acted on by in-path filters. ... veying host-based information to the network, we i

Multi Utility Sensor Network: Host Software Cover Page
Jul 26, 2005 - can communicate with other nodes and the host over the RS-485 Bus. MAX485 ... The Web server runs a PHP based application that is Ajax enabled. The PHP ...... 10, SWT.BOLD));. sShell.setSize(new org.eclipse.swt.graphics. ...... Elecram

Network Connectivity Graph for Malicious Traffic Dissection - PORTO ...
For instance, the same host could visit a legitimate web page, poll the mail server, and .... Algorithm 1 Create Network Connectivity Graph. input args s: seed.

A Network Traffic Reduction Method for Cooperative ...
Wireless positioning has been providing location-based ser- vices in ... Let us consider a wireless network with two types of ..... Cambridge University Press,.

Filtering Network Traffic Based on Protocol ... - Fulvio Risso
Let's put the two together and create a new automaton that models our filter tcp in ip* in ipv6 in ethernet startproto ethernet ip ipv6 tcp http udp dns. Q0. Q3. Q1.

A Network Traffic Reduction Method for Cooperative ...
[11], our work in [9] operates across the physical layer and the medium access layer. In this paper, we extend our previous work by developing a network traffic ...

traffic grooming in wdm network using ilp
WDM Networks. [4,5,6].However most of them have formulated the traffic grooming problem for different networks as an ILP problem. For larger networks, ILP ...

Network Connectivity Graph for Malicious Traffic ...
poll the mail server, and upload files on a cloud storage service, ... 1. Example of events generated by a host as seen from the network. Graph (CG), which ...

Traffic Grooming in WDM Network Using ILP - Semantic Scholar
Sitesh Shrivastava. Department of Computer Science, ... (2) Dense WDM with nearly 160 wavelengths per fiber ... In recent years, Dense Wavelength Division Multiplexing is widely ..... Sitesh Shrivastava is at present pursuing his B.E. degree.

Filtering Network Traffic Based on Protocol Encapsulation Rules
Fulvio Risso, Politecnico di Torino – ICNC 2013. 1/15. Filtering Network Traffic Based on. Protocol Encapsulation Rules. Fulvio Risso, Politecnico di Torino, Italy.

[READ] Mobi Router Security Strategies: Securing IP Network Traffic ...
section provides case studies from both the enterprise network and the service provider network perspectives. In this way, the individual IP traffic plane security techniques reviewed in the second section of the book are brought together to help you

Traffic Grooming in WDM Network Using ILP - Semantic Scholar
1. Traffic Grooming in WDM Network Using ILP. Partha Paul. Dept. of Computer Science,. Birla Institute of Technology, Mesra, Ranchi, India. Sitesh Shrivastava.

network access traffic manager project report
3.3 Video Conferencing Traffic rates(384kbps Session Example . . . . . . . . ..... Such applications include FTP, e-mail, backup operations, database synchronizing.

Traffic Based Clustering in Wireless Sensor Network
Traffic Based Clustering in Wireless Sensor. Network ... Indian Institute of Information Technology ... Abstract- To increase the lifetime and scalability of a wireless.