Toward Measurement of IP Hosts DongJin Lee and Nevil Brownlee Department of Computer Science The University of Auckland [email protected] , [email protected]

Introduction

< srcIP, dstIP, srcPort, dstPort, protocol > … < A, B, 2222, 80, 6 > < B, A, 80, 2222, 6 > < A, B, 2223, 80, 6 > …

Hosts

Internet traffic continues to grow, in step with the number of Internet users and the bandwidth available to them. Since 1995 we have used 5-tuple IP flows [1] as a way to reduce packet data and analyze traffic. Today operators have many tools available to produce and analyze flow data, e.g. those based on NetFlow [2] and IPFIX [3]. Considering the traffic in terms of flows, however, provides less understanding of how network hosts behave. Every host will have interactions with at least one other host, those interactions are sustained by a series of flows between the two hosts. On a larger scale, a host will interact with several other hosts. Analyzing network traffic per-host could narrow the focus toward a user perspective, as the flows are grouped by an even more common characteristic (i.e., a common end-point). This would allow network operators to work on higher abstract levels, as well as producing simpler observations with a more diverse host behavior analysis. A growing number of ISPs are now interested in understanding user behaviors so as to manage their services, and to detect behavior changes that might affect their revenue. For example, in addition to just byte counts, operators might also prioritize their network traffic based on keeping ‘per-host’ states in their management system. We consider that host behaviors need to be better understood. Here, we propose a study of host behaviors in the context of network traffic measurement. Rather than solely building on 5-tuple flows, we further aggregate them to higher levels; 2-tuple interactions and 1-tuple hosts so as to gather richer information from the network. In particular, we study the lifetime of hosts and their reentry behaviors. Our study demonstrates the feasibility of using a higher abstraction method, in conjunction of flow studies, to better understand network traffic. In our further work, we plan to analyze in-depth host behaviors that can benefit network researchers and operators.

1-tuple

Interactions 2-tuple Flows 5-tuple

H-Table … A B … …

I-Table … A B … …

F-Table … 2222:80:6 80:222:6 2223:80:6

… Packets

Fig. 1 Left: An illustration of hosts, interactions and flows. Right: An example of table manipulation

Constructing Hosts Our proposed aggregation scheme may be taken at different levels, e.g., traffic-meter or manager. It may also work directly with exported flow-data or raw packets. Network traffic at its lowest level consists of packets as shown in Fig.1(left). An interaction is a group of the 5tuple flows that shares the same 2-tuple source and destination IP address. Similarly, a host is a group of the interactions that shares the same 1-tuple source IP address. We aggregate packets to form flows, aggregate those into interactions, and finally into hosts. Thus, each host could produce multiple interactions (referring to multiple entries in the interaction table), and each interaction could have multiple flows (referring to multiple entries in the flow table). Fig.1(right) is an example of table manipulation. When the packets for a flow are no longer seen for a particular (fixed) expiry time [1], our meter assumes that the flow is inactive, and can be expired from the memory . To maintain consistency with the 5-tuple flow definition, we follow a similar concept for interaction and host. 2tuple interaction is assumed to be inactive when the flows for it are no longer seen after a particular expiry time. Similarly, a 1-tuple host is assumed to be inactive when the interactions for it are no longer seen after a particular expiry time. We use 30s, 60s and 120s as expiry times for flows, interactions and hosts respectively. We find that this setting is effective, and different expiry times hardly change our results.

0.4

flow flow_size interaction interaction_size host host_sizes

0.8

0.3

CDF

0.6

Fractions

1 min

1 hour

0.4

0.4

Auck07 Bell‐I Leip‐II NZIX‐II Wits06

0.2

Auck07 [64%] Bell‐I [79%] Leip‐II [63%] NZIX‐II [86%] Wits06 [63%]

0.3 Fractions

1

0.2

0.1

0.1

0

0

1 sec

0.2

0.5 1 day 0 0.001

0.1

10 lifetime (s)

1000

100000

Re‐entry

Re‐entry 

Fig. 2 Left: Cumulative distributions of Auck07 flow, interaction and host numbers and sizes. Center: Fractions of host Reentry. Right: Fractions of Reentry host volume.

Host Lifetime and Reentry Behavior Fig.2(left) shows one-day CDF plot of flow, interaction and host numbers and sizes vs their lifetimes. We observe that many flows are short-lived contributing very small volume (dragonflies [4]), and similarly for the 2-tuple interactions. Generally, since hosts produce more than one flow, their lifetimes also tend to be longer lived than the flows. Yet, observing long-lived 1-tuple hosts, the distributions are radically different; only a tiny fraction (0.01%) of hosts contribute a very large fraction of traffic volume, i.e., up to 60% of volume is contributed by the hosts that lasted for more than 12 hours. We also find that such elephant hosts are responsible for most of the (many) elephant flows. We note that network traffic now contains enormous amounts of malicious traffic. In particular, flows containing only a single packet contribute the majority of mice traffic. Their lifetime is also 0s since the traffic meter requires at least two packets to compute a flow’s duration. On our measured network, between 38% and 72% of the total (unique IP) hosts re-enter the network. Also, between 63% and 86% of total volume originates from the reentering hosts. Fig. 2(center) is a histogram plot showing fractions of hosts and their re-entry counts. A large host proportion re-enters once, fewer re-enter twice, and so on. Intriguingly there are hosts that reentered more than 512 times. Also, analyzing longer traces (Bell-I and NZIX-II) clearly shows that hosts tend to re-enter more than just once. We further observe that NZIX-II had an unusually high number of hosts that re-enter between 64 and 127 times; presumably because of some network-specific host behaviors. Fig. 2(right) plots those re-entered hosts’ volume contribution. As mentioned, their volumes are not insignificant; this network view shows that volumes are not just dominated by long-lived hosts, but bandwidth is mostly consumed by those re-entering hosts. Also, there are no clear volume distributions contributed by the re-entering hosts. In Bell-I, for instance, about 30% of total volumes are consumed by hosts that re-entered for between 4 and 7 times. We also measure idle time for those re-entered hosts. We find that at most 20% of hosts are likely to return in 30

minutes, and 80% return in 12 hours. Clearly, longer analyzed traces show more contributions of longer idle times. We also measure each host's interactions when it re-enters the network, to observe whether it interacts with the same hosts or different hosts, by matching its destination IP addresses. In this, we find that many hosts only interact with hosts that they have already interacted with. For example, a user visits only certain websites, or a user repeatedly interacts with popular file-servers.

Summary We consider our measurement scheme can potentially reveal several significant host behaviors. Given that every flow originates from some host IP addresses, it is important to measure host behavior as a whole. Rather than studying only at a flow level, our scheme incorporates a bigger picture. We have analyzed several different aspects of hosts, e.g., correlations, and find that host behaviors are a lot more unpredictable than flow behaviors. Analyzing at a host-level provides a new approach to understanding network behavior, which should be of interest to both researchers and network operators. Nevertheless, understanding host, interaction and flow relationships is a challenging study, and one we plan to continue in the future.

Reference [1] K. C. Claffy, H. W. Braun, and G. C. Polyzos, "A parameterizable methodology for Internet traffic flow profiling," Selected Areas in Communications, IEEE Journal on, vol. 13, pp. 1481-1494, 1995. [2] "Cisco IOS NetFlow," http://www.cisco.com/en/US/products/ps6601/products_ios_pr otocol_group_home.html. [3] "IP Flow Information Export (ipfix) Charter," http://www.ietf.org/html.charters/ipfix-charter.html. [4] N. Brownlee and K. C. Claffy, "Understanding Internet traffic streams: dragonflies and tortoises," Communications Magazine, IEEE, vol. 40, pp. 110-117, 2002.