Generic Host Behaviour

Get Complete Project Material File(s) Now! »

ONE–WAY AND TWO–WAY FLOWS

Introduction and Motivation

The majority of the communications between IP hosts are bidirectional (packets are exchanged in both directions). A TCP connection by default is a bidirectional flow since it relies on data packets being acknowledged. While other protocols (such as UDP) may not acknowledge messages, communicating IP hosts mostly recognize each other by exchanging packets at some point in time. In the edge network where both inbound and outbound packets can be observed by the traffic meter, two flow types are observed: unidirectional and bidirectional flows. Understanding flow directions can provide insights into the application behaviours, such as flow durations and anomalies. We find that most short-lived flows are the packets travelling only in one direction. These flows often indicate anomalies; a malicious host producing a large numbers of nonresponsive flows (e.g. port-scans, DDoS). Experiments conducted by Barford and Plonka (2001) characterized traffic flow anomalies by separating the flows as inbound vs outbound. Also Kreibich et al. (2005) examined the ratio of inbound and outbound packet transmission (Packet Symmetry) to restrain malicious traffic.
This chapter therefore explores a different approach to gain insights into flow lifetimes from the host aspect by aggregating packets into two types of flow. Other than known flow characteristics (e.g. mice, tortoises) aforementioned, they may have another dimension in their connection patterns. In this, we study a flow aggregation technique that focuses on one-way (unidirectional) and two-way (bidirectional) flows and their changes over the years. We find that previous studies do not consider flow directionality in lifetime observation and we believe that separating two flow types can yield insights into their behaviours. In particular, we examine two notions in this chapter: 1) How the network IP hosts may produce one-way and two-way flows, leading toward a different perception of 5-tuple aggregation. 2) Changes of flow sizes in bytes/packets, their lifetimes, and their applications/protocols.

Examples of One-way and Two-way Flows

As illustrated earlier in Figure 2.7, our traffic meter regards flows for which the packets travel only in one direction as one-way. We use a timeout of 64 seconds to expire the flows. Though it is inevitable to encounter the traffic meter’s ‘start-up’ and ‘finish-off’ effect²⁰, three broad situations could trigger one-way flows. 1) A host may be unable to respond to a transmission when it encounters a system halt (e.g. busy CPU cycles, erroneous protocol implementation) or physical faults (e.g. shutdown, network cable cut-off). Similarly, a traffic meter may fail to capture all of the packets during traffic peaks. 2) There are applications that mostly send or receive packets in one direction only; a few or no packets are produced from the other side. DNS or ICMP packets may follow this type of behaviour when the destination hosts do not respond, e.g. due to malformed DNS requests, or routers rejecting an ICMP request. 3) There are packets passing the links that are blocked by NAT devices, gateway firewalls or by hosts’ software firewalls, e.g. restricted applications such as p2p or malicious flows.
Conversely, a one-way flow becomes a two-way flow when the traffic meter observes packets for it in both directions (e.g. request/response packets). Many applications use either TCP or UDP to communicate between a pair of hosts. For example, an FTP transaction (port 20 and 21) flows would be four unidirectional flows; two for the sender and other two for the receiver. Since these flows are bidirectional, only two two-way flow entries are made in our table (when configured for bidirectional match as illustrated in Figure 2.4.1).
Note that network topology can affect the flow types. If links are asymmetric routed or load balanced, then the traffic meter may only observe in one direction or misses out portions of packet streams due to route changes. This can affect the way the meter measures about end-to-end transaction, and we consider that it is useful to address network topology so as to minimize erroneous inference.

One Day Observations

Figure 3.1 [top] shows traffic plotted by NLANR PMA for an active connection from Auck-03 with a high spike of TCP flows observed at about 10:30 NZDT. However, we find that the vertical axis was not actually ‘active connection’ as indicated on the NLANR plot, but the number of active (unidirectional) TCP flows. The same dataset is measured using our approach, showing the active flow counts of one-way and two-way flows (Figure 3.1 [centre]). The high spikes which lasted for 9 minutes were caused by both inbound malicious flows and by our outbound proxy hosts being shut down, causing many hosts to produce (retry) one-way flows. The other smaller spikes were also identified as one-way flows. In total, these spikes caused more than 50% of total flows, and about four times as many one-way flows appeared from the inbound traffic as compared to outbound.
As years pass, more spikes are frequently observed. Figure 3.1 [bottom] shows that an enormous number of single-packet UDP flows appeared in 2006, contributing more than 70% of the total one-way flows. This behaviour was observed by Brownlee (2005) as plagues of dragonflies, a sudden rise of short-lived flows. Closer analysis has revealed that all of the high spikes were caused by attack flows, widely known malicious packets targeting UDP port 1026-1029, 135, 445, 4899 and so on (enumerating various hosts inside our network). These scanning flow sizes varied between 200 to 1000 bytes. Also, a few attack flows that were using well-known ports (e.g. 25, 53 and 80) were discovered.

1. Introduction
1.1. Problem Statement and Motivation
1.2. Contributions and Overview
1.3. Background
1.4. Network Traffic Flow
1.5. Empirical and Theoretical Models
1.6. Summary
2. Host Conceptual Model
2.1. Generic Host Behaviour
2.2. Aggregation
2.3. Tuple consideration
2.4. Conceptual Implementation
2.5. Expiry Timeout
2.6. Observation
2.7. Selected Hosts Observation
2.8. Variation
2.9. Summary
3. One-way and Two-way Flows
3.1. Introduction and Motivation
3.2. Examples of One-way and Two-way Flows
3.3. One Day Observations
3.4. Distributions
3.5. One-way Flow Lifetime and Volume
3.6. Two-way Flow Lifetime and Volume
3.7. Summary
4. IP Host Measurement
4.1. Introduction and Motivation
4.2. Host Arrivals
4.3. Host Re-entry
4.4. Per-Host, Interaction and Flow Observation
4.5. Host Lifetime
4.6. Host Attribute Correlations
4.7. Summary
5. Host Interaction Variation
5.1. Introduction and Motivation
5.2. Related Study
5.3. Host and Interaction
5.4. Interaction Variations
5.5. Correlations
5.6. Application Examples
5.7. Summary
6. Extracting Significant IP Hosts
6.1. Introduction and Motivation
6.2. Related Study
6.3. Concept of Significant Host
6.4. Host Score Observations
6.5. Score and Rank Variations
6.6. Selecting significant hosts
6.7. Attribute Weights
6.8. Summary
7. Conclusion 
7.1. IP Host Measurement
7.2. Host Application
7.3. Future Work
GET THE COMPLETE PROJECT
Toward Empirical IP Host Traffic Measurementin Passive Network Measurement