Tapping multiple points across a network can result in the same packet being copied several times, particularly with East/West traffic and traffic from SPAN ports. This can negatively affect the performance of various tools as they not only need to process the same traffic multiple times, but in many cases further expend processing resources performing deduplication themselves. In instances where packet data is being recording to storage or a ring buffer, duplicate packets simply consume space without additional benefit. In some cases, the duplicate packets could even cause false positives or skew reporting.
The Solution From Cubro
Cubro provides several options capable of performing deduplication, according to user-selectable parameters, prior to forwarding traffic to monitoring tools. This offers customers flexibility in selecting units that meet their needs for port density and bandwidth without expending unnecessary budget. Offloading this resource intensive task frees other tools processing resources leading to increased efficiency in performing the functions they were designed to handle.
The Cubro EXA8 is a versatile and cost-effective device that offers deduplication for 10Gbps links to eliminate identical packets and thus protect monitoring equipment from being overloaded. With the deduplication function enabled, the EXA8 will calculate a checksum, or hash-key, using the full packet. Using every bit in the packet ensures that the resulting hash represents the maximum amount of uniqueness in the packet. This is important in preventing a different packet, that varies in only the slightest way, from being discarded because the hashing algorithm only used a subset of the packet data. The hash-key itself is an MD5 calculation and results in a 16 byte long string. Hash-keys are stored (one per packet) and every incoming packet has its hash-key checked against the hashes stored in memory. If the hash of an arriving packet is found to have a match already stored in memory the incoming packet is dropped, ensuring that duplicated packets within the specified time windows are not forwarded to the output.
For more information, read the application note
- Improved efficiency of monitoring and analytic tools.
- Maintain accuracy of tool reporting
- Offload function from tools to conserve processing resources