cubro blog headerpic with technical details, icons

Useless Feature in a Network Packet Broker - Deduplication

Useless Feature in a Network Packet Broker - Deduplication

Additional features are typically used in any industry as a means to give a product more apparent value but, of course, for a much higher price.

Example: A base-model car has 5 speakers, but you can upgrade to 8 or even 12 speakers. Did you really need them though? In a car, it is a question of the luxury feeling whether to pay a premium on top.

A network packet broker is a tool, it should do its job well and in the background. Let's look at the packet deduplication feature of a network packet broker.


Packet deduplication is an important application which should be done by the network packet broker. Safe removal of redundant data to save time and processing power. Removing duplicate traffic improves monitoring tool efficiency, accuracy, and recording space requirements while lowering overall costs.


The general idea is not a bad one but it is technically very difficult, costs a lot of money, and can be altogether avoided in an intelligent TAP deployment design! 


1:) There is only one real situation that produces duplicate traffic and necessitates this feature.

"When you use SPAN ports from active network elements like routers, (no switches, see next slide) the problem results due to the fact that they can change the L3 information, in this situation no other filter option will work."

2:) If TAPs and SPAN ports are used, duplicates can be avoided or removed by simple L3 filters. Yes, sometimes this will result in more filters but a good NPB will always have enough filters to support this.

3:) Deduplication must be done in CPU modules which costs money and creates a lot of issues, especially when it comes to highly loaded links. Typically deduplication only works on 10 Gbit or slower links because of performance problems.

Load Balancing will not help because if you have the same traffic with different L3 information, the traffic will most likely be load balanced to two different ports, in which case there are no duplicates on these ports to deduplicate in the first place.

4:) The biggest issue is that if something goes wrong in deduplication your troubleshooting solutions will never find the issue. This is because the traffic is being compromised by the tool which is supposed to be helping you!


That is the reason why Cubro's stance is to avoid this feature! It is possible, ask our experts. 

Leave a comment

You are commenting as guest.

always up to date stempel

  • Cubro Network Visibility
  • Ghegastraße 3, 1030 Vienna Austria
  • Tel.:+43 1 29826660
  • Fax: +43 1 2982666399
  • Email: This email address is being protected from spambots. You need JavaScript enabled to view it.