The impact of excluding common blocks for approximate matching

back to overview


Moia, V. H., Breitinger, . F., & Henriques, M. A. (2020). The impact of excluding common blocks for approximate matching. Computers & Security, 89. (ISI_2016: 3.062; ISI_2016_5year: 3.476)

Publication type

Article in Scientific Journal


Approximate matching functions allow the identification of similarity (bytewise level) in a very efficient way, by creating and comparing compact representations of objects (a.k.a. digests). However, many similarity matches occur due to common data that repeats over many different files and consist of inner structure, header and footer information, color tables, font specifications, etc.; data created by applications and not generated by users. Most of the times, this sort of information is less relevant from an investigator perspective and should be avoided. In this work, we show how the common data can be identified and filtered out by using approximate matching, as well as how they are spread over different file types and their frequency. We assess the impact on similarity when removing it (i.e., in the number of matches) and the effects on performance. Our results show that for a small price on performance, a reduction about 87% on the number of matches can be achieved when removing such data.


Organizational Units

  • Institute of Information Systems
  • Hilti Chair for Data and Application Security

Original Source URL