18

I have been trying to find information on fuzzy hashes, and I have found surprisingly little.

I know that fuzzy hashes can be used to detect if a piece of malware has changed slightly. I would rather not develop my own algorithm since that seems like a terrible idea.

I found ssdeep which looks interesting, but I don't see any alternative.

Are there other fuzzy hash algorithms? Does anyone use these?

Jeff
  • 666
  • 1
  • 6
  • 12
  • 1
    To migrating voters - The migration guideline state that you should not vote to migration questions with answers. – Xander Mar 18 '16 at 18:29
  • @Xander That's only if there are answers that fully cover the topic, which is not the case — unsurprisingly, because this question is too broad. There is a reason why this question should not be migrated, which is that it is too broad. The only way to satisfactorily answer it would be to list all known fuzzy hashing algorithms. If the question mentioned a specific application in mind, it might be answerable (and it would be on-topic here, too). “Detect if a piece of malware has changed slightly” doesn't cut it: whole books can be written on malware recognition. – Gilles 'SO- stop being evil' Mar 18 '16 at 20:11
  • @Gilles how should I rework this question? I need to test a number of different fuzzy hashing algorithms and their effectiveness. I was only able to find ssdeep, and even after gave few suggestions, I wasn't able to implement those yet. – Jeff Mar 21 '16 at 14:23

1 Answers1

12

Here are a few other "fuzzy hash" algorithms:

  1. Nilsimsa: was developed for email spam detection. However it results in more false positives then some others.

  2. TLSH: (https://github.com/trendmicro/tlsh) A Trend Micro fuzzy hash.

  3. Sdhash: another fuzzy hash like algorithm. http://roussev.net/sdhash/sdhash.html

These are used for several things. One is to forensically locate similar files. Another is to identify malware, even if some of the code is changed. A third use is in spam detection

AstroDan
  • 2,238
  • 15
  • 25
  • Is a rolling hash the same thing? – Jeff Mar 18 '16 at 18:13
  • Similar, rolling hashes are often to match subsections (think plagiarism detection, searching for similar matches) while fuzzy hashes are used to compare how different larger sets are (think how different two files are). – AstroDan Mar 18 '16 at 18:18
  • 1
    A fuzzy hash is not good at detecting a few (say a 100 identical characters out of 100000 characters), in fact it should read as two completely separate files, but a properly setup rolling hash can. – AstroDan Mar 18 '16 at 18:27
  • 1
    Also, it looks like sdhash might be dead. – Jeff Mar 18 '16 at 18:29
  • 2
    It has been a few years since the last commit. But it is open source with a reasonable license (Apache) and still works just fine. Unlike a lot of things related to security pattern matching algorithms have a fairly long half life. My favorite string matching algorithm is Damerau–Levenshtein and is rather older then 2013. – AstroDan Mar 18 '16 at 18:36