Locating Encrypted Data Hidden among Non-Encrypted Data Using Statistical Tools

This research tests the security of software protection techniques that use encryption to protect code segments containing critical algorithm implementation to prevent reverse engineering. Using the National Institute of Standards and Technology (NIST) Tests for Randomness encrypted regions hidden among non-encrypted bits of a binary executable file are located. The location of ciphertext from four encryption algorithms (AES, DES, RSA, and TEA) and three block sizes (10, 100, and 500 32-bit words) were tested during the development of the techniques described in this research. The test files were generated from the Win32 binary executable file of Adobe's Acrobat Reader version 7.0.9. The culmination of this effort developed a technique capable of locating 100% of the encryption regions with no false negative error and minimal false positive error with a 95% confidence. The encrypted region must be encrypted with a strong encryption algorithm whose ciphertext appears statistically random to the NIST Tests for Randomness, and the size of the encrypted region must be at least 100 32-bit words (3,200 bits).