Binary Analysis with Immunity Debugger

Introduction

Reverse engineering represents a critical discipline within the security research domain. During my graduate studies in Cyber Security and through various CTF competitions, I've developed fundamental skills in this area that continue to inform my approach to security architecture and threat analysis.

This article examines the process of analyzing an executable provided during an advanced Malware Analysis seminar led by Jaanus Kääp. The exercise demonstrates systematic binary analysis techniques using command-line tools and Immunity Debugger. All files referenced are available in the Task Repo.

Analysis Environment

Sectools describes Immunity Debugger as "a debugger designed specifically for writing exploits, analyzing malware, and reverse engineering binary files. It features function graphing, advanced heap analysis capabilities, and an extensive Python API for extensibility."

The analysis environment comprised:

• Host Machine: Windows 10
• Virtualization: Oracle VirtualBox
• Analysis VMs:
  • Windows 7 (free Windows VMs available from Microsoft)
  • Kali Linux (for supplementary analysis)

Challenge Parameters

The project documentation included essential context for the analysis:

The target was a straightforward serial generation executable that produces a serial code from a provided username. The objective was to analyze the underlying algorithm and potentially develop a keygen—a common task in advanced security assessments and penetration testing.

Preliminary Analysis

Initial analysis suggested the implementation used standard English character manipulation. Input testing was performed to establish operational parameters.

The tests established clear constraints:

• No numerical values accepted
• No special characters accepted
• Only lowercase English alphabet characters [a-z] permitted

Static Analysis

Loading the target into Immunity Debugger revealed the executable's structure:

The investigation began by examining all referenced text strings, a technique that often reveals operational logic and execution flow in compiled binaries:

Algorithm Identification

The critical section of code was located in the validation routine preceding a comparison operation. Analysis revealed:

• A loop executing precisely once for each character in the input string
• Instructions including SAR, MOVZX, MOVSX with IMUL—commonly associated with bit manipulation operations
• Testing with controlled input "aaaa" showed a consistent transformation pattern where each 'a' character (hex 61) was transformed to 'k' (hex 6b)

Through systematic instruction tracing and register analysis, the algorithm was determined to be:

For each char in input:
    Serial += englishletters[char.index + 10]

The algorithm implements a Caesar cipher with a shift value of 10 positions in the lowercase English alphabet.

Keygen Implementation

With the algorithm fully characterized, I developed a verification tool using Python with the Tkinter library for the interface. The implementation applied the identified algorithm to validate the reverse engineering analysis.

Verification

Testing confirmed the accuracy of the reverse-engineered algorithm:

Conclusion

This analysis demonstrates the systematic approach required for effective binary analysis:

1. Establish a secure, isolated test environment
2. Perform preliminary black-box testing to understand input/output constraints
3. Conduct static analysis to identify key code sections
4. Execute dynamic analysis to observe runtime behavior
5. Develop a hypothesis about the underlying algorithm
6. Verify through independent implementation

While this example focused on a relatively straightforward algorithm, the methodologies applied are directly relevant to complex malware analysis, security assessments, and vulnerability research scenarios encountered in enterprise security operations.