HookCase: Tool for reverse engineering macOS/OS X
HookCase
HookCase is a tool for debugging and reverse engineering applications on macOS (aka OS X), and the operating system itself. It re-implements and extends Apple’s DYLD_INSERT_LIBRARIES functionality. It can be used to hook any method in any module (even non-exported ones, and even those that don’t have an entry in their own module’s symbol table). In a single operation, it can be applied to a parent process and all its child processes, whether or not the child processes inherit their parent’s environment. So HookCase is considerably more powerful than DYLD_INSERT_LIBRARIES. It also doesn’t have the restrictions Apple has placed on DYLD_INSERT_LIBRARIES. So, for example, HookCase can be used with applications that have entitlements.
HookCase supports interpose hooks. But it also supports another, more powerful kind of hook that we call “patch hooks”. These can hook calls to a method named in its module’s symbol table, including ones that come from the same module. They can also hook calls to an unnamed method (one that isn’t in its module’s symbol table), by specifying the method’s address in its module. So they can be used with non-exported (aka private) methods (named and unnamed) — ones not intended for use by external modules.
Patch hooks are so-called because we set them up by “patching” the beginning of an original method with a software interrupt instruction (int 0x30). HookCase’s kernel extension handles the interrupt to implement the hook. This is analogous to what a debugger does when it sets a breakpoint (though it uses int 3 instead of int 0x30). Software interrupts are mostly not used on BSD-style operating systems like macOS and OS X, so we have plenty to choose among. For now, we’re using those in the range 0x30-0x34.
Whatever their disadvantages, interpose hooks are very performant. They’re implemented by changing a pointer, so they impose no performance penalty whatsoever (aside from the cost of whatever additional code runs inside the hook). Patch hooks can be substantially less performant — if we have to unset the breakpoint on every call to the hook, then reset it afterward (and protect these operations from race conditions). But this isn’t needed for methods that start with a standard C/C++ prologue in machine code (which is most of them). So most patch hooks run with only a very small performance penalty (that of a single software interrupt).
HookCase is compatible with DYLD_INSERT_LIBRARIES and doesn’t stomp on any of the changes it may have been used to make. So a DYLD_INSERT_LIBRARIES hook will always override the “same” HookCase interpose hook. This is because Apple often uses DYLD_INSERT_LIBRARIES internally, in ways it doesn’t document. HookCase would likely break Apple functionality if it could override Apple’s hooks. But this doesn’t apply to patch hooks. Since Apple doesn’t use them, we don’t need to worry about overriding any that Apple may have set. If an interpose hook doesn’t seem to work, try a patch hook instead. (Unless you write them to do so, neither interpose hooks nor patch hooks inherently change the behavior of the methods they hook.)
HookCase is compatible with lldb and gdb: Any process with HookCase’s interpose or patch hooks can run inside these debuggers. But you may encounter trouble if you set a breakpoint and a patch hook on the same method, or try to step through code that contains a patch hook.
HookCase runs on OS X 10.9 (Mavericks) through macOS 15 (Sequoia).
