Clang __usercall: towards native support for user defined calling conventions

Jared Q. Widberg, Sashank Narain, Yimin Chen

Abstract: In reverse engineering interfacing with C/C++ functions is of great interest because it provides much more flexibility for product development and security purpose. However, it has been a great challenge when interfacing functions with user defined calling conventions due to the lack of sufficient and user-friendly tooling. In this work, we design and implement Clang __usercall, which aims to provide programmers with an elegant and familiar syntax to specify user defined calling conventions on functions in C/C++ source code. Our key novelties lie in mimicing the most popular syntax and adapting Clang for interfacing purpose. Our preliminary user study shows that our solution outperforms the existing ones in multiple key aspects including user experience and required lines of code. Clang __usercall is already added to the Compiler Explorer website as well.