Corellium Technical Articles

Kernel Debugging | Inspect and Modify Under User Control | Guide

Written by Corellium | May 16, 2024 6:15:51 PM

One of the major benefits of using a virtual environment is the ability to inspect and modify the state of the whole system under user control. The primary interface to these features is through the TCP-based GDB remote protocol compatible stub.

Important Notes on the Kernel Debug Stub​

  • The kernel debug stub presents the multiple CPUs in the system as 'threads' of the process being debugged. For instance, on an iPhone 6, there are two threads: 1 and 2, corresponding to CPU 0 and CPU 1. (GDB protocol does not allow for a thread called 0.)
  • Only all-stop mode is supported: if one CPU stops, the others stop as well.
  • Single-stepping is supported by using AArch64 single-stepping features. In the case of vCont packets, if a CPU single-steps and other CPUs do not have actions specified, they do not perform a step.
  • Hardware breakpoints and watchpoints are supported. (Up to 4 of each at a time.) Software breakpoint packets issued to the stub are converted into hardware breakpoints.
  • Memory addresses from the debugger are passed through virtual-to-physical address translation - this is necessary to make it work. Also, only actual RAM is visible to the debugger; access to MMIO regions is ignored and returns 0.
  • Only one concurrent debug stub connection is supported per VM.
  • High latency can cause the VM to visually “freeze” or “stutter” as the underlying kernel is setting breakpoints and communicating with your GDB. If you use features like watch / awatch / rwatch with conditions, every breakpoint is sent to your local machine, the condition is calculated, and then resumed if it was meant to ignore. This is how GDB functions, but it is typically not noticeable for local kernel debugging. On the local GDB end, nothing will visually indicate that this is happening unless the condition is met, in which case you’ll receive a prompt.

Initial Setup​

  1. Set up your virtual device.
  2. Connect to the VPN if you are using the cloud product.
  3. Grab the Corellium vmlinux file with symbols appropriate for your device:

Connecting to the Kernel Debug Stub​

Using GDB / LLDB​

To connect using GDB / LLDB, check out our article on GDB / LLDB.

To Connect to the Stub​

Note: The address and port provided here are for example purposes only. You will need to use the address and port for your particular virtual device. You can find the address and port for your device at the end of the "kernel gdb" link.

Example 

To Switch CPUs (in this case, to CPU 1)

To Access Monitor Commands

Otherwise, use regular GDB commands to control the debug stub.

To Connect to the Stub Without the Binary​

Note: The address and port provided here are for example purposes only. You will need to use the address and port for your particular virtual device. You can find the address and port for your device at the end of the "kernel gdb" link.

Example 

To Connect to the Stub with the Binary​

Example

The GDB stub represents CPU cores as threads.

While LLDB is specified in the UI, users are free to use GDB as well.

You can use the regular LLDB commands to control the debug stub.

Your debugger will work as if it was attached to a hardware debugger (think OpenOCD).

Using IDA​

The following instructions are for IDA 7.0 versions.

  1. Select Debugger | Switch debugger... from the main menu, then pick Remote GDB debugger in the dialog box.

  2. Then, again from the main menu, select Debugger then Debugger options.... Click the Set specific options button and make sure the Use stepping support checkbox is checked.

  3. Finally, select Debugger | Process options... from the main menu, enter the stub's address in the Hostname and Port fields. After this setup, which is saved in the IDA database, select Debugger | Attach to process... to attach to the running device.

To access monitor commands from IDA, locate the GDB command line bar at the bottom of the window (just above the status bar, next to a GDB button). Enter the monitor commands there, without the word "monitor" itself. For instance, instead of monitor sr, simply write sr and press Enter. The output will appear in IDA's text output window above.

Looking to debug the kernel on a virtual device? Book a meeting to deep dive into TCP-based GDB remote protocol usage, handle multiple CPUs, and manage breakpoints with precision.

Advance Your Mobile Security Research with Corellium

Experience Corellium’s groundbreaking virtualization technology for mobile devices and discover never-before-possible mobile vulnerability and threat research for iOS and Android phones. Book a meeting today to explore how our platform can optimize mobile security research and malware analysis.
View full post