kgdb: Source level debugging for linux kernel

What's New: kgdb for 2.4 kernels is available, checkout linux-2.4.2-kgdb.patch

kgdb is a patch to the linux kernel. It allows use of gdb for source level debugging of a linux kernel. One can place breakpoints, step through code and observe variables in the kernel similar to debugging a program.

For using kgdb you need two machines. One of these machines is the development machine and the other is tests machine. The kernel to be debugged runs on the tests machine. gdb runs on the development machine. The machines are connected through a serial line. The serial line is used by gdb to communicate to the kernel being debugged.

kgdb patch adds following to a linux kernel

gdb stub - This component talks to gdb. It communicates to gdb using remote debugging protocol used by gdb.
modifications to fault handlers - Kernel gives control on occurance of an unexpected fault to gdb. ( instead of doing a register dump). Registers (including instruction pointer) can be modified through gdb to get around the fault.
serial communication - This component uses a serial driver in the kernel and offers an interface to gdb stub in the kernel. It is responsible for sending and recieving data from a serial line. Control break sent through the serial line is also handled by this component.

Patches
Current kgdb patch is for kernel 2.4.2.
This patch includes following features in addition to basic source level debugging:
1. Support for multiple threads - This enables use of thread related commands in gdb to observe stack trace, variables etc. in different kernel threads (processes).
2. ia-32 hardware debug register support.
3. MP capabilities - The debugger forces all cpus to enter the debugger when one of the cpu hits a breakpoint.
4. Console output - kgdb can be configured to get console output in gdb.

kgdb patch for 2.2.17 kernel also supports above features except for console output. This patch will need SMP support to be enabled while configuring the kernel. SMP support is needed because the patch was ported from 2.4 kernels and 2.2 kernels have some of the required definitions available only if SMP is enabled. A kernel compiled with SMP support will work fine on a UP machine.

GDB Script for using hw debugging

gdbinit file contains gdb macros for using hardware debugging provided by kgdb. Copy this file to .gdbinit file in gdb starting directory (e.g. /usr/src/linux if you start gdb from that directory).
Please read this file to find macros for hw debugging. It also includes few utility macros to list processes and modules.

Debugging kernel modules

For debugging kernel modules, you can load module object files in gdb using gdb command add-symbol-file. This command needs addresses of sections in the module file. module loading script loadmodule.sh
can be used to automate the process of loading a module into the kenel and generate a gdb script that will load the module file into gdb. It will need modutils-2.3.19 or later. modutils can be downloaded from ( http://www.kernel.org/pub/linux/utils/kernel/modutils/ or its mirrors).

gdb had a bug that caused incorrect calculation addresses of symbols loaded via add-symbol-file command. It has been fixed in developement versions of gdb. Unfortunately this fix is not present in gdb-5.0 Please use a developement version of gdb for debugging kernel modules. You may use a gdb built for RH6.2 from gdb developement sources. This is a developement version of gdb and may contain bugs. It is recommended that you use it for DEBUGGING KERNEL MODULES ONLY. Please go to gdb official site for information on gdb.
2
Credits
Dave Grothe wrote kgdb patch originally. He is the guy who started source level debugging on linux kernel. Thanks to him for making kernel developer's lives easier :-)
2Thanks to John H. Hartman and Lincoln Myers for helping me in integrating kgdb into newer 2.4 kernels.
8Thanks to Duane Voth for backporting kgdb to 2.2 kernels.

TODO

kgdb for 2.2.17 kernels crashes frequently. Need to check this.
gdbstart program can be executed one once. Second time it fails. gdb_hook needs to be modified so that it does not do initializations which can be done only once (e.g. grabbing serial port irq).

Please send queries, bug-reports, enhancements to Amit S. Kale

Last updated on 16th March, 2001.