Advanced debugging with gdb (Reverse execution , Pretty-Printer and Asan)

90% of coding is debugging. The other 10% is writing bugs. Therefore the debugger is an essential tool when programming in i.e. the Linux environment. Here I will go through an example where we use qt-creators pretty printer, reverse execution and the address sanitizer to find a stack overwrite.

I will start by exploring howto add pretty print capabilities for two string classes, QString and IString.

.gdbinit

Put this in the directory where you want to debug your program.

This is the .gdbinit I use in this example. It will use the nice formatting used by qtcreator

# Pretty print (pp)
python import sys
python sys.path.insert(1,'<path/to/qtcreator>/share/qtcreator/debugger/')
# normally /usr/share/qtcreator/debugger
python from gdbbridge import *
handle SIGPIPE nostop
# Normal arguments to 
set args -o test.txt
set substitute-path [original_path] [new_path]
set history filename ~/.gdb_history
set history save on
set history size 1000
source my_pp.py
# Use dir if code is referenced relative i.e. ../src
dir /home/where/source/is/found

The configuration used by your system can be examined with

gdb --configuration

Normally you can find the gdb python files here,

/usr/share/gdb/python/gdb/

In this example I will use this program with some bugs. A class IString will be used to create a pretty printer of the data.

#include <string.h>
class IString
{
public:
  IString(const char *txt) : pBuffer(text) {
    strcpy(text,txt);
  }
  char *pBuffer;
  int   numChars;
  char  text[10];
};
void fun() {
  IString string1("This is a string that will not fit");
}
int main(int, char**)
{
  IString string1("This string also do not fit");
  char test[20];
fun();
return 0;
}

The IString debug-helper, you can add your own pretty printer:

# my_pp.py
from dumper import *
def qdump__IString(d, value):
    d.putValue("[{0}, {1}]".format(value["Buff"], value["n"]))
    d.putNumChild(2)
    if d.isExpanded():
        with Children(d):
           d.putSubItem("Buff", value["pBuffer"])
           d.putSubItem("n", value["numChars"])

This code uses the qt-creator debugger helpers. To use (pp)

http://nikosams.blogspot.com/2009/10/gdb-qt-pretty-printers.html

‘info pretty-printer’

(gdb) set print pretty 1

Reverse

If you limit your recording to be close to the interesting part of where you want to debug and can accept that only a limited number of instructions, as well as there will be a significant slowdown, so put in a breakpoint as close as possible before the problem happens.

record fullStart recording each instruction step

record stopStop recording

rnSimilar to the n command, but in reverse

rsSimilar to the s command, but in reverse

rsiSimilar to the step instruction command, but in reverse

When using Visual studio code

In DEBUG CONSOLE

Use set exec-direction reverse and

set exec-direction forward

Info

info

info argsPrint function arguments

info localsPrint local variables

info line *0xffff prints out the source line and file where the given assembly address maps to.

info vtbl this print virtual functions (c++)

More tricks

ptype $_siginfo
p $_siginfo._sifields._sigfault.si_addrp $_exception
C-Arrays
int *array = (int *) malloc (len * sizeof (int));
p *array@len
p/x (short[2])0x12345678

CentOS 7

Default install comes with GCC-4.8.5 printers, which are not compatible with STL structures from i.e. GCC7.3.

Solution: install newer GDB (e.g. from devtoolset-9):

yum install centos-release-scl
yum install devtoolset-9-gdb

Newer gdb is then available at: /opt/rh/devtoolset-9/root/usr/bin/gdb

python
import sys 
sys.path.insert(0, '/usr/share/gcc-4.8.5/python')
from libstdcxx.v6.printers import register_libstdcxx_printers
register_libstdcxx_printers (None)
end

Asan

To compile with asan

-g3 -fsanitize=address,undefined

(gdb) b __asan_report_error

Try this, to stop at the location of the error

(gdb) rbreak ^__ubsan_handle_ __asan_report_error
(gdb) commands
(gdb) finish
(gdb) end

When debugging c++ you also might want to use,

(gdb) catch throw

(gdb) catch catch