Open in app

Sign in

Write

Sign in

Ghidra debug tool with Qemu

Olof Astrand
9 min readJan 22, 2024

--

Here we will look at debugging binaries on a remote target with qemu. First I try the ghidra debug tool, without success. So we try ret-sync instead with limited success.

Starting Qemu

We are emulating the raspberry zero

qemu-system-arm  \
   -M raspi0   \
    -d unimp,guest_errors  -s -S  \
    -kernel  u-boot.elf \
    -trace "bcm*" \
    -cpu arm1176  \
    -sd disk.img \
    -m 0.5G -smp 1 \
    -serial tcp::12344,server,nowait -serial tcp::12345,server,nowait \
    -usb -device usb-mouse -device usb-kbd \
  -device usb-net,netdev=net0 \
  -netdev user,id=net0,hostfwd=tcp::5555-:22

The disk.img file is a dump of an image that runs on a raspberry zero which is similar in hardware on the raspberry pi. It boots the sofware with a script called boot.scr which contains the u-boot boot commands. The -S -s options starts a gdb-server that ghidra will connect to.

U-Boot boot.scr

fatload mmc 0 0x00200000 kernel.bin;
go 0x00200000

This instruct u-boot to boot kernel.bin into adress 0x00200000 then start at 0x00200000 (go).

Use the Debugger-> locally gdb IN-VM

Enter the following in the interpreter tab

set architecture arm
set arm fallback-mode thumb
set arm force-mode thumb
target extended-remote 127.0.0.1:1234
break *0x200000

Connect on serial

In a terminal, connect to the serial device.

nc 127.0.0.1 12344

Continue in debugger (interpreter tab)

(gdb) c

Unfortunately I never got the debug tool to work with remote targets.

Retsync

Instead we use retsync,

GitHub - bootleg/ret-sync: ret-sync is a set of plugins that helps to synchronize a debugging…

ret-sync is a set of plugins that helps to synchronize a debugging session (WinDbg/GDB/LLDB/OllyDbg2/x64dbg) with…

github.com

Check issues on how to build for Ghidra 11.0

Here we create a .sync file that allows the ghidra retsync plugin to sychronize with gdb.

.sync

Normally retsync would check with the operating system for mapping of adresses with information from /proc/pid/maps. This is not possible for an embedded target with an unkown operatingsystem.

Instead retsunc will use the .sync file.

[INIT]
context = {
  "pid": 200,
  "mappings": [
    [ 0x000000, 0x001fffff, 0x001fffff, "u-boot.elf" ],
    [ 0x200000, 0x2a70ac8, 0xa70ac8, "u-boot.elf" ],
    # Decompressed program area 
    [0xfe000000, 0xfe0fffff, 0x000fffff, "u-boot.elf"],
  ]
 }


[GENERAL]
use_raw_addr=true

Retsync plugin in ghidra

After pressing listen (+)

Searching .sync in /home/olas/testGhidra11Project.rep
[>] Searching .sync in /home/olas
[>] loading configuration file /home/olas/.sync
[>] failed to parse conf file: parse error (at line: 5): [0x200000, 0x2c70ac7, 0x2a70ac8, "u-boot.elf"],
[>] programOpened: u-boot.elf
    imageBase: 0x0
[>] programActivated: kernel.bin
    local addr: 00000000, remote: unknown
[>] ret-sync enable
[>] server listening 
[>] server started

Now you can connect the running (gdb) with sync.

In gdb

The most important line is (gdb) source ret-sync/ext_gdb/sync.py

(gdb) target extended-remote 127.0.0.1:1234
Remote debugging using 127.0.0.1:1234
_start () at arch/arm/lib/vectors.S:88

(gdb) source ret-sync/ext_gdb/sync.py

[sync] configuration file loaded from: /home/olas/.sync
       general: use_tmp_logging_file is True
[sync] initialization context:
{
    "pid": 200,
    "mappings": [
        [
            0,
            2097152,
            2097152,
            "u-boot.elf"
        ],
        [
            2097152,
            46598855,
            44501704,
            "u-boot.elf"
        ],
        [
            4261412864,
            4262461439,
            1048575,
            "u-boot.elf"
        ]
    ]
}

[sync] init
[sync] 18 commands added
(gdb) sync
[sync] initializing tunnel to IDA using localhost:9100...
[sync] sync is now enabled with host localhost
(gdb) b *0x200000
(gdb) c
Window RetSyncPlugin

Now u-boot is loading

MMC:   sdhci@7e300000: 0
Loading Environment from FAT... *** Warning - bad CRC, using default environment

In:    serial
Out:   vidconsole
Err:   vidconsole
Net:   No ethernet found.
starting USB...
USB0:   Can't get reset: -524
scanning bus 0 for devices... 5 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found
Hit any key to stop autoboot:  0
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
Found U-Boot script /boot.scr
128 bytes read in 38 ms (2.9 KiB/s)
## Executing script at 02400000
44501704 bytes read in 51707 ms (839.8 KiB/s)
## Starting application at 0x00200000 ...

And we get a breakpoint in gdb after a long bootup time,

Breakpoint 1, 0x00200000 in ?? ()
[sync] pid: 200
[sync] unknown module at current PC: 0x200000
[sync] NOTE: will resume sync when at a known module address
(gdb) sync
(update)
(gdb) dump binary memory ram_dump.bin 0xfe000000 0xfe0fffff

Note that we are not syncing here either (unknown module at current PC), We dump som raw memory with gdb. It seems like the program has decompressed and loaded itself into som memory.

dump binary memory ram_dump.bin 0xfe000000 0xfe0fffff

After decompressing the kernel we dump the memory, and import it to ghidra.

File -> Add to program

Click options

Then we update the .sync to include the decompressed function.

Ghidra plugin shortcuts.

F2 - Set breakpoint at cursor address
Ctrl-F2 - Set hardware breakpoint at cursor address
Alt-F3 - Set one-shot breakpoint at cursor address
Ctrl-F3 - Set one-shot hardware breakpoint at cursor address
Alt-F2 - Translate (rebase in debugger) current cursor address
F5 - Go
Alt-F5 - Run (GDB only)
F10 - Single step
F11 - Single trace
Retsync syncing with gdb

Raspberry Pi Zero memory map

The MMU, creates a more complicated map but basically you get this.

 The Raspberry Pi Zero, which uses the Broadcom BCM2835 SoC:

Hardware Address (7E Range):

Usage: When programming peripheral devices like the DMA controller to interact with hardware registers (e.g., PWM), use addresses based on the 0x7E000000 range.
Context: This is the address range as seen by the hardware, including the GPU.
ARM Processor Address (20/3F Range):

Usage: When the ARM CPU is interfacing with hardware registers, use addresses based on the 0x20000000 (Pi Zero) range.
Context: These addresses are what the ARM processor directly sees and interacts with.
GPU Cache and RAM Addressing:

1 GB RAM Space: The RAM space, although 512MB on the Pi Zero, is represented multiple times (4x) in the hardware address space.
Cache Control: The two highest bits in the address determine the GPU cache usage in BCM2835.
Direct Uncached Access: For direct uncached access from peripherals to RAM (e.g., DMA transferring audio data to PWM), add 0xC0000000 to the RAM address for use in the DMA control block.
ARM MMU and Address Mapping:

MMU Functionality: The ARM Memory Management Unit (MMU) can remap these addresses. In certain environments (like Ultibo), it’s set 1:1 to the ARM physical addresses.
Linux Kernel Mapping: In the Linux kernel environment, there's further remapping, typically indicated in the third gray column in memory layout diagrams.

mon info mtree

This information comes from qemu, about the emulated device.

address-space: dwc2
  0000000000000000-00000000ffffffff (prio 0, i/o): bcm2835-gpu
    0000000000000000-000000001fffffff (prio 0, ram): alias bcm2835-gpu-ram-alias[*] @ram 0000000000000000-000000001fffffff
    0000000040000000-000000005fffffff (prio 0, ram): alias bcm2835-gpu-ram-alias[*] @ram 0000000000000000-000000001fffffff
    000000007e000000-000000007effffff (prio 1, i/o): alias bcm2835-peripherals @bcm2835-peripherals 0000000000000000-0000000000ffffff
    0000000080000000-000000009fffffff (prio 0, ram): alias bcm2835-gpu-ram-alias[*] @ram 0000000000000000-000000001fffffff
    00000000c0000000-00000000dfffffff (prio 0, ram): alias bcm2835-gpu-ram-alias[*] @ram 0000000000000000-000000001fffffff

address-space: cpu-memory-0
address-space: cpu-secure-memory-0
address-space: memory
  0000000000000000-ffffffffffffffff (prio 0, i/o): system
    0000000000000000-000000001fffffff (prio 0, ram): ram
    0000000020000000-0000000020ffffff (prio 1, i/o): bcm2835-peripherals
      0000000020003000-000000002000301f (prio 0, i/o): bcm2835-sys-timer
      0000000020004000-0000000020004fff (prio -1000, i/o): bcm2835-txp
      0000000020006000-0000000020006fff (prio 0, i/o): mphi
      0000000020007000-0000000020007fff (prio 0, i/o): bcm2835-dma
      000000002000b200-000000002000b3ff (prio 0, i/o): bcm2835-ic
      000000002000b400-000000002000b43f (prio -1000, i/o): bcm2835-sp804
      000000002000b800-000000002000bbff (prio 0, i/o): bcm2835-mbox
      0000000020100000-00000000201001ff (prio 0, i/o): bcm2835-powermgt
      0000000020101000-0000000020102fff (prio 0, i/o): bcm2835-cprman
      0000000020104000-000000002010400f (prio 0, i/o): bcm2835-rng
      0000000020200000-0000000020200fff (prio 0, i/o): bcm2835_gpio
      0000000020201000-0000000020201fff (prio 0, i/o): pl011
      0000000020202000-0000000020202fff (prio 0, i/o): bcm2835-sdhost
      0000000020203000-00000000202030ff (prio -1000, i/o): bcm2835-i2s
      0000000020204000-000000002020401f (prio -1000, i/o): bcm2835-spi0
      0000000020205000-000000002020501f (prio -1000, i/o): bcm2835-i2c0
      000000002020f000-000000002020f07f (prio -1000, i/o): bcm2835-otp
      0000000020212000-0000000020212007 (prio 0, i/o): bcm2835-thermal
      0000000020214000-00000000202140ff (prio -1000, i/o): bcm2835-spis
      0000000020215000-00000000202150ff (prio 0, i/o): bcm2835-aux
      0000000020300000-00000000203000ff (prio 0, i/o): sdhci
      0000000020600000-00000000206000ff (prio -1000, i/o): bcm2835-smi
      0000000020804000-000000002080401f (prio -1000, i/o): bcm2835-i2c1
      0000000020805000-000000002080501f (prio -1000, i/o): bcm2835-i2c2
      0000000020900000-0000000020907fff (prio -1000, i/o): bcm2835-dbus
      0000000020910000-0000000020917fff (prio -1000, i/o): bcm2835-ave0
      0000000020980000-0000000020990fff (prio 0, i/o): dwc2
        0000000020980000-0000000020980fff (prio 0, i/o): dwc2-io
        0000000020981000-0000000020990fff (prio 0, i/o): dwc2-fifo
      0000000020c00000-0000000020c00fff (prio -1000, i/o): bcm2835-v3d
      0000000020e00000-0000000020e000ff (prio -1000, i/o): bcm2835-sdramc
      0000000020e05000-0000000020e050ff (prio 0, i/o): bcm2835-dma-chan15

address-space: I/O
  0000000000000000-000000000000ffff (prio 0, i/o): io

address-space: bcm2835-mbox-memory
  0000000000000000-000000000000008f (prio 0, i/o): bcm2835-mbox
    0000000000000010-000000000000001f (prio 0, i/o): bcm2835-fb
    0000000000000080-000000000000008f (prio 0, i/o): bcm2835-property

memory-region: ram
  0000000000000000-000000001fffffff (prio 0, ram): ram

memory-region: bcm2835-peripherals
  0000000020000000-0000000020ffffff (prio 1, i/o): bcm2835-peripherals
    0000000020003000-000000002000301f (prio 0, i/o): bcm2835-sys-timer
    0000000020004000-0000000020004fff (prio -1000, i/o): bcm2835-txp
    0000000020006000-0000000020006fff (prio 0, i/o): mphi
    0000000020007000-0000000020007fff (prio 0, i/o): bcm2835-dma
    000000002000b200-000000002000b3ff (prio 0, i/o): bcm2835-ic
    000000002000b400-000000002000b43f (prio -1000, i/o): bcm2835-sp804
    000000002000b800-000000002000bbff (prio 0, i/o): bcm2835-mbox
    0000000020100000-00000000201001ff (prio 0, i/o): bcm2835-powermgt
    0000000020101000-0000000020102fff (prio 0, i/o): bcm2835-cprman
    0000000020104000-000000002010400f (prio 0, i/o): bcm2835-rng
    0000000020200000-0000000020200fff (prio 0, i/o): bcm2835_gpio
    0000000020201000-0000000020201fff (prio 0, i/o): pl011
    0000000020202000-0000000020202fff (prio 0, i/o): bcm2835-sdhost
    0000000020203000-00000000202030ff (prio -1000, i/o): bcm2835-i2s
    0000000020204000-000000002020401f (prio -1000, i/o): bcm2835-spi0
    0000000020205000-000000002020501f (prio -1000, i/o): bcm2835-i2c0
    000000002020f000-000000002020f07f (prio -1000, i/o): bcm2835-otp
    0000000020212000-0000000020212007 (prio 0, i/o): bcm2835-thermal
    0000000020214000-00000000202140ff (prio -1000, i/o): bcm2835-spis
    0000000020215000-00000000202150ff (prio 0, i/o): bcm2835-aux
    0000000020300000-00000000203000ff (prio 0, i/o): sdhci
    0000000020600000-00000000206000ff (prio -1000, i/o): bcm2835-smi
    0000000020804000-000000002080401f (prio -1000, i/o): bcm2835-i2c1
    0000000020805000-000000002080501f (prio -1000, i/o): bcm2835-i2c2
    0000000020900000-0000000020907fff (prio -1000, i/o): bcm2835-dbus
    0000000020910000-0000000020917fff (prio -1000, i/o): bcm2835-ave0
    0000000020980000-0000000020990fff (prio 0, i/o): dwc2
      0000000020980000-0000000020980fff (prio 0, i/o): dwc2-io
      0000000020981000-0000000020990fff (prio 0, i/o): dwc2-fifo
    0000000020c00000-0000000020c00fff (prio -1000, i/o): bcm2835-v3d
    0000000020e00000-0000000020e000ff (prio -1000, i/o): bcm2835-sdramc
    0000000020e05000-0000000020e050ff (prio 0, i/o): bcm2835-dma-chan15

https://www.raspberrypi.org/app/uploads/2012/02/BCM2835-ARM-Peripherals.pdf

RAM (Random Access Memory)

Start Address: Typically, 0x00000000.
Size: The Raspberry Pi Zero comes with 512MB of RAM. However, some of this RAM is reserved for the GPU, so the available system memory is less than this.
End Address: This depends on the actual available system memory. If 512MB is fully available, the end address would be around 0x1FFFFFFF.

Peripherals:

Base Address: 0x20000000 for Raspberry Pi Zero (this is different for later models like the Raspberry Pi 3 and 4, where it starts at 0x3F000000 and 0xFE000000 respectively).
Size: The peripheral address space typically spans 0x01000000 (16MB).
Range: From 0x20000000 to 0x20FFFFFF.

GPU Memory and I/O:

Address space overlapping with the upper region of the RAM might be allocated for GPU and other system purposes.

Conclusion

No big success. I also have been more successful, running retsync with both qemu and real arm hardware before. A better .sync file helped but I was not able to get the Ghidra debug tool up and running. Some lesons can however be learned by article. Fixing the .sync file is a good idea, as we added some binaries to the original elf file afterwards.

Here you can read about using the ghidra debug tool with a local binary instead.

https://olof-astrand.medium.com/running-ghidra-debugger-in-vm-81beae3a1239

Ghidra
Debugging Tools
Resync
Raspberry Pi
Qemu

--

--

Olof Astrand

Written by Olof Astrand

29 Followers

Software Engineer.

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams