Coder Perfect

How do I analyze a program’s core dump file with GDB when it has command-line parameters?

Problem

This is how my program works:

exe -p param1 -i param2 -o param3

It crashed and a core dump file, core.pid, was created.

I’d like to examine the core dump file in detail.

gdb ./exe -p param1 -i param2 -o param3 core.pid

GDB, on the other hand, recognizes the EXE file’s arguments as GDB’s input.

In this case, how can I evaluate a core dump file?

Asked by Treper

Solution #1

You can use the core file with GDB in a variety of ways, but providing parameters that should be supplied to the executable to GDB is not one of them. It’s also possible that this is why you got the error. The core file can be used in the following ways:

gdb executable> core-file> or gdb executable> -c core-file> or gdb executable> -c core-file>

gdb <executable>
...
(gdb) core <core-file>

You don’t need to give arguments when utilizing the core file. In GDB, the crash scenario is depicted (checked with GDB version 7.1 on Ubuntu).

For example:

$ ./crash -p param1 -o param2
Segmentation fault (core dumped)
$ gdb ./crash core
GNU gdb (GDB) 7.1-ubuntu
...
Core was generated by `./crash -p param1 -o param2'. <<<<< See this line shows crash scenario
Program terminated with signal 11, Segmentation fault.
#0  __strlen_ia32 () at ../sysdeps/i386/i686/multiarch/../../i586/strlen.S:99
99    ../sysdeps/i386/i686/multiarch/../../i586/strlen.S: No such file or directory.
    in ../sysdeps/i386/i686/multiarch/../../i586/strlen.S
(gdb)

Use —args to send options to the executable that will be debugged in GDB.

For example:

$ gdb --args ./crash -p param1 -o param2
GNU gdb (GDB) 7.1-ubuntu
...
(gdb) r
Starting program: /home/@@@@/crash -p param1 -o param2

Program received signal SIGSEGV, Segmentation fault.
__strlen_ia32 () at ../sysdeps/i386/i686/multiarch/../../i586/strlen.S:99
99    ../sysdeps/i386/i686/multiarch/../../i586/strlen.S: No such file or directory.
    in ../sysdeps/i386/i686/multiarch/../../i586/strlen.S
(gdb)

Other GDB options can be found on the man pages.

The following are the most useful commands:

Answered by another.anon.coward

Solution #2

GDB may be used to debug coredump files in a simple way:

gdb <executable_path> <coredump_file_path>

A “process” coredump file is created as a “core.pid” file.

Once you’ve gotten into the GDB prompt (by running the above command), type:

...
(gdb) where

This will provide you with stack information, allowing you to investigate the source of the crash/fault. Another command that can be used for the same goals is:

...
(gdb) bt full

This is the same as the last example. It lists the entire stack information by convention (which ultimately leads to the crash location).

Answered by parasrish

Solution #3

Minimal runnable example using objdump and gdb

TL;DR:

The following is the complete educational exam setup:

main.c

#include <stddef.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int myfunc(int i) {
    *(int*)(NULL) = i; /* line 7 */
    return i - 1;
}

int main(int argc, char **argv) {
    /* Setup some memory. */
    char data_ptr[] = "string in data segment";
    char *mmap_ptr;
    char *text_ptr = "string in text segment";
    (void)argv;
    mmap_ptr = (char *)malloc(sizeof(data_ptr) + 1);
    strcpy(mmap_ptr, data_ptr);
    mmap_ptr[10] = 'm';
    mmap_ptr[11] = 'm';
    mmap_ptr[12] = 'a';
    mmap_ptr[13] = 'p';
    printf("text addr: %p\n", text_ptr);
    printf("data addr: %p\n", data_ptr);
    printf("mmap addr: %p\n", mmap_ptr);

    /* Call a function to prepare a stack trace. */
    return myfunc(argc);
}

To generate core, compile and run:

gcc -ggdb3 -std=c99 -Wall -Wextra -pedantic -o main.out main.c
ulimit -c unlimited
rm -f core
./main.out

Output:

text addr: 0x4007d4
data addr: 0x7ffec6739220
mmap addr: 0x1612010
Segmentation fault (core dumped)

GDB pinpoints the specific line where the segmentation fault occurred, which is exactly what most debugging users want:

gdb -q -nh main.out core

then:

Reading symbols from main.out...done.
[New LWP 27479]
Core was generated by `./main.out'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000000000400635 in myfunc (i=1) at main.c:7
7           *(int*)(NULL) = i;
(gdb) bt
#0  0x0000000000400635 in myfunc (i=1) at main.c:7
#1  0x000000000040072b in main (argc=1, argv=0x7ffec6739328) at main.c:28

It directs us to line 7 of the buggy.

Arguments to the CLI are saved in the core file and do not need to be given again.

To respond to the specific CLI argument questions, we can observe that changing the cli arguments, for example, with:

rm -f core
./main.out 1 2

Then, without making any changes to our commands, this is reflected in the previous bactrace:

Reading symbols from main.out...done.
[New LWP 21838]
Core was generated by `./main.out 1 2'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000564583cf2759 in myfunc (i=3) at main.c:7
7           *(int*)(NULL) = i; /* line 7 */
(gdb) bt
#0  0x0000564583cf2759 in myfunc (i=3) at main.c:7
#1  0x0000564583cf2858 in main (argc=3, argv=0x7ffcca4effa8) at main.c:2

Take note of how argc=3 has changed. As a result, this must imply that the information is stored in the core file. I’m presuming it just stores it as the parameters of main, just like any other function’s arguments.

This makes sense when you consider that the core dump must be storing the program’s whole memory and register state, and so has all of the information needed to calculate the value of function arguments on the current stack.

How to inspect the environment variables is less obvious: What is the best way to get environment variables from a core dump? Environment variables are also present in memory so the objdump does contain that information, but I’m not sure how to list all of them in one go conveniently, one by one as follows did work on my tests though:

p __environ[0]

Binutils analysis

We may bulk dump information included in the core file, such as the memory state, using binutils utilities like readelf and objdump.

Most/all of it should be viewable in GDB as well, but those binutils utilities provide a more bulky approach that is useful for some use situations, whilst GDB is better for a more interactive examination.

First:

file core

indicates that the core file is in fact an ELF file:

core: ELF 64-bit LSB core file x86-64, version 1 (SYSV), SVR4-style, from './main.out'

As a result, we may investigate it more directly with standard binutils tools.

A short examination of the ELF standard reveals that it has its own ELF type:

Elf32_Ehd.e_type == ET_CORE

More information about the format can be found at:

man 5 core

Then:

readelf -Wa core

provides some information about the file’s structure. Regular program headers appear to be memory-contained:

Program Headers:
  Type           Offset   VirtAddr           PhysAddr           FileSiz  MemSiz   Flg Align
  NOTE           0x000468 0x0000000000000000 0x0000000000000000 0x000b9c 0x000000     0
  LOAD           0x002000 0x0000000000400000 0x0000000000000000 0x001000 0x001000 R E 0x1000
  LOAD           0x003000 0x0000000000600000 0x0000000000000000 0x001000 0x001000 R   0x1000
  LOAD           0x004000 0x0000000000601000 0x0000000000000000 0x001000 0x001000 RW  0x1000

There is also some more metadata in the notes area, most notably prstatus, which contains the PC:

Displaying notes found at file offset 0x00000468 with length 0x00000b9c:
  Owner                 Data size       Description
  CORE                 0x00000150       NT_PRSTATUS (prstatus structure)
  CORE                 0x00000088       NT_PRPSINFO (prpsinfo structure)
  CORE                 0x00000080       NT_SIGINFO (siginfo_t data)
  CORE                 0x00000130       NT_AUXV (auxiliary vector)
  CORE                 0x00000246       NT_FILE (mapped files)
    Page size: 4096
                 Start                 End         Page Offset
    0x0000000000400000  0x0000000000401000  0x0000000000000000
        /home/ciro/test/main.out
    0x0000000000600000  0x0000000000601000  0x0000000000000000
        /home/ciro/test/main.out
    0x0000000000601000  0x0000000000602000  0x0000000000000001
        /home/ciro/test/main.out
    0x00007f8d939ee000  0x00007f8d93bae000  0x0000000000000000
        /lib/x86_64-linux-gnu/libc-2.23.so
    0x00007f8d93bae000  0x00007f8d93dae000  0x00000000000001c0
        /lib/x86_64-linux-gnu/libc-2.23.so
    0x00007f8d93dae000  0x00007f8d93db2000  0x00000000000001c0
        /lib/x86_64-linux-gnu/libc-2.23.so
    0x00007f8d93db2000  0x00007f8d93db4000  0x00000000000001c4
        /lib/x86_64-linux-gnu/libc-2.23.so
    0x00007f8d93db8000  0x00007f8d93dde000  0x0000000000000000
        /lib/x86_64-linux-gnu/ld-2.23.so
    0x00007f8d93fdd000  0x00007f8d93fde000  0x0000000000000025
        /lib/x86_64-linux-gnu/ld-2.23.so
    0x00007f8d93fde000  0x00007f8d93fdf000  0x0000000000000026
        /lib/x86_64-linux-gnu/ld-2.23.so
  CORE                 0x00000200       NT_FPREGSET (floating point registers)
  LINUX                0x00000340       NT_X86_XSTATE (x86 XSAVE extended state)

objdump can dump all memory with ease using:

objdump -s core

which contains:

Contents of section load1:

 4007d0 01000200 73747269 6e672069 6e207465  ....string in te
 4007e0 78742073 65676d65 6e740074 65787420  xt segment.text 

Contents of section load15:

 7ffec6739220 73747269 6e672069 6e206461 74612073  string in data s
 7ffec6739230 65676d65 6e740000 00a8677b 9c6778cd  egment....g{.gx.

Contents of section load4:

 1612010 73747269 6e672069 6e206d6d 61702073  string in mmap s
 1612020 65676d65 6e740000 11040000 00000000  egment..........

This is identical to the stdout value in our test.

Ubuntu 16.04 amd64, GCC 6.4.0, and binutils 2.26.1 were used to test this.

Answered by Ciro Santilli 新疆再教育营六四事件法轮功郝海东

Solution #4

Simply ignore the settings. They aren’t required by GDB:

gdb ./exe core.pid

Answered by Nialscorva

Solution #5

RMS’s GDB debugger tutorial says:

prompt > myprogram
Segmentation fault (core dumped)
prompt > gdb myprogram
...
(gdb) core core.pid
...

Make sure your file is a core image by running it through file.

Answered by Brian Cain

Post is based on https://stackoverflow.com/questions/8305866/how-do-i-analyze-a-programs-core-dump-file-with-gdb-when-it-has-command-line-pa