How to detect if the current process is being run by GDB

C Problem Overview

The standard way would be the following:

if (ptrace(PTRACE_TRACEME, 0, NULL, 0) == -1)
  printf("traced!\n");

In this case, ptrace returns an error if the current process is traced (e.g., running it with GDB or attaching to it).

But there is a serious problem with this: if the call returns successfully, GDB may not attach to it later. Which is a problem since I'm not trying to implement anti-debug stuff. My purpose is to emit an 'int 3' when a condition is met (e.g., an assert fails) and GDB is running (otherwise I get a SIGTRAP which stops the application).

Disabling SIGTRAP and emitting an 'int 3' every time is not a good solution because the application I'm testing might be using SIGTRAP for some other purpose (in which case I'm still screwed, so it wouldn't matter, but it's the principle of the thing :))

C Solutions

Solution 1 - C

On Windows there is an API, IsDebuggerPresent, to check if process is under debugging. At Linux, we can check this with another way (not so efficient).

Check "/proc/self/status" for "TracerPid" attribute.

Example code:

#include <sys/stat.h>
#include <string.h>
#include <fcntl.h>
#include <unistd.h>
#include <ctype.h>

bool debuggerIsAttached()
{
    char buf[4096];

    const int status_fd = ::open("/proc/self/status", O_RDONLY);
    if (status_fd == -1)
        return false;

    const ssize_t num_read = ::read(status_fd, buf, sizeof(buf) - 1);
    ::close(status_fd);

    if (num_read <= 0)
        return false;

    buf[num_read] = '\0';
    constexpr char tracerPidString[] = "TracerPid:";
    const auto tracer_pid_ptr = ::strstr(buf, tracerPidString);
    if (!tracer_pid_ptr)
        return false;

    for (const char* characterPtr = tracer_pid_ptr + sizeof(tracerPidString) - 1; characterPtr <= buf + num_read; ++characterPtr)
    {
        if (::isspace(*characterPtr))
            continue;
        else
            return ::isdigit(*characterPtr) != 0 && *characterPtr != '0';
    }

    return false;
}

Solution 2 - C

The code I ended up using was the following:

int
gdb_check()
{
  int pid = fork();
  int status;
  int res;

  if (pid == -1)
  {
    perror("fork");
    return -1;
  }

  if (pid == 0)
  {
    int ppid = getppid();

    /* Child */
    if (ptrace(PTRACE_ATTACH, ppid, NULL, NULL) == 0)
    {
      /* Wait for the parent to stop and continue it */
      waitpid(ppid, NULL, 0);
      ptrace(PTRACE_CONT, NULL, NULL);

      /* Detach */
      ptrace(PTRACE_DETACH, getppid(), NULL, NULL);

      /* We were the tracers, so gdb is not present */
      res = 0;
    }
    else
    {
      /* Trace failed so GDB is present */
      res = 1;
    }
    exit(res);
  }
  else
  {
    waitpid(pid, &status, 0);
    res = WEXITSTATUS(status);
  }
  return res;
}

A few things:

When ptrace(PTRACE_ATTACH, ...) is successful, the traced process will stop and has to be continued.
This also works when GDB is attaching later.
A drawback is that when used frequently, it will cause a serious slowdown.
Also, this solution is only confirmed to work on Linux. As the comments mentioned, it won't work on BSD.

Solution 3 - C

You could fork a child which would try to PTRACE_ATTACH its parent (and then detach if necessary) and communicates the result back. It does seem a bit inelegant though.

As you mention, this is quite costly. I guess it's not too bad if assertions fail irregularly. Perhaps it'd be worthwhile keeping a single long-running child around to do this - share two pipes between the parent and the child, child does its check when it reads a byte and then sends a byte back with the status.

Solution 4 - C

I had a similar need, and came up with the following alternatives

static int _debugger_present = -1;
static void _sigtrap_handler(int signum)
{
    _debugger_present = 0;
    signal(SIGTRAP, SIG_DFL);
}

void debug_break(void)
{
    if (-1 == _debugger_present) {
        _debugger_present = 1;
        signal(SIGTRAP, _sigtrap_handler);
        raise(SIGTRAP);
    }
}

If called, the debug_break function will only interrupt if a debugger is attached.

If you are running on x86 and want a breakpoint which interrupts in the caller (not in raise), just include the following header, and use the debug_break macro:

#ifndef BREAK_H
#define BREAK_H

#include <stdio.h>
#include <stdlib.h>
#include <signal.h>

int _debugger_present = -1;
static void _sigtrap_handler(int signum)
{
    _debugger_present = 0;
    signal(SIGTRAP, SIG_DFL);
}

#define debug_break()                       \
do {                                        \
    if (-1 == _debugger_present) {          \
        _debugger_present = 1;              \
        signal(SIGTRAP, _sigtrap_handler);  \
        __asm__("int3");                    \
    }                                       \
} while(0)

#endif

Solution 5 - C

I found that a modified version of the file descriptor "hack" described by Silviocesare and blogged by xorl worked well for me.

This is the modified code I use:

#include <stdio.h>
#include <unistd.h>

// gdb apparently opens FD(s) 3,4,5 (whereas a typical prog uses only stdin=0, stdout=1,stderr=2)
int detect_gdb(void)
{
    int rc = 0;
    FILE *fd = fopen("/tmp", "r");

    if (fileno(fd) > 5)
    {
        rc = 1;
    }

    fclose(fd);
    return rc;
}

Solution 6 - C

If you just want to know whether the application is running under GDB for debugging purposes, the simplest solution on Linux is to readlink("/proc/<ppid>/exe"), and search the result for "gdb".

Solution 7 - C

This is similar to terminus' answer, but uses pipes for communication:

#include <unistd.h>
#include <stdint.h>
#include <sys/ptrace.h>
#include <sys/wait.h>

#if !defined(PTRACE_ATTACH) && defined(PT_ATTACH)
#  define PTRACE_ATTACH PT_ATTACH
#endif
#if !defined(PTRACE_DETACH) && defined(PT_DETACH)
#  define PTRACE_DETACH PT_DETACH
#endif

#ifdef __linux__
#  define _PTRACE(_x, _y) ptrace(_x, _y, NULL, NULL)
#else
#  define _PTRACE(_x, _y) ptrace(_x, _y, NULL, 0)
#endif

/** Determine if we're running under a debugger by attempting to attach using pattach
 *
 * @return 0 if we're not, 1 if we are, -1 if we can't tell.
 */
static int debugger_attached(void)
{
    int pid;

    int from_child[2] = {-1, -1};

    if (pipe(from_child) < 0) {
        fprintf(stderr, "Debugger check failed: Error opening internal pipe: %s", syserror(errno));
        return -1;
    }

    pid = fork();
    if (pid == -1) {
        fprintf(stderr, "Debugger check failed: Error forking: %s", syserror(errno));
        return -1;
    }

    /* Child */
    if (pid == 0) {
        uint8_t ret = 0;
        int ppid = getppid();

        /* Close parent's side */
        close(from_child[0]);

        if (_PTRACE(PTRACE_ATTACH, ppid) == 0) {
            /* Wait for the parent to stop */
            waitpid(ppid, NULL, 0);

            /* Tell the parent what happened */
            write(from_child[1], &ret, sizeof(ret));

            /* Detach */
            _PTRACE(PTRACE_DETACH, ppid);
            exit(0);
        }

        ret = 1;
        /* Tell the parent what happened */
        write(from_child[1], &ret, sizeof(ret));

        exit(0);
    /* Parent */
    } else {
        uint8_t ret = -1;

        /*
         *    The child writes a 1 if pattach failed else 0.
         *
         *    This read may be interrupted by pattach,
         *    which is why we need the loop.
         */
        while ((read(from_child[0], &ret, sizeof(ret)) < 0) && (errno == EINTR));

        /* Ret not updated */
        if (ret < 0) {
            fprintf(stderr, "Debugger check failed: Error getting status from child: %s", syserror(errno));
        }

        /* Close the pipes here, to avoid races with pattach (if we did it above) */
        close(from_child[1]);
        close(from_child[0]);

        /* Collect the status of the child */
        waitpid(pid, NULL, 0);

        return ret;
    }
}

Trying the original code under OS X, I found waitpid (in the parent) would always return -1 with an EINTR (System call interrupted). This was caused by pattach, attaching to the parent and interrupting the call.

It wasn't clear whether it was safe to just call waitpid again (that seemed like it might behave incorrectly in some situations), so I just used a pipe to do the communication instead. It's a bit of extra code, but will probably work reliably across more platforms.

This code has been tested on OS X v10.9.3 (Mavericks), Ubuntu 14.04 (Trusty Tahr) (3.13.0-24-generic) and FreeBSD 10.0.

For Linux, which implements process capabilities, this method will only work if the process has the CAP_SYS_PTRACE capability, which is typically set when the process is run as root.

Other utilities (gdb and lldb) also have this capability set as part of their filesystem metadata.

You can detect whether the process has effective CAP_SYS_PTRACE by linking against -lcap,

#include <sys/capability.h>

cap_flag_value_t value;
cap_t current;

/*
 *  If we're running under Linux, we first need to check if we have
 *  permission to to ptrace. We do that using the capabilities
 *  functions.
 */
current = cap_get_proc();
if (!current) {
    fprintf(stderr, "Failed getting process capabilities: %s\n", syserror(errno));
    return -1;
}

if (cap_get_flag(current, CAP_SYS_PTRACE, CAP_PERMITTED, &value) < 0) {
    fprintf(stderr, "Failed getting permitted ptrace capability state: %s\n", syserror(errno));
    cap_free(current);
    return -1;
}

if ((value == CAP_SET) && (cap_get_flag(current, CAP_SYS_PTRACE, CAP_EFFECTIVE, &value) < 0)) {
    fprintf(stderr, "Failed getting effective ptrace capability state: %s\n", syserror(errno));
    cap_free(current);
    return -1;
}

Solution 8 - C

C++ version of Sam Liao's answer (Linux only):

// Detect if the application is running inside a debugger.
bool being_traced()
{
  std::ifstream sf("/proc/self/status");
  std::string s;
  while (sf >> s)
  {
    if (s == "TracerPid:")
    {
      int pid;
      sf >> pid;
      return pid != 0;
    }
    std::getline(sf, s);
  }

  return false;
}

Content Type	Original Author	Original Content on Stackoverflow
Question	terminus	View Question on Stackoverflow
Solution 1 - C	Sam Liao	View Answer on Stackoverflow
Solution 2 - C	terminus	View Answer on Stackoverflow
Solution 3 - C	Hugh	View Answer on Stackoverflow
Solution 4 - C	badeip	View Answer on Stackoverflow
Solution 5 - C	pestophagous	View Answer on Stackoverflow
Solution 6 - C	Employed Russian	View Answer on Stackoverflow
Solution 7 - C	Arran Cudbard-Bell	View Answer on Stackoverflow
Solution 8 - C	Carlo Wood	View Answer on Stackoverflow