VM networking tip

If you are setting up VMs using libvirt then it’s a good idea to change the address of the virtual network to something other than the default. Why? Because if you don’t, and you create a guest which itself starts up libvirt and uses NetworkManager then at least some of the time your VM will start up with its networking hosed.

If the host is using the default network (192.168.122.0/24) and the guests also want to use that network then there is a race between NetworkManager bringing up eth0 and libvirt bringing up virbr0. libvirt checks for existing interfaces using the network it is configured for before starting up virbr0, so if NetworkManager brings up eth0 first then virbr0 will not be set up on the guest and everything will be fine. But, if eth0 is not set up by the time libvirt runs the check, then virbr0 will take 192.168.122.0/24, then eth0 will come up on 192.168.122.something, and you’ll have a VM with two separate interfaces connected to two separate networks that both have the same address range… and it won’t work!

The easy way to solve this is to not install libvirt on the guest, but you may not be able to change this until after the guest is running, and if libvirt starts up during a guest’s installer then you may need to complete parts of the installation with no networking. This may or may not be ok for you and your OS. I’m using VMs to set up clean test environments for GDB, and at the moment I’m setting up three or four new “machines” every day (and throwing them away when I’m done) so I want the process as streamlined as possible. If you only occasionally set up new VMs then some extra tasks during the installation may not be a problem, but it is pretty simple to change the network on the host and you only have to do it once:

  1. As root, run virsh net-edit default
  2. Change every occurence of 192.168.122 to something else
  3. Stop any running guests
  4. Run virsh net-destroy default
  5. Run virsh net-start default

It’s a shame this can’t be fixed more conclusively elsewhere, but NetworkManager brings up the interfaces asynchronously at boot time which makes it impossible to definitively schedule libvirt’s startup to happen after NetworkManager.

Thank you Laine Stump for helping me out with this.

Only dest dir longer than base dir not supported

Hello,

Have you experienced the mysterious error, “Only dest dir longer than base dir not supported”?

I have.

The Problem

When you build an rpm, the code is built in %{_builddir}, which usually evaluates as %{_topdir}/BUILD, which in turn evaluates as something like /home/you/rpmbuild/BUILD. The built code (if built with GCC) ends up with loads of /home/you/rpmbuild/BUILD paths embedded in it, and the script /usr/lib/rpm/debugedit rewrites these paths to /usr/src/debug so that the debuginfo rpms work. /usr/lib/rpm/debugedit cannot extend strings, it can only shrink them. If you are seeing “Only dest dir longer than base dir not supported” then, somewhere in your build system, %{_builddir} is defined as something that expands to a string shorter than /usr/src/debug.

Example

In my case, I was building a glibc rpm in a VM that turned out not to have enough disk. I created a new disk, mounted it on /mnt, and added the line %_topdir /mnt to my ~/.rpmmacros. The result? “Only dest dir longer than base dir not supported”. I fixed it by editing ~/.rpmmacros to say %_topdir /mnt/rpmbuild.

Briefly

/usr/src/debug       # the reference
/mnt/BUILD           # too short!
/mnt/rpmbuild/BUILD  # long enough

Thank you,
The Mgt.

Saving money

I have a pair of set-top box PCs I’ve been using as always-on servers. I used them because they’re silent, but lately I’ve been thinking about power consumption. They were pretty good when I bought them in 2006 and 2008, but there’s much better stuff available now. I spent £60 on a Raspberry Pi and some supporting bits; given that it uses roughly a tenth the power of one of the set-top boxes it will have paid for itself in about two months.

While reorganising everything I also decommissioned an old Netgear switch which was likely costing £100 a year to run. Maybe it’s time you looked in your networking cupboard too!

Easy things to do with GDB #1

I’ve been meaning to write some introductory articles to GDB ever since back in February at FOSDEM, where I was surprised by just how many people do not use GDB–or even know what it is! It’s taken me a while to figure out a nice example, but I finally found them.

Last week I wanted to extend ogg123, a tool for playing music files on the commandline. It supports several formats–Ogg Vorbis, FLAC and Speex–and the way it abstracts that is by passing around a struct format_t full of function pointers to various callbacks. I was adding a new format, and wanted to know what kind of arguments it was calling its callbacks with. I could have tried to figure it out by reading the code, but it was easier to just run it in GDB and see what was happening. Here’s how:

  1. Build a copy of ogg123 with debugging support:
    wget http://downloads.xiph.org/releases/vorbis/vorbis-tools-1.4.0.tar.gz
    tar xf vorbis-tools-1.4.0.tar.gz
    cd vorbis-tools-1.4.0
    CFLAGS="-g -O0" ./configure
    make

    The important bit here is the CFLAGS="-g -O0". This causes configure to pass these extra options to GCC in the makefiles it generates. -g instructs GCC to include debugging information in the files it generates to allow GDB to relate the executable files you’re debugging to the source files they were generated from. -O0 instructs GCC not to optimise the code, which makes for easier debugging.

  2. Start up GDB on the ogg123 you just built:
    gdb -args ogg123/ogg123 ~/music/Various/Disco\ Demands/4-08\ -\ Give\ It\ Up.flac

    Everything after the -args option is the command you’d normally type to run the program. GDB will print out some stuff as it starts up, then present you with a (gdb) prompt and wait for you to enter some commands:

    GNU gdb (GDB) 7.4.50.20120313-cvs
    Copyright (C) 2012 Free Software Foundation, Inc.
    License GPLv3+: GNU GPL version 3 or later 
    This is free software: you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
    and "show warranty" for details.
    This GDB was configured as "x86_64-unknown-linux-gnu".
    For bug reporting instructions, please see:
    <http://www.gnu.org/software/gdb/bugs/>...
    Reading symbols from /home/gary/vorbis-tools/ogg123/ogg123...done.
    (gdb) 
  3. Ok, the callback I’m interested in is the “read” callback, and we’re playing a FLAC file so the callback is called flac_read. I’d like to run the program until flac_read is called, so I set a breakpoint on flac_read by entering break flac_read and then using the run command to set ogg123 going:
    (gdb) break flac_read 
    Breakpoint 1 at 0x40dde5: file flac_format.c, line 253.
    (gdb) run
    Starting program: /home/gary/vorbis-tools/ogg123/ogg123 /home/gary/music/Various/Disco\ Demands/4-08\ -\ Give\ It\ Up.flac
    [Thread debugging using libthread_db enabled]
    Using host libthread_db library "/lib64/libthread_db.so.1".
    [New Thread 0x7fffeb863700 (LWP 10312)]
    [Thread 0x7fffeb863700 (LWP 10312) exited]
    
    Audio Device:   PulseAudio Output
    
    Playing: /home/gary/music/Various/Disco Demands/4-08 - Give It Up.flac
    
    Breakpoint 1, flac_read (decoder=0x637560, ptr=0x6182a0, nbytes=30240, 
        eos=0x7fffffffe37c, audio_fmt=0x7fffffffe380) at flac_format.c:253
    253	  flac_private_t *priv = decoder->private;
    (gdb)
  4. GDB now has a running copy of ogg123 stopped at the very start of flac_read. Pretty jazzy huh? You can see the values of the arguments in the second-to-last line it printed, so for example you can see that the caller has requested to read 30240 bytes of data. The various structures (decoder=0x637560 etc) aren’t so useful as they’re pointers, but we can deference them with the print command:
    (gdb) print *decoder
    $1 = {source = 0x637060, request_fmt = {big_endian = 0, word_size = 2, 
        signed_sample = 1, rate = 0, channels = 0, matrix = 0x0}, actual_fmt = {
        big_endian = 0, word_size = 2, signed_sample = 1, rate = 0, channels = 0, 
        matrix = 0x0}, format = 0x617ec0, callbacks = 0x7fffffffe3a0, 
      callback_arg = 0x0, private = 0x6375d0}

    That’s pretty nice, we can see for example what format it wants the data in: little endian, signed, etc. Also, notice the line of code it stopped on:

    253	  flac_private_t *priv = decoder->private;

    That’s the next line of code GDB will execute if we set the program going again in some way. We can advance over just that line with the next command:

    (gdb) next
    254	  decoder_callbacks_t *cb = decoder->callbacks;

    Now it’s initialised the local variable priv, which we can also print out:

    (gdb) p *priv
    $2 = {decoder = 0x637000, is_oggflac = 0, channels = 2, rate = 44100, 
      bits_per_sample = 16, totalsamples = 13337016, currentsample = 0, 
      samples_decoded = 4096, samples_decoded_previous = 0, bytes_read = 24576, 
      bytes_read_previous = 0, comments = 0x638240, bos = 1, eos = 0, 
      buf = 0x637020, buf_len = 4096, buf_start = 0, buf_fill = 4096, stats = {
        total_time = 0, current_time = 0, instant_bitrate = 0, avg_bitrate = 0}}

    Did you see what I did there? Most GDB commands have abbreviated forms, and the abbreviation for print is p. The commands I’ve introduced in this article are some of the most commonly used, and their abbreviated forms are all simply their first letter: b for breakpoint, r for run and n for next. You can also repeat the previous command by pressing Return, which is handy for doing a load of next commands one after the other, for instance.

Ok, that’s all for today. Go and have a play!

Working on GDB

For future reference, and because I keep forgetting things, here is how I work on GDB:

  1. Check out a copy of the sources. Some of this is described on the branch management page of the GDB wiki:
    mkdir /somewhere/to/work
    cd /somewhere/to/work
    git clone ssh://sourceware.org/git/archer.git src
    cd src
    git remote add gdb git://sourceware.org/git/gdb.git
  2. Point the master branch to the correct place. The real one is in the GDB repo, but there’s an old one in Archer that is no longer used. Open /somewhere/to/work/src/.git/config in your favourite editor and change remote = origin to remote = gdb in the [branch "master"] section. Then probably do a git pull to fetch the actual code you’ll be working on.
  3. Create yourself a nice new branch to make your changes in:
  4. git branch --track archer-gbenson-lazy-skip-inline-frames gdb/master
    git checkout archer-gbenson-lazy-skip-inline-frames
  5. (make your changes here)
  6. Build it. I build out-of-tree, like this:
    mkdir /somewhere/to/work/build
    cd /somewhere/to/work/build
    CFLAGS="-g -O0" ../src/configure --with-separate-debug-dir=/usr/lib/debug
    make
  7. Run the testsuite. I like to run the tests twice, once without and once with the changes, then I diff the results and pipe it through a script I wrote to hide the unimportant bits:
    make check >& build-20110921-3.log
    diff -u baseline-20110919.log build-20110921-3.log | gdbtestdiff
  8. If a test fails, you’ll want to hook gdb into it. To do that you run the specific test to create a transcript:
    cd /somewhere/to/work/build/gdb/testsuite
    runtest TRANSCRIPT=1 ../../../src/gdb/testsuite/gdb.opt/inline-cmds.exp
    gdb ../gdb
  9. Then you pipe the transcript into gdb:
    set prompt (top-gdb) 
    b internal_error
    r -nx <transcript.1

Ok, I think that is enough for now.