I finished converting everything to 64-bit. It was way easier than I expected and I’m very glad I took the time to do it because what I thought were vast and insurmountable differences turned out to be pretty minuscule. I managed to tuck everything away in macros, the ABI differences in prolog and epilog, and enter (née function_entry_point) and call, and the register-size differences in much simpler ones like load and store which just map to lwz/ld and stw/std respectively. Hiding the ugly stuff in the assembler keeps the generators happily free of conditionals.

My next job is designing the interpreter calling convention. There’s no real reason for the interpreter to follow the platform’s ABI, and the fact that Java is essentially stack-based and PPC is essentially register-based is a very good reason not to. So I’m trying to figure out how to arrange the stack.

The general layout of stack frames is the same under both 32- and 64-bit ABIs:

    | ...                  |  high addresses
+-> | Link area            |
|   +----------------------+
|   | Register save area   |
|   | Local variable space |
|   | Parameter list space |
+---+ Link area            |  low addresses
    +----------------------+

The stack grows downwards, and the stack pointer, r1, points to the first word of the link area, the lowest address, such that all accesses into the stack are relative to r1 with a positive offset. The ABIs are pretty relaxed about what happens in the stack, but one thing they’re firm about is that 0(r1) points to the previous frame — essentially it’s where you save your caller’s r1. This is slightly irritating, because I think the interpreter would like an open-ended stack, but the requirement to maintain a valid link area at the very top of the stack would seem to preclude this. Aside from anything else, if r1 isn’t pointing to a valid link area then gdb cannot unwind the stack and produce backtraces. I discovered this empirically a while back ;)

My thinking at the moment is to leave r1 alone, and to use another register (r31 maybe) as the interpreter’s stack pointer. That way the interpreter can extend the stack however it likes, without thought to link areas and alignment, on the assumption that if it ever jumps out into C-land then it must first create a valid stack frame around it’s own data to protect it. Specifically, this will be a frame with no register save area, meaning the interpreter’s stuff falls neatly into the local variable space.

I’m not sure how stack walking will work under this scenario. It may be that it’s better to do this stack-shuffling every time the interpreter calls a new method, such that each method call, be it Java or C, has it’s own valid ABI stack frame. This will undoubtedly resolve itself as I progress.

I’ve been working on converting my existing 32-bit ppc code to ppc64. A lot of the ABI is the same (and my funky StackFrame class hides a lot of the differences) but one weird thing is that on ppc64 function pointers do not point to the actual code but to some descriptor thing. OpenJDK has loads of places that do the following, and they’re all wrong:

void (*function)() = generate_some_code();
...
(*function)();

I think the descriptors are supposed to be in some table that the linker creates, but I sneaked around it by making an assembler macro, function_entry_point(), that creates a function descriptor that points to the address immediately following it, the idea being that you place it at the start of anything you’re going to jump into from C. I’m not entirely happy with it: I’m not sure what the implications of mixing descriptors and code like this, and it makes the stubs non-relocatable, and omitting the function_entry_point() will cause a mysterious segfault as the processor dereferences the first two instructions of your stub and jumps to wherever they point. But needs must…

So as if to atone from my awful disassembler hack I wrote a really neat stack frame class. Say you’re writing some function that needs some non-volatile registers and maybe a local variable. You define the frame like this:

StackFrame frame = StackFrame();

const Register start_addr     = frame.get_register();
const Register end_addr       = frame.get_register();
const FloatRegister mr_floaty = frame.get_float_register();

const Address variable = frame.get_local_variable();

Once you defined the frame you just wrap your code with my funky new prolog and epilog macros and all the storing and restoring and stack pointer setting and alignment stuff gets done for you:

__ prolog (frame);

__ addi (end_addr, start_addr, 8);
__ lfd (mr_floaty, variable);
__ call (some_func);

__ epilog (frame);

As far as I can see the other OpenJDK architectures just do this kind of thing manually which for someone as slapdash as me is a whole world of pain.

Fun function of the week:

void disassemble(address start, address end)
{
  const char *fmt = "/tmp/aztec-%d.%c";
  char c_file[BUFSIZ], o_file[BUFSIZ];
  sprintf(c_file, fmt, getpid(), 'c');
  sprintf(o_file, fmt, getpid(), 'o');

  FILE *fp = fopen(c_file, "w");
  if (fp == NULL)
    fatal("%s:%d: can't write file", __FILE__, __LINE__);

  fputs("unsigned char start[] = {", fp);
  for (address a = start; a < end; a++) {
    if (a != start)
      fputc(',', fp);
    fprintf(fp, "0x%02x", *a);
  }
  fputs("};\n", fp);
  fclose(fp);

  char cmd[BUFSIZ];
  sprintf(cmd, "gcc -c %s -o %s", c_file, o_file);
  if (system(cmd) != 0)
    fatal("%s:%d: can't compile file", __FILE__, __LINE__);

  putchar('\n');
  sprintf(cmd, "objdump -D -j .data %s | grep '^....:'", o_file);
  if (system(cmd) != 0)
    fatal("%s:%d: can't disassemble file", __FILE__, __LINE__);
  putchar('\n');

  unlink(c_file);
  unlink(o_file);
}

It actually did work too!

#0  report_unimplemented (
    file_name=0xfb3d624 "aztec/hotspot/src/cpu/ppc/vm/icache_ppc.cpp",
    line_no=65)
    at aztec/hotspot/src/share/vm/utilities/debug.cpp:241
#1  0xf5b8405c in ?? ()
#2  0x0f760d84 in AbstractICache::invalidate_range (start=0xf5b86400 "",
    nbytes=15)
    at aztec/hotspot/src/share/vm/runtime/icache.cpp:102

Obviously this is like “so what?” for all you JIT guys but I’m kind of stunned here.

Well, my assembler worked, I think:

3c 60 0f b4     lis     r3, file@hi
60 63 68 e4     ori     r3, r3, file@lo
38 80 00 3a     li      r4, line
3d 40 0f 6e     lis     r10, report_unimplemented@hi
61 4a 82 60     ori     r10, r10, report_unimplemented@lo
7d 48 03 a6     mtlr    r10
4e 80 00 21     blrl
3d 40 0f e5     lis     r10, abort@hi
61 4a 7b a0     ori     r10, r10, abort@lo
7d 48 03 a6     mtlr    r10
4e 80 00 21     blrl

I finally bit the bullet and threw myself into PPC assembly. And what better way to ease you into your first assembly since 1998 than writing an assembler! It’s been a long couple of days but finally — finally! — it compiled and ran:

3c600fb4606368e43880003a3d400f6e614a82607d48
03a64e8000213d400fe5614a7ba07d4803a64e800021

Check back Monday to see if that’s garbage…

I’ve been busy filling in stubs recently, but something didn’t feel right. The stubs all say Unimplemented() — and that was calling itself. I’d been working towards the short term goal of getting Unimplemented() to work when I discovered that what lay beneath Unimplemented() and assert() was the most complicated error handling and debugging system in the world.

First it scans the command line to see if you told it to ignore the particular error it’s trying to report. Then it prints a message to tell you the command line options to make it skip that particular error. Then it checks to see if this is the first error it’s seen, if any other threads are trying to report errors, if any other threads are trying to report this particular error, whether this particular error is being reported as the first error by another thread, and whether this error is being reported as a result of an error in the error reporting system. Then it prepares messages of varying detail to print to a file if possible and the screen if not. It creates and executes shell scripts (!) to do who knows what. Then it thinks about quitting.

This is great, except that all this wacky stack and thread stuff is calling unimplemented code, and filling in stubs that the error reporting system is calling was getting the error reporting system working — not the VM!

The error reporting system now looks like this:

fprintf(stderr, "error: %s: %s:%d\n", _message, _filename, _lineno);
exit(EXIT_FAILURE);

Om Shanti.