gbenson.net

The segfault I was debugging the other day turned out a stack overflow caused by the Unimplemented() in Atomic::cmpxchg_ptr (void *, void *, void *) calling itself to generate the error message for the Unimplemented() in Atomic::xchg (jint, jint *). I guess this means it’s time to start writing assembler.

I had a little tangent yesterday while I tried to figure out what sizes all the different types were. aph and mjw pointed me at some stuff [1, 2 and 3]. Who knew?

	ILP32	LP64
int	32	32
long	32	64
void *	32	64
jint	32	32
jlong	64	64
intptr_t	32	64

I fixed the build on ppc64 while I was experimenting too.

Here’s a fun thing: all the commands in OpenJDK will execve() themselves to set LD_LIBRARY_PATH if it’s not what they want it to be. This totally unhooks your gdb session.

Here’s another fun thing: setting _JAVA_LAUNCHER_DEBUG=1 prints out a load of stuff so you can see this bit of tomfoolery. I added an extra bit of debugging in my repository, a dump of the environment so I can see what I need to be exporting to keep gdb hooked in.

Om Shanti.

Not much to report today, other than that I’ve switched from my weird half-IcedTea, half-IBM build environment to using entirely IcedTea. It felt like I was storing up some future hell for myself. Mostly I’ve been trying to figure out how to integrate my porting work into IcedTea so I can get it into a public repository.

I always plan to write about what I’ve been doing at the end of the day, but I never quite get round to it. Usually my head is too full of code, or something. So I end up with a piece of paper with scribble all over it to write about in the morning. Here’s yesterday’s notes.

1. There are two VMs, client and server. When you build the C++ interpreter it’s the server JIT it gets built into.
2. The templater that makes the stubs has some magic to support multi-model architectures. It works like this. To generate the stubs for ppc and ppc64 you run the templater like this:

   python porter/generate.py ppc

   Whether a ppc64-specific file is generated depends on a template’s filename. If it has CPU in it, you get ppc only; if it has CPUS you get ppc and ppc64. So from the template:

   porter/hotspot/src/cpu/CPU/vm/assembler_CPU.cpp

   you get:

   hotspot/src/cpu/ppc/vm/assembler_ppc.hpp

   but from the template:

   porter/hotspot/build/linux/makefiles/CPUS.make

   you get:

   hotspot/build/linux/makefiles/ppc.make
   hotspot/build/linux/makefiles/ppc64.make

   It’s clunky but it works.

3. This is possibly obvious, but sneaky too. Everything in a CPU template is evaluated once, for the main cpu (the one you specify on the command line). A consequence of this is that the templater function is64bit(cpu) will always return False if it’s in a CPU template. You need to use other methods to conditionalize this, #ifdef _LP64 for example.
4. My bootstrap environment is… interesting. XSLT with IcedTea’s gij/ecj combo produces mangled jvmti headers on ppc, but the IBM JDK from RHEL5 doesn’t understand version 50 class files. So I’m using the IBM JDK with IcedTea’s javac wrapper.

Ok, I just saw this:“Note that ARCH_DATA_MODEL is really only needed on Solaris to indicate you want to built the 64-bit version. And before the Solaris 64-bit binaries can be used, they must be merged with the binaries from a separate 32-bit build. The merged binaries may then be used in either 32-bit or 64-bit mode, with the selection occurring at runtime with the -d32 or -d64 options.”

I was away all last week on a training course where I caught a cold despite sneaking off between labs for yoga breaks. Now I’m sat here feeling like death and trying to figure out where I left off.

Ok, where I left off was that I finally made enough stubs for the build to finish building Hotspot and to start failing on the class library instead. I had been about to throw myself into the class library failures, but I found out that ppc64 is not the sleek modern version of ppc but rather its fat ugly sister so I rearranged the build system to favour ppc instead.

It’s quite neat. There’s an environment variable, ARCH_DATA_MODEL, which allows 32-bit VMs to be built on amd64 and 64-bit VMs to be built on sparc. It flips ARCH — though not everywhere! — to i486 (on amd64 with ARCH_DATA_MODEL=32) and to sparcv9 (on sparc with ARCH_DATA_MODEL=64). I think it’s only used on Solaris at present, but the best bit for me is that there is no sparcv9 directory — it shares with sparc. I fixed it so ppc/ppc64 and s390/s390x do the same thing, so you’ll get a ppc64/s390x build if you set ARCH_DATA_MODEL=64 and a ppc/s390 build otherwise. So now I only have three sets of stubs to worry about, not five.

Yesterday marked an important milestone in my portable interpreter work in that I finally got enough stubs written to get a libjvm.so with no undefined references.

I have mixed feelings about this portable interpreter work at the moment. On the one hand it’s going great: every day feels like I’m progressing faster and faster, partly because I’ve written tools to do the awkward bits and partly because now when I get build errors I have a much better idea of where to fix them. But on the other hand it’s a little daunting, because it’s becoming clear that some big things have changed since it was last used and the lack of revision history means I can’t just go back and see how they were changed in the JITs. So I don’t know what to think.

One cool thing is that before I started trying to build the portable interpreter I toyed with the idea of implementing the JVM interface in a free VM and using it as a replacement for libjvm.so — and it turns out twisti is doing just that! So my “maybe this would work” fallback has turned into a “this will work” without me lifting a finger!

This past few days I’ve been wrestling with a minor flaw in my porting plan, namely that some files are reference not methods but constants, and I have no idea what values to assign to them. I’ve been picking random values to get the build to progress, but that’s just storing up hard to debug errors for when I finally get to run the thing. But I finally figured out how to do this properly:

• Every time make complains about a missing file, touch it.
• Every time something complains about a missing method, stub it with something that throws an error.
• Every time something complains about a missing constant, create one randomly but mark it with XXX EVIL.

Then:

• Remove all evil constants, and stub everything that needed them with the same error-throwing thing as before.
• Every time a stub gets hit, write the real method.

I’m glad I figured this one out, it’s been bugging me.

Yesterday I made a templater to generate machine-specific files for OpenJDK. I’m not 100% convinced that all the files make says are missing are necessary — the portable interpreter is old code, and it’s possible there are files listed as dependencies that are only required for JITs — and I don’t intend writing assembler that isn’t used so the way I plan to proceed is:

• Every time make complains about a missing file, touch it.
• Every time something complains about a missing method or whatever, stub it.
• Every time a stub gets hit, write the real method.

The great thing about this is that the first two steps can be done in the templater. Nominally I’m only supposed to be getting it running on ppc, but there’s four other platforms that we care about and others that other distributions want. The idea is that the longer I can keep my work generic the easier individual ports will be.