New stack overflow code

It’s funny, but I’d kind of thought that the version of Zero that got upstreamed was pretty much cut and dried. It hadn’t had any real changes for months, after all, and the S/390 compatibility work required was so minimal—in terms of code changed, if not in terms of time taken!—that I didn’t think there was much more to do. But in this last couple of months I’ve made a ton of fixes, including one big change that’s upstream (improved stack overflow handling) and one that’s hopefully heading that way (some frame anchor fixes).

The new stack overflow code is pretty awesome (if I say so myself!) The old code jumped through some pretty bizarre hoops in order to throw the exception from the exact frame of the method being called. This is more or less how it happens in “classic” HotSpot, which decides which method you’re in from the ABI stack, but wasn’t workable for Zero which might have run out of Zero stack with no room to put the frame of the method from which you want to throw the exception. Argh! So I wrote this look-ahead detection thing, to throw the exception when you still had some stack to put your frame on, which worked, but was easily fooled. A valid Java method could have a frame far larger than the default thread stack size on most systems, so the lookahead worked on the assumption that a frame probably wouldn’t be that big. There were a number of places where the stack could overflow, in which case you’d hit an Unimplemented and the VM would bomb out. Not good, and not very secure either.

The new code is awesome. I realised that it doesn’t really matter exactly which frame the StackOverflowError got thrown from, only that it got thrown. I wrote a little method, ZeroStack::overflow_check, which is inline and can be called from anywhere—any VM state, etc. It does the check, and throws the exception, via ZeroStack::handle_overflow. That one is pretty complicated, but I did say I wanted the overflow check to be callable from anywhere, so it has to set up the frame anchor if it isn’t already, and cope with whatever thread state it finds itself in. The best thing about ZeroStack::handle_overflow? You can call it from Shark, which let me blow away all the ugly stack overflow code I was griping about in my last post.

Those Unimplementeds in the stack overflow detector were my least favourite thing about Zero, my guilty secret (in as much as open source code published on the internet is secret). And now they’re gone! No more ugly code in Zero! Until a few days later, when I discovered some stuff in the frame anchor that was reversed. But that’s for another day.