The IRIX LLVM Port - Part 1

Back in ~June of 2021, I began porting clang and LLD (and by extension, LLVM) to Silicon Graphics' IRIX operating system. It was my first time ever hacking on any compiler or linker, and even though I was flying blind a lot of the time, it was incredibly interesting, and I learned a lot from the process. Along the way, I wrote a series of forum posts over on IRIXNet which described my progress as well as bugs I ran into while enabling IRIX support. These forum posts, mostly unchanged, are going to make up the contents of this series. As for the LLVM port itself, even though I ended up having to drop out of the effort due to real life circumstances, Vladimir Vukicevic ended up finishing the port, and tackled plenty of hard bugs related to ELF relocations and MIPS codegen

Hey all, I started working on getting clang/LLVM to output ELF binaries for IRIX a few weeks ago. I noticed a while back that LLVM already has 99% of what you need to get working binaries for IRIX. Its MIPS support is pretty good and it already supports a wide range of MIPS processors, along with support for the 3 major MIPS ABIs that IRIX uses, o32, n32, and n64. As a matter of fact, you don’t even need to necessarily modify LLVM in any way to be able to get freestanding object files that can theoretically be linked on IRIX. To my understanding, IRIXNet user Eschaton attempted to use LLVM in this way, compiling object files on Linux under clang, then trying to link under SGI’s linker. I wanted to take it a step further however, and get full blown IRIX support integrated into clang and LLVM, as well as to be able to natively host the full LLVM suite on IRIX itself.

I started off by attempting to get just LLVM, without clang or LLD, to build & run on IRIX. The main changes here were just small #ifdef __sgi patches to some files to fix some build errors, however I pretty quickly ran into a brick wall, linking all the LLVM static libraries to produce the various executables quickly caused the n32 GCC/Binutils to run out of memory. After banging my head against the wall for a few days trying to get a full n64 GCC/Binutils toolchain. I gave up and switched gears to instead try and teach clang about IRIX. This was actually relatively easy, LLVM has a clean & easy to read codebase, and it already has plenty of OS drivers built in. For IRIX, I decided to mostly copy Hurd’s driver file, and make some changes here and there to specify IRIX’s dynamic loader and system directories. There were some other miscellaneous changes I had to make in other files, mostly relating to target triplets, although I did also have to specify an IRIX OS Target, which again, wasn’t hard at all. 

Finally I built LLVM with only clang enabled, and I tested it… my hello world program didn’t build… clang was specifying that SGUG’s LD link using elf32btsmipn32, which it didn’t support. I made a quick clang patch to instead pass elf32bmipn32 if the target OS is IRIX. This time the build failed again, but for a different reason; for some reason binutils couldn’t find the C runtime or libgcc even though I had them in the sysroot I had passed to binutils when I built it. This gave me an idea though, I could just have clang output object files, then link them on a real SGI, similar to what the OpenVMS developers did when they were porting LLVM for x86. I uploaded helloworld.o to an Octane, then ran gcc on it to link it, and whaddya know, it printed “Hello world from LLVM on an SGI!”

Building helloworld.c on my desktop

Linking and running helloworld.o on IRIX

That was pretty cool, but I wanted to see if my clang/LLVM patches in combo with SGUG binutils on the Octane could handle something a bit more complex, so I got to work on building figlet. I immediately ran into some compilation errors. These were related to types, so I assumed that I needed to use the same include-fixed that GCC uses on IRIX. I added a system include to the IRIX driver for a /usr/include-fixed in addition to the usual /usr/include. However even with these fixes, figlet still didn’t build.

I was beginning to think that maybe I’d just gotten lucky after all with the hello world program. But then I realized that the IRIX C standard library headers enable/disable various types and prototypes depending on what flags the compiler has enabled by default. I didn’t feel like putting in the work to add them to clang yet, so I instead dumped all the predefines GCC has for IRIX, then stuck all of them into an irix-defines.h file, and added that to the top of all figlet source files so all of the predefines would apply before any IRIX headers were included. This actually worked, although it resulted in incredibly spammy output during the build since a lot of preprocessor macros were getting redefined by clang, or by the various system headers. I got my figlet object files and uploaded them all to the Octane. Figlet linked and actually ran!

There’s plenty of work to be done to get a useful LLVM toolchain for IRIX, for one, the various predefines GCC has for IRIX need to be added to clang so you don’t have to stick an irix-defines.h into every source file you want to compile. I also want to have clang/LLVM ported directly to IRIX so you don’t need to compile software using two different computers. Finally, I want to eventually be able to have a full LLVM toolchain that uses LLD, as well as the LLDB debugger. A few longer term goals would be to have full C ABI compatibility with MIPSPro so there aren’t any issues with struct alignment, to get LLD to emit debugging symbols that can be read by DBX, and finally, to clean up the patches enough to upstream IRIX support.

Towards the end of the original post, I expressed confusion over the fact that although though I was compiling n32 binaries using mips-sgi-irix6.5 as my target triplet, clang’s internal target triplet was converting this to mips64-sgi-irix6.5, but still outputting n32 binaries. IRIXNet user robespierre commented that n32 is a 64 bit ABI, and, well, turns out he is indeed correct. Up until then, I had assumed that n32 was simply a newer fully 32 bit ABI. I didn’t actually realize that it was more akin to AMD64’s x32 ABI, i.e. the program is fully 64 bit with the one exception of using 32 bit pointers, the reasoning being that the amount of memory occupied by pointers is cut in half.