Is it possible to write a JIT compiler (to native code) entirely in a managed .NET language

I'm toying with the idea of writing a JIT compiler and am just wondering if it is even theoretically possible to write the whole thing in managed code. In particular, once you've generated assembler into a byte array how do you jump into it to begin execution?

Yes, you can. In fact, it's my job :)

I've written GPU.NET entirely in F# (modulo our unit tests) -- it actually disassembles and JITs IL at run-time, just like the .NET CLR does. We emit native code for whatever underlying acceleration device you want to use; currently we only support Nvidia GPU's, but I've designed our system to be retargetable with a minimum of work so it's likely we'll support other platforms in the future.

As for performance, I have F# to thank -- when compiled in optimized mode (with tailcalls), our JIT compiler itself is probably about as fast as the compiler within the CLR (which is written in C++, IIRC).

For execution, we have the benefit of being able to pass control to hardware drivers to run the jitted code; however, this wouldn't be any harder to do on the CPU since .NET supports function pointers to unmanaged/native code (though you'd lose any safety/security normally provided by .NET).

And for the full proof of concept here is a fully capable translation of Rasmus' approach to JIT into F#

open System
open System.Runtime.InteropServices

type AllocationType =
    | COMMIT=0x1000u

type MemoryProtection =
    | EXECUTE_READWRITE=0x40u

type FreeType =
    | DECOMMIT = 0x4000u

[<DllImport("kernel32.dll", SetLastError=true)>]
extern IntPtr VirtualAlloc(IntPtr lpAddress, UIntPtr dwSize, AllocationType flAllocationType, MemoryProtection flProtect);

[<DllImport("kernel32.dll", SetLastError=true)>]
extern bool VirtualFree(IntPtr lpAddress, UIntPtr dwSize, FreeType freeType);

let JITcode: byte[] = [|0x55uy;0x8Buy;0xECuy;0x8Buy;0x45uy;0x08uy;0xD1uy;0xC8uy;0x5Duy;0xC3uy|]

[<UnmanagedFunctionPointer(CallingConvention.Cdecl)>] 
type Ret1ArgDelegate = delegate of (uint32) -> uint32

[<EntryPointAttribute>]
let main (args: string[]) =
    let executableMemory = VirtualAlloc(IntPtr.Zero, UIntPtr(uint32(JITcode.Length)), AllocationType.COMMIT, MemoryProtection.EXECUTE_READWRITE)
    Marshal.Copy(JITcode, 0, executableMemory, JITcode.Length)
    let jitedFun = Marshal.GetDelegateForFunctionPointer(executableMemory, typeof<Ret1ArgDelegate>) :?> Ret1ArgDelegate
    let mutable test = 0xFFFFFFFCu
    printfn "Value before: %X" test
    test <- jitedFun.Invoke test
    printfn "Value after: %X" test
    VirtualFree(executableMemory, UIntPtr.Zero, FreeType.DECOMMIT) |> ignore
    0

that happily executes yielding

Value before: FFFFFFFC
Value after: 7FFFFFFE

The trick should be VirtualAlloc with the EXECUTE_READWRITE-flag (needs P/Invoke) and Marshal.GetDelegateForFunctionPointer.

Here is a modified version of the rotate integer example (note that no unsafe code is needed here):

[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
public delegate uint Ret1ArgDelegate(uint arg1);

public static void Main(string[] args){
    // Bitwise rotate input and return it.
    // The rest is just to handle CDECL calling convention.
    byte[] asmBytes = new byte[]
    {        
      0x55,             // push ebp
      0x8B, 0xEC,       // mov ebp, esp 
      0x8B, 0x45, 0x08, // mov eax, [ebp+8]
      0xD1, 0xC8,       // ror eax, 1
      0x5D,             // pop ebp 
      0xC3              // ret
    };
    
    // Allocate memory with EXECUTE_READWRITE permissions
	IntPtr executableMemory = 
        VirtualAlloc(
            IntPtr.Zero, 
            (UIntPtr) asmBytes.Length,    
            AllocationType.COMMIT,
            MemoryProtection.EXECUTE_READWRITE
        );

    // Copy the machine code into the allocated memory
	Marshal.Copy(asmBytes, 0, executableMemory, asmBytes.Length);

    // Create a delegate to the machine code.
    Ret1ArgDelegate del = 
        (Ret1ArgDelegate) Marshal.GetDelegateForFunctionPointer(
            executableMemory, 
            typeof(Ret1ArgDelegate)
        );

    // Call it
    uint n = (uint)0xFFFFFFFC;
    n = del(n);
    Console.WriteLine("{0:x}", n);

    // Free the memory
    VirtualFree(executableMemory, UIntPtr.Zero, FreeType.DECOMMIT);
 }

Full example (now works with both X86 and X64).

Using unsafe code, you can "hack" a delegate and make it point to an arbitrary assembly code that you generated and stored in an array. The idea is that delegate has a _methodPtr field, which can be set using Reflection. Here is some sample code:

• Inline x86 ASM in C#

This is, of course, a dirty hack that may stop working at any time when the .NET runtime changes.

I guess that, in principle, fully managed safe code cannot be allowed to implement JIT, because that would break any security assumptions that the runtime relies on. (Unless, the generated assembly code came with a machine-checkable proof that it does not violate the assumptions...)