I&#39;m looking at writing a JIT compiler for a hobby virtual machine I&#39;ve been working on recently. I know a bit of assembly, (I&#39;m mainly a C programmer. I can read most assembly with  reference for opcodes I don&#39;t understand, and write some simple programs.) but I&#39;m having a hard time understanding the few examples of self-modifying code I&#39;ve found online.

This is one such example: http://asm.sourceforge.net/articles/smc.html

The example program provided does about four different modifications when run, none of which are clearly explained. Linux kernel interrupts are used several times, and aren&#39;t explained or detailed. (The author moved data into several registers before calling the interrupts. I assume he was passing arguments, but these arguments aren&#39;t explained at all, leaving the reader to guess.)

What I&#39;m looking for is the simplest, most straightforward example in code of a self-modifying program. Something that I can look at, and use to understand how self-modifying code in x86 assembly has to be written, and how it works. Are there any resources you can point me to, or any examples you can give that would adequately demonstrate this?

I&#39;m using NASM as my assembler.

EDIT: I&#39;m also running this code on Linux.

How to write self-modifying code in x86 assembly

EDIT: unfortunately LuaJIT was taken out of the comparison in the link below.

This &lt;a href=&quot;http://shootout.alioth.debian.org/u64/which-programming-languages-are-fastest.php&quot;&gt;comparison&lt;/a&gt; of programming languages shows that LuaJIT has an over tenfold improvement over the normal Lua implementation.
Why is the change so big? Is there something specific about Lua that makes it benefit a lot from JIT compilation?
Python is dynamically typed and compiled to bytecode as well, so why doesn&#39;t PyPy (that has JIT now, I believe) show such a large jump in performance?

Why is LuaJIT so good?

If I am using a constant that is **needed only in a method**, is it best to declare the const within the method scope, or in the class scope? Is there better performance declaring it in the method? If that is true, I think it&#39;s more standard to define them at class scope (top of file) to change the value and recompile easier.

    public class Bob
    {
       private const int SomeConst = 100; // declare it here?
       public void MyMethod()
       {
          const int SomeConst = 100; // or declare it here?
          // Do something with SomeConst
       }
    }

Defining Local Variable const vs Class const

Many dynamic languages implement (or want to implement) a JIT Compiler in order to speed up their execution times. Inevitably, someone from the peanut gallery asks why they don&#39;t use LLVM. The answer is often, &quot;LLVM is unsuitable for building a JIT.&quot; (For Example, Armin Rigo&#39;s comment [here.][1])

**Why is LLVM Unsuitable for building a JIT?**

Note: I know LLVM has its own JIT. If LLVM used to be unsuitable, but now is suitable, please say what changed. I&#39;m not talking about running LLVM Bytecode on the LLVM JIT, I&#39;m talking about using the LLVM libraries to implement a JIT for a dynamic language.


  [1]: http://morepypy.blogspot.com/2008/04/float-operations-for-jit.html?showComment=1208773800000#c6322995253226647458

Why is llvm considered unsuitable for implementing a JIT?

We are making some `kNN` and `SVD` implementations in Python.  Others picked Java. Our execution times are very different. I used cProfile to see where I make mistakes but everything is quite [fine][1] actually. Yes, I use `numpy` also. But I would like to ask simple question.

    total = 0.0
    for i in range(9999): # xrange is slower according 
        for j in range(1, 9999):            #to my test but more memory-friendly.
            total += (i / j)
    print total

This snippet takes 31.40s on my computer.

Java version of this code takes 1 second or less on the same computer. Type checking is a main problem for this code, I suppose. But I should make lots of operation like this for my project and I think 9999*9999 is not so big number.

I think I am making mistakes because I know Python is used by lots of scientific projects. But why is this code so slow and how can I handle problems bigger than this?

Should I use a JIT compiler such as `Psyco`?

## EDIT ##

I also say that this loop problem is only an example. The code is not as simple as like this and It may be tough to put into practice your improvements/code samples.

Another question is that can I implement lots of data mining &amp; machine learning algorithms with `numpy` and `scipy` if I use it correctly? 

 


  [1]: http://wiki.python.org/moin/PythonSpeed/PerformanceTips

Why Python is so slow for a simple for loop?

Both of them pretty much do the same thing. Identify that the method is hot and compile it instead of interpreting. With OSR, you just move to the compiled version right after it gets compiled, unlike with JIT, where the compiled code gets called when the method is called for the second time. 

Other than this, are there any other differences?

Differences between Just in Time compilation and On Stack Replacement

What is the difference between the JIT compiler and CLR? If you compile your code to il and CLR runs that code then what is the JIT doing? How has JIT compilation changed with the addition of generics to the CLR?

CLR vs JIT

I have heard that Java must use a JIT to be fast. This makes perfect sense when comparing to interpretation, but why can&#39;t someone make an ahead-of-time compiler that generates fast Java code? I know about `gcj`, but I don&#39;t think its output is typically faster than Hotspot for example.

Are there things about the language that make this difficult? I think it comes down to just these things:

 * Reflection
 * Classloading

What am I missing? If I avoid these features, would it be possible to compile Java code once to native machine code and be done?

Why is Java faster when using a JIT vs. compiling to machine code?

Surprisingly, the following code outputs:
```none
/
-1
```
The code:
```
public class LoopOutPut {

    public static void main(String[] args) {
        LoopOutPut loopOutPut = new LoopOutPut();
        for (int i = 0; i &lt; 30000; i++) {
            loopOutPut.test();
        }

    }

    public void test() {
        int i = 8;
        while ((i -= 3) &gt; 0) ;
        String value = i + &quot;&quot;;
        if (!value.equals(&quot;-1&quot;)) {
            System.out.println(value);
            System.out.println(i);
        }
    }

}
```

I tried many times to determine how many times this would occur, but, unfortunately, it was ultimately uncertain, and I found that the output of -2 sometimes turned into a period. In addition, I also tried to remove the while loop and output -1 without any problems. Who can tell me why?

----------------------------------

JDK version information:
```none
HopSpot 64-Bit 1.8.0.171
IDEA 2019.1.1
```

The output -1 becomes a slash in the loop

According to the computer language benchmark game, the LuaJIT implementation [seems to beat every other JIT-ed dynamic language][1] (V8, Tracemonkey, PLT Scheme, Erlang HIPE) by **an order of magnitude**. 

I know that these benchmarks are not representative (as they say: &quot;Which programming language implementations have the fastest benchmark programs?&quot;), but this is still really impressive.

In practice, is it really the case? Someone have tested that Lua implementation?


  [1]: http://shootout.alioth.debian.org/u32/which-programming-languages-are-fastest.php?gpp=on&amp;luajit=on&amp;sbcl=on&amp;mzscheme=on&amp;hipe=on&amp;v8=on&amp;vw=on&amp;lua=on&amp;calc=chart

Is LuaJIT really faster than every other JIT-ed dynamic languages?

The following Java program takes on average between 0.50 secs and 0.55 secs to run:

    public static void main(String[] args) {
        long startTime = System.nanoTime();
        int n = 0;
        for (int i = 0; i &lt; 1000000000; i++) {
            n += 2 * (i * i);
        }
        System.out.println((double) (System.nanoTime() - startTime) / 1000000000 + &quot; s&quot;);
        System.out.println(&quot;n = &quot; + n);
    }

If I replace `2 * (i * i)` with `2 * i * i`, it takes between 0.60 and 0.65 secs to run. How come?

I ran each version of the program 15 times, alternating between the two. Here are the results:

     2*(i*i)  |  2*i*i
    ----------+----------
    0.5183738 | 0.6246434
    0.5298337 | 0.6049722
    0.5308647 | 0.6603363
    0.5133458 | 0.6243328
    0.5003011 | 0.6541802
    0.5366181 | 0.6312638
    0.515149  | 0.6241105
    0.5237389 | 0.627815
    0.5249942 | 0.6114252
    0.5641624 | 0.6781033
    0.538412  | 0.6393969
    0.5466744 | 0.6608845
    0.531159  | 0.6201077
    0.5048032 | 0.6511559
    0.5232789 | 0.6544526

The fastest run of `2 * i * i` took longer than the slowest run of `2 * (i * i)`. If they had the same efficiency, the probability of this happening would be less than `1/2^15 * 100% = 0.00305%`.


Why is 2 * (i * i) faster than 2 * i * i in Java?

I&#39;ve been hearing a lot about the [PyPy][1] project. They claim it is 6.3 times faster than the [CPython][2] interpreter on [their site][3].

Whenever we talk about dynamic languages like Python, speed is one of the top issues. To solve this, they say PyPy is 6.3 times faster.

The second issue is parallelism, the infamous [Global Interpreter Lock][4] (GIL). For this, PyPy says it [can give GIL-less Python][5].

If PyPy can solve these great challenges, what are its weaknesses that are preventing wider adoption? That is to say, what&#39;s preventing someone like me, a typical Python developer, from switching to PyPy *right now*? 

  [1]: http://en.wikipedia.org/wiki/PyPy
  [2]: http://en.wikipedia.org/wiki/CPython
  [3]: http://speed.pypy.org
  [4]: http://en.wikipedia.org/wiki/Global_Interpreter_Lock
  [5]: http://pypy.org/tmdonate.html


Why shouldn&#39;t I use PyPy over CPython if PyPy is 6.3 times faster?

What does a JIT compiler specifically do as opposed to a non-JIT compiler? Can someone give a succinct and easy to understand description?



What does a just-in-time (JIT) compiler do?

The canonical JVM implementation from Sun applies some pretty sophisticated optimization to bytecode to obtain near-native execution speeds after the code has been run a few times. 

The question is, why isn&#39;t this compiled code cached to disk for use during subsequent uses of the same function/class?

As it stands, every time a program is executed, the JIT compiler kicks in afresh, rather than using a pre-compiled version of the code. Wouldn&#39;t adding this feature add a significant boost to the initial run time of the program, when the bytecode is essentially being interpreted?

Why doesn&#39;t the JVM cache JIT compiled code?

Let&#39;s say the bottleneck of my Java program really is some tight loops to compute a bunch of vector dot products. Yes I&#39;ve profiled, yes it&#39;s the bottleneck, yes it&#39;s significant, yes that&#39;s just how the algorithm is, yes I&#39;ve run Proguard to optimize the byte code, etc.

The work is, essentially, dot products. As in, I have two `float[50]` and I need to compute the sum of pairwise products. I know processor instruction sets exist to perform these kind of operations quickly and in bulk, like SSE or MMX.

Yes I can probably access these by writing some native code in JNI. The JNI call turns out to  be pretty expensive.

I know you can&#39;t guarantee what a JIT will compile or not compile. Has anyone *ever* heard of a JIT generating code that uses these instructions? and if so, is there anything about the Java code that helps make it compilable this way?

Probably a &quot;no&quot;; worth asking.

Do any JVM&#39;s JIT compilers generate code that uses vectorized floating point instructions?

I&#39;ve become a bit confused about the details of how the JIT compiler works. I know that C# compiles down to IL. The first time it is run it is JIT&#39;d. Does this involve it getting translated into native code? Is the .NET runtime (as a Virtual Machine?) interact with the JIT&#39;d code? I know this is naive, but I&#39;ve really confused myself. My impression has always been that the assemblies are not interpreted by the .NET Runtime but I don&#39;t understand the details of the interaction. 

C# JIT compiling and .NET

Is there some way to see the native code produces by the JIT in a JVM?

How to see JIT-compiled code in JVM?

I was going through a document which talks about [just-in-time compiler][1] (JIT) optimization techniques for Java. One of them was &quot;loop inversion&quot;. And the document says:

&gt; You replace a regular `while` loop with a `do-while` loop. And the
&gt; `do-while` loop is set within an `if` clause. This replacement leads to two fewer jumps.

How does loop inversion work and how does it optimize our code path?

**N.B.:** *It would be great if somebody can explain with an example of Java code and how JIT optimizes it to native code and why is it optimal in modern processors.*

  [1]: http://en.wikipedia.org/wiki/Just-in-time_compilation


What is the loop inversion technique?

I found that when I ask something more to Python, python doesn&#39;t use my machine resource at 100% and it&#39;s not really fast, it&#39;s fast if compared to many other interpreted languages, but when compared to compiled languages i think that the difference is really remarkable.

Is it possible to speedup things with a Just In Time (JIT) compiler in Python 3?

Usually a JIT compiler is the only thing that can improve performances in interpreted languages, so i&#39;m referring to this one, if other solutions are available i would love to accept new answers.

Does the Python 3 interpreter have a JIT feature?

**Intro:**  I write high-performance code in C#.  Yes, I know C++ would give me better optimization, but I still choose to use C#.  I do not wish to debate that choice.  Rather, I&#39;d like to hear from those who, like me, are trying to write high-performance code on the .NET Framework.

**Questions:**

 - Why is the operator in the code below slower than the equivalent
   method call??
 - Why is the method passing two doubles in the code below
   faster than the equivalent method passing a struct that has two
   doubles inside? (A: older JITs optimize structs poorly)
 - Is there a way to get the .NET JIT Compiler to treat
   simple structs as efficiently as the members of the struct? (A: get newer JIT)

**What I think I know:**
The original .NET JIT Compiler would not inline anything that involved a struct.  Bizarre given structs should only be used where you need small value types that should be optimized like built-ins, but true.  Fortunately, in .NET 3.5SP1 and .NET 2.0SP2, they made some improvements to the JIT Optimizer, including improvements to inlining, particularly for structs.  (I am guessing they did that because otherwise the new Complex struct that they were introducing would have performed horribly... so the Complex team was probably pounding on the JIT Optimizer team.)  So, any documentation prior to .NET 3.5 SP1 is probably not too relevant to this issue.

**What my testing shows:**
I have verified that I do have the newer JIT Optimizer by checking that C:\Windows\Microsoft.NET\Framework\v2.0.50727\mscorwks.dll file does have version &gt;= 3053 and so should have those improvements to the JIT optimizer.  However, even with that, what my timings and looks at the disassembly both show are:

The JIT-produced code for passing a struct with two doubles is far less efficient than code that directly passes the two doubles.

The JIT-produced code for a struct method passes in &#39;this&#39; far more efficiently than if you passed a struct as an argument.

The JIT still inlines better if you pass two doubles rather than passing a struct with two doubles, even with the multiplier due to being clearly in a loop.

**The Timings:**
Actually, looking at the disassembly I realize that most of the time in the loops is just accessing the test data out of the List&lt;Element&gt;.  The difference between the four ways of making the same calls is dramatically different if you factor out the overhead code of the loop and the accessing of the data.  I get anywhere from 5x to 20x speedups for doing PlusEqual(double, double) instead of PlusEqual(Element).  And 10x to 40x for doing PlusEqual(double, double) instead of operator +=.  Wow.  Sad.

Here&#39;s one set of timings:

    Populating List&lt;Element&gt; took 320ms.
    The PlusEqual() method took 105ms.
    The &#39;same&#39; += operator took 131ms.
    The &#39;same&#39; -= operator took 139ms.
    The PlusEqual(double, double) method took 68ms.
    The do nothing loop took 66ms.
    The ratio of operator with constructor to method is 124%.
    The ratio of operator without constructor to method is 132%.
    The ratio of PlusEqual(double,double) to PlusEqual(Element) is 64%.
    If we remove the overhead time for the loop accessing the elements from the List...
    The ratio of operator with constructor to method is 166%.
    The ratio of operator without constructor to method is 187%.
    The ratio of PlusEqual(double,double) to PlusEqual(Element) is 5%.

**The Code:**

    namespace OperatorVsMethod
    {
      public struct Element
      {
        public double Left;
        public double Right;
    
        public Element(double left, double right)
        {
          this.Left = left;
          this.Right = right;
        }
    
        public static Element operator +(Element x, Element y)
        {
          return new Element(x.Left + y.Left, x.Right + y.Right);
        }
    
        public static Element operator -(Element x, Element y)
        {
          x.Left += y.Left;
          x.Right += y.Right;
          return x;
        }    
    
        /// &lt;summary&gt;
        /// Like the += operator; but faster.
        /// &lt;/summary&gt;
        public void PlusEqual(Element that)
        {
          this.Left += that.Left;
          this.Right += that.Right;
        }    
    
        /// &lt;summary&gt;
        /// Like the += operator; but faster.
        /// &lt;/summary&gt;
        public void PlusEqual(double thatLeft, double thatRight)
        {
          this.Left += thatLeft;
          this.Right += thatRight;
        }    
      }    
      
      [TestClass]
      public class UnitTest1
      {
        [TestMethod]
        public void TestMethod1()
        {
          Stopwatch stopwatch = new Stopwatch();
    
          // Populate a List of Elements to multiply together
          int seedSize = 4;
          List&lt;double&gt; doubles = new List&lt;double&gt;(seedSize);
          doubles.Add(2.5d);
          doubles.Add(100000d);
          doubles.Add(-0.5d);
          doubles.Add(-100002d);
    
          int size = 2500000 * seedSize;
          List&lt;Element&gt; elts = new List&lt;Element&gt;(size);
    
          stopwatch.Reset();
          stopwatch.Start();
          for (int ii = 0; ii &lt; size; ++ii)
          {
            int di = ii % seedSize;
            double d = doubles[di];
            elts.Add(new Element(d, d));
          }
          stopwatch.Stop();
          long populateMS = stopwatch.ElapsedMilliseconds;
    
          // Measure speed of += operator (calls ctor)
          Element operatorCtorResult = new Element(1d, 1d);
          stopwatch.Reset();
          stopwatch.Start();
          for (int ii = 0; ii &lt; size; ++ii)
          {
            operatorCtorResult += elts[ii];
          }
          stopwatch.Stop();
          long operatorCtorMS = stopwatch.ElapsedMilliseconds;
    
          // Measure speed of -= operator (+= without ctor)
          Element operatorNoCtorResult = new Element(1d, 1d);
          stopwatch.Reset();
          stopwatch.Start();
          for (int ii = 0; ii &lt; size; ++ii)
          {
            operatorNoCtorResult -= elts[ii];
          }
          stopwatch.Stop();
          long operatorNoCtorMS = stopwatch.ElapsedMilliseconds;
    
          // Measure speed of PlusEqual(Element) method
          Element plusEqualResult = new Element(1d, 1d);
          stopwatch.Reset();
          stopwatch.Start();
          for (int ii = 0; ii &lt; size; ++ii)
          {
            plusEqualResult.PlusEqual(elts[ii]);
          }
          stopwatch.Stop();
          long plusEqualMS = stopwatch.ElapsedMilliseconds;
    
          // Measure speed of PlusEqual(double, double) method
          Element plusEqualDDResult = new Element(1d, 1d);
          stopwatch.Reset();
          stopwatch.Start();
          for (int ii = 0; ii &lt; size; ++ii)
          {
            Element elt = elts[ii];
            plusEqualDDResult.PlusEqual(elt.Left, elt.Right);
          }
          stopwatch.Stop();
          long plusEqualDDMS = stopwatch.ElapsedMilliseconds;
    
          // Measure speed of doing nothing but accessing the Element
          Element doNothingResult = new Element(1d, 1d);
          stopwatch.Reset();
          stopwatch.Start();
          for (int ii = 0; ii &lt; size; ++ii)
          {
            Element elt = elts[ii];
            double left = elt.Left;
            double right = elt.Right;
          }
          stopwatch.Stop();
          long doNothingMS = stopwatch.ElapsedMilliseconds;
    
          // Report results
          Assert.AreEqual(1d, operatorCtorResult.Left, &quot;The operator += did not compute the right result!&quot;);
          Assert.AreEqual(1d, operatorNoCtorResult.Left, &quot;The operator += did not compute the right result!&quot;);
          Assert.AreEqual(1d, plusEqualResult.Left, &quot;The operator += did not compute the right result!&quot;);
          Assert.AreEqual(1d, plusEqualDDResult.Left, &quot;The operator += did not compute the right result!&quot;);
          Assert.AreEqual(1d, doNothingResult.Left, &quot;The operator += did not compute the right result!&quot;);
    
          // Report speeds
          Console.WriteLine(&quot;Populating List&lt;Element&gt; took {0}ms.&quot;, populateMS);
          Console.WriteLine(&quot;The PlusEqual() method took {0}ms.&quot;, plusEqualMS);
          Console.WriteLine(&quot;The &#39;same&#39; += operator took {0}ms.&quot;, operatorCtorMS);
          Console.WriteLine(&quot;The &#39;same&#39; -= operator took {0}ms.&quot;, operatorNoCtorMS);
          Console.WriteLine(&quot;The PlusEqual(double, double) method took {0}ms.&quot;, plusEqualDDMS);
          Console.WriteLine(&quot;The do nothing loop took {0}ms.&quot;, doNothingMS);
    
          // Compare speeds
          long percentageRatio = 100L * operatorCtorMS / plusEqualMS;
          Console.WriteLine(&quot;The ratio of operator with constructor to method is {0}%.&quot;, percentageRatio);
          percentageRatio = 100L * operatorNoCtorMS / plusEqualMS;
          Console.WriteLine(&quot;The ratio of operator without constructor to method is {0}%.&quot;, percentageRatio);
          percentageRatio = 100L * plusEqualDDMS / plusEqualMS;
          Console.WriteLine(&quot;The ratio of PlusEqual(double,double) to PlusEqual(Element) is {0}%.&quot;, percentageRatio);
    
          operatorCtorMS -= doNothingMS;
          operatorNoCtorMS -= doNothingMS;
          plusEqualMS -= doNothingMS;
          plusEqualDDMS -= doNothingMS;
          Console.WriteLine(&quot;If we remove the overhead time for the loop accessing the elements from the List...&quot;);
          percentageRatio = 100L * operatorCtorMS / plusEqualMS;
          Console.WriteLine(&quot;The ratio of operator with constructor to method is {0}%.&quot;, percentageRatio);
          percentageRatio = 100L * operatorNoCtorMS / plusEqualMS;
          Console.WriteLine(&quot;The ratio of operator without constructor to method is {0}%.&quot;, percentageRatio);
          percentageRatio = 100L * plusEqualDDMS / plusEqualMS;
          Console.WriteLine(&quot;The ratio of PlusEqual(double,double) to PlusEqual(Element) is {0}%.&quot;, percentageRatio);
        }
      }
    }

**The IL:** (aka. what some of the above gets compiled into)

    public void PlusEqual(Element that)
        {
    00000000 push    ebp 
    00000001 mov     ebp,esp 
    00000003 push    edi 
    00000004 push    esi 
    00000005 push    ebx 
    00000006 sub     esp,30h 
    00000009 xor     eax,eax 
    0000000b mov     dword ptr [ebp-10h],eax 
    0000000e xor     eax,eax 
    00000010 mov     dword ptr [ebp-1Ch],eax 
    00000013 mov     dword ptr [ebp-3Ch],ecx 
    00000016 cmp     dword ptr ds:[04C87B7Ch],0 
    0000001d je     00000024 
    0000001f call    753081B1 
    00000024 nop       
          this.Left += that.Left;
    00000025 mov     eax,dword ptr [ebp-3Ch] 
    00000028 fld     qword ptr [ebp+8] 
    0000002b fadd    qword ptr [eax] 
    0000002d fstp    qword ptr [eax] 
          this.Right += that.Right;
    0000002f mov     eax,dword ptr [ebp-3Ch] 
    00000032 fld     qword ptr [ebp+10h] 
    00000035 fadd    qword ptr [eax+8] 
    00000038 fstp    qword ptr [eax+8] 
        }
    0000003b nop       
    0000003c lea     esp,[ebp-0Ch] 
    0000003f pop     ebx 
    00000040 pop     esi 
    00000041 pop     edi 
    00000042 pop     ebp 
    00000043 ret     10h 
     public void PlusEqual(double thatLeft, double thatRight)
        {
    00000000 push    ebp 
    00000001 mov     ebp,esp 
    00000003 push    edi 
    00000004 push    esi 
    00000005 push    ebx 
    00000006 sub     esp,30h 
    00000009 xor     eax,eax 
    0000000b mov     dword ptr [ebp-10h],eax 
    0000000e xor     eax,eax 
    00000010 mov     dword ptr [ebp-1Ch],eax 
    00000013 mov     dword ptr [ebp-3Ch],ecx 
    00000016 cmp     dword ptr ds:[04C87B7Ch],0 
    0000001d je     00000024 
    0000001f call    75308159 
    00000024 nop       
          this.Left += thatLeft;
    00000025 mov     eax,dword ptr [ebp-3Ch] 
    00000028 fld     qword ptr [ebp+10h] 
    0000002b fadd    qword ptr [eax] 
    0000002d fstp    qword ptr [eax] 
          this.Right += thatRight;
    0000002f mov     eax,dword ptr [ebp-3Ch] 
    00000032 fld     qword ptr [ebp+8] 
    00000035 fadd    qword ptr [eax+8] 
    00000038 fstp    qword ptr [eax+8] 
        }
    0000003b nop       
    0000003c lea     esp,[ebp-0Ch] 
    0000003f pop     ebx 
    00000040 pop     esi 
    00000041 pop     edi 
    00000042 pop     ebp 
    00000043 ret     10h 

Why are operators so much slower than method calls?  (structs are slower only on older JITs)

I&#39;m toying with the idea of writing a JIT compiler and am just wondering if it is even theoretically possible to write the whole thing in managed code. In particular, once you&#39;ve generated assembler into a byte array how do you jump into it to begin execution?


Is it possible to write a JIT compiler (to native code) entirely in a managed .NET language

I was benchmarking some code, and I could not get it to run as fast as with [`java.math.BigInteger`][1], even when using the exact same algorithm.
So I copied [`java.math.BigInteger`][1] source into my own package and tried this:

    //import java.math.BigInteger;

    public class MultiplyTest {
    	public static void main(String[] args) {
	    	Random r = new Random(1);
     		long tm = 0, count = 0,result=0;
    		for (int i = 0; i &lt; 400000; i++) {
    			int s1 = 400, s2 = 400;
    			BigInteger a = new BigInteger(s1 * 8, r), b = new BigInteger(s2 * 8, r);
    			long tm1 = System.nanoTime();
    			BigInteger c = a.multiply(b);
    			if (i &gt; 100000) {
    				tm += System.nanoTime() - tm1;
    				count++;
    			}
    			result+=c.bitLength();
    		}
    		System.out.println((tm / count) + &quot;nsec/mul&quot;);
    		System.out.println(result); 
    	}
    }

When I run this (jdk 1.8.0_144-b01 on MacOS) it outputs:

    12089nsec/mul
    2559044166

When I run it with the import line uncommented:

    4098nsec/mul
    2559044166

It&#39;s almost three times as fast when using the JDK version of BigInteger versus my version, even if it&#39;s using the exact same code.

I&#39;ve examined the bytecode with javap, and compared compiler output when running with options:

    -Xbatch -XX:-TieredCompilation -XX:+PrintCompilation -XX:+UnlockDiagnosticVMOptions 
    -XX:+PrintInlining -XX:CICompilerCount=1

and both versions seem to generate the same code.
So is hotspot using some precomputed optimisations that I can&#39;t use in my code? I always understood that they don&#39;t.
What explains this difference?


  [1]: https://docs.oracle.com/javase/7/docs/api/java/math/BigInteger.html

Does Java JIT cheat when running JDK code?

I&#39;ve heard this term used, but I&#39;m not entirely sure what it means, so:

* What DOES it mean and what DOESN&#39;T it mean?
* What are some examples of what IS and ISN&#39;T microbenchmarking?
* What are the dangers of microbenchmarking and how do you avoid it?
   * (or is it a good thing?)

What is microbenchmarking?

The following code gives different output when running the release inside Visual Studio, and running the release outside Visual Studio. I&#39;m using Visual Studio 2008 and targeting .NET 3.5. I&#39;ve also tried .NET 3.5 SP1.

When running outside Visual Studio, the JIT should kick in. Either (a) there&#39;s something subtle going on with C# that I&#39;m missing or (b) the JIT is actually in error. I&#39;m doubtful that the JIT can go wrong, but I&#39;m running out of other possiblities...

Output when running inside Visual Studio:

        0 0,
        0 1,
        1 0,
        1 1,

Output when running release outside of Visual Studio:

        0 2,
        0 2,
        1 2,
        1 2,

What is the reason?

    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;

    namespace Test
    {
        struct IntVec
        {
            public int x;
            public int y;
        }

        interface IDoSomething
        {
            void Do(IntVec o);
        }

        class DoSomething : IDoSomething
        {
            public void Do(IntVec o)
            {
                Console.WriteLine(o.x.ToString() + &quot; &quot; + o.y.ToString()+&quot;,&quot;);
            }
        }

        class Program
        {
            static void Test(IDoSomething oDoesSomething)
            {
                IntVec oVec = new IntVec();
                for (oVec.x = 0; oVec.x &lt; 2; oVec.x++)
                {
                    for (oVec.y = 0; oVec.y &lt; 2; oVec.y++)
                    {
                        oDoesSomething.Do(oVec);
                    }
                }
            }

            static void Main(string[] args)
            {
                Test(new DoSomething());
                Console.ReadLine();
            }
        }
    }


.NET 3.5 JIT not working when running the application

I&#39;ve been thinking about it lately, and it seems to me that most advantages given to **JIT** compilation should more or less be attributed to the intermediate format instead, and that jitting in itself is not much of a good way to generate code.

So these are the main **pro-JIT** compilation arguments I usually hear:

1. **Just-in-time compilation allows for greater portability.** Isn&#39;t that attributable to the intermediate format? I mean, nothing keeps you from compiling your virtual bytecode into native bytecode once you&#39;ve got it on your machine. Portability is an issue in the &#39;distribution&#39; phase, not during the &#39;running&#39; phase.
1. **Okay, then what about generating code at runtime?** Well, the same applies. Nothing keeps you from integrating a just-in-time compiler for a real just-in-time need into your native program.
1. **But the runtime compiles it to native code just once anyways, and stores the resulting executable in some sort of cache somewhere on your hard drive.** Yeah, sure. But it&#39;s optimized your program under time constraints, and it&#39;s not making it better from there on. See the next paragraph.

It&#39;s not like **ahead-of-time** compilation had no advantages either. **Just-in-time** compilation has time constraints: you can&#39;t keep the end user waiting forever while your program launches, so it has a tradeoff to do somewhere. Most of the time they just optimize less. A friend of mine had _profiling evidence_ that inlining functions and unrolling loops &quot;manually&quot; (obfuscating source code in the process) had a positive impact on performance on his **C#** number-crunching program; doing the same on my side, with my **C** program filling the same task, yielded no positive results, and I believe this is due to the extensive transformations my compiler was allowed to make. 

And yet we&#39;re surrounded by jitted programs. **C#** and **Java** are everywhere, Python scripts can compile to some sort of bytecode, and I&#39;m sure a whole bunch of other programming languages do the same. There must be a good reason that I&#39;m missing. So what makes **just-in-time** compilation so superior to **ahead-of-time** compilation?


----------


**EDIT** To clear some confusion, maybe it would be important to state that I&#39;m all for an intermediate representation of executables. This has a lot of advantages (and really, most arguments for **just-in-time** compilation are actually arguments for an intermediate representation). My question is about how they should be compiled to native code.

Most runtimes (or compilers for that matter) will prefer to either compile them just-in-time or ahead-of-time. As **ahead-of-time** compilation looks like a better alternative to me because the compiler has more time to perform optimizations, I&#39;m wondering why Microsoft, Sun and all the others are going the other way around. I&#39;m kind of dubious about profiling-related optimizations, as my experience with **just-in-time** compiled programs displayed poor basic optimizations.

I used an example with C code only because I needed an example of **ahead-of-time** compilation versus **just-in-time** compilation. The fact that **C** code wasn&#39;t emitted to an intermediate representation is irrelevant to the situation, as I just needed to show that **ahead-of-time** compilation can yield better immediate results.

What are the advantages of just-in-time compilation versus ahead-of-time compilation?

Using `java -XX:+PrintFlagsFinal` I found the `TieredCompilation` flag, and I read about it a bit online.

Yet, I still don&#39;t know _exactly_ what happens when setting it to `false`.

I know that the compilation system supports 5 execution levels, basically splitted into interpreter, C1 and C2:

 *  level 0 - interpreter
 *  level 1 - C1 with full optimization (no profiling)
 *  level 2 - C1 with invocation and backedge counters
 *  level 3 - C1 with full profiling (level 2 + MDO)
 *  level 4 - C2

Source: http://hg.openjdk.java.net/jdk8u/jdk8u/hotspot/file/2b2511bd3cc8/src/share/vm/runtime/advancedThresholdPolicy.hpp#l34

Two questions:

(1) By setting `-XX:-TieredCompilation`, are some of this levels just disabled? If yes, which?

(2) Is there some flag to decide whether to disable C1 or C2, or to not compile at all?

What exactly does -XX:-TieredCompilation do?

I was trying to figure out hands-on how tail calls are handled by the C# compiler.

(Answer: [They&#39;re not.][1] But the *64bit JIT(s)* WILL do TCE (tail call elimination). [Restrictions apply][2].)

So I wrote a small test using a recursive call that prints how many times it gets called before the `StackOverflowException` kills the process.

    class Program
    {
        static void Main(string[] args)
        {
            Rec();
        }

        static int sz = 0;
        static Random r = new Random();
        static void Rec()
        {
            sz++;

            //uncomment for faster, more imprecise runs
            //if (sz % 100 == 0)
            {
                //some code to keep this method from being inlined
                var zz = r.Next();  
                Console.Write(&quot;{0} Random: {1}\r&quot;, sz, zz);
            }
            
            //uncommenting this stops TCE from happening
            //else
            //{
            //    Console.Write(&quot;{0}\r&quot;, sz);
            //}

            Rec();
        }

Right on cue, the program ends with SO Exception on any of:

- &#39;Optimize build&#39; OFF (either Debug or Release)
- Target: x86
- Target: AnyCPU + &quot;Prefer 32 bit&quot; (this is new in VS 2012 and the first time I saw it. [More here][3].) 
- Some seemingly innocuous branch in the code (see commented &#39;else&#39; branch).

Conversely, using &#39;Optimize build&#39; ON + (Target = x64 or AnyCPU with &#39;Prefer 32bit&#39; OFF (on a 64bit CPU)), TCE happens and the counter keeps spinning up forever (ok, it arguably spins *down* each time its value overflows).

**But I noticed a behaviour I can&#39;t explain** in the `StackOverflowException` case: it never (?) happens at *exactly* the same stack depth. Here are the outputs of a few 32-bit runs, Release build:

    51600 Random: 1778264579
    Process is terminated due to StackOverflowException.

    51599 Random: 1515673450
    Process is terminated due to StackOverflowException.

    51602 Random: 1567871768
    Process is terminated due to StackOverflowException.

    51535 Random: 2760045665
    Process is terminated due to StackOverflowException.

And Debug build:

    28641 Random: 4435795885
    Process is terminated due to StackOverflowException.

    28641 Random: 4873901326  //never say never
    Process is terminated due to StackOverflowException.

    28623 Random: 7255802746
    Process is terminated due to StackOverflowException.

    28669 Random: 1613806023
    Process is terminated due to StackOverflowException.

The stack size is constant ([defaults to 1 MB][4]). The stack frames&#39; sizes are constant. 

So then, what can account for the (sometimes non-trivial) variation of stack depth when the `StackOverflowException` hits? 


UPDATE
--
Hans Passant raises the issue of `Console.WriteLine` touching P/Invoke, interop and possibly non-deterministic locking. 

So I simplified the code to this: 

    class Program
    {
        static void Main(string[] args)
        {
            Rec();
        }
        static int sz = 0;
        static void Rec()
        {
            sz++;
            Rec();
        }
    }

I ran it in Release/32bit/Optimization ON without a debugger. When the program crashes, I attach the debugger and check the value of the counter. 

And it *still* isn&#39;t the same on several runs. (Or my test is flawed.)

UPDATE: Closure
---
As suggested by fejesjoco, I looked into ASLR (Address space layout randomization). 

It&#39;s a security technique that makes it hard for buffer overflow attacks to find the precise location of (e.g.) specific system calls, by randomizing various things in the process address space, including the stack position and, apparently, its size.

**The theory sounds good. Let&#39;s put it into practice!**

In order to test this, I used a Microsoft tool specific for the task: [EMET or The Enhanced Mitigation Experience Toolkit][5]. It allows setting the ASLR flag (and a lot more) on a system- or process-level.  
(There is also a [system-wide, registry hacking alternative][6] that I didn&#39;t try)

![EMET GUI][7]

In order to verify the effectiveness of the tool, I also discovered that [Process Explorer][8] duly reports the status of the ASLR flag in the &#39;Properties&#39; page of the process. Never saw that until today :)

![enter image description here][9]

Theoretically, EMET can (re)set the ASLR flag for a single process. In practice, it didn&#39;t seem to change anything (see above image).

However, I disabled ASLR for the entire system and (one reboot later) I could finally verify that indeed, the SO exception now always happens at the same stack depth.

BONUS
---
ASLR-related, in older news: [How Chrome got pwned][10]


  [1]: https://stackoverflow.com/questions/491376/why-doesnt-net-c-optimize-for-tail-call-recursion
  [2]: https://web.archive.org/web/20130321234831/http://blogs.msdn.com/b/davbr/archive/2007/06/20/tail-call-jit-conditions.aspx
  [3]: https://stackoverflow.com/questions/12066638/what-is-the-purpose-of-the-prefer-32-bit-setting-in-visual-studio-2012-and-how
  [4]: http://msdn.microsoft.com/en-us/library/windows/desktop/ms686774%28v=vs.85%29.aspx
  [5]: http://technet.microsoft.com/en-us/security/jj653751
  [6]: https://stackoverflow.com/questions/9560993/how-do-you-disable-aslr-address-space-layout-randomization-on-windows-7-x64
  [7]: http://i.stack.imgur.com/Uzf8i.png
  [8]: http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx
  [9]: http://i.stack.imgur.com/LcsBf.png
  [10]: http://9to5google.com/2012/03/07/vupen-conquers-chrome-at-pwn2own-security-conference-google-must-pay-cash-reward-for-french-teams-hack/

Why does a recursive call cause StackOverflow at different stack depths?

I couldn&#39;t find the difference between JIT and Interpreters. 

Jit is intermediary to Interpreters and Compilers. During runtime, it converts byte code to machine code ( JVM or Actual Machine ?) For the next time, it takes from the cache and runs
Am I right?

Interpreters will directly execute bytecode without transforming it into machine code. Is that right?

How the real processor in our pc will understand the instruction.?

Please clear my doubts.

JIT vs Interpreters

Under a heavy load, our application is making a beefy server go to 100% CPU usage. Reading the process dump, looking at the threads, some of them are 10 minutes up. None of them give me any insight when using !CLRStack.

The !runaway is giving me:

    0:030&gt; !runaway
     User Mode Time
      Thread       Time
      53:2e804      0 days 0:10:04.703
      30:31894      0 days 0:07:51.593
      33:47100      0 days 0:07:24.890
      42:11e54      0 days 0:06:45.875
      35:35e18      0 days 0:06:07.578
      41:54464      0 days 0:05:49.796
      47:57700      0 days 0:05:45.000
      44:3c2d4      0 days 0:05:44.265
      32:3898c      0 days 0:05:43.593
      50:54894      0 days 0:05:41.968
      51:5bc58      0 days 0:05:40.921
      43:14af4      0 days 0:05:40.734
      48:35074      0 days 0:05:40.406
      ...

Calling !DumpStack on one of those threads, I am getting:

    0000001ab442f900 00007ff9ef4c1148 KERNELBASE!WaitForSingleObjectEx+0x94, calling ntdll!NtWaitForSingleObject
    0000001ab442f980 00007ff9e920beb2 clr!SVR::gc_heap::compute_new_dynamic_data+0x17b, calling clr!SVR::gc_heap::desired_new_allocation
    0000001ab442f9a0 00007ff9e90591eb clr!CLREventWaitHelper2+0x38, calling kernel32!WaitForSingleObjectEx
    0000001ab442f9b0 00007ff9e90e0d2c clr!WriteBarrierManager::UpdateEphemeralBounds+0x1c, calling clr!WriteBarrierManager::NeedDifferentWriteBarrier
    0000001ab442f9e0 00007ff9e9059197 clr!CLREventWaitHelper+0x1f, calling clr!CLREventWaitHelper2
    0000001ab442fa40 00007ff9e9059120 clr!CLREventBase::WaitEx+0x70, calling clr!CLREventWaitHelper
    0000001ab442fa70 00007ff9ef4c149c KERNELBASE!SetEvent+0xc, calling ntdll!NtSetEvent
    0000001ab442faa0 00007ff9e90ef1e1 clr!SVR::gc_heap::set_gc_done+0x22, calling clr!CLREventBase::Set
    0000001ab442fad0 00007ff9e90e9331 clr!SVR::gc_heap::gc_thread_function+0x8a, calling clr!CLREventBase::WaitEx
    0000001ab442fb00 00007ff9e92048e7 clr!SVR::gc_heap::gc_thread_stub+0x7a, calling clr!SVR::gc_heap::gc_thread_function
    0000001ab442fb60 00007ff9e91a0318 clr!Thread::CLRSetThreadStackGuarantee+0x48, calling kernel32!SetThreadStackGuaranteeStub
    0000001ab442fb90 00007ff9e91a01ef clr!Thread::CommitThreadStack+0x10, calling clr!Thread::CLRSetThreadStackGuarantee
    0000001ab442fbd0 00007ff9e910df0b clr!ClrFlsSetValue+0x57, calling kernel32!SetLastErrorStub
    0000001ab442fc00 00007ff9e92048dc clr!SVR::gc_heap::gc_thread_stub+0x6f, calling clr!_chkstk
    0000001ab442fc40 00007ff9f0d316ad kernel32!BaseThreadInitThunk+0xd
    0000001ab442fc70 00007ff9f1e54409 ntdll!RtlUserThreadStart+0x1d


What is it telling me? I see a lot of calls to the CLR, but I can&#39;t understand where would the problem be. After the .reload (suggested by Thomas) now I can see the GC calls.

***Update 1***

After running xperf, each w3wp.exe is consuming something about 45% of CPU. Filtering by one of them and grouping by Function, there is a function labeled as &quot;?&quot; that is responsible for 13.62%, the others are 2.67% or less. How do I manage to know what is this &quot;?&quot;?

***Update 2***

Ran xperf again, and function `JIT_MonEnterWorker_InlineGetThread_GetThread_PatchLabel` is responsible for 12.31% of CPU usage. The &quot;?&quot; function still stays there.

Grouping by Stack:

    Line #, Stack, Count, Weight (in view), TimeStamp, % Weight
    2,   |- ?!?, 501191, 501222.365294, , 35.51
    3,   |    |- clr.dll!JITutil_MonContention, 215749, 215752.552227, , 15.28
    4,   |    |- clr.dll!JIT_MonEnterWorker_InlineGetThread_GetThread_PatchLabel, 170804, 170777.100191, , 12.10

As you can see, those two are responsible for more than 27% of CPU usage (for each process, so it is significant).

***Update 3***

After using wpr.exe (suggestion by @magicandre1981):

    wpr.exe -start cpu and wpr -stop result.etl

I found out that FormsAuthentication and some unnecessary calls to Ninject on critical path were contributing to around 16% of CPU usage. I still don&#39;t understand the threads running gor 10 minutes or more.


***Update 4***

Tried DebugDiag (suggestion from @leppie) and it just confirmed that the threads hanging are all similar to:

    Thread ID: 53     Total CPU Time: 00:09:11.406     Entry Point for Thread: clr!Thread::intermediateThreadProc 
    Thread ID: 35     Total CPU Time: 00:07:26.046     Entry Point for Thread: clr!SVR::gc_heap::gc_thread_stub 
    Thread ID: 50     Total CPU Time: 00:07:01.515     Entry Point for Thread: clr!SVR::gc_heap::gc_thread_stub 
    Thread ID: 29     Total CPU Time: 00:06:02.264     Entry Point for Thread: clr!SVR::gc_heap::gc_thread_stub 
    Thread ID: 31     Total CPU Time: 00:06:41.281     Entry Point for Thread: clr!SVR::gc_heap::gc_thread_stub 

or due to StackExchange.Redis:


    DomainBoundILStubClass.IL_STUB_PInvoke(Int32, IntPtr[], IntPtr[], IntPtr[], TimeValue ByRef)+e1 
    [[InlinedCallFrame] (StackExchange.Redis.SocketManager.select)] StackExchange.Redis.SocketManager.select(Int32, IntPtr[], IntPtr[], IntPtr[], TimeValueByRef) 
    StackExchange.Redis.SocketManager.ReadImpl()+889 
    StackExchange.Redis.SocketManager.Read()+66 

or

    [[GCFrame]] 
    [[HelperMethodFrame_1OBJ] (System.Threading.Monitor.ObjWait)] System.Threading.Monitor.ObjWait(Boolean, Int32, System.Object) 
    mscorlib_ni!System.Threading.Monitor.Wait(System.Object, Int32)+19 
    StackExchange.Redis.ConnectionMultiplexer.ExecuteSyncImpl[[System.__Canon, mscorlib]](StackExchange.Redis.Message, StackExchange.Redis.ResultProcessor`1, StackExchange.Redis.ServerEndPoint)+24f 
    StackExchange.Redis.RedisBase.ExecuteSync[[System.__Canon, mscorlib]](StackExchange.Redis.Message, StackExchange.Redis.ResultProcessor`1, StackExchange.Redis.ServerEndPoint)+77 
    [[StubHelperFrame]] 
    StackExchange.Redis.RedisDatabase.SetMembers(StackExchange.Redis.RedisKey, StackExchange.Redis.CommandFlags)+ee 



xperf WinDBG C# .NET 4.5.2 Application - Understanding process dump

When I run `composer --version` in the macOS terminal, I get the following errors.

&gt; PHP Warning:  preg_match(): JIT compilation failed: no more memory in
&gt; phar:///usr/local/bin/composer.phar/vendor/symfony/console/Application.php
&gt; on line 755
&gt; 
&gt;     
&gt;     Warning: preg_match(): JIT compilation failed: no more memory in phar:///usr/local/bin/composer.phar/vendor/symfony/console/Application.php
&gt; on line 755
&gt;     PHP Warning:  preg_match(): JIT compilation failed: no more memory in
&gt; phar:///usr/local/bin/composer.phar/vendor/symfony/console/Application.php
&gt; on line 759
&gt;     
&gt;     Warning: preg_match(): JIT compilation failed: no more memory in phar:///usr/local/bin/composer.phar/vendor/symfony/console/Application.php
&gt; on line 759
&gt;     PHP Warning:  preg_split(): JIT compilation failed: no more memory in
&gt; phar:///usr/local/bin/composer.phar/vendor/symfony/console/Application.php
&gt; on line 654
&gt;     
&gt;     Warning: preg_split(): JIT compilation failed: no more memory in phar:///usr/local/bin/composer.phar/vendor/symfony/console/Application.php
&gt; on line 654
&gt;     PHP Warning:  preg_split(): JIT compilation failed: no more memory in
&gt; phar:///usr/local/bin/composer.phar/vendor/symfony/console/Application.php
&gt; on line 1091
&gt;     
&gt;     Warning: preg_split(): JIT compilation failed: no more memory in phar:///usr/local/bin/composer.phar/vendor/symfony/console/Application.php
&gt; on line 1091
&gt;     PHP Warning:  preg_replace(): JIT compilation failed: no more memory in
&gt; phar:///usr/local/bin/composer.phar/vendor/symfony/console/Formatter/OutputFormatter.php
&gt; on line 36
&gt;     
&gt;     Warning: preg_replace(): JIT compilation failed: no more memory in phar:///usr/local/bin/composer.phar/vendor/symfony/console/Formatter/OutputFormatter.php
&gt; on line 36
&gt;                                                                 
&gt;       [ErrorException]                                          
&gt;       preg_match_all(): JIT compilation failed: no more memory

All Jit Solutions on Gang of Coders

Total of 30 Jit Solutions