I&#39;m writing an application which reads large arrays of floats and performs some simple operations with them. I&#39;m using floats, because I thought it&#39;d be faster than doubles, but after doing some research I&#39;ve found that there&#39;s some confusion about this topic. Can anyone elaborate on this?


Are doubles faster than floats in C#?

I am building a multi-class classifier with Keras 2.02 (with Tensorflow backend)，and I do not know how to calculate precision and recall in Keras. Please help me.

How to calculate precision and recall in Keras

From what I have read, a value of data type double has an approximate precision of 15 decimal places.  However, when I use a number whose decimal representation repeats, such as 1.0/7.0, I find that the variable holds the value of 0.14285714285714285 - which is 17 places (via the debugger).

I would like to know why it is represented as 17 places internally, and why a precision of 15 is always written at ~15?

Double precision - decimal places

I have the following column specified in a database: decimal(5,2)

How does one interpret this?

According to the properties on the column as viewed in SQL Server Management studio I can see that it means: decimal(Numeric precision, Numeric scale).

What do precision and scale mean in real terms?

It would be easy to interpret this as a decimal with 5 digits and two decimals places...ie 12345.12

P.S. I&#39;ve been able to determine the correct answer from a colleague but had great difficulty finding an answer online. As such, I&#39;d like to have the question and answer documented here on stackoverflow for future reference.

How do I interpret precision and scale of a number in a database?

How can I exactly construct a time stamp of actual time with milliseconds precision? 

I need something like 16.4.2013 9:48:00:123. Is this possible? I have an application, where I sample values 10 times per second, and I need to show them in a graph.


Get DateTime.Now with milliseconds precision

Why do some numbers lose accuracy when stored as floating point numbers?

For example, the decimal number `9.2` can be expressed exactly as a ratio of two decimal integers (`92/10`), both of which can be expressed exactly in binary (`0b1011100/0b1010`). However, the same ratio stored as a floating point number is never exactly equal to `9.2`:

    32-bit &quot;single precision&quot; float: 9.19999980926513671875
    64-bit &quot;double precision&quot; float: 9.199999999999999289457264239899814128875732421875

How can such an apparently simple number be &quot;too big&quot; to express in **64 bits** of memory?

Why are floating point numbers inaccurate?

I keep seeing people using doubles in C#. I know I read somewhere that doubles sometimes lose precision. 
My question is when should a use a double and when should I use a decimal type?
Which type is suitable for money computations? (ie. greater than $100 million)

decimal vs double! - Which one should I use and when?

Given a double, I want to round it to a given number of points of precision *after the decimal point*, similar to PHP&#39;s round() function.

The closest thing I can find in the Dart docs is double.toStringAsPrecision(), but this is not quite what I need because it includes the digits *before* the decimal point in the total points of precision.

For example, using toStringAsPrecision(3):

    0.123456789 rounds to 0.123  
    9.123456789 rounds to 9.12  
    98.123456789 rounds to 98.1  
    987.123456789 rounds to 987  
    9876.123456789 rounds to 9.88e+3

As the magnitude of the number increases, I correspondingly lose precision after the decimal place.



How do you round a double in Dart to a given degree of precision AFTER the decimal point?

I am trying to get the sum of `1 + 2 + ... + 1000000000`, but I&#39;m getting funny results in PHP and [Node.js][1].

**PHP**

    $sum = 0;
    for($i = 0; $i &lt;= 1000000000 ; $i++) {
        $sum += $i;
    }
    printf(&quot;%s&quot;, number_format($sum, 0, &quot;&quot;, &quot;&quot;));   // 500000000067108992

**Node.js**

    var sum = 0;
    for (i = 0; i &lt;= 1000000000; i++) {
        sum += i ;
    }
    console.log(sum); // 500000000067109000

The correct answer can be calculated using

    1 + 2 + ... + n = n(n+1)/2

Correct answer = **500000000500000000**, so I decided to try another language.

**GO**

    var sum , i int64
    for i = 0 ; i &lt;= 1000000000; i++ {
        sum += i
    }
    fmt.Println(sum) // 500000000500000000

But it works fine! So what is wrong with my PHP and Node.js code?

Perhaps this a problem of interpreted languages, and that&#39;s why it works in a compiled language like Go? If so, would other interpreted languages such as Python and Perl have the same problem?

  [1]: http://en.wikipedia.org/wiki/Node.js
  [2]: http://en.wikipedia.org/wiki/Python_%28programming_language%29
  [3]: http://en.wikipedia.org/wiki/Perl


Unexpected results when working with very big integers on interpreted languages

Recently I have had to serialize a double into text, and then get it back. The value seems to not be equivalent:

    double d1 = 0.84551240822557006;
    string s = d1.ToString(&quot;R&quot;);
    double d2 = double.Parse(s);
    bool s1 = d1 == d2;
    // -&gt; s1 is False

But according to [MSDN: Standard Numeric Format Strings][1], the &quot;R&quot; option is supposed to guarantee round-trip safety.

&gt; The round-trip (&quot;R&quot;) format specifier is used to ensure that a numeric value that is converted to a string will be parsed back into the same numeric value

Why did this happen?

  [1]: https://msdn.microsoft.com/en-us/library/dwhawy9k(v=vs.110).aspx#Anchor_8

Why is a round-trip conversion via a string not safe for a double?

What is the difference between a single precision floating point operation and double precision floating operation?

I&#39;m especially interested in practical terms in relation to video game consoles. For example, does the Nintendo 64 have a 64 bit processor and if it does then would that mean it was capable of double precision floating point operations? Can the PS3 and Xbox 360 pull off double precision floating point operations or only single precision and in general use is the double precision capabilities made use of (if they exist?).



What&#39;s the difference between a single precision and double precision floating point operation?

    public class doublePrecision {
        public static void main(String[] args) {

            double total = 0;
            total += 5.6;
            total += 5.8;
            System.out.println(total);
        }
    }

The above code prints:

&lt;!-- language: none --&gt;

    11.399999999999

How would I get this to just print (or be able to use it as) 11.4?

Retain precision with double in Java

No, this is not another _&quot;Why is (1/3.0)*3 != 1&quot;_ question.

I&#39;ve been reading about floating-points a lot lately; specifically, how the **same calculation might give different results** on different architectures or optimization settings.

This is a problem for video games which store replays, or are [peer-to-peer networked][1] (as opposed to server-client), which rely on all clients generating exactly the same results every time they run the program - a small discrepancy in one floating-point calculation can lead to a drastically different game-state on different machines (or even [on the same machine!][2])

This happens even amongst processors that &quot;follow&quot; [IEEE-754][3], primarily because some processors (namely x86) use [double extended precision][4].  That is, they use 80-bit registers to do all the calculations, then truncate to 64- or 32-bits, leading to different rounding results than machines which use 64- or 32- bits for the calculations.

I&#39;ve seen several solutions to this problem online, but all for C++, not C#:

 * Disable double extended-precision mode (so that all `double` calculations use IEEE-754 64-bits) using [`_controlfp_s`][5] (Windows), `_FPU_SETCW` (Linux?), or [`fpsetprec`][6] (BSD).
 * Always run the same compiler with the same optimization settings, and require all users to have the same CPU architecture (no cross-platform play).  Because my &quot;compiler&quot; is actually the JIT, which **may optimize differently every time the program is run**, I don&#39;t think this is possible.
 * Use fixed-point arithmetic, and avoid `float` and `double` altogether. `decimal` would work for this purpose, but would be much slower, and none of the `System.Math` library functions support it.

---

So, **is this even a problem in C#?**  What if I only intend to support Windows (not Mono)?

If it is, **is there any way to force my program to run at normal double-precision?**

If not, **are there any libraries that would help** keep floating-point calculations consistent?
 


  [1]: https://gamedev.stackexchange.com/questions/15192/rts-game-protocol
  [2]: http://www.parashift.com/c%2B%2B-faq-lite/newbie.html#faq-29.18
  [3]: http://en.wikipedia.org/wiki/IEEE_754-2008
  [4]: http://en.wikipedia.org/wiki/Extended_precision
  [5]: http://msdn.microsoft.com/en-us/library/c9676k6h.aspx
  [6]: http://www.gsp.com/cgi-bin/man.cgi?section=3&amp;topic=fpsetprec

Is floating-point math consistent in C#? Can it be?

I know the `0.1` decimal number cannot be represented exactly with a finite binary number ([explanation][1]), so `double n = 0.1` will lose some precision and will not be exactly `0.1`. On the other hand `0.5` can be represented exactly because it is `0.5 = 1/2 = 0.1b`.

Having said that it is understandable that adding `0.1` **three times** will not give exactly `0.3` so the following code prints `false`:

    double sum = 0, d = 0.1;
    for (int i = 0; i &lt; 3; i++)
        sum += d;
    System.out.println(sum == 0.3); // Prints false, OK

But then how is it that adding `0.1` **five times** will give exactly `0.5`? The following code prints `true`:

    double sum = 0, d = 0.1;
    for (int i = 0; i &lt; 5; i++)
        sum += d;
    System.out.println(sum == 0.5); // Prints true, WHY?

If `0.1` cannot be represented exactly, how is it that adding it 5 times gives exactly `0.5` which can be represented precisely?


  [1]: http://www.exploringbinary.com/why-0-point-1-does-not-exist-in-floating-point/

Why does adding 0.1 multiple times remain lossless?

I&#39;m new to JavaScript and just discovered `toFixed()` and `toPrecision()` to round numbers. However, I can&#39;t figure out what the difference between the two is.

What is the difference between `number.toFixed()` and `number.toPrecision()`?



Difference between toFixed() and toPrecision()?

There is an option in R to get control over digit display. For example:

    options(digits=10)

is supposed to give the calculation results in 10 digits till the end of R session. In the help file of R, the definition for digits parameter is as follows:

&gt; digits: controls the number of digits
&gt; to print when printing numeric values.
&gt; It is a suggestion only. Valid values
&gt; are **1...22** with default **7**

So, it says this is a suggestion only. What if I like to always display 10 digits, not more or less?

My second question is, what if I like to display more than 22 digits, i.e. for more precise calculations like 100 digits? Is it possible with base R, or do I need an additional package/function for that?

**Edit:** Thanks to jmoy&#39;s suggestion, I tried `sprintf(&quot;%.100f&quot;,pi)` and it gave

    [1] &quot;3.1415926535897931159979634685441851615905761718750000000000000000000000000000000000000000000000000000&quot;

which has 48 decimals. Is this the maximum limit R can handle?



Controlling number of decimal digits in print output in R

This isn&#39;t a question as it is more of a be aware. I updated an application that uses `json_encode()` to PHP7.1.1 and I was seeing an issue with floats being changed to sometimes extend out 17 digits. According to documentation, PHP 7.1.x started to use `serialize_precision` instead of precision when encoding double values. I&#39;m guessing this caused an example value of 

&gt; 472.185 

to become 

&gt;472.18500000000006 

after that value went through `json_encode()`. Since my discovery, I have reverted back to PHP 7.0.16 and I no longer have the issue with `json_encode()`. I also tried to update to PHP 7.1.2 before reverting back to PHP 7.0.16.

The reasoning behind this question does stem from https://stackoverflow.com/questions/3726721/php-floating-number-precision, however the end all reason for this is because of the change from precision to serialize_precision usage in `json_encode()`.

If anyone does know of a solution to this problem, I&#39;d be more than happy to listen in on the reasoning/fix.

Excerpt from multidimensional array (before): 

    [staticYaxisInfo] =&gt; Array
                        (
                            [17] =&gt; stdClass Object
                                (
                                    [variable_id] =&gt; 17
                                    [static] =&gt; 1
                                    [min] =&gt; 0
                                    [max] =&gt; 472.185
                                    [locked_static] =&gt; 1
                                )

                        )
and after going through `json_encode()`...

    &quot;staticYaxisInfo&quot;:
				{
					&quot;17&quot;:
					{
						&quot;variable_id&quot;: &quot;17&quot;,
						&quot;static&quot;: &quot;1&quot;,
						&quot;min&quot;: 0,
						&quot;max&quot;: 472.18500000000006,
						&quot;locked_static&quot;: &quot;1&quot;
					}
				},

PHP7.1 json_encode() Float Issue

This was a question I was asked at my recent interview and I want to know *(I don&#39;t actually remember the theory of the numerical analysis, so please help me :)*

If we have some function, which accumulates floating-point numbers:

    std::accumulate(v.begin(), v.end(), 0.0);

`v` is a `std::vector&lt;float&gt;`, for example.

- **Would it be better to sort these numbers before accumulating them?**

- **Which order would give the most precise answer?**

I suspect that **sorting** the numbers in ascending order would actually make the numerical error **less**, but unfortunately I can&#39;t prove it myself.

*P.S. I do realize this probably has nothing to do with real world programming, just being curious.*


In which order should floats be added to get the most precise result?

I know you can&#39;t rely on equality between double or decimal type values normally, but I&#39;m wondering if 0 is a special case.

While I can understand imprecisions between 0.00000000000001 and 0.00000000000002, 0 itself seems pretty hard to mess up since it&#39;s just nothing.  If you&#39;re imprecise on nothing, it&#39;s not nothing anymore.

But I don&#39;t know much about this topic so it&#39;s not for me to say.

    double x = 0.0;
    return (x == 0.0) ? true : false;

Will that always return true?

Is it safe to check floating point values for equality to 0?

I just ran into some unexpected behavior with DateTime.UtcNow while doing some unit tests. It appears that when you call DateTime.Now/UtcNow in rapid succession, it seems to give you back the same value for a longer-than-expected interval of time, rather than capturing more precise millisecond increments.

I know there is a Stopwatch class that would be better suited for doing precise time measurements, but I was curious if someone could explain this behavior in DateTime?  Is there an official precision documented for DateTime.Now (for example, precise to within 50&amp;nbsp;ms?)? Why would DateTime.Now be made less precise than what most CPU clocks could handle? Maybe it&#39;s just designed for the lowest common denominator CPU?

    public static void Main(string[] args)
    {
        var stopwatch = new Stopwatch();
        stopwatch.Start();
        for (int i=0; i&lt;1000; i++)
        {
            var now = DateTime.Now;
            Console.WriteLine(string.Format(
                &quot;Ticks: {0}\tMilliseconds: {1}&quot;, now.Ticks, now.Millisecond));
        }

        stopwatch.Stop();
        Console.WriteLine(&quot;Stopwatch.ElapsedMilliseconds: {0}&quot;,
            stopwatch.ElapsedMilliseconds);

        Console.ReadLine();
    }


C# DateTime.Now precision

Are there data types with better precision than float?

Double precision floating values in Python?

I&#39;ve read about the difference between double precision and single precision. However, in most cases, `float` and `double` seem to be interchangeable, i.e. using one or the other does not seem to affect the results. Is this really the case? When are floats and doubles interchangeable? What are the differences between them?

What is the difference between float and double?

I am facing a strange behavior of the `round()` function:

    for i in range(1, 15, 2):
    	n = i / 2
    	print(n, &quot;=&gt;&quot;, round(n))

This code prints:

    0.5 =&gt; 0
    1.5 =&gt; 2
    2.5 =&gt; 2
    3.5 =&gt; 4
    4.5 =&gt; 4
    5.5 =&gt; 6
    6.5 =&gt; 6

I expected the floating values to be always rounded up, but instead, it is rounded to the nearest even number.

Why such behavior, and what is the best way to get the correct result?

I tried to use the [`fractions`](https://docs.python.org/3.5/library/fractions.html) but the result is the same.

How to properly round-up half float numbers?

I want `a` to be rounded to *13.95*. I tried using [`round`][1]:

    &gt;&gt;&gt; a
    13.949999999999999
    &gt;&gt;&gt; round(a, 2)
    13.949999999999999


  [1]: https://docs.python.org/2/library/functions.html#round

Limiting floats to two decimal points

This is something that&#39;s been on my mind for years, but I never took the time to ask before.

Many (pseudo) random number generators generate a random number between 0.0 and 1.0. Mathematically there are infinite numbers in this range, but `double` is a floating point number, and therefore has a finite precision.

So the questions are:

1. Just how many `double` numbers are there between 0.0 and 1.0?
2. Are there just as many numbers between 1 and 2? Between 100 and 101? Between 10^100 and 10^100+1?

Note: if it makes a difference, I&#39;m interested in Java&#39;s definition of `double` in particular.

How many double numbers are there between 0.0 and 1.0?

    $a = &#39;35&#39;;
    $b = &#39;-34.99&#39;;
    echo ($a + $b);

Results in 0.009999999999998

What is up with that? I wondered why my program kept reporting odd results.

Why doesn&#39;t PHP return the expected 0.01?

PHP - Floating Number Precision

I&#39;m using BigDecimal for my numbers in my application, for example, with JPA. I did a bit of researching about the terms &#39;precision&#39; and &#39;scale&#39; but I don&#39;t understand what are they exactly.

Can anyone explain me the meaning of &#39;precision&#39; and &#39;scale&#39; for a BigDecimal value?

    @Column(precision = 11, scale = 2)

Thanks!

BigDecimal, precision and scale

I am working on a project which incorporates computing a sine wave as input for a control loop.

The sine wave has a frequency of 280 Hz, and the control loop runs every 30&amp;nbsp;&#181;s and everything is written in C for an [Arm Cortex-M7][1].

At the moment we are simply doing:

    double time;
    void control_loop() {
        time += 30e-6;
        double sine = sin(2 * M_PI * 280 * time);
        ...
    }

Two problems/questions arise:
1. When running for a long time, `time` becomes bigger. Suddenly there is a point where the computation time for the sine function increases drastically (see image). Why is this? How are these functions usually implemented? Is there a way to circumvent this (without noticeable precision loss) as speed is a huge factor for us? We are using sin from math.h (Arm GCC).
[![sin function profiling][2]][2]
2. How can I deal with time in general? When running for a long time, the variable `time` will inevitably reach the limits of double precision. Even using a counter ```time = counter++ * 30e-6;``` only improves this, but it does not solve it. As I am certainly not the first person who wants to generate a sine wave for a long time, there must be some ideas/papers/... on how to implement this fast and precise.

  [1]: https://en.wikipedia.org/wiki/ARM_Cortex-M
  [2]: https://i.stack.imgur.com/pBq3w.png




Endless sine generation in C

&gt; **Possible Duplicate:**  
&gt; [Why can&amp;#39;t I return a double from two ints being divided](https://stackoverflow.com/questions/7571326/why-cant-i-return-a-double-from-two-ints-being-divided)  

&lt;!-- End of automatically inserted text --&gt;

My C++ program is truncating the output of my integer devision even when I try and place the output into a float. How can I prevent this whilst keeping those to variables (a &amp; b) as integers?

    user@box:~/c/precision$ cat precision.cpp
    #include &lt;iostream&gt;
    #include &lt;iomanip&gt;
    using namespace std;
    
    int main()
    {
      int a = 10, b = 3;
      float ans = (a/b);
      cout&lt;&lt;fixed&lt;&lt;setprecision(3);
      cout &lt;&lt; (a/b) &lt;&lt; endl;
      cout &lt;&lt; ans &lt;&lt; endl;
      return 0;
    }
    
    user@box:~/c/precision$ g++ -o precision precision.cpp 
    user@box:~/c/precision$ ./precision 
    3
    3.000


Dividing two integers to produce a float result

I know a little bit about how floating-point numbers are represented, but not enough, I&#39;m afraid.

The general question is: 

&gt; For a given precision (for my purposes, the number of accurate decimal places in base 10), what range of numbers can be represented for 16-, 32- and 64-bit IEEE-754 systems?

Specifically, I&#39;m only interested in the range of 16-bit and 32-bit numbers accurate to +/-0.5 (the ones place) or +/- 0.0005 (the thousandths place).

What range of numbers can be represented in a 16-, 32- and 64-bit IEEE-754 systems?

I wanted to display a number to 2 decimal places.

I thought I could use `toPrecision(2)` in JavaScript .

However, if the number is `0.05`, I get `0.0500`. I&#39;d rather it stay the same.

See it on [JSbin][1].


What is the best way to do this?

I can think of coding a few solutions, but I&#39;d imagine (I hope) something like this is built in?



  [1]: http://jsbin.com/eguvo/edit

JavaScript displaying a float to 2 decimal places

I have some clients getting weird bills. I was able to isolate the core problem:

    SELECT 199.96 - (0.0 * FLOOR(CAST(1.0 AS DECIMAL(19, 4)) * CAST(199.96 AS DECIMAL(19, 4)))) -- 200 what the?
    SELECT 199.96 - (0.0 * FLOOR(1.0 * CAST(199.96 AS DECIMAL(19, 4)))) -- 199.96
    SELECT 199.96 - (0.0 * FLOOR(CAST(1.0 AS DECIMAL(19, 4)) * 199.96)) -- 199.96

    SELECT 199.96 - (CAST(0.0 AS DECIMAL(19, 4)) * FLOOR(CAST(1.0 AS DECIMAL(19, 4)) * CAST(199.96 AS DECIMAL(19, 4)))) -- 199.96
    SELECT 199.96 - (CAST(0.0 AS DECIMAL(19, 4)) * FLOOR(1.0 * CAST(199.96 AS DECIMAL(19, 4))))                         -- 199.96
    SELECT 199.96 - (CAST(0.0 AS DECIMAL(19, 4)) * FLOOR(CAST(1.0 AS DECIMAL(19, 4)) * 199.96))                         -- 199.96

    -- It gets weirder...
    SELECT (0 * FLOOR(CAST(1.0 AS DECIMAL(19, 4)) * CAST(199.96 AS DECIMAL(19, 4)))) -- 0
    SELECT (0 * FLOOR(1.0 * CAST(199.96 AS DECIMAL(19, 4))))                         -- 0
    SELECT (0 * FLOOR(CAST(1.0 AS DECIMAL(19, 4)) * 199.96))                         -- 0

    -- so... ... 199.06 - 0 equals 200... ... right???
    SELECT 199.96 - 0 -- 199.96 ...NO....

Has anyone a clue, what the heck is happening here? I mean, it has certainly something to do with the decimal datatype, but I can&#39;t really wrap my head around it...

------------------------------------------

There was a lot of confusion about of what datatype the number literals were, so I decided to show the real line:

    PS.SharePrice - (CAST((@InstallmentCount - 1) AS DECIMAL(19, 4)) * CAST(FLOOR(@InstallmentPercent * PS.SharePrice) AS DECIMAL(19, 4))))

    PS.SharePrice DECIMAL(19, 4)

    @InstallmentCount INT

    @InstallmentPercent DECIMAL(19, 4)

I made sure that the result of each operation having an operand of a type different than `DECIMAL(19, 4)` is cast explicitly before applying it to the outer context.

Nevertheless, the result remains `200.00`.

------------------

I have now created a boiled down sample you guys can execute on your computer.

    DECLARE @InstallmentIndex INT = 1
    DECLARE @InstallmentCount INT = 1
    DECLARE @InstallmentPercent DECIMAL(19, 4) = 1.0
    DECLARE @PS TABLE (SharePrice DECIMAL(19, 4))
    INSERT INTO @PS (SharePrice) VALUES (599.96)

    -- 2000
    SELECT
      IIF(@InstallmentIndex &lt; @InstallmentCount,
      FLOOR(@InstallmentPercent * PS.SharePrice),
      1999.96)
    FROM @PS PS

    -- 2000
    SELECT
      IIF(@InstallmentIndex &lt; @InstallmentCount,
      FLOOR(@InstallmentPercent * CAST(599.96 AS DECIMAL(19, 4))),
      1999.96)
    FROM @PS PS

    -- 1996.96
    SELECT
      IIF(@InstallmentIndex &lt; @InstallmentCount,
      FLOOR(@InstallmentPercent * 599.96),
      1999.96)
    FROM @PS PS

    -- Funny enough - with this sample explicitly converting EVERYTHING to DECIMAL(19, 4) - it still doesn&#39;t work...
    -- 2000
    SELECT
      IIF(@InstallmentIndex &lt; @InstallmentCount,
      FLOOR(@InstallmentPercent * CAST(199.96 AS DECIMAL(19, 4))),
      CAST(1999.96 AS DECIMAL(19, 4)))
    FROM @PS PS

-------------

Now I&#39;ve got something...

    -- 2000
    SELECT
      IIF(1 = 2,
      FLOOR(CAST(1.0 AS decimal(19, 4)) * CAST(199.96 AS DECIMAL(19, 4))),
      CAST(1999.96 AS DECIMAL(19, 4)))

    -- 1999.9600
    SELECT
      IIF(1 = 2,
      CAST(FLOOR(CAST(1.0 AS decimal(19, 4)) * CAST(199.96 AS DECIMAL(19, 4))) AS INT),
      CAST(1999.96 AS DECIMAL(19, 4)))

What the hell - floor is supposed to return an integer anyway. What&#39;s going on here? :-D

------------

I think I now managed to really boil it down to the very essence :-D

    -- 1.96
    SELECT IIF(1 = 2,
      CAST(1.0 AS DECIMAL (36, 0)),
      CAST(1.96 AS DECIMAL(19, 4))
    )

    -- 2.0
    SELECT IIF(1 = 2,
      CAST(1.0 AS DECIMAL (37, 0)),
      CAST(1.96 AS DECIMAL(19, 4))
    )

    -- 2
    SELECT IIF(1 = 2,
      CAST(1.0 AS DECIMAL (38, 0)),
      CAST(1.96 AS DECIMAL(19, 4))
    )


Why is 199.96 - 0 = 200 in SQL?

In my [earlier question](https://stackoverflow.com/questions/553974/why-does-int55-54-in-c) I was printing a `double` using `cout` that got rounded when I wasn&#39;t expecting it.  How can I make `cout` print a `double` using full precision?

How do I print a double value with full precision using cout?

*This question arose out of something strange that I noticed after investigating [this question][1] further...*

I always understood MATLAB variables to be [double-precision][2] by default. So, if I were to do something like declare a variable with 20 digits after the decimal point:

    &gt;&gt; num = 2.71828182845904553488;
    &gt;&gt; class(num)  % Display the variable type
    ans =
    double

I would expect the last 4 digits to be ignored, since the [floating-point relative accuracy][3] is on the order of 10&lt;sup&gt;-16&lt;/sup&gt;:

    &gt;&gt; eps(num)
    ans =
        4.440892098500626e-016

If I try to display the number with more than 16 digits after the decimal point (using either [`fprintf`][4] or [`sprintf`][5]), I get what I expect to see:

    &gt;&gt; fprintf(&#39;%0.20f\n&#39;, num)
    2.71828182845904550000
    &gt;&gt; sprintf(&#39;%0.20f&#39;, num)
    ans =
    2.71828182845904550000

In other words, digits 17 through 20 are all 0.

But things get weird when I pass `num` to the [variable precision arithmetic function][6] in the [Symbolic Toolbox][7], telling it to represent the number using 21 digits of precision:

    &gt;&gt; vpa(num, 21)
    ans =
    2.71828182845904553488

**WHAT?!** Those last 4 digits have reappeared! Shouldn&#39;t they have been lost when the original number I entered was stored as a double-precision variable `num`? Since `num` is a double-precision variable when it is passed to `vpa`, how did `vpa` know what they were?

My best guess as to what is happening is that MATLAB internally represents `num` with more precision than a double since I initialized it to a number with more digits past the decimal point than a double-precision variable could handle. Is this really what is happening, or is something else going on?

&lt;br&gt;

----------


**BONUS:** And here&#39;s an additional source of confusion if you don&#39;t already have a migraine from the above...

    &gt;&gt; num = 2.71828182845904553488;  % Declare with 20 digits past the decimal
    &gt;&gt; num = 2.718281828459045531;    % Re-declare with 18 digits past the decimal
    &gt;&gt; vpa(num, 21)
    ans =
    2.71828182845904553488  % It&#39;s the original 20-digit number!!!


  [1]: https://stackoverflow.com/q/4222257/52738
  [2]: https://en.wikipedia.org/wiki/Double-precision_floating-point_format
  [3]: https://www.mathworks.com/help/matlab/ref/eps.html
  [4]: https://www.mathworks.com/help/matlab/ref/fprintf.html
  [5]: https://www.mathworks.com/help/matlab/ref/sprintf.html
  [6]: https://www.mathworks.com/help/symbolic/vpa.html
  [7]: https://www.mathworks.com/help/symbolic/index.html

In MATLAB, are variables REALLY double-precision by default?

So, I know I can get current time in milliseconds using JavaScript. But, is it possible to get the current time in nanoseconds instead?

Is there any way to get current time in nanoseconds using JavaScript?

I am writing a program where I need to delete duplicate points stored in a matrix. The problem is that when it comes to check whether those points are in the matrix, MATLAB can&#39;t recognize them in the matrix although they exist.

In the following code, `intersections` function gets the intersection points:

    [points(:,1), points(:,2)] = intersections(...
        obj.modifiedVGVertices(1,:), obj.modifiedVGVertices(2,:), ...
        [vertex1(1) vertex2(1)], [vertex1(2) vertex2(2)]);

The result:

    &gt;&gt; points
    points =
       12.0000   15.0000
       33.0000   24.0000
       33.0000   24.0000

    &gt;&gt; vertex1
    vertex1 =
        12
        15

    &gt;&gt; vertex2    
    vertex2 =
        33
        24

Two points (`vertex1` and `vertex2`) should be eliminated from the result. It should be done by the below commands:

    points = points((points(:,1) ~= vertex1(1)) | (points(:,2) ~= vertex1(2)), :);
    points = points((points(:,1) ~= vertex2(1)) | (points(:,2) ~= vertex2(2)), :);

After doing that, we have this unexpected outcome:

    &gt;&gt; points
    points =
       33.0000   24.0000

The outcome should be an empty matrix. As you can see, the first (or second?) pair of `[33.0000 24.0000]` has been eliminated, but not the second one.

Then I checked these two expressions:

    &gt;&gt; points(1) ~= vertex2(1)
    ans =
         0
    &gt;&gt; points(2) ~= vertex2(2)
    ans =
         1   % &lt;-- It means 24.0000 is not equal to 24.0000?

What is the problem?

---

More surprisingly, I made a new script that has only these commands:

    points = [12.0000   15.0000
              33.0000   24.0000
              33.0000   24.0000];

    vertex1 = [12 ;  15];
    vertex2 = [33 ;  24];

    points = points((points(:,1) ~= vertex1(1)) | (points(:,2) ~= vertex1(2)), :);
    points = points((points(:,1) ~= vertex2(1)) | (points(:,2) ~= vertex2(2)), :);

The result as expected:

    &gt;&gt; points
    points =  
       Empty matrix: 0-by-2


Why is 24.0000 not equal to 24.0000 in MATLAB?

These lines in C#

    decimal a = 2m;
    decimal b = 2.0m;
    decimal c = 2.00000000m;
    decimal d = 2.000000000000000000000000000m;
    
    Console.WriteLine(a);
    Console.WriteLine(b);
    Console.WriteLine(c);
    Console.WriteLine(d);

Generates this output:

    2
    2.0
    2.00000000
    2.000000000000000000000000000

So I can see that creating a decimal variable from a literal allows me to control the precision.

* Can I adjust the precision of decimal variables without using literals?
* How can I create b from a?  How can I create b from c?  


Adjusting decimal precision, .net

My question is whether all integer values are guaranteed to have a perfect double representation. 

Consider the following code sample that prints &quot;Same&quot;:

    // Example program
    #include &lt;iostream&gt;
    #include &lt;string&gt;
    
    int main()
    {
      int a = 3;
      int b = 4;
      double d_a(a);
      double d_b(b);
      
      double int_sum = a + b;
      double d_sum = d_a + d_b;
      
      if (double(int_sum) == d_sum)
      {
          std::cout &lt;&lt; &quot;Same&quot; &lt;&lt; std::endl;
      }
    }

Is this guaranteed to be true for any architecture, any compiler, any values of `a` and `b`? Will any integer `i` converted to `double`, always be represented as `i.0000000000000` and not, for example, as `i.000000000001`?

I tried it for some other numbers and it always was true, but was unable to find anything about whether this is coincidence or by design.


Note: This is different from [this question][1] (aside from the language) since I am adding the two integers. 


  [1]: https://stackoverflow.com/questions/12629087/is-floatinteger-integer-guaranteed-to-be-equal-in-c

Are all integer values perfectly represented as doubles?

I am writing a program for an embedded hardware that only supports 32-bit single-precision floating-point arithmetic. The algorithm I am implementing, however, requires a 64-bit double-precision addition and comparison. I am trying to emulate `double` datatype using a tuple of two `float`s. So a `double d` will be emulated as a `struct` containing the tuple: `(float d.hi, float d.low)`.

The comparison should be straightforward using a lexicographic ordering. The addition however is a bit tricky because I am not sure which base should I use. Should it be `FLT_MAX`? And how can I detect a carry?

How can this be done?

--------------------
*Edit (Clarity):* I need the extra significant digits rather than the extra range.

Emulate &quot;double&quot; using 2 &quot;float&quot;s

I have this code in C where I&#39;ve declared 0.1 as double.

    #include &lt;stdio.h&gt; 
    int main() {
        double a = 0.1;

        printf(&quot;a is %0.56f\n&quot;, a);
        return 0;
    }
This is what it prints, `a is 0.10000000000000001000000000000000000000000000000000000000`

Same code in C++,

    #include &lt;iostream&gt;
    using namespace std;
    int main() {
        double a = 0.1;

        printf(&quot;a is %0.56f\n&quot;, a);
        return 0;
    }

This is what it prints, `a is 0.1000000000000000055511151231257827021181583404541015625`

What is the difference? When I read both are alloted 8 bytes? How does C++ print more numbers in the decimal places?

Also, how can it go until 55 decimal places? IEEE 754 floating point has only 52 bits for fractional number with which we can get 15 decimal digits of precision. It is stored in binary. How come its decimal interpretation stores more? 


Why does double in C print fewer decimal digits than C++?

Is it possible to set the display precision of a float in Ruby?

Something like:

    z = 1/3
    z.to_s    #=&gt; 0.33333333333333
    z.to_s(3) #=&gt; 0.333
    z.to_s(5) #=&gt; 0.33333

Or do I have to override the `to_s` method of `Float`?

Set the display precision of a float in Ruby

I am looking for the best way to convert a Number to a BigDecimal.

Is this good enough?

    Number number;
    BigDecimal big = new BigDecimal(number.toString());

Can we lose precision with the `toString()` method ?

All Precision Solutions on Gang of Coders

Total of 42 Precision Solutions