Double, double toil and trouble or, Corner-Cases of Comparing Clojure Numbers

Let’s talk about Clojure.

In Clojure, comparing two numbers can throw an exception.

Check this out:

(< 1/4 0.5M) ;=> true ; as expected (< 1/3 0.5M) ; Execution error (ArithmeticException) at java.math.BigDecimal/divide (BigDecimal.java:1783). ; Non-terminating decimal expansion; no exact representable decimal result.

But why? Why would comparing two perfectly cromulent numbers throw an ArithmeticException?! Everybody knows that ⅓

Well, the problem is that we’re comparing a ratio to a BigDecimal (a decimal number of arbitrary precision). Java doesn’t offer a built-in way of comparing these (Clojure’s ratios aren’t part of the Java standard library), so it has to coerce one into the other. It chooses to coerce the ratio into a BigDecimal, so divides (bigdec 1) by (bigdec 3)…

…and that throws! The decimal representation of ⅓ is infinite, so you can’t keep all the digits in finite memory.

You may ask: how exactly does Clojure know what coercions to apply and how to produce the result? Let’s look at the code.

The implementation of clojure.core/ calls the Java method clojure.lang.Numbers.lt, which is implemented like this:

static public boolean lt(Object x, Object y){ return ops(x).combine(ops(y)).lt((Number)x, (Number)y); }

What’s ops? It’s an implementation of the Ops interface, which has methods for addition, subtraction, etc.; each number class has its own implementation: there is a LongOps, RatioOps, BigDecimalOps etc.

The combine method can alter the behaviour of an Ops depending on the type of the other argument – for example, RatioOps switches to BigDecimalOps if the other argument is a BigDecimal. It’s like a poor man’s implementation of multiple dispatch, which Java doesn’t have.

BigDecimalOps.lt calls toBigDecimal on both arguments, and it’s that method that performs the failing division:

static BigDecimal toBigDecimal(Object x) { // ... other cases ... if (x instanceof Ratio) { Ratio r = (Ratio)x; return (BigDecimal)divide(new BigDecimal(r.numerator), r.denominator); } }

Incidentally, this used to produce the expected result in Clojure up to 1.2.1. At that version, Clojure already used the Ops-based multiple dispatch, but combining RatioOps with BigDecimalOps would yield the former, not the latter.

Is the current behaviour a bug? I’m not sure. It seems so, but maybe 1.3.0’s optimizations warrant this behaviour in the admitedly rare case. There’s an ongoing discussion on the Ask Clojure Q&A.

So, in current Clojure, how do you compare ratios to bigdecs? Simple, you think: just coerce the bigdec to a double!

(< 1/3 (double 0.5M)) ;=> true (> 2/3 (double 0.5M)) ;=> true (= 1/2 (double 0.5M)) ;=> false

Wait, WHAT?

Yep. Comparing ratios to doubles for inequality works fine, but a ratio is never equal to a double (nor a bigdec), even if said double is an exact representation of the ratio.

This one is documented, but often forgotten about (and not hinted at by the docstring). From Clojure’s equality guide:

Clojure’s = is true when called with two immutable scalar values, if:

• Both arguments are nil, true, false, the same character, or the same string (i.e. the same sequence of characters).
• Both arguments are symbols, or both keywords, with equal namespaces and names.
• Both arguments are numbers in the same 'category', and numerically the same, where category is one of:
  • integer or ratio
  • floating point (float or double)
  • BigDecimal.

And indeed, the code for Numbers.equal has a check for both operands’ categories before it delves to the Ops business that we’ve seen. Remember also that Clojure has a numbers-only == which doesn’t trigger that category check:

(== 1/2 (double 0.5M)) ;=> true ; yay

Corollary: if you want to compare a ratio to a BigDecimal, you could coerce the bigdec to a double. That can return an incorrect result only in a very narrow range of cases: when the BigDecimal’s value is close enough to the ratio that it would be lost in the double conversion.

For 100% certainty, the only way I’m aware of is to remember to always use == when comparing for equality, and explicitly coerce the bigdec to ratio:

(defn exactly-equals? [ratio bigdec] (== (* 1 (clojure.lang.Numbers/toRatio bigdec)) ratio)) (exactly-equals? 1/18446744073709551616 5.42101086242752217003726400434970855712890625E-20M) ;=> true ; correct even in this pathological case!

(Multiplying by 1 forces Clojure to normalize the ratio. Otherwise, converting 0.5M would have yielded 5/10 which doesn’t test == to 1/2. Go figure.)