Arithmetic

Decimal vs. Binary

Stak uses unnormalized decimal arithmetic. Most computer arithmetic is binary, which causes problems in converting input and output to the decimal format that humans can understand. For this reason, most calculators do calculations in decimal format.

Many calculator programs use binary arithmetic anyway, because they use the built-in floating point support on the platform. This may cause problems such as 0.1 + 0.1 + 0.1 - 0.3 not being 0, but 5.551115123E-17 because 0.1 cannot be represented exactly in binary floating point.

I have implemented decimal arithmetic since many phones do not have floating point support anyway.

Unnormalized arithmetic

Unlike other calculators, Stak uses unnormalized arithmetic. This basically means that trailing zeros are remembered, so that "2" + "2.0" equals "4.0", not "4" or "4.0000000". This is a bit more like the way people do arithmetic on paper.

Stak's arithmetic is based on Mike Cowlishaw's General Decimal Arithmetic Specification. Mike has a Decimal Arithmetic FAQ explaining the advantages of decimal arithmetic and why it is unnormalized.

This specification is also the basis of the upcoming IEEE 754R standard, which defines an arithmetic that can be implemented portably and efficiently in hardware.

Compliance

Mike Cowlishaw has defined a large number of testcases, of which my implementation passes 16015, fails 38 and skips 19466. 28 of the failures are because my implementation returns the correct remainder even if it is not possible to divide the numbers with the current precision, and the rest are minor differences in the implementation of some flags. The skipped testcases require larger precision or larger exponents than my implementation supports.

Stak does not follow the encoding described on Mike Cowlishaw's website and in the IEEE 754R draft (a kind of Huffman-compressed BCD), but uses a simpler format where the coefficient is an 50-bit unsigned binary number and the exponent is a 10 bit signed two's complement number. This gives a maximum precision of 15 digits instead of 16, and a maximum exponent of 999 instead of 384.

Fudging the result

Unnormalized arithmetic has one disadvantage for a calculator: it makes it harder to cover up the inevitable loss of precision caused by the limits on the precision. Most calculators use hidden digits that are part of the calculation but are not shown in the display. This hides problems such as 1 / 3 * 3 giving "0.99999999" instead of 1, because the display rounds the hidden digits and shows "1.0000000". Hiding digits does not make much sense with unnormalized arithmetic because much of the point is to keep the result with exactly the precision available.

Some Casio calculators even fix the result (and not just the number shown in the display) if it is close to a nice decimal number, assuming that this is the result of a rounding error. This can cause problems because some correct results may be fixed inappropriately (note: this Google link may be broken or show the whole thread and not just John Meyer's comment).

Stak provides an explicit "FUDGE" operation that works the same way as Casio's "feature", but must be explicitly invoked by the user.