Support Board
Date/Time: Tue, 24 Jun 2025 00:58:47 +0000
Post From: Rounding error on spreadsheet?
| [2022-02-08 00:22:36] |
| 1+1=10 - Posts: 270 |
|
@User104854. So we're accustomed to numbers being stored in Base10, and a perfect example we all know of what's going on here is that you can't represent 1/3 in base 10. We use 1.3333 repeating but it is not exact. In computers numbers are generally represented in binary -- base 2 -- and it is only specific numbers that can't be represented exactly. All that to say, it is very possible to have 'C3=J9' work with many numbers but not all. To give you two examples, neither 0.1 nor 0.2 can be represented exactly in computers (using binary base-2) so if you try 0.1=0.1 in a programing language it will be false. A spreadsheet program uses a programming language under the hood, but a dedicated program like Excel will smooth out this problem for the user so a workaround is not needed. SC could have chosen to store all prices in cents, as whole numbers, which can always be perfectly represented, but then there would have been a slight performance cost to convert the number to a format suitable for display. Anyway, good luck with your trading! |