Hi Matrix Team,
Im having some problems with Float Rounding. I have attached an example file.
What am i doing wrong? Only one of the roundings seem to do something.
Thanks,
Wayne M
Float Rounding Problem
-
- Posts: 94
- http://meble-kuchenne.info.pl
- Joined: Fri Dec 04, 2020 6:12 pm
- Has thanked: 1 time
- Been thanked: 9 times
-
- Matrix Staff
- Posts: 1760
- Joined: Mon Dec 07, 2020 10:06 am
- Has thanked: 447 times
- Been thanked: 607 times
Re: Float Rounding Problem
Hello Wayne,
It looks like it's working in the simulation, but maybe the precision of storing and displaying the number is not ideal. I beleive this is the limitation of the float format rather than anything else.
This is a nice tool as it allows you to enter a real number and it will show you the floating point representation of the number.
https://www.h-schmidt.net/FloatConverter/IEEE754.html
100.7 is actually stored as 100.6999969482421875
100.68 as 100.68000030517578125
etc
It looks like it's working in the simulation, but maybe the precision of storing and displaying the number is not ideal. I beleive this is the limitation of the float format rather than anything else.
This is a nice tool as it allows you to enter a real number and it will show you the floating point representation of the number.
https://www.h-schmidt.net/FloatConverter/IEEE754.html
100.7 is actually stored as 100.6999969482421875
100.68 as 100.68000030517578125
etc
Regards Ben Rowland - MatrixTSL
Flowcode Online Code Viewer (Beta) - Flowcode Product Page - Flowcode Help Wiki - My YouTube Channel
Flowcode Online Code Viewer (Beta) - Flowcode Product Page - Flowcode Help Wiki - My YouTube Channel
-
- Posts: 94
- Joined: Fri Dec 04, 2020 6:12 pm
- Has thanked: 1 time
- Been thanked: 9 times
Re: Float Rounding Problem
Hi everyone
What I wanted was to only display to two places after the decimal point.
I thought in the past when you did a round to two decimal places you would loose all the other numbers.
So how do you do it.
Thanks
Wayne m
What I wanted was to only display to two places after the decimal point.
I thought in the past when you did a round to two decimal places you would loose all the other numbers.
So how do you do it.
Thanks
Wayne m
-
- Posts: 94
- Joined: Fri Dec 04, 2020 6:12 pm
- Has thanked: 1 time
- Been thanked: 9 times