#truncate *float value* to specific decimal place

1 messages · Page 1 of 1 (latest)

static turret
#

@trunc(x * pow(10, n)) / pow(10, n)

static turret
#

you're using strict equality on a floating point number, don't

#

floating point numbers can't store all numbers exactly, it can only get close

#

and every operation does this rounding

#

this is only a floating point problem

#

expectEqual does ==, that's strict equality

#

floating point numbers, and ONLY floating point numbers

#

anything that checks if floating point numbers are exactly equal

gilded pendant
#

expectEqual isnt special, for floats the definition is literally just

if (actual != expected) {
  print("expected {}, found {}\n", .{ expected, actual });
  return error.TestExpectedEqual;
}
#

comparing two (non-representable) floats with == isnt reliable, sometimes it works sometimes it doesnt, it might have to do with that not passing a function boundry and being able to be evaluated easier. you should use expectApproxEqRel (or expectApproxEqAbs if youre working with values close to 0)

#

theres std.math.approxEql(Abs/Rel) too for non-testing purposes

autumn pagoda