I was having dinner with Steve and Alan at SC08. Alan mentioned that the trouble with floating point is that it is difficult to define the "right" answer to certain computations. Starting very early in life we learn that in math, answers are either right or wrong. But with floating-point math, there is no "right" or "wrong".
I responded that floating-point numbers must live in a moral gray area. They deny the existence of God, yet they try to do good.