leandrof/linux
1
0
Fork 0
forked from Minki/linux
Code Issues Pull Requests Packages Projects Releases Wiki Activity

math-emu: fix floating-point to integer unsigned saturation

Browse Source 
The math-emu macros _FP_TO_INT and _FP_TO_INT_ROUND are supposed to
saturate their results for out-of-range arguments, except in the case
rsigned == 2 (when instead the low bits of the result are taken).
However, in the case rsigned == 0 (converting to unsigned integers),
they mistakenly produce 0 for positive results and the maximum
unsigned integer for negative results, the opposite of correct
unsigned saturation.  This patch fixes the logic.

Signed-off-by: Joseph Myers <joseph@codesourcery.com>
Signed-off-by: Scott Wood <scottwood@freescale.com>
This commit is contained in:
Joseph Myers 2013-11-04 16:53:50 +00:00 committed by Scott Wood
parent 28414a6def
commit 4f6db5efff
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
include/math-emu/op-common.h
View File

@ -685,7 +685,7 @@ do { \
else \
{ \
r = 0; \
if (X##_s) \
if (!X##_s) \
r = ~r; \
} \
FP_SET_EXCEPTION(FP_EX_INVALID); \
@ -762,7 +762,7 @@ do { \
if (!rsigned) \
{ \
r = 0; \
if (X##_s) \
if (!X##_s) \
r = ~r; \
} \
else if (rsigned != 2) \