Skip to content

Commit 787652b

Browse files
jswaldenararslan
authored andcommitted
msun: signed overflow in atan2
As a component of atan2(y, x), the case of x == 1.0 is farmed out to atan(y). The current implementation of this comparison is vulnerable to signed integer underflow (that is, undefined behavior), and it's performed in a somewhat more complicated way than it need be. Change it to not be quite so cute, rather directly comparing the high/low bits of x to the specific IEEE-754 bit pattern that encodes 1.0. Note that while there are three different e_atan* files in the relevant directory, only this one needs fixing. e_atan2f.c already compares against the full bit pattern encoding 1.0f, while e_atan2l.cuses bitwise-ands/ors/nots and so doesn't require a change. Incorporated from FreeBSD source tree. See: freebsd/freebsd-src@b21ccf6
1 parent a844d58 commit 787652b

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

src/e_atan2.c

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,7 @@ __ieee754_atan2(double y, double x)
7171
if(((ix|((lx|-lx)>>31))>0x7ff00000)||
7272
((iy|((ly|-ly)>>31))>0x7ff00000)) /* x or y is NaN */
7373
return x+y;
74-
if(((hx-0x3ff00000)|lx)==0) return atan(y); /* x=1.0 */
74+
if(hx==0x3ff00000&&lx==0) return atan(y); /* x=1.0 */
7575
m = ((hy>>31)&1)|((hx>>30)&2); /* 2*sign(x)+sign(y) */
7676

7777
/* when y = 0 */

0 commit comments

Comments
 (0)