quotearg: don't attempt to store 1 << 31 into an "int"

* lib/quotearg.c (quotearg_buffer_restyled): Building coreutils with
gcc's new -fsanitize=undefined and running its tests triggered some
new test failures due to undefined behavior, all with this diagnostic:
  lib/quotearg.c:629:62: runtime error: left shift of 1 by 31 places \
    cannot be represented in type int
Rather than shifting "1" left to form a mask, shift the bits right and
simply use "1" as the mask.
Co-authored-by: Paul Eggert <eggert@cs.ucla.edu>

diff --git a/ChangeLog b/ChangeLog
index 3567503..16443b3 100644 (file)
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,15 @@
+2013-11-18  Jim Meyering  <meyering@fb.com>
+       and Paul Eggert  <eggert@cs.ucla.edu>
+
+       quotearg: don't attempt to store 1 << 31 into an "int"
+       * lib/quotearg.c (quotearg_buffer_restyled): Building coreutils with
+       gcc's new -fsanitize=undefined and running its tests triggered some
+       new test failures due to undefined behavior, all with this diagnostic:
+         lib/quotearg.c:629:62: runtime error: left shift of 1 by 31 places \
+           cannot be represented in type int
+       Rather than shifting "1" left to form a mask, shift the bits right and
+       simply use "1" as the mask.
+
 2013-11-21  Paul Eggert  <eggert@cs.ucla.edu>
 
        error: depend on stdio

diff --git a/lib/quotearg.c b/lib/quotearg.c
index 40114d7..e280e6e 100644 (file)
--- a/lib/quotearg.c
+++ b/lib/quotearg.c
@@ -626,7 +626,7 @@ quotearg_buffer_restyled (char *buffer, size_t buffersize,
 
       if (! ((backslash_escapes || elide_outer_quotes)
              && quote_these_too
-             && quote_these_too[c / INT_BITS] & (1 << (c % INT_BITS)))
+             && quote_these_too[c / INT_BITS] >> (c % INT_BITS) & 1)
           && !is_right_quote)
         goto store_c;