st.c: fix crashes on huge hash tables

From: Vladimir Makarov <vmakarov@redhat.com>

By Vladimir's estimation, this requires at least 64 GB of memory
to reproduce this bug due to the hash sizes required.  So there
is no new test case (and I am unable to test it, myself).

* st.c (get_bins_num): avoid out-of-bounds on shift by using correct type
  [ruby-core:78139] [Bug #12939]
* st.c (get_allocated_entries): ditto

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56793 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

1 files changed, 2 insertions, 2 deletions

diff --git a/st.c b/st.c
index 7bf5e13284..8cb3f7fef7 100644
--- a/st.c
+++ b/st.c
@@ -417,7 +417,7 @@ get_size_ind(const st_table *tab) {
 /* Return the number of allocated bins of table TAB.  */
 static inline st_index_t
 get_bins_num(const st_table *tab) {
-    return 1<<tab->bin_power;
+    return ((st_index_t) 1)<<tab->bin_power;
 }
 
 /* Return mask for a bin index in table TAB.  */
@@ -436,7 +436,7 @@ hash_bin(st_hash_t hash_value, st_table *tab) {
 /* Return the number of allocated entries of table TAB.  */
 static inline st_index_t
 get_allocated_entries(const st_table *tab) {
-    return 1<<tab->entry_power;
+    return ((st_index_t) 1)<<tab->entry_power;
 }
 
 /* Return size of the allocated bins of table TAB.  */