[rubygems/rubygems] Mask the file mode when extracting files

When extracting files from the tarball, a mode is retrieved from
the header. Occasionally you'll encounter a gem that was packaged
on a system whose permission bits result in a value that is larger
than the value that File.chmod will allow (anything >= 2^16). In
that case the extraction fails with a RangeError, which is pretty
esoteric.

If you extract the tarball with the tar and gunzip utilities, the
file permissions end up being just the bottom 16 bits masked off
from the original value. I've mirrored that behavior here. Per the
tar spec:

> Modes which are not supported by the operating system restoring
> files from the archive will be ignored.

I think that basically means what I've done here.

---

This commit also changes the behavior very slightly with regard to
when the chmod is called. Previously it was called while the file
descriptor was still open, but after the write call.

When write flushes, the file permissions are changed to the mode
value from the File.open call, undoing the changes made by
FileUtils.chmod. CRuby appears to flush the buffer after the
chmod call, whereas TruffleRuby flushes before the chmod call.
So the file permissions can change depending on implementation.
Both implementations end up getting the correct file permissions
for the bottom 9 bits (user, group, world), but differ with
regard to the sticky bit in the next 3.

To get consistent behavior, this commit changes it to close the
file descriptor before attempting to chmod anything, which makes
it consistent because the write flushes in both cases.

https://github.com/rubygems/rubygems/commit/22ce076e99

2 files changed, 15 insertions, 0 deletions

diff --git a/test/rubygems/packages/Bluebie-legs-0.6.2.gem b/test/rubygems/packages/Bluebie-legs-0.6.2.gem
new file mode 100644
index 0000000000..60918f3bc5
--- /dev/null
+++ b/test/rubygems/packages/Bluebie-legs-0.6.2.gem
diff --git a/test/rubygems/test_gem_package.rb b/test/rubygems/test_gem_package.rb
index 9295f42dba..eebe4d86d0 100644
--- a/test/rubygems/test_gem_package.rb
+++ b/test/rubygems/test_gem_package.rb
@@ -510,6 +510,21 @@ class TestGemPackage < Gem::Package::TarTestCase
     assert_path_exist @destination
   end
 
+  def test_extract_file_permissions
+    pend "chmod not supported" if win_platform?
+
+    gem_with_long_permissions = File.expand_path("packages/Bluebie-legs-0.6.2.gem", __dir__)
+
+    package = Gem::Package.new gem_with_long_permissions
+
+    package.extract_files @destination
+
+    filepath = File.join @destination, "README.rdoc"
+    assert_path_exist filepath
+
+    assert_equal 0104444, File.stat(filepath).mode
+  end
+
   def test_extract_tar_gz_absolute
     package = Gem::Package.new @gem