Fix .sprockets-manifest and gzip reproducibility issues

When building Rails assets with Nix, I found they are almost
completely reproducible save the manifest file, and the gzipped
files. This commit fixes those issues. For more information about why
it is good for a build to be reproducible see:

https://reproducible-builds.org/

Here are the reproducibility issues and how they were fixed:

1. .sprckets-manifest contained the time when the assets were
generated. Instead use timestamp 1.
2. gzip encoded the file mtime in the archive, which is not
reproducible. Instead use timestamp 1.
3. .sprockets-manifest generating a random path for this file is not
reproducible. Instead use the file's data to generate a digest.

CHANGELOG.md

@@ -4,6 +4,7 @@ Get upgrade notes from Sprockets 3.x to 4.x at https://github.com/rails/sprocket
 
 - Add support for Rack 3.0. Headers set by sprockets will now be lower case. [#758](https://github.com/rails/sprockets/pull/758)
 - Make `Sprockets::Utils.module_include` thread safe on JRuby. [#759](https://github.com/rails/sprockets/pull/759)
+- Improve reproducibility. [#761](https://github.com/rails/sprockets/pull/761)
 
 ## 4.1.0
 

lib/sprockets/manifest.rb

@@ -4,8 +4,6 @@
 
 require 'concurrent'
 
-require 'sprockets/manifest_utils'
-
 module Sprockets
   # The Manifest logs the contents of assets compiled to a single directory. It
   # records basic attributes about the asset for fast lookup without having to
@@ -17,7 +15,6 @@ module Sprockets
   # that don't have sprockets loaded. See `#assets` and `#files` for more
   # information about the structure.
   class Manifest
-    include ManifestUtils
 
     attr_reader :environment
 
@@ -51,19 +48,14 @@ def initialize(*args)
       # Default dir to the directory of the filename
       @directory ||= File.dirname(@filename) if @filename
 
-      # If directory is given w/o filename, pick a random manifest location
-      if @directory && @filename.nil?
-        @filename = find_directory_manifest(@directory, logger)
-      end
-
-      unless @directory && @filename
+      unless @directory
         raise ArgumentError, "manifest requires output filename"
       end
 
       data = {}
 
       begin
-        if File.exist?(@filename)
+        if !@filename.nil? && File.exist?(@filename)
           data = json_decode(File.read(@filename))
         end
       rescue JSON::ParserError => e
@@ -172,7 +164,7 @@ def compile(*args)
       end
 
       assets_to_export.each do |asset|
-        mtime = Time.now.iso8601
+        mtime = Time.at(1).utc.to_datetime.iso8601 # for reproducibility
         files[asset.digest_path] = {
           'logical_path' => asset.logical_path,
           'mtime'        => mtime,
@@ -277,6 +269,13 @@ def clobber
     # Persist manifest back to FS
     def save
       data = json_encode(@data)
+
+      # If directory is given w/o filename, use reproducible manifest location
+      if @directory && @filename.nil?
+        etag = DigestUtils.pack_hexdigest(DigestUtils.digest(@data))
+        @filename = File.join(@directory, ".sprockets-manifest-#{etag}.json")
+      end
+
       FileUtils.mkdir_p File.dirname(@filename)
       PathUtils.atomic_write(@filename) do |f|
         f.write(data)

lib/sprockets/utils/gzip.rb

@@ -12,7 +12,7 @@ class Gzip
       module ZlibArchiver
         def self.call(file, source, mtime)
           gz = Zlib::GzipWriter.new(file, Zlib::BEST_COMPRESSION)
-          gz.mtime = mtime
+          gz.mtime = 1 # for reproducibility
           gz.write(source)
           gz.close
 

test/test_manifest.rb

@@ -50,11 +50,10 @@ def teardown
     assert_equal filename, manifest.path
   end
 
-  test "specify manifest directory yields random .sprockets-manifest-*.json" do
+  test "specify manifest directory reproducible .sprockets-manifest-*.json" do
     manifest = Sprockets::Manifest.new(@env, @dir)
 
     assert_equal @dir, manifest.directory
-    assert_match(/^\.sprockets-manifest-[a-f0-9]{32}.json/, File.basename(manifest.filename))
 
     manifest.save
     assert_match(/^\.sprockets-manifest-[a-f0-9]{32}.json/, File.basename(manifest.filename))