Problem: A client has requested that we use SHA-512 to create file hashes. They have many files that they intend to use long-term and want a hash that is sufficiently long and secure to work for many years to come.

Proposed solution: Add a 'sha512' field to the filehash table and include it as one of the available options on the filehash settings page.

Issue fork filehash-3088648

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

beeyayjay created an issue. See original summary.

beeyayjay’s picture

Status: Active » Needs review
StatusFileSize
new2.66 KB

Patch implementing proposed solution.

mfb’s picture

Can you resolve the coding standards messages at https://www.drupal.org/pift-ci-job/1439666

beeyayjay’s picture

StatusFileSize
new2.65 KB

Fixes for coding standards.

beeyayjay’s picture

StatusFileSize
new2.64 KB

Fixes remaining indentation issues.

mfb’s picture

Status: Needs review » Needs work

Your specification of the index in filehash_update_7001() isn't working, maybe it needs an "indexes" key?

beeyayjay’s picture

StatusFileSize
new2.62 KB

Not sure why that didn't work, but changed to add index using db_add_index(). That seem to do it

mfb’s picture

There is still one coding standards message left to resolve.

mfb’s picture

The one downside of this patch is that the table size will be significantly larger.. changing to binary storage of hashes would be a nice future improvement.

beeyayjay’s picture

StatusFileSize
new2.63 KB

Yes, SHA-512 is really long. :-(

Anyway, here's the coding standards fix.

beeyayjay’s picture

Status: Needs work » Needs review
StatusFileSize
new2.62 KB

Fix coding standards indentation issue.

mfb’s picture

Version: 7.x-1.x-dev » 8.x-1.x-dev
Status: Needs review » Patch (to be ported)

Great! Thanks!

If you also have time to work on a Drupal 8 version of the patch, that would be amazing...

beeyayjay’s picture

I'll take a look.

beeyayjay’s picture

StatusFileSize
new4.21 KB

Here's the D8 version.

beeyayjay’s picture

StatusFileSize
new4.21 KB

Sorry, coding standards again. This should do it.

mfb’s picture

Status: Patch (to be ported) » Needs review
StatusFileSize
new4.13 KB

Rerolled

mfb’s picture

StatusFileSize
new3.96 KB
new969 bytes

Fixes broken addIndex in the update function (it passed in the full schema array rather than just the "filehash" element).

mfb’s picture

StatusFileSize
new4.07 KB
new602 bytes

Wrap long line in README file.

mfb’s picture

StatusFileSize
new4.06 KB

Very minor reroll due to README changes.

mfb’s picture

A while back, PHP added support for the "sha512/256" algorithm.

I wonder if that would meet the needs of this feature request?

Or is full sha512 a requirement for some users?

On the other hand, perhaps we should support dynamic schema specification when algorithms are enabled, and allow anyone to enable whatever hash algorithm they need.

mfb’s picture

StatusFileSize
new20.49 KB

Ok here's an improved patch that makes the rest of the SHA-2 and SHA3 algorithms available, modifying the table schema dynamically via the ConfigEvents::SAVE event.

Unfortunately ended up being a kinda big patch, as the new PHP hash algorithm identifiers are not "safe" alphanumeric strings. So I had to add logic to deal w/ that.

mfb’s picture

StatusFileSize
new20.97 KB

Update module help text and a couple other fixups.

mfb’s picture

StatusFileSize
new20.97 KB

Fix config change logic to handle NULL, e.g. config deleted in Drush.

mfb’s picture

StatusFileSize
new21.79 KB

Ok a bunch more changes needed :p

We need to add new column and index in two separate method calls, to work around SQLite issue (wanting the current state of full filehash schema).

Arbitrary config could theoretically be created, so I added some helper functions to validate the config before passing it in to table alter statements. Also separated the translated strings into their own function so these aren't loaded unless needed.

Views data cache was being invalidated incorrectly.

mfb’s picture

StatusFileSize
new21.8 KB

Use a nicer version of algorithm string for SQL column comment.

mfb’s picture

StatusFileSize
new21.76 KB

Ok turns out we cannot rely on the "original" config when handling config save events - particularly re: config import, which sets the original to NULL for some reason (#2605144: Original configuration is missing during configuration imports).

So revamping all the config state change logic to try to drop any columns that shouldn't be there, and create any columns that should be present but aren't...

mfb’s picture

StatusFileSize
new27.53 KB

Move any column dropping to a separate batch process - as it can be slow on larger databases and isn't really necessary.

(Adding a column should be fast in many cases thanks to "instant add column")

mfb’s picture

StatusFileSize
new28.54 KB

Report the name of the hash algorithm column currently being deleted to batch engine, and start adding some test coverage for new functionality.

mfb’s picture

StatusFileSize
new29.43 KB

Miscellaneous code cleanup. Guess next time I'll use a merge request, but I think this is getting close

mfb’s picture

Sadly we don't (yet) have a test for content-addressable storage w/ File (Field) Paths module, but I tested with [file:filehash-sha512_256-pair-1]/[file:filehash-sha512_256-pair-2] tokens and appears to work fine.

mfb’s picture

StatusFileSize
new29.24 KB

Cleaning up redundant logic in filehash.tokens.inc that calls hash_file() - I don't think this was ever needed, but certainly shouldn't be needed now.

mfb’s picture

StatusFileSize
new29.25 KB

Tweak strings from "unused" to "disabled". All we really know is they're disabled.

  • mfb committed 6c13128 on 8.x-1.x
    Issue #3088648 by mfb, beeyayjay: Add SHA-512 et al. to list of...
mfb’s picture

Version: 8.x-1.x-dev » 7.x-1.x-dev
Status: Needs review » Patch (to be ported)

Back to 7.x-1.x for backport

mfb’s picture

Status: Patch (to be ported) » Needs review
StatusFileSize
new13.56 KB
mfb’s picture

StatusFileSize
new14.43 KB

In case of config imported via features, drush, etc. catch PDOException and add missing database column, if required.

mfb’s picture

StatusFileSize
new15.08 KB

Some fixes for duplicate check.

mfb’s picture

StatusFileSize
new15.21 KB

Remove any invalid configuration elements before using in table alter statements.

mfb’s picture

StatusFileSize
new15.29 KB

Fix index name.

  • mfb committed 8870261 on 7.x-1.x
    Issue #3088648 by mfb, beeyayjay: Add SHA-512 et al. to list of...
mfb’s picture

Status: Needs review » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.