Problem
- The current password hashing library is a custom fork of phpass.
- It has to be maintained by Drupal. Drupal should not be in the business of developing/maintaining a password hashing library.
- The hashing algorithm is 100% custom. 0% interoperability.
- The next time we upgrade our hash algorithm or iterations count, we have to deal with it all over again. PHP's password_hash() has forward-upgrading built in to its design
Proposed resolution
- Replace the custom password hashing library with PHP 5.5's
password_hash()
. - Use
password_hash()
with default parameters (i.e.$algo = PASSWORD_DEFAULT
and$options = []
). As a result, Drupal follows improvements to the defaults made in subsequent PHP releases automatically. - Sites with special needs may specify
$algo
and$options
by overriding the arguments of thepassword
service. - Extract the existing hashing mechanism to validate passwords of user entities created with Drupal prior to version 10.1.x into an
@internal
abstract base class. - Rewrite
Drupal\Core\Password\PhpassHashedPassword
to extend the newly introduced abstract base class and deprecate it. - Additionally introduce a separate core module (
phpass
). - Enable the
phpass
core module in apost_update
hook for all existing sites. - Add
phpass
as a dependency tomigrate_drupal
. This ensures that passwords migrated from an existing site can be verified without any further action. - Keep the
phpass
core module disabled for new sites. - In Drupal 11 remove
Drupal\Core\Password\PhpassHashedPassword
(but not the code moved to thephpass
module).
Note: Whether it is acceptable to disable the phpass
module is highly individual for each existing website. Some sites have thousands of active users and breaking their logins all at the same time will result in a huge support nightmare. On other sites there is only a small circle of admin accounts. Migrating their passwords to the PHP password_hash()
format might be accomplished in a few weeks without any manual intervention and the phpass
module can be disabled without causing any problems in a subsequent deployment.
Drupal should deprecated the phpass
module in some future release and move it to contrib. That decision could be taken based on actual usage numbers - which are expected to decline over time.
Compliance
In order to simplify adoption by US government agencies, a big effort was made to eliminate most usage of md5()
shortly before Drupal 7 was released (#723802: convert to sha-256 and hmac from md5 and sha1). Especially the change in password.inc
was disputed quite a bit within the community. The switch from md5
to sha512
in the password hashing code was perceived as being technically unnecessary and political to some degree. At that time the change was justified with FIPS 180-3 and FIPS 198-1 compliance (which went into effect in 2010).
NIST published SP 800-132 Recommendation for Password-Based Key Derivation in 2010 as well. That document is currently being reviewed. They are gathering feedback on the industry need for new password-based standards, including memory-hard password-based key derivation functions and password hashing schemes.
There is PHP #81326 Support a NIST SP 800-63B compatible password hash algorithm arguing that none of the password_hash()
algorithms is NIST compliant.
OWASP currently recommends Argon2id with a minimum configuration of 19 MiB of memory, an iteration count of 2, and 1 degree of parallelism.
Remaining tasks
None.
API changes
- A new password service (
Drupal\Core\Password\PhpPassword
) is introduced which essentially wrapspassword_hash()
,password_verify()
andpassword_needs_rehash
). - The implementation of
Drupal\Core\Password\PhpassHashedPassword
is moved to a newphpass
module. - A deprecated subclass is left at
Drupal\Core\Password\PhpassHashedPassword
and removed in Drupal 11.0
Data model changes
None
Release notes snippet
Drupal now uses the default PHP password_hash() and password_verify() functions in order to store and verify passwords securely. Backwards compatibility is provided by the new phpass module that will be installed on existing sites via an update.
Comment | File | Size | Author |
---|---|---|---|
#297 | 1845004-nr-bot.txt | 90 bytes | needs-review-queue-bot |
#296 | interdiff.1845004.265-295.txt | 18.47 KB | longwave |
Issue fork drupal-1845004
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #1
RobLoachComment #2
Owen Barton CreditAttribution: Owen Barton commentedOur implementation has diverged from phpass due to using sha512 primitive so the hashes are no longer compatible, so I don't think we can make a direct switch to this library (without extending it in a fairly non-trivial way).
If we are going to switch/improve implementations, we should use the php 5.5 built in password hashing API, together with the backward compatibility wrapper (which supports all PHP versions D8 supports). This is stronger than our sha512 based phpass (for the same hashing workload time) due to bcrypt usage, and also an agreed standard, rather than a defacto one, so encourages broader comparability between web frameworks/platforms.
See discussion at #1201444-39: Strenghten password hashing mechanism.
https://wiki.php.net/rfc/password_hash
https://github.com/ircmaxell/password_compat
http://blog.nic0.me/post/63180966453/php-5-5-0s-password-hash-api-a-deep...
Comment #3
Crell CreditAttribution: Crell commentedI agree with #2. If we're going to change our password hashing logic at all, it makes the most sense to just go straight to the PHP 5.5 API with a known BC layer written by the same author. No sense reinventing a wheel that is already right there in front of us.
Comment #4
sunTotally +1. Re-titling accordingly.
Just the aspect that our code still has "Phpass" in its name is a very terrible joke.
Implementing this is easy. But we need a plan for migrating those custom hashed passwords from D7. :-/
Comment #5
sunComment #6
claudiu.cristeaLet's start with a patch. It's just a very rough approach. I ran no tests yet and the D7>D8 password rehashing is still something that needs attention (I wrote the D6>D8 migration for user password rehash).
Comment #7
claudiu.cristeaComment #9
claudiu.cristeaRemoved the D7 password service injection.
Comment #11
claudiu.cristeaOops! Wrong patch, sorry.
Comment #13
killua99 CreditAttribution: killua99 commentedYou're starting with a patch for PHP 5.5 ? or you're doing this for PHP 5.5 + the fallback for PHP 5.4 ?
Comment #14
claudiu.cristeaNew version. Still under heavy development.
@killua99, yes I'm trying that approach.
Comment #15
claudiu.cristea\Drupal\Core\Password\Password
to more meaningful\Drupal\Core\Password\PhpPassword
Still @todo:
Comment #16
claudiu.cristeaSome fixes on D7 hashed password detection when migrating.
Note:
Reviews are welcomed.
Comment #18
claudiu.cristeaI cancelled manually the test on
replace_d7_password-1845004-16-wo-library-and-composer-gen.patch
. Forgot to name it in the right way.Comment #19
claudiu.cristeaAdded test for first time login after the password has been migrated from D6 or D7 and fixed the rehashing from D7 (that has a bug in previous patch).
Comment #20
claudiu.cristeaSwitching back to the previous title. It is better.
Comment #21
claudiu.cristeaTagging.
Comment #22
killua99 CreditAttribution: killua99 commentedThe patch that doesn't have the library "password.php" mean is ready for PHP 5.5 ?
Comment #23
claudiu.cristea@killua99, no. That one is a simplified version for reviewers.
Comment #24
Crell CreditAttribution: Crell at Palantir.net commentedI'd like to have this vetted by the security team; length-limited passwords are usually regarded with derision by most security gurus, although in this case it may be logical if the number is big enough.
If it is kept, move the magic value to a class constant rather than make it inline?
If I'm reading this correctly, we don't rehash an old password. That is, a user with an ancient D6 password will still have that password in the DB until they change it, rather than it being updated to the more secure modern hash at login. Is that true? Don't we want to upgrade all of those old hashes? (I don't know what we did with this in D7 so I may be barking up the wrong tree here.)
So it is supposed to happen, but I don't see it happening above. Am I just missing it, or do we need more docs?
Split this into 2 separate tests. They have little to do with each other.
Invalid docblock. One line please.
The "We want..." sentence can just be moved to its own line, after an extra linebreak.
Comment #25
Berdir@Crell: The password length check was a security issue a while ago: #2378703: Port denial of service fixes from SA-CORE-2014-006 to Drupal 8 It's just ported over from the current implementation.
Comment #26
claudiu.cristea@Crell,
Short answer: No. In every moment we have D8 hashed passwords in the DB.
Here's how it works:
What passwords are rehashed?
Password rehashing can be offered ONLY if the users are migrated using the migrate in core. Otherwise the migrate manager should take care of its own custom rehashing policy.
What happens during migration?
When the destination of the migration is a Drupal 8 user —
\Drupal\migrate\Plugin\migrate\destination\EntityUser
that class swaps the main password hasher with\Drupal\migrate\MigratePassword
. This one has the main hasher injected in the constructor so he can use the D8 hashing mechanism. The service swapping is handled by\Drupal\migrate\MigrateServiceProvider
that is magically (yes, we're still using magic!) called.What
\Drupal\migrate\MigratePassword
does?Now we're safe. In order to break the passwords a hacker needs to break the D8 new hash that is based on
password_hash()
.Rehashing the password
When a user authenticates for the first time we also check the hashed prefix
EDIT (ADD): After a successful authenticating we simply use the plain pass and recreate the hash with D8 hasher.
Comment #27
claudiu.cristeaFew small fixes and a more compact patch.
Comment #28
pwolanin CreditAttribution: pwolanin commentedMaybe this should be a motivation for 5.5 as the minimum supported version?
Comment #29
claudiu.cristea@pwolanin, I fully agree :)
Comment #30
Crell CreditAttribution: Crell at Palantir.net commentedThanks, Claudiu, that helps. I don't see the "reset on first login" logic, though. Is that just elsewhere outside this patch's context?
I wouldn't be opposed to requiring 5.5, but I don't know that it simplifies any code here. The BC library from ircmaxell is a straight drop in, and most of the code here is around old passwords. What would be the 5.5 benefit here, specifically? (This would be a catch and/or Alex decision either way.)
Comment #31
claudiu.cristeaComment #32
claudiu.cristeaAdded a comment discussed on IRC. Needs... English language review :)
Comment #33
Crell CreditAttribution: Crell at Palantir.net commented"... will not comply WITH the expected..."
English looks good otherwise. ;-)
Comment #35
claudiu.cristeaComment #36
Crell CreditAttribution: Crell at Palantir.net commentedAs soon as the bot is happy with it, so am I. Thanks, Claudiu!
Comment #37
alexpottSo why back to 10? There does not seem to be any discussion of this on the issue.
Lets put this on PasswordInterface
From one of the blogpost's linked: Anthony Ferrara (@ircmaxell) was kind enough to point out that bcrypt actually limits passwords to 72 characters internally, so it’s impossible for someone to take down your server by sending very long passwords. He also provided me with a StackOverflow link with more information on the matter.
I guess it is okay to leave this in.
Not used
Not used.
Comment #38
pwolanin CreditAttribution: pwolanin commented@Crell - you are assuming that the library is correct, etc. it would be nice to reduce the number of code forks and we shouldn't ship Drupal 8 supporting php 5.4 anyhow.
I'm not sure I'm happy with the all the code changes. Can you explain:
Also - the flag to enable MD5 is a bit odd and seems overkill - you should just read the algorithm from the setting string?
Comment #39
pwolanin CreditAttribution: pwolanin commentedComment #40
Crell CreditAttribution: Crell at Palantir.net commentedpwolanin: Well both the C function and the BC lib were written by ircmaxell, in parallel, so I am assuming that yes, they are compatible as advertised.
Comment #41
claudiu.cristea@alexpott
#37.1
That is not the same parameter as in D7 hashing. So we are not downgrading the complexity. This is the
cost
ofpassword_hash()
new function (see http://php.net/manual/en/function.password-hash.php). I used the default value for this option as it's used by the PHP function. It's true that there was only one discussion on that. I asked Anthony Ferrara (@ircmaxell) what is his opinion about that. He answered me: "It's enough". Other opinions would be valuable. Just keep in mind that the increase with 1 of this value doubles the time needed for a hash. Once I tested with 15 and I felt the delay.#37.2 and #37.3: You're right. It's stated in PHP documentation:
Moved the constant to interface. In fact is useless now. Even somebody will use a password between 72 and 512, he will not be aware that only the first 72 chars were used. In fact this can lead to the next hypothetical and fun situation: A user is entering a password > 72 but he can login with a different password as long the first 72 chars are correct. Shouldn't we add a constraint to user password field limiting the length to 72?
@pwolanin
Yes, sure. When a D7 user logs in for the first time in D8 we need to hash his plain password first with D7 hash. But we need to use the same salt as it has been used when originally his password was hashed in D7. That's why, after the "D7" prefix we are adding the D7 salt. When we are hashing with D7 algorithm we need to be able to pass the salt (extracted from the prefix) to the hashing engine of D7. Unfortunately the only method on the interface (public) is
PhpassHashedPassword::hash()
. But that method doesn't take the salt as parameter. We need access to a method that can accept the salt. The straight way seems to me makingPhpassHashedPassword::crypt()
public so that I can access the hashing mechanism from outside and pass the salt that I want (not a random generated one).That I inherited from D6 > D7 migration. Why is "U"? I have no idea. I can replace it with "D6" but in this case we have maybe to be more flexible because we can use it to rehash from Joomla, and also other old systems that uses MD5. Open to discussion.
Comment #42
claudiu.cristeaTagging
Comment #45
claudiu.cristeaNeeds reroll.
Comment #46
neclimdulReroll.
Comment #47
neclimdulignore interdiff... I was considering using lazy but doesn't seem needed. during upload it disappeared from the files list though so I couldn't remove it.
Comment #48
claudiu.cristea#37, #38 were addressed, so back to RTBC (per #36)
Comment #50
neclimdulreroll
Comment #51
claudiu.cristeaBack to RTBC.
Comment #54
catch+1 to getting back to standard phpass passwords.
Few questions though:
1. This doesn't remove PhpassHashedPassword.php at all - so we're adding the compat library but not actually getting rid of any of our custom code? Nor is any of it marked deprecated
2. Issue needs a beta evaluation. Lack of interoperability is a (minor) bug so I think this issue is fine just for that, but could use documenting
3. Found a couple of nits in the patch.
This says the cost value is 10, but we set it to 16.
typo: an old Drupal
Comment #55
claudiu.cristea@catch,
#54.1 : Right. We need to keep the mechanisms that makes possible the rehashing of a plain password to Drupal 6 and Drupal 7 hashes. Otherwise we cannot rehash the passwords imported (migrated) from D6 and 7. While with D6 it's easy (because the hashing mechanism is
md5()
), we need to keep the code that make possible to hash a D7 plain password and that isPhpassHashedPassword.php
. Please read #26 for a detailed explanation.#54.2: Added.
#54.3.1: Those two values are different things. "16" is a D7 hashing mechanism parameter called "password stretching iteration count", referred also as "countLog2" in code. "10" is something different, related to the new hasher we are implementing, and is called cost. You can see in core.services.yml that we have 2 different services:
password
(the new D8 hasher) anddrupal7_password
(the D7 hasher — that we still need for migrate). "16" is allocated todrupal7_password
and "10" is the cost allocated topassword
.#54.3.2: Fixed.
Comment #56
catchOK that makes sense for the 7.x passwords. I think we should add additional docs to that class to explain why it's there now, and possibly rename it though?
Also all the password hashing services are good candidates to mark as lazy.
Comment #57
claudiu.cristea@catch,
OK.
First I renamed but then I preferred to keep the patch footprint small. Renaming will make a bigger patch. Maybe as a followup?
OK. Let's see if passes the tests.
Comment #61
claudiu.cristea@catch,
The problem with making LAZY the 2 services is that they need
\Drupal\user\UserInterface
(thank you @dawehner for clarifying me this!). Unfortunately proxies of module interfaces don't work! Also, conceptually, it is wrong that a class in\Drupal\Core
depends on an interface that is in a module (\Drupal\user
).That said, and taking other comments in this issues (@catch: rename the D7 hasher class, etc.), I decided to clean up more and remove
\Drupal\user\UserInterface
from\Drupal\Core\Password\PasswordInterface
. The methods that are requesting now the user arePasswordInterface::check()
andPasswordInterface::userNeedsNewHash()
. In fact we don't need the user object to be passed to the hashing engine. We need only the hashed password extracted from the user.It's true that passing the entire user object was made with this assumption: by extending the class somebody is able to swap the mechanism and use other backend to lookup for the hash. (eg. use the uid from the account object and query an external backend). But this assumption is wrong because this interface should not be in business of abstracting the authentication authority. To switch to other authentication authority (credential database) the developer should implement the
\Drupal\user\UserAuthInterface
. The shipped implementation\Drupal\user\UserAuth
is the place where the user database is queried.PasswordInterface
must be responsible only for hashing. This is resolved now in the patch.But after I dropped the reference to
\Drupal\user\UserInterface
in\Drupal\Core\Password\PasswordInterface
I found out that there’s no more Drupal bootstrapped context for this interface. So, yes, this interface defines a now Component. The new namespaced name is\Drupal\Component\Password\PasswordInterface
.I also performed some docs cleanup and the LAZY stuff just works. Method
::userNeedsNewHash()
is renamed to::needsRehash()
.Comment #63
claudiu.cristeaForgot two occurrences...
Read the comment and interdiff from #61.
Comment #64
dawehnerI like that solution! It allows you to reuse the service in more places!
This now introduces a couple of API changes ... so tagging.
Mh, so afaik in components, there should be no Drupal specifics, what about not naming the Class Drupal7Passsword but rather keep the name PhpassHashedPasword?
Do you mind trying to not clean everything up? Don't fix the docs here, please, it makes the patch way bigger than actually needed. Thank you
So yeah, the only really needed change is this hunk, I think
Can we point to some documentation, what this means? Is it scaling linear?
So yeah I'd argue that this bit is specific to Drupal, so it should not be a component.
Comment #65
znerol CreditAttribution: znerol commentedThe API change makes a lot of sense and also the refactoring into a component.
Yes, in fact the parameter documentation of the old D7 generator has an appropriate comment. Perhaps copy that over. Another option would be to point to the PHP documentation, however that one references the crypt docs.
Btw. I do not see any reason why we would want to lower the default cost to 10. Is there any evidence that either the D7 value was to high or that the security of the new implementation with algorithmic cost 10 actually matches the security of the old implementation?
Please no. This patch was RTBC several times and the doc changes never were rejected before. So let's not throw away work which is already done.
How about introducing a core password service which wraps around both (the d7 and the new password service) in order to ensure backwards compatibility on migrated sites?
Comment #66
dawehnerSorry, well fair
Comment #67
catchYes I also think we should set this to 16 unless there's a reason not to.
Timing the old vs. new password hashing ought to give us an idea how computationally expensive they are compared to each other.
Comment #68
claudiu.cristea@catch, @znerol,
I responded to this question back in #41 & #55. There's no decrease, they are simply different parameters. Also, i discussed with Anthony Ferrara / @ircmaxell (the developer of https://github.com/ircmaxell/password_compat — the PHP < 5.5 compatibility library) about the value of cost. The "cost = 10" is the default value used by
password_hash()
and he said "It's enough". Keep in mind that an increase of this value with 1 doubles the computation time for hashing.Comment #69
Damien Tournoud CreditAttribution: Damien Tournoud at Centarro commentedGiven that those are two completely different implementation, the easiest way would be to benchmark the old implementation vs the new one and come up with a cost value that leads to roughly comparable results. That would be a net increase in security, as the new implementation is expected to be much faster. Just @ircmaxell choosing 10 as the default value is a good reason enough for us to chose the same value.
Comment #70
claudiu.cristea@dawehner
#64.1, #64.5: Ok, ok. For me this is behaving like a component because it doesn't need a Drupal bootstrapped context but, it's true, smells as Drupal a lot. So, back to
\Drupal\Core
. But now, because is core, I kept the name :)#64.2, #64.3: Thank you @znerol, @dawehner :)
#64.4: Added.
Comment #71
znerol CreditAttribution: znerol commentedThe cost parameter of the PHP password hash function is exactly the same as the count_log2 parameter in our implementation. The only thing which changed is the hashing function (sha256 vs blowfish). We need to ensure that the amount of work necessary to crack passwords does not decrease. I highly suspect that 2^10 rounds of blowfish is much cheaper than 2^15 (drupal 7) rounds of sha256.
It is important to understand that this is excatly what we want. The parameter needs to be set as high as tolerable.
Comment #72
neclimdul@catch can we not rename this. Technically its the D7 hash algorithm but really its the forked phpass implementation.
And other cleanups. I second dawehner on leaving this out. It has not been rtbc'd it was added in #61
Comment #73
catchYeah it's not a massive deal for me. However some indication of what it's there for now would be good. Eventually we should be able to remove this code - although it'll need to be there for 7-9 migrated password hashes so not before Drupal 10.
Comment #74
claudiu.cristea@znerol,
#71: I'm not gonna disagree with you. Then I expect an agreed value for the cost. I do not know in detail about hashing algorithms. I only had a discussion with @ircmaxell and his input was that 10 is OK.
@neclimdul,
#72.1: I think that
Drupal7Password
tells better what is that.PhpassHashedPassword
was diverged from PhPass. So it's not PhpPass.#72.2: Oh, please :(
Comment #75
neclimdul#1203852-4: Increase hashing iterations for D7 and D8. wall time. 50-100ms
10 actually falls between this but it varies from the original implementation depending on various factors(its a different algorithm).
From this, I think 10 is actually in our target range and fine. I'm open to more review from other people in the security team though so I will mention this to them.
Comment #76
claudiu.cristea@neclimdul, thank you for benchmarking. I'm just confused a little looking in the script. For D8 you are using every time the same cost:
new PhpPassword(10, $d7hash);
?Comment #77
pwolanin CreditAttribution: pwolanin commentedThe question is not really how fast it is in PHP really, but how fast could you make some optimized cracking code.
Looks like at least 11 would be safer from your numbers,
also I would consider this wrong (patch needs work):
Just leave it as
$S$
Unless there is a crypt hash with id
$S$
that we might be consuming. I don't see any collision with http://php.net/manual/en/function.crypt.phpComment #78
pwolanin CreditAttribution: pwolanin commentedI think the whole wrapping logic of the D7 class is flawed - we should just have one implementation that has a case statement on the hash indetifier
We should also set PASSWORD_BCRYPT as the algorithm, or we run the risk that a DB dump from Drupal 8 on PHP 7.1 will not work on Drupal 8 on PHP 5.6. Then we can look for the expected "$2y$" identifier
Comment #79
claudiu.cristea@pwolanin,
No, that is not wrong. That is not a D7 hash but a D7 hash over-hashed with D8 hash and prefixed with 'D7' in the moment of migration (see
\Drupal\migrate\MigratePassword
). That should remain as it is (or we can replace the "D7" prefix with something else more meaningful?). Please read #26 to understand how the D6 and D7 passwords are migrated and rehashed.I'm OK with cost = 11 but keep in mind that the count_log2 in D7 is now 15. In D8 was already increased to 16. So, I think 10 is acceptable now but can be increased later, in future releases.
Comment #80
claudiu.cristeaRe #78
We need the D7 hashing mechanism in D8. Why? We need to rehash the plain password when the user logs in for the very first time. But before rehashing, we need to authenticate the user (is he a valid user with a valid password?). In that moment we have his password hashed with D7 service and over-hashed with D8 (on migrate). So, we need the D7 hashing service to validate his plain entered password. I don't say we cannot mess up everything in a single service. We can, but that will be very ugly. In the patch we have a very clean, separated logic for the 2 hashers. In fact they are 2 implementations of the same interface — and that sounds good.
Comment #81
pwolanin CreditAttribution: pwolanin commentedI think re-hashing the hashed 7 password with the 8 algorithm is pointless work. I do not think that should go in core. It also makes the D7 -> D8 migration much slower than it needs to be.
I'd rather see here a small class that routes to a checker pased on hash ID.
Comment #82
pwolanin CreditAttribution: pwolanin commentedOverall, I basically think the approach of patch is confusing and needlessly complex since the PhpPassword class needs to have the other instance injected and needs to know how to delegate to it.
If you need to support old + new, there should be a "routing" class that does that. The PhpPassword class should stand alone and not know anything about hashes other than php bcrypt.
Comment #83
claudiu.cristea@pwolanin,
Should D8 store MD5 hashed passwords (from D6)? Of course not! We need to align the security of those imported/migrated passwords to D8 standards. If, for some reasons, the database data leaks, we have to make sure that the hashes are strong enough.
The same applies to D7 > D8. Why keeping in backend passwords hashed with different algorithms? After migrating all users, in D8 we will have only D8 hashed passwords. Yes, some of them will be rehashed when the user will send us the plain pass for the first time. But in every moment we are aligned to
password_hash()
standards in terms of computation effort needed to compute the hash.Setting the
PASSWORD_BCRYPT
as explicit algo sounds good. Let's see other opinions on this, but to me this would assure that we are consistent across different PHP versions.Comment #84
Crell CreditAttribution: Crell at Palantir.net commentedclaudiu: I think the point is that D6 passwords are too weak of a hash, so wrapping those in another hash is sensible, but the D7 passwords are already securely hashed, just with a different algorithm; wrapping those doesn't improve security of the old passwords.
Comment #85
BerdirI'd say it would also considerably lower the barrier of this getting committed if the switch could happen on existing D8 site without having to manually re-hash all existing passwords first. Just because we don't officially support an upgrade path yet doesn't mean we have to try hard to break existing sites ;)
Wondering what exactly would happen on an D8 site that already has users migrated from D6 with re-hashed passwords. (Purely theoretical question of course. I of course wouldn't have such a site).
Comment #86
claudiu.cristea@Crell, in both cases (wrapping the D7 hash or not) we need the D7 hashing service. And, conceptually, each (D7 & D8 service) should be implementations of an interface. Well, not over-hashing the already D7 hashes is a point. I don't see what is the gain. I started from the D6>D7 model but, yes, it's true that D7 hashes are stronger.
@Berdir, I'm not sure I understood well. The patch doesn't propose a manual rehash. What would lower the committing barrier? Sorry.
Comment #87
claudiu.cristea@Crell, in both cases (wrapping the D7 hash or not) we need the D7 hashing service. And, conceptually, each (D7 & D8 service) should be implementations of an interface. Well, not over-hashing the already D7 hashes is a point. I don't see what is the gain. I started from the D6>D7 model but, yes, it's true that D7 hashes are stronger.
@Berdir, I'm not sure I understood well. The patch doesn't propose a manual rehash. What would lower the committing barrier? Sorry.
Comment #88
pwolanin CreditAttribution: pwolanin commentedWe don't need a D7 hashing service, we just need the D7 hashing code as a utility class that can compare an existing hash to an incoming plaintext (or already md5'd) password
Comment #89
neclimdulre #76, the hash changes the iterations
re #87, Berdir is pointing out that d8 sites in the wild have phpass password and this patch would leave the sites broken.
Comment #90
neclimdulTalking to pwolanin earlier today he said he'd be happy with a class containing delegation logic and classes for the hashing algorithms. This patch implements that. This has a nice side effect that you could trivially use only the php_password implementation on new installs. Also, I set it up to support parse phpass hashes so existing sites don't break which leads to the next point.
I've also kicked the can with this patch and removed all D7 migration related code(other then tests that are going to fail). Sorry about the test I just ran out of steam and wanted to get some thoughts on this.
Let me explain why I did this.
1) We don't have a migration path for D7 users so we're guessing.
2) There seems to be strong feelings that the D7 hashed passwords are strong enough for now which would mean we can just store them as is.
We can handle those two concerns in follow ups.
Comment #92
claudiu.cristeaConceptually:
Drupal7Password
can be an exception but we can keep it asPhpassHashedPassword
. Renaming this was only a matter of telling that this is not really a PhPass algo.Comment #93
neclimdul#92.1 right on
#92.2 ditto
#92.3 this seems to be the solution proposed b znerol and pwolanin. What were the service concerns?
I'm also open to better names. I literally went "Man don't know what to call this... Whatever this will work for now ." legacy delegator or something along those lines?
Comment #94
pwolanin CreditAttribution: pwolanin commentedre: services - I don't wee why there ever needs to be more than 1 service registered. A service is something that may be swapped, right?
Let me take a shot at further simplifying.
Comment #95
pwolanin CreditAttribution: pwolanin commentedOk, rewrite to 1 service and hack the phpass class down to some minimal static methods.
We should probably generate a few Drupal 7 hashes and migrated Drupal 6 -> 7 hashes as test fixtures for a unit test since this code can no longer generate new Drupal 7 hashes.
Comment #97
claudiu.cristeaSo, we are not implementing
PasswordInterface
anymore. Now we have 3 classes but only one service. I really don't understand what's the gain here. Now the whole logic lost its simplicity.Comment #98
pwolanin CreditAttribution: pwolanin commentedThere is absolutely no reason for the back-compatibility code to implement the whole interface.
There is no reason for multiple services wither. We just need 1 service. If you do not want to support old/migrated/imported hashes, you would just replace the main service and use \Drupal\Core\Password\PhpPassword
Comment #99
neclimdulThis isn't consistent...
Catch requested this change
Why static?
completely gone? The interface is broken.
Comment #100
neclimdulre #94 I think that's too limiting a definition. Swapped out or dependency. Possibly other reasons.
re #98
Again I disagree.
1) the code is more then a backwards compatibility service, its a working implementation. It could be used to integrate with phpBB for example.
2) Logic is consistent and clear
3) You can still swap out the implementation with the 3 service approach.
Comment #101
neclimdulReroll and test fixes for #90.
Comment #102
neclimdulsorry. testbot.
Comment #103
claudiu.cristeare #101
I still don't understand why we need 3 classes. We have the main hashing service but we want to be able to rehash from D6 and D7. For that we need to keep the old hashing engines. With D6 it's easy because the hasher is in PHP, it's
md5()
. With D7 we have to keep the old hashing service as it is and make it available for the main service. But how? By injecting the D7 hashing service into the main service. Simple and clean.Comment #104
pwolanin CreditAttribution: pwolanin commentedSo, I disagree that maintaining the full D7 (crufty and unused) implementation of the interface is useful. That's making core harder to maintain.
I also think the 3 services setup makes the logic here needlessly complex. You can swap the one service if you need something different.
@claudiu.cristea - the code to check an existing D7 hash can be 100% static. There is no reason to inject a service. This is not something you would every swap out.
Comment #105
neclimdulIt is used and will need to be maintained regardless in all discussed approaches because we'll have migrated d7 passwords and old d8 passwords. We're talking about a handful of additional methods that will largely match their ported d7 counterparts.
The password service could have 100 dependencies and if you replace 'password' its replaced and the dependencies go unused. I changed to the '.' separated values for the service names to clarify this.
Being static really depends on keeping the interface on the d7 class. We don't want to mix static and global state obviously.
But again I don't agree with "no reason to inject." The PhPass implementation is a dependency therefore we should inject it. Making it static just hides that fact that it is a dependency which is not necessarily a good thing. For one, it makes certain forms of unit testing very difficult.
Comment #106
pwolanin CreditAttribution: pwolanin commentedI'm not sure I see why static code (within the same namespace) could make unit testing harder.
If so then let's put the remaining code into the single "routing" class.
DI makes sense where you might inject a different implementation using the same interface. That's not true here as far as I can tell. If core is going to use the php password implementation, then it's ok to simply instantiate an instance.
Comment #107
Crell CreditAttribution: Crell at Palantir.net commentedVia pwolanin:
Incorrect. A service is a stateless object that does something self-contained. If using a DIC correctly they can (almost) always be swapped out, but that is not the only reason to use them. "I don't know why you would swap this out" is not in any way a reason that something should not be a service. There may be other reasons to not makg something a service, but that is not one of them.
Because a static call always means code that you cannot ever not use. It means that the smallest "unit" that can be tested is the aggregate of the service in question AND the class it's statically calling... and anything else that calls. That chain can get long very quickly if you're not careful. It's a very slippery slope. (I view every single "static utility class" we have in core in D8 as a self-inflicted bug for this reason. We should fix that in Drupal 9, or even 8.x.) While in this specific case one could argue that the chain doesn't go very far, that's not the point. If you cannot test the D8 password logic without running D7 password logic, there is something seriously wrong with the definition of "unit" and you need to fix your code.
That said, there is also no reason, strictly speaking, why the D7 password class needs to implement the same interface as the D8 password class. There are extremely few people that would be using D7's logic on D8; I would go as far as arguing that is a case we don't even want to support. (People running beta sites knew what they were getting into; Worst case, you tell everyone to use the password-request feature and you're done. kthxbye.)
The problem with having the D7 hasher be a dependency of the D8 hasher is that we then need to load it on every single login. That's code that will *never* run on a site installed fresh, and will run less and less on an upgrade site over time. The 95% case is that the D7 hasher is not needed, so we don't need to spend the time on it.
Comment #108
pwolanin CreditAttribution: pwolanin commented@Crell - so take a look at my patch - I assume the static code is only loaded if you ever hit the relevant branch in the case statement?
I also agree that we should not be supporting/enabling people to use the old algorithm if we are moving to a new and more standard one.
Comment #109
pwolanin CreditAttribution: pwolanin commentedAlso - I'm remembering now that we made the interface take an account object on purpose so we could authenticate using some scheme other than local password hash comparison if desired. There doesn't seem to be any justification for this API change in the issue summary.
Given that this feature could be added to 8.1, etc, with no migration path needed, I'm really questioning why we need to do it for 8.0.0.
Comment #110
claudiu.cristeaThat is wrong as architecture. This service should do only one thing: provide hashing service. The same thing is now implemented in PHP >= 5.5.0 by the suite of password_*() functions. If you want to swap the authentication authority that should be done by implementing
\Drupal\user\UserAuthInterface
. The shipped implementation\Drupal\user\UserAuth
is the place where the backend is queried for the stored hash. And that is correct. Also we cannot make the services "lazy" because #2408371: Proxies of module interfaces don't work.Comment #111
xjmI'm on the fence as to whether this is a feature request (because the goal of the issue is to add interoperability) or a task (because why?). It's definitely not a bug, so changing that claim in the beta eval.
I'm also inclined to agree with @pwolanin that this should be postponed in either case due to its disruption. Even with migration paths and BC this is a risky change to introduce during the beta and liable to add technical debt, so my perspective is that the disruption outweighs any impact in terms of interoperability or potential security hardening.
Comment #112
xjmComment #113
neclimdulI'm happy to discuss disruption.
First, I think we can roll back the method name changes and user object to string change if that's a blocker. Those are the real "API" changes and they where done to support the lazy tag on the service. Really we are probably not going to gain a lot from a lazy proxy because we don't do anything in the constructors of these classes so the cost of instantiating them over their proxies is probably negligible.
Also, I think some of the other disruptions are a bit overstated.
1) migration path. Technically there is not update hook or migration code. Earlier d8 hashes and d7 hashes get rehashed the same way we would rehash them if we upped the cost value. Its is just using a new hashing library for the new hash.
2) Technical debt. Why technically adding any code is technical debt, really we are trying to removing technical debt by deprecating a hashing library we maintain internally in favor of one maintained by PHP.
Comment #114
Crell CreditAttribution: Crell at Palantir.net commentedxjm: Is the disruption greater in beta than in 8.1? "We changed all your passwords" seems like a more invasive thing to do in a point release than in beta...
Comment #115
pwolanin CreditAttribution: pwolanin commentedSo, claudiu.cristea makes a valid point about this interface being the wrong place to manage external auth.
Perhaps that's a bug and I'd support changing just the interface to take the plain hash so we could revisit the rest of this in 8.1.
@Crell - the stored hash is only changed when a user logs in, so it's not really disruptive.
Comment #116
giorgio79 CreditAttribution: giorgio79 commentedWow! Could the patch differentiate between users logging in via the regular interface vis a vis a backend login such as Services ? Where I work, Drupal is integrated via https://www.drupal.org/project/ldap, so there is no way the password can be changed upstream.
Comment #117
pwolanin CreditAttribution: pwolanin commented@giorgio79 - you'd probalby be working at a higher level like \Drupal\user\UserAuthInterface to handle local vs remote users?
Comment #118
pwolanin CreditAttribution: pwolanin commentedBased on discussion with catch, created this issue to split out just the interface change: #2503083: Simplify PasswordInterface so it's not coupled to UserInterface
Comment #119
dawehnerNote: We are going with 5.5 now so I think we shoudl no longer provide a wrapper here.
Comment #120
claudiu.cristea@dawehner, right. Changing the title and IS.
Comment #121
claudiu.cristeaOops! That auto-complete :)
Comment #122
pwolanin CreditAttribution: pwolanin commentedWe should fix the interface in the other issue and postpone changing the code for 8.1
Comment #123
claudiu.cristeaAgree with postponing but, as @Crell said in #114, beta is a better moment to make this switch than 8.1.x. Moving back to 8.0.x.
Comment #124
Crell CreditAttribution: Crell at Palantir.net commentedOne of the reasons for going 5.5 is to avoid pulling the rug out from under people in a point release. The same argument applies here. Let's do it now for 8.0.0.
Comment #125
cilefen CreditAttribution: cilefen commented@claudiu.cristea @Crell The Beta Evaluation could be improved to explain the need to get this in now.
Comment #126
Crell CreditAttribution: Crell at Palantir.net commentedUpdating issue summary accordingly.
Comment #127
claudiu.cristeaHere's a patch built on top of #2503083: Simplify PasswordInterface so it's not coupled to UserInterface. The patch is based on #70 but with the next changes:
PASSWORD_BCRYPT
algo. See #78.::check()
ofPasswordInterface
was renamed to::verify()
to be consistent with PHP 5.5 password functions namingPasswordInterface
methodpassword_hash()
::hash()
password_verify()
::verify()
password_needs_rehash()
::needsRehash()
There's a patch built on top of #2503083: Simplify PasswordInterface so it's not coupled to UserInterface and a non-testing relative patch. Anyway the test will fail because the bot is still on PHP 5.4.
Comment #128
claudiu.cristeaJust to run the test. I will switch it back later.
Comment #130
claudiu.cristeaComment #131
JeroenTSimplify PasswordInterface so it's not coupled to UserInterface Is in.
Comment #132
claudiu.cristeaYes, but is still postponed on #2508231: Raise minimum required version of PHP to 5.5.9.
Comment #133
pwolanin CreditAttribution: pwolanin commented@Crell - I don't understand your argument about point releases.
Comment #134
pwolanin CreditAttribution: pwolanin commentedAlso, I still don't think the last patch is the right approach (there is no need to add a service, and the code to make D7 hashes should be removed), and as a feature request this should still wait for 8.1
Comment #135
claudiu.cristeaIt was wrongly changed to Feature request. It's a Task because:
What (I think) @Crell pointed out in #124 and I agree is that this switch introduces a kind of disruption. The passwords will be rehashed in the backend on first successful login. Better to do this on BETA phase not, later between 8.0.x and 8.1.x, when will affect (maybe) one million of websites. Even the transition of passwords seems smooth let's take this precaution. Doing the switch now will affect about 6,000 websites (according to https://www.drupal.org/project/usage/drupal, most of them not in production in this moment.
Comment #136
pwolanin CreditAttribution: pwolanin commentedSorry, but I disagree with most of those points, and in #111 the core committer agreed it was a new feature and should be postponed to 8.1
Comment #137
cilefen CreditAttribution: cilefen commentedA release manager has already stated their opinion on this. Unless there are new facts, we should go with that decision.
Comment #138
znerol CreditAttribution: znerol commented@claudiu.cristea: Would you be so kind and post the documentation fixes as a separate issue?
Comment #139
xjmThanks everyone!
@catch and I chatted about this issue a bit more this morning. We decided that we should consider this issue a task, and that there is long-term value in making the change before 9.x without blocking this on feature thaw. We also agreed that it would be better still to postpone this change to 8.1.x (or any minor), even if it's slightly more work to update core then, because we are in a late beta and should be restricting changes now.
@catch also suggested that we could open a separate issue now just for prefixing the 8.x hashes.
Comment #140
pwolanin CreditAttribution: pwolanin commented@xjm - not sure I follow. I don't think there is any new prefixing that's needed?
Comment #141
catchIf we could actually remove it that would be great, but we can't do that until the release that drops core support for upgrading from Drupal 7, which unless something changes with migrate/core cycles is going to be Drupal 10. This just means making the change in a Drupal 8 release, any Drupal 8 release, before 7.x support is dropped. Since we need the library to handle Drupal 7 hashes anyway.
And even when we drop support for upgrading from Drupal 7, there will be some users who did not log in for a long time, and still have old password hashes.
Those users would then not be able to log in at all, and would have to reset their passwords.
Since this is a problem with every Drupal 7 site, a release or two of Drupal 8 sites doesn't make a huge difference here IMO. We're still going to end up re-hashing potentially billions of passwords in the wild over the next 6-7 years either way.
Comment #142
neclimdulfor 8.0 releases I guess we can just use https://www.drupal.org/project/php_password
Comment #143
claudiu.cristea@neclimdul, nice. But I wonder what happens if the admin uninstalls that module :)
Comment #144
neclimdulNo one can log in. The module should probably take some pain's to make that exceptionally clear but that's the decision we made. Its not really any different from any other module that decided to implement PasswordInterface and provide a password hashing replacement.
Comment #145
znerol CreditAttribution: znerol commentedFiled #2511806: Fix documentation in password hashing class
Comment #148
pfrenssen8.1.x has come and gone, guess this is no longer postponed :)
Comment #149
slasher13try to re-roll #127
Comment #152
slasher13Comment #153
slasher13Comment #156
slasher13Comment #157
claudiu.cristea@slasher13, the approach from #127 is no more actual. We cannot go with that solution anymore. The issue is assigned to me, meaning I'll come with a proposal shortly.
Comment #159
claudiu.cristeaBecause we missed this on 8.0.x BETA release, now we have to keep some BC. Updated the IS accordingly.
No intediff has been provided as is not relevant.
Comment #160
claudiu.cristeaComment #161
catchWe can make this change in 8.x if we add a new interface extending the old interface and deprecate the old method. Doesn't need to happen here, but maybe open an issue for that and remove the 9.x @todo? Need to get into the habit of this for #2822727: [policy, no patch] Adopt a continuous API upgrade path for major version changes.
Any particular reason for the Legacy rename, is it just to make it clear it shouldn't be used by itself?
Comment #162
claudiu.cristea@catch, but I'm not very happy. Looks like a BC break in UserAuth. If a module has swapped that service, it will expect a PasswordInterface but will get a PasswordHashInterface.
Comment #167
claudiu.cristeaComment #169
catchPasswordHashInterface extends PasswordInterface, so it still gets a PasswordInterface?
Also just a note I don't think this needs to happen in this issue at all, just said we should do it in an 8.x issue rather than adding the 9.x @todo.
Comment #170
claudiu.cristea@catch,
Added the followup #2828794: Deprecate PasswordInterface::check() in favour of ::verify().
Regarding the naming of
LegacyPassword
. In earlier patches I renamed that asDrupal7Password
. But that make no sense anymore because that engine was used also in Drupal < 8.3.0. Also the original name has no clear meaning. I think, yes, we should signal somehow that it should not be used by contrib. But i'm open to any better naming.Comment #171
claudiu.cristeaComment #172
catchI think LegacyPassword is OK, 782Password came to mind but is worse...
Comment #173
claudiu.cristeaComment #174
claudiu.cristeaSmall IS change.
Comment #175
klausiwhy is this a kernel parameter? Can we just hard code '10' as parameter to the service instead? Ah, I think I understand: make it a kernel parameter so that modules do not have to override the service signature of the password service because it might change constructors as we do with this patch. Maybe we should have a comment here why this is a kernel parameter?
That does not make sense to me. If the password is too long then we should just cut it off at the maximum length and continue. That way we support passwords with arbitrary length.
But we are doing that in the old implementation as well, so should probably be a separate issue.
why do you set the cost to 4 here? Please add a comment. Can we set it to 1 also?
typo: "the others"
usage of password_hash() parameters: Ideally we should use PASSWORD_DEFAULT to be automatically future safe. The docs say we should have a large enough DB column for that (255 characters). Which is exactly the size of our DB column - so why can't we use PASSWORD_DEFAULT?
Otherwise looks almost ready to me! The test case updates look good as well.
Comment #176
klausihttp://php.net/manual/en/function.password-hash.php says about the $password parameter: "Caution
Using the PASSWORD_BCRYPT as the algorithm, will result in the password parameter being truncated to a maximum length of 72 characters."
So PHP will already cut this down to 72 characters, which makes the strlen() check pointless here. We should instead leave a comment that the length check is not necessary.
Comment #177
colanThis issue is discussed in On the (in)security of popular open source Content Management Systems written in PHP: Drupal uses SHA512Crypt which is Sub-Optimal:
Comment #179
tvlooy CreditAttribution: tvlooy commentedThe default right now is bcrypt but PHP 7.2 will get libsodium so Argon2 will also be available as a hash and will probably be a better default. A benefit from using the API is that people can automatically get stonger passwords when upgrading to newer PHP versions (if password_needs_rehash() is implemented).
Comment #181
landsman CreditAttribution: landsman commentedI think that PASSWORD_BCRYPT should be used.
Drupal 9 can have BC to drop old algorithm of hashing passwords, why waiting?
Comment #183
sunA slightly less ambitioned proposal: #2939888: Allow verification of imported user passwords using other Crypt schemes
Comment #191
mcdruidCame here from #3110670: Increase password hashing iterations.
phpass's author recommends:
https://www.openwall.com/phpass/
It'd be good to land this. What's blocking it?
Comment #192
catchI think this needs a re-roll incorporating some of the feedback from #1845004-175: Replace custom password hashing library with PHP password_hash() (mostly minor changes + opening follow-ups), and otherwise it was looking ready in 2016 but I haven't done a full re-review since the last one.
Comment #194
andypostnot sure this indent is valid
I'm sure it should be new class and old one is deprecated, we can't rename because of BC policy
Moreover it could fail on upgrade as old container will fail to load file
needs to decide if it allowed for 9.5 or should go for 10.1
Comment #195
tvb CreditAttribution: tvb commentedThis is a reroll.
Additionally:
Additionally:
Comment #197
tvb CreditAttribution: tvb commentedRerolled from #195.
Comment #198
pradhumanjain2311 CreditAttribution: pradhumanjain2311 at OpenSense Labs for DrupalFit commentedTry to fix CS and PHPStan errors in patch #197.
Comment #199
NivethaSubramaniyan CreditAttribution: NivethaSubramaniyan as a volunteer and at Srijan | A Material+ Company for Drupal India Association commentedFixing CCF errors
Comment #200
NivethaSubramaniyan CreditAttribution: NivethaSubramaniyan as a volunteer and at Srijan | A Material+ Company for Drupal India Association commentedComment #201
SpokjeComment #202
SpokjeLet's first try to get a green patch against 10.1.x before addressing any left feedback.
Comment #203
SpokjeComment #204
SpokjeComment #205
volegerAdded initial version of CR https://www.drupal.org/node/3322420
Comment #206
andypostThe cost increase issue could be linked #3110670: Increase password hashing iterations
8.3.0 needs replace and not sure all this mentions are required
Comment #207
volegerstr_starts_with
function can be used instead ofsubstr() === 'string'
expressionWe can use Constructor Promotion here
https://www.php.net/manual/en/language.oop5.decon.php#language.oop5.deco...
Comment #208
volegerAlso, add type declaration for properties, class constants, return types, and method arguments where possible.
Comment #209
Ratan Priya CreditAttribution: Ratan Priya at OpenSense Labs for DrupalFit commentedAddressed comment #206.
Please review.
Comment #210
SpokjeThanks @voleger and @andypost.
I'll try to address all/most of your issues during the my attempt to get this issue back on track again.
EDIT: @Ratan Priya, let's not complicate this issue even more by having 2 persons adding patches simultaneously. When I'm done, I'll un-assign this issue and I will be looking forward to your additions then. :)
Comment #212
SpokjeSwitching to MR workflow.
Comment #215
SpokjeGotta let this one go for now, have to urgently jump on something else.
Comment #216
SpokjeComment #217
SpokjeAnd a happy 10th anniversary for this issue...
Comment #218
viappidu CreditAttribution: viappidu as a volunteer commentedROFL
(even if it's not right...)
Comment #221
SpokjeMessed up whilst rebasing the old MR, created new one from HEAD of 10.1.x and cherry-picked all commits from the old to the new MR.
Comment #222
znerol CreditAttribution: znerol commentedRegarding the 72 bytes limit, this can be tested with the following snipped:
Output:
Comment #223
znerol CreditAttribution: znerol commentedThis would also resolve #2939888: Allow verification of imported user passwords using other Crypt schemes while the current patch wouldn't.
Comment #224
znerol CreditAttribution: znerol commentedAdditionally I suggest to not bother making the services
lazy
. The change has been suggested in #47 and #56 but has been subsequently questioned a couple of times including in #113.The effect of marking those services
lazy
is that proxy classes are instantiated in place of the actual service classes. But since the service classes do not have any additional dependencies themselves, the net gain of instantiating a proxy vs the actual service is zero. In this case, having to instantiate a proxy in order to get to the service class adds complexity and overhead for no benefit at all.Comment #225
znerol CreditAttribution: znerol commentedComment #226
neclimdulThe decorator idea is what php_password module has been doing for 7 years. Its a good model. https://git.drupalcode.org/project/php_password/-/blob/8.x-1.x/src/Passw...
Comment #228
znerol CreditAttribution: znerol commentedTook a stab on this. I've opened up a new MR 3245 since the approach departs a little bit from the current ideas.
core/lib/Drupal/Core/Password/PhpPassword.php
has no knowledge of the legacy stuff.phpass
module.phpass
module is enabled in apost_upgrade
hook. That way new sites will start out with the legacy stuff disabled. Existing sites may disablephpass
when all active users have logged in again and most passwords were migrated.core/lib/Drupal/Core/Password/PhpassHashedPassword.php
will be removed$corePassword
parameter ofcore/modules/phpass/src/Password/PhpassHashedPassword.php
will become mandatory. As a result,hash()
andneedsRehash()
will start calling through to$corePassword
service unconditionally.phpass
module can be moved to contrib.Comment #229
znerol CreditAttribution: znerol commentedComment #230
znerol CreditAttribution: znerol commentedTests pass, issue summary is up-to-date. Needs framework manager review now in order to decide on the approach.
Comment #231
gappleTook me a moment to come around to MR3245 adding an additional module, but I like that it gives new sites a minimal install and legacy sites can opt in to disabling old password hashes when they deem best (e.g. after monitoring that all hashes have been updated, or perhaps by sending a forced password reset to all users).
I don't think
PhpPassword
should be explicitly set toPASSWORD_BCRYPT
; as it is in MR3245, it would be necessary to implement a new class to use a different algorithm (and provide it with the corresponding options), rather than just altering the service arguments.I think @neclimdul's recent work on the php_password 2.x branch is better, and keeping the arguments in the service definition empty so that it uses
PASSWORD_DEFAULT
and the default options (unless overridden by a site implementer) would allow them to follow changes in new versions of PHP without Drupal core needing to make any additional changes.(see https://git.drupalcode.org/project/php_password/-/blob/2.0.x/src/Passwor...)
Comment #232
znerol CreditAttribution: znerol commentedFully agree with that.
Comment #233
catchHow does this affect user migrations from Drupal 6 and 7? I think we'd need those sites to enable the module, but how can we inform them?
A possibility would be to make it a dependency of migrate_drupal so it gets installed on those sites, then once they've uninstalled migrate_drupal, it'd still be enabled but uninstallable. Not perfect, but can't think of another way at the moment.
Comment #234
neclimdulGlad you like the approach! :-D
I was wondering about the same thing catch mentioned. But its a bit worse then a migration issue because, migrate_drupal only layers the hash in a more secure one because the only _real_ way to rehash is with the original password. While it seems ideal to have the legacy logic removed from new sites, it seems like a very difficult thing to fully remove legacy hash logic from most existing sites.
Comment #235
znerol CreditAttribution: znerol commentedTrue. This should cover most cases.
Exactly. I am quite sure that this depends a lot on the user population of a site. Some existing sites will wish to keep the module enabled for years while on other sites password migration will be over in a couple of weeks. For that reason it is preferable to keep the logic around in a module, so it is easier for a site to opt-out on an appropriate point in time.
Comment #236
znerol CreditAttribution: znerol commentedNote to reviewers: In order to examine changes in moved files (especially
core/{tests/Drupal/Tests/Core/Password => modules/phpass/tests/src/Tests}/PasswordHashingTest.php
), usegit diff
with -B -M flags combined.Comment #237
znerol CreditAttribution: znerol commentedComment #238
znerol CreditAttribution: znerol commentedComment #239
znerol CreditAttribution: znerol commentedComment #240
znerol CreditAttribution: znerol commentedAdressed #233 and #234, issue summary is up-to-date.
Comment #242
SpokjeClosed original MR since we're going in a different direction.
Comment #243
znerol CreditAttribution: znerol commentedManually tested a Drupal 7 to Drupal 10.1 (+patch) upgrade. The sequence of steps is more or less the following:
Prepare D7:
composer create-project drupal-composer/drupal-project:7.x-dev /tmp/drupal-7 --no-interaction
/tmp/drupal-7
./vendor/bin/drush core:quick:drupal --yes --no-interaction
Prepare D10.1:
git clone -b 1845004-phpass-module https://git.drupalcode.org/issue/drupal-1845004.git /tmp/drupal-10
cd /tmp/drupal-10
composer install
php ./core/scripts/drupal quick-start standard
/tmp/drupal-7/web/sites/default/files/web.sqlite
), enter the path to the document root for public files (e.g.,/tmp/drupal-7/web/sites/default/files
, click on Review upgrade.Test login (with
phpass
enabled):Test login (with
phpass
disabled):phpass
was still enabled)Comment #244
znerol CreditAttribution: znerol commentedFinally managed to manually test a Drupal 6 to Drupal 10.1 (+patch) upgrade. The sequence of steps is analogous to #243. The result is identical as well.
Note: quickly spinning up Drupal 6 isn't that easy nowadays. I went with PHP 5.6 installed from deb.sury.org and a combination of
php -S localhost:8888
anddrush.phar rs
(drush 8). @pcambra points out thatddev
should be able to do the job as well.That said, I think that the PR is ready now. Next step is the change record.
Comment #245
znerol CreditAttribution: znerol commentedMade a CR draft.
Comment #247
smustgrave CreditAttribution: smustgrave at Mobomo commentedClosed #3110670: Increase password hashing iterations as a duplicate moving over credit
Comment #248
longwaveAdded some review comments.
Comment #249
znerol CreditAttribution: znerol commentedThanks @longwave, addressed.
Comment #250
longwaveThis is looking pretty good to me. I will raise this issue with the security team regarding a review.
Added an extra comment where one of the deprecation messages has gone a bit wrong.
I think we also need an upgrade path test, to prove that a user created in Drupal 10.0 can still log in in Drupal 10.1 after the phpass module has been enabled.
There also doesn't seem to be a test case that covers the deprecation of \Drupal\Core\Password\PhpassHashedPassword.
Comment #251
znerol CreditAttribution: znerol commentedThanks @longwave.
Right. Fixed.
This is covered by
PasswordVerifyTest::testPasswordCheckSupported()
. Do you have something different in mind?Comment #252
longwaveUsually if we have a post_update hook we write an update test that extends UpdatePathTestBase. This ensures that a populated database can successfully upgrade and confirms that the post_update hook works as expected. In this case I think a good test would be to directly set a Phpass password hash before performing the update, and attempt to log in with it following the update.
Also please point me to this if I'm incorrect, but I think we don't have explicit test coverage to ensure passwords hashed in Drupal 10.0 or earlier are upgraded to the new hash format on login? Perhaps this should be part of the same test.
Regarding the security team review, @mlhess and @catch gave me some points to consider. Here are a list of points I came up with; some require further discussion:
password_hash()
? I would assume not, given that this is built into the language itself - but then as far as I have found via Google, bcrypt/Blowfish is not recommended by NIST or other government agencies. NIST appears to recommend PBKDF2 instead. But the docs for hash_pbkdf2() say "The PBKDF2 method can be used for hashing passwords for storage. However, it should be noted that password_hash() or crypt() with CRYPT_BLOWFISH are better suited for password storage." so I am not sure what the correct answer is here.Comment #253
longwaveRegarding #252.3 I just thought of something that might be useful: we could report statistics on how many passwords are hashed with new vs legacy, and present this to the site owner. I think this could be done with a relatively simple database query, given that we only need to check the first three characters of the hash. This would then help indicate whether it is safe to disable the legacy module. Unsure if this is a job for core or contrib, and it shouldn't hold up this issue, but I think it's worth discussing?
Comment #254
longwaveRegarding #252.4 I found a discussion at https://news.ycombinator.com/item?id=7286057 which suggests bcrypt is probably better, but PBKDF2 is referred to in standards, so the question about considerations for government users still applies.
Comment #255
catchI thought about what the criteria for this might be, but hadn't wrote it out yet. At the beginning of writing this comment I didn't have any idea, but writing it out pushed me towards one a bit:
For each site, there is a window between them first installing a version of Drupal that includes the new password hashing algorithm, and when they uninstall the phpass module (or have to install the contrib version explicitly if we've dropped support but they haven't). If that window is two years or five years, then their users just have to log in once during that window to get a new password hash. If they haven't, they'll need to do a password reset. Long login cookie lifetimes could mean that their users don't have to log in for months after they initially update, so it really needs to be years not months to have a chance of catching even most active users on a site.
IMO this is not an easy concept for site owners to understand, much less site visitors - the fact that the hash can only be updated when the user actually logs in and the way this is spread over time, so it'd be quite possible for sites to uninstall the module, lock out their visitors, and not realise what they've done.
For Drupal core / us, there is the point at which we can assume that all Drupal sites should be on 10.1 or higher, this will only be when 10.0, 9.5, and Drupal 7 are out of support. It's very likely that Drupal 7 will be the last of those to go EOL.
Since the old password hashing algorithm is going to be an integral part of the Drupal 7 migration path, then we can't drop it until we move the Drupal 6 and 7 migrations to contrib, and consensus seems to be that this should only happen when Drupal 7 is no longer supported.
However, if we have a major release that supports migrating from Drupal 7, then to allow sites and their users to have a password migration window long enough after they've migrated, I think we should probably consider supporting the old password hashes for one additional major release cycle after the Drupal 7 migration path is moved to contrib.
This would mean:
Drupal 7 site updates to Drupal x
Drupal x+1 is released after Drupal 7 goes EOL.
Drupal x site updates to Drupal x+1 (Phpass still supported in core)
Drupal x+1 site updates to Drupal x+2 (uninstall Phpass or install the contrib version)
That way if you go from Drupal 7 to a mid-release major version (i.e. just before the next major is released or maybe just after that), you still have a solid couple of years or more before we lock out all your users.
This is definitely worth a core issue. If we decided not to put it in core, it might be a good thing for upgrade status even?
Related to that, I wonder if we should have a fallback that detects the old password hash fallback when Phpass isn't installed, and when it finds an incompatible hash, tell the user 'You haven't logged in for a while, please reset your password'. Should also be split to its own issue though.
Comment #256
znerol CreditAttribution: znerol commentedLooks like NIST is currently reviewing SP 800-132. The original document NIST Special Publication 800-132 Recommendation for Password-Based Key Derivation is from 2010. In the meantime Argon2 won the password hashing competition in 2015 and became the recommended hashing algorithm by OWASP.
If I'm not totally mistaken, the current Drupal password hashing algorithm isn't NIST approved neither. My interpretation of #723802: convert to sha-256 and hmac from md5 and sha1 is that government agencies require the absence of SHA-1 and MD5 (which is reasonable) and not explicitly the presence of PBKDF2. Would be great if somebody with relations to government agencies could verify.
Comment #257
znerol CreditAttribution: znerol commentedIt looks like
UpdatePathTestBaseFilledTest::testUpdatedSite()
is doing exactly that (line 97).Comment #258
znerol CreditAttribution: znerol commentedI'm going to add a test which reproduces #243 explicitly (including an attempt to try logging in with a user whose password hash hasn't been updated after disabling the
phpass
module).Comment #259
longwaveRe #257 just for an extra confirmation should we add phpass to the installed module list in that test (lines 346-411)?
This will also need some consideration in Drupal 11 as to how we handle legacy passwords when we create the Drupal 10 dump database for upgrade tests; we should ensure we still have a legacy hash and ensure that it is still correctly managed. This makes me think that perhaps we should write an explicit test for this now?
edit: crosspost with the above
Comment #260
znerol CreditAttribution: znerol commentedComment #261
znerol CreditAttribution: znerol commentedRe #253
I made a start here: Password Stats. Its a very simple
drush
command for the moment.Comment #262
znerol CreditAttribution: znerol commentedComment #263
znerol CreditAttribution: znerol commentedComment #264
longwaveNot sure what happened with the GitLab comments, I don't seem to be able to dismiss my previous review.
I added three extra comments (the last three above) that need a tiny bit more work, but after that I'm happy to mark this RTBC to get more core committer eyes on it.
Comment #265
znerol CreditAttribution: znerol commentedComment #266
znerol CreditAttribution: znerol commentedComment #267
longwaveThanks - this has been back and forth a lot now, I have no further comments so marking RTBC for other committers to take a look.
Comment #268
neclimdulPretty sure the code locations are inverted and we shouldn't be referencing an module's classes in Core. That would lead to possible fatal errors if referenced and the module wasn't enabled.
Comment #269
longwaveDo we have precedent for moving a core service into a module? We want the deprecation error on the core side only.
Maybe we need to temporarily move the code into e.g.
\Drupal\Core\Password\LegacyPhpassHashedPassword
then\Drupal\Core\Password\PhpassHashedPassword
and\Drupal\phpass\Password\PhpassHashedPassword
can extend both, with only the former issuing a deprecation, then in Drupal 11 we delete the core classes and move the code tophpass
?Comment #270
znerol CreditAttribution: znerol commentedYes. The database modules #3129043: Move core database drivers to modules of their own. I used that as the model.
Comment #271
znerol CreditAttribution: znerol commentedSetting this to needs review. There is no obvious next step to work on here.
Comment #272
DamienMcKennaDue to the critical bug (https://bugs.php.net/bug.php?id=81744) that was fixed in PHP 8.1.16 (https://www.php.net/ChangeLog-8.php#8.1.16), should this require 8.1.16 or newer to avoid the bug?
Comment #273
znerol CreditAttribution: znerol commentedWeird, the GHSA is rated with severity low. The CVE is still missing most info.
Comment #274
andypostCVE-2023-0567 is 2 commits
- https://github.com/php/php-src/commit/c840f71524067aa474c00c3eacfb83bd86...
- https://github.com/php/php-src/commit/a92acbad873a05470af1a47cb785a18ead...
Comment #275
znerol CreditAttribution: znerol commentedThere is also another GHSA (
DoS vulnerability when parsing multipart request body) which is rated high affecting the same PHP versions. That one is broadly exploitable over the network without authentication.
Comment #276
neclimdulre #272, I suspect no because of the way OS's versions their PHP versions. Using say Ubuntu's PHP 8.1.13 which will have this patched should be allowed.
My understanding of how we handle language version requirements is to tie them to features that would be required for the site to work. It is up to the site maintainers to keep their OS and language patched against security releases. Don't know of any other security release that's bumped our version requirements so this would be the same.
Re #270 I believe I complained about that too and it _does_ cause fatal errors that are really hard to recover from if you do things incorrectly so I stand behind that being the wrong approach.
Comment #277
znerol CreditAttribution: znerol commentedWhat would be a reproduction for that case? I'd be interested in testing this out.
Comment #281
smustgrave CreditAttribution: smustgrave at Mobomo commentedClosed #1850638: Add default parameter to PhpassHashedPassword constructor in favor of this.
Comment #282
needs-review-queue-bot CreditAttribution: needs-review-queue-bot as a volunteer commentedThe Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".
Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.
Comment #283
znerol CreditAttribution: znerol commentedRebased after #3296293: Apply SensitiveParameter attribute. Also added the
#[\SensitiveParameter]
toPasswordInterface::needsRehash()
. That method obviously was missed in the original change.Comment #284
znerol CreditAttribution: znerol commentedRe #276, I found one upgrade path which is breaking a standard install. The idea here is to do as much as possible through the web ui and avoid the command line altogether.
1845004-phpass-module
branch.post_upgrade
hook hasn't had the chance to run.This update procedure has some other usability problems as well (even without the
phpass
patch). E.g., the themes loose all their CSS and it is quite hard to navigate the way through to the status site and the upgrade script in the first place.In order to allow an admin to login after the code base was deployed and before upgrade scripts were run, it is necessary to ensure that the legacy password checking method remains in place for updates sites.
Comment #285
znerol CreditAttribution: znerol commentedComment #287
znerol CreditAttribution: znerol commentedComment #288
needs-review-queue-bot CreditAttribution: needs-review-queue-bot as a volunteer commentedThe Needs Review Queue Bot tested this issue. It fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".
Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.
Comment #289
znerol CreditAttribution: znerol commentedComment #290
znerol CreditAttribution: znerol commentedUpdated the issue summary in order to match the state of the MR. Addressed #268 by reversing the dependencies (i.e.,
Drupal\phpass\Password\PhpassHashedPassword
now inherits fromDrupal\Core\Password\PhpassHashedPasswordBase
).Also introduced a
password.core_backward_compat
service which wraps thepassword
service on sites in a state where the post update hook had no chance to run yet. This addresses #284. However, the introduction of this service makes it impossible to deprecateDrupal\Core\Password\PhpassHashedPassword
in this release cycle. Regrettably it is necessary to postpone the deprecation to the first release which drops support for direct upgrades from Drupal 10.0.x.This also means that sites newly installed with Drupal 10.1.x will still have the old legacy password hashing class in the code path. Unless they choose to alter away the
password.core_backward_compat
service, e.g. using a custom or contrib module. Fortunately, new sites will nevertheless start out with passwords in PHPpassword_hash()
format for all accounts.Now that the
password.core_backward_compat
decorator is in place without any chance for quick deprecation within the current major release cycle, thephpass
module is somewhat redundant. Thus, there is some room for minimizing change / complexity here by just dropping thephpass
module from this PR. I'd appreciate your thought on this idea.Comment #291
znerol CreditAttribution: znerol commentedComment #292
znerol CreditAttribution: znerol commentedComment #293
catch@znerol
This is unsupported, if you want to run updates via the browser without being logged in, you have to set
$settings['update_free_access'] = TRUE
, so for me it's not a problem with the patch here - any number of things can render browsing a site unusable when it has pending updates to run.Comment #294
znerol CreditAttribution: znerol commentedComment #295
znerol CreditAttribution: znerol commentedRight, removed
password.core_backward_compat
again, that makes deprecation easier and more predictable.Still leaving the
@internal
abstract base classDrupal\Core\Password\PhpassHashedPasswordBase
in tree. So, #268 remains resolved.Comment #296
longwaveAttaching an interdiff of the changes between #265 and #295, which fixes the concerns in #268, to make it simpler for other reviewers. This looks clean to me and I have no further comments.
Also removing the "needs security review" tag; as a member of the security team I believe all concerns that have been raised so far have been addressed adequately, and I have no new concerns to add. Adding "needs followup" for the followup requested by @catch in #255 - we should inform users of when it is safe to uninstall the legacy module, but that can be done after this issue lands.
Marking RTBC as I believe there is no further work to do here, but this is still tagged for framework manager review.
Comment #297
needs-review-queue-bot CreditAttribution: needs-review-queue-bot as a volunteer commentedThe Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".
Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.
Comment #298
znerol CreditAttribution: znerol commentedRebased. Resolved one conflict in
core/modules/system/system.post_update.php
.Comment #299
catchI think this is a good state, but since I've been reviewing it on and off for years, and I'm extremely pro the issue landing because I was extremely opposed to the current password implementation (the phpass fork) that we're deprecating here, I've asked if another framework manager can also review it who's possibly less over-enthusiastic.
Comment #300
VladimirAus+1. Patch applies and passes all the tests.
Comment #301
andypostPasswords still can be null, so added related #3305807: Password is null if user has never logged in which causes PHP 8 warning
Comment #302
alexpottLeft a review on the MR - I think we can tighten up the abstract base class to make it narrower and therefore easier to remove once it is no longer needed. We're also doing work that can now be done in a union typehint - something that was not possible when this issue started.
Comment #303
znerol CreditAttribution: znerol commentedBack to NR.
Comment #304
longwaveAll review points were addressed, back to RTBC.
Comment #305
alexpottI opened #3352089: Work out what needs to happen due to core using PHP's password_hash() as a follow-up for the contrib module.
Added credit for comments that had impact on the solution, are part of release management and pertained to security issues.
Comment #306
alexpottCommitted 375376d and pushed to 10.1.x. Thanks!
Added a release note based on the CR.
Comment #308
longwaveComment #309
andypost10 years! congrats!
is the CR is not published for reason?
PS: follow-up filed in #305 - #3352089: Work out what needs to happen due to core using PHP's password_hash()
Comment #310
DamienMcKennaI published the change notice.
Comment #311
bnjmnmAdjusting issue credit as credit is not provided for button-click rebases without additional contribution.
Comment #313
andypostUnppostponed follow-up #2828794: Deprecate PasswordInterface::check() in favour of ::verify()