Problem/Motivation

We have many url-related helper functions in core, that kill unit-testing..e.g the new RedrectResponseSubscriber went in without unit tests because it calls those procedural helpers.

Proposed resolution

Group all url related functions i a Url Component under Drupal\Component\utility\Url.
There is an existing UrlValidator component, which would now make sense to merge inside the Url one.

drupal_get_query_parameters() => Url::filterQueryParameters()
drupal_get_query_array() => Url::getArray()
drupal_parse_url() => Url::parse()
drupal_encode_path() => Url::encodePath()
_external_url_is_local() => Url::externalIsLocal()
 url_is_external() => Url::isExternal()

Remaining tasks

We got a green patch, commit it

User interface changes

None

API changes

The old procedural functions will stay as a wrapper, but will be marked as deprecated. Url component should be called directly

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

dawehner’s picture

Status: Active » Needs review
FileSize
6.52 KB

Let's start.

I think we should though try to rely more on just the php functions and helper stuff from symfony.

Crell’s picture

+++ b/core/lib/Drupal/Component/Utility/HttpQuery.php
@@ -0,0 +1,68 @@
+   * This differs from http_build_query() as we need to rawurlencode() (instead of
+   * urlencode()) all query parameters.

Why? No one has ever explained to me why we have to do something different here.

+++ b/core/lib/Drupal/Component/Utility/HttpQuery.php
@@ -0,0 +1,68 @@
+  public static function getArray($query) {
+    $result = array();
+    parse_str($query, $result);
+    return $result;

This is too small to bother making a utility function. Really.

dawehner’s picture

Well, I think we can replace drupal_get_query_array() by this function completely without an issue but some backward-compatible-layer would be great to make some progress here.

Why? No one has ever explained to me why we have to do something different here.

I don't know either, but maybe it is written somewhere on #578520: Make $query in url() only accept an array or is just not the case anymore in Drupal 8.

Damien Tournoud’s picture

+++ b/core/lib/Drupal/Component/Utility/HttpQuery.php
@@ -0,0 +1,68 @@
+   * This differs from http_build_query() as we need to rawurlencode() (instead of
+   * urlencode()) all query parameters.

Why? No one has ever explained to me why we have to do something different here.

rawurlencode() is RFC3986 compliant, and as a consequence RFC3987 compliant. The latter defines the required format of "URLs" in HTML5.

urlencode() is almost the same as rawurlencode(), except that it encodes spaces as "+" instead of "%20". This makes its result non compliant to RFC3986 and as a consequence non compliant to RFC3987 and as a consequence not valid as a "URL" in HTML5.

That said, http_build_query() has an optional $enc_type parameter since PHP 5.4, that allows it to generate URLs compliant with HTML5.

Crell’s picture

Sigh. If we need it essentially as a BC wrapper, then we should document that (per #4) and if on PHP 5.4 just use http_build_query() directly. That way when we eventually move to just 5.4 we can drop this function entirely.

dawehner’s picture

Component: simpletest.module » base system
Issue tags: +PHPUnit Blocker
FileSize
1.6 KB
7.23 KB

Thanks for the explanation!

Status: Needs review » Needs work
Issue tags: -PHPUnit Blocker

The last submitted patch, drupal-2010024-6.patch, failed testing.

dawehner’s picture

Status: Needs work » Needs review

#6: drupal-2010024-6.patch queued for re-testing.

Status: Needs review » Needs work
Issue tags: +PHPUnit Blocker

The last submitted patch, drupal-2010024-6.patch, failed testing.

dawehner’s picture

Now that the url generator went in ... should this live on the url generator?

katbailey’s picture

The generator already has an httpBuildQuery() method:
http://drupalcode.org/project/drupal.git/blob/2432c02138861423b7db365265...

ParisLiakos’s picture

Imo drupal_parse_url and httpBuildQuery should be together in one component

dawehner’s picture

What about drupal_get_query_array?

ParisLiakos’s picture

that too..anything that has to do with query/url besides generating should be in the same component...so that people can find them immediately..maybe merge UrlValidator too

damiankloip’s picture

Status: Needs work » Needs review
FileSize
17.65 KB

Ok, how about we move the whole lot of methods into one Url class? This patch is still rough around the edges, be warned.

Status: Needs review » Needs work

The last submitted patch, 2010024-15.patch, failed testing.

dawehner’s picture

Status: Needs work » Needs review
+++ b/core/lib/Drupal/Component/Utility/Url.phpundefined
@@ -20,6 +20,122 @@ class UrlValidator {
+   * @param $query
...
+   * @param $exclude
...
+   * @param $parent
...
+   * @return

Need some docs.

ParisLiakos’s picture

Title: Convert drupal_get_query_array and drupal_http_build_query to a component. » move url related functions to a new Url component

retitling

Crell’s picture

+++ b/core/lib/Drupal/Component/Utility/Url.php
@@ -20,6 +20,122 @@ class UrlValidator {
+    // Set defaults, if none given.
+    if (!isset($query)) {
+      $query = $_GET;
+    }

No way. $query should be required. Code in Component should not be touching global anything.

damiankloip’s picture

I said be warned, and rough around the edges. I thought it was pretty obvious I just quickly c & p'ed a few things.

damiankloip’s picture

FileSize
12.03 KB
33.1 KB

ok, I've moved some more things. I'm not sure about moving _external_url_is_local, as it needs the base path. Thoughts?

I also spoke to Paris about merging in #2020811: Move _external_url_is_local and url_is_external to UrlValidator and putting everything in this Url class.

Status: Needs review » Needs work

The last submitted patch, 2010024-21.patch, failed testing.

damiankloip’s picture

Status: Needs work » Needs review
FileSize
1.09 KB
33.11 KB

Seems the version_compare logic was backwards.

Status: Needs review » Needs work

The last submitted patch, 2010024-23.patch, failed testing.

damiankloip’s picture

Status: Needs work » Needs review
FileSize
595 bytes
33.11 KB

Sorry, elementary error there.

dawehner’s picture

+++ b/core/lib/Drupal/Component/Utility/Url.phpundefined
@@ -46,8 +46,8 @@ class Url {
+      //return http_build_query($query, NULL, NULL, PHP_QUERY_RFC3986);

Commented out?

+++ b/core/includes/common.incundefined
@@ -441,34 +441,10 @@ function drupal_get_feeds($delimiter = "\n") {
 function drupal_get_query_parameters(array $query = NULL, array $exclude = array(), $parent = '') {

@@ -481,14 +457,7 @@ function drupal_get_query_parameters(array $query = NULL, array $exclude = array
 function drupal_get_query_array($query) {

@@ -574,44 +542,7 @@ function drupal_get_destination() {
 function drupal_parse_url($url) {

@@ -626,7 +557,7 @@ function drupal_parse_url($url) {
+  Url::encodePath($path);

@@ -709,7 +640,7 @@ function valid_email_address($mail) {
+  return Url::isValid($url, $absolute);

@@ -1314,11 +1245,7 @@ function url($path = NULL, array $options = array()) {
+  return Url::isExternal($path);

There should be a @deprecated tag on that as well.

+++ b/core/includes/common.incundefined
@@ -441,34 +441,10 @@ function drupal_get_feeds($delimiter = "\n") {
     $query = $_GET;

While here we could do Drupal::request()->query->all() but this is optional, i think.

+++ b/core/lib/Drupal/Component/Utility/Url.phpundefined
@@ -0,0 +1,356 @@
+   *   $options = drupal_parse_url($_GET['destination']);

Would be cool to adapt this comment.

+++ b/core/lib/Drupal/Component/Utility/Url.phpundefined
@@ -0,0 +1,356 @@
+   * Note that url() takes care of calling this function, so a path passed to
...
+   * check_url() or filter_xss(), but those functions return an HTML-encoded
...
+   * to be a plain-text string for passing to t(), l(),
...
+   * check_plain() separately.

Maybe another one which makes sense to change.

+++ b/core/lib/Drupal/Component/Utility/Url.phpundefined
@@ -0,0 +1,356 @@
+   *   THe encoded path.

"THe", must be important.

+++ b/core/tests/Drupal/Tests/Component/Utility/UrlTest.phpundefined
@@ -0,0 +1,51 @@
+ * Tests the http query methods.
+ */
+class UrlTest extends UnitTestCase {
...
+   * Tests HttpQuery::buildQuery().

An @see for the url would be nice. I guess we have to kill the references to the http query class.

+++ b/core/lib/Drupal/Component/Utility/Url.phpundefined
@@ -46,8 +46,8 @@ class Url {
+      //return http_build_query($query, NULL, NULL, PHP_QUERY_RFC3986);

Commented out?

Status: Needs review » Needs work

The last submitted patch, 2010024-25.patch, failed testing.

ParisLiakos’s picture

ok, I've moved some more things. I'm not sure about moving _external_url_is_local, as it needs the base path. Thoughts

you can see my patch over @ #2020811: Move _external_url_is_local and url_is_external to UrlValidator
Just add one more parameter. $base_url

damiankloip’s picture

Status: Needs work » Needs review
FileSize
13.8 KB
37.52 KB

Thanks for the review! Yep, accidentally left that line commented out :) Added the @see, @deprecated tags, and fixed up the other part of the doc block (I think). I also moved everything from UrlValidatorTest to UrlTest.

I have left check_url() for now, as it uses check plains the url string. Do you think we should move that into this class too? and just rely on string? or keep it out of there?

damiankloip’s picture

Sorry Paris, I x posted with your comment, here is a new patch with that rolled in.

ParisLiakos’s picture

Issue tags: +PHPUnit

Great, looks awesome
just a thing which will make bot fail probably

+++ b/core/includes/common.incundefined
@@ -624,9 +563,11 @@ function drupal_parse_url($url) {
-  return str_replace('%2F', '/', rawurlencode($path));
+  Url::encodePath($path);

you dont return the value anymore

Thank you!

Status: Needs review » Needs work

The last submitted patch, 2010024-30.patch, failed testing.

damiankloip’s picture

FileSize
390 bytes
39.3 KB

That'll do it!

damiankloip’s picture

Status: Needs work » Needs review
dawehner’s picture

Really nice!!

+++ b/core/includes/common.incundefined
@@ -531,7 +505,7 @@ function drupal_get_destination() {
+    $query = Url::buildQuery(Url::filterQueryParameters($_GET));

It would be cool to use the request object here as well.

+++ b/core/tests/Drupal/Tests/Component/Utility/UrlTest.phpundefined
@@ -2,29 +2,62 @@
+   * Provides test data for testDrupalHttpBuildQuery().

Sorry, but this referes to the old method.

damiankloip’s picture

FileSize
2.14 KB
40.11 KB

Thank you, I changed drupal_get_destination to use the Request object instead. I also changed all the data provider docblocks, so they are consistent.

dawehner’s picture

Status: Needs review » Reviewed & tested by the community

Perfect!

ParisLiakos’s picture

FileSize
420 bytes
40.15 KB

one $_GET escaped:P die!
heh
damiankloip++

alexpott’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs issue summary update

@todo is not an accpetable issue summary :D

ParisLiakos’s picture

Title: move url related functions to a new Url component » Move url related functions to a new Url component
Status: Needs work » Needs review
Issue tags: -Needs issue summary update

Updated issue summary...not sure what the status should be...letss put it in needs review

ParisLiakos’s picture

Issue summary: View changes

Proper issue summary

Crell’s picture

Status: Needs review » Reviewed & tested by the community

Back to RTBC. I couldn't find anything in the new class that violated Component rules, so...

dawehner’s picture

Thank you for writing an issue summary!

alexpott’s picture

Status: Reviewed & tested by the community » Needs work
+++ b/core/lib/Drupal/Component/Utility/Url.phpundefined
@@ -0,0 +1,373 @@
+  /**
+   * Splits a URL-encoded query string into an array.
+   *
+   * @param string $query
+   *   The query string to split.
+   *
+   * @return array
+   *   An array of URL decoded couples $param_name => $value.
+   */
+  public static function getArray($query) {
+    $result = array();
+    parse_str($query, $result);
+    return $result;
+  }
+++ b/core/includes/common.incundefined
@@ -479,16 +457,11 @@ function drupal_get_query_parameters(array $query = NULL, array $exclude = array
 function drupal_get_query_array($query) {
-  $result = array();
-  if (!empty($query)) {
-    foreach (explode('&', $query) as $param) {
-      $param = explode('=', $param);
-      $result[$param[0]] = isset($param[1]) ? rawurldecode($param[1]) : '';
-    }
-  }
-  return $result;
+  return Url::getArray($query);
 }

As Url::getArray just calls the native PHP function we just use this instead so lets deprecate this for parse_str and not Just Another Wrapper (tm)

dawehner’s picture

Status: Needs work » Needs review
FileSize
3.57 KB
43.71 KB

Let's do that and get rid of the problematic issue at the same time.

There are tests failures with php 5.4, so it seems to be that http_build_query() can't be used

dawehner’s picture

FileSize
1.94 KB
43.09 KB

Let's remove the actual code as well.

ParisLiakos’s picture

Status: Needs review » Reviewed & tested by the community

yeah i can reproduce the failures in PHP 5.4, so yes lets just stick with the old code for now.
removing the wrapper is awesome though!
thanks:)

alexpott’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs reroll

Needs a reroll

curl https://drupal.org/files/vdc-2010024-45.patch | git a
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 44121  100 44121    0     0  33760      0  0:00:01  0:00:01 --:--:-- 38976
error: patch failed: core/includes/common.inc:3
error: core/includes/common.inc: patch does not apply
damiankloip’s picture

Status: Needs work » Reviewed & tested by the community
FileSize
43.09 KB

Pure git reroll.

alexpott’s picture

Priority: Normal » Critical
Status: Reviewed & tested by the community » Active

Committed 91c32e3 and pushed to 8.x. Thanks!

We need a change notice to announce the removal of drupal_get_query_array and to tell people to use the built in parse_str()

ParisLiakos’s picture

Title: Move url related functions to a new Url component » Change notice: Move url related functions to a new Url component
Issue tags: -Needs reroll +Needs change record

yay!

dlu’s picture

Component: base system » routing system

Moved to routing system per #2050763-16: Refine "base system" component (notes on refactoring of "base system" category here: https://docs.google.com/a/acquia.com/spreadsheet/ccc?key=0AusehVccVSq2dF...).

dawehner’s picture

Status: Active » Needs review
Crell’s picture

Title: Change notice: Move url related functions to a new Url component » Move url related functions to a new Url component
Priority: Critical » Normal
Status: Needs review » Fixed
Issue tags: -Needs change record

I shifted the list of functions to the body, not the title, so that it's actually readable. :-) You should still be able to search by the function name, though. We're good.

Status: Fixed » Closed (fixed)
Issue tags: -PHPUnit, -PHPUnit Blocker

Automatically closed -- issue fixed for 2 weeks with no activity.

Anonymous’s picture

Issue summary: View changes

typo

David_Rothstein’s picture

Title: Move url related functions to a new Url component » Change notice: Move url related functions to a new Url component
Priority: Normal » Major
Issue summary: View changes
Status: Closed (fixed) » Active
Issue tags: +Needs change record

We need a change notice to announce the removal of drupal_get_query_array and to tell people to use the built in parse_str()

This is still missing from the change notice (https://drupal.org/node/2079005).

(Coming here from #2143461: drupal_get_query_array encodes not enough if query parameter contains a equal sign (=); was trying to decide if that patch needed to go into Drupal 8 too and had to read this issue rather than the change notice to figure out that it didn't...)

star-szr’s picture

Issue tags: +DrupalWorkParty

Tagging for the task of updating the change record as described in #55.

SebCorbin’s picture

Priority: Major » Normal
Status: Active » Fixed

Change record updated.

SebCorbin’s picture

Removing tags

Crell’s picture

Title: Change notice: Move url related functions to a new Url component » Move url related functions to a new Url component
star-szr’s picture

Issue tags: +DrupalWorkParty

Thanks @SebCorbin! I think we can keep this tag for posterity.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.