To make the patch in #470258: Groupwise maximum ('representative') relationships work properly, I had to slightly alter the behavior of the add_orderby method of views_plugin_query_default.
Currently, when the views query is built, add_orderby adds an aliased field into the SELECT clause (by adding a field to the View object) and then ORDERing by the aliased field: SELECT sort_field as sort_field_alias ... ORDER BY sort_field_alias).
As far as I know, the ORDER BY clause can simply use the table alias instead: table_alias.field.
Adding a field to the View (and consequently the select clause) should at best be an optional side-effect, and it seems to e like it just shouldn't happen there at all. My concern is that other code may depend on the field being added when add_orderby is called.
I generated a few views to test it out, a couple with complex relationships -- nothing broke, and the generated queries matched my hand-written queries pretty closely. But I definitely haven't tested it thoroughly.
| Comment | File | Size | Author |
|---|---|---|---|
| #13 | 844910-views.dont-add-field-in-orderby.patch | 978 bytes | davideads |
| #8 | 844910-views.dont-add-field-in-orderby.patch | 982 bytes | davideads |
| #1 | views.dont-add-field-in-orderby.patch | 636 bytes | davideads |
Comments
Comment #1
davideads commentedWhoops, preview ate my attachment, apparently.
Comment #2
dawehnerWe cannot remove the add_field, this would be a major api change. I suggest you to look at this export
Before sql:
After sql:
Comment #3
merlinofchaos commentedANSI SQL requires that any field in the ORDER BY clause must also be in the SELECT clause. The add_field() is therefore not optional.
Comment #4
joachim commented> ANSI SQL requires that any field in the ORDER BY clause must also be in the SELECT clause
Wha...?
MySQL doesn't need this, and PostGRESQL (or however the capitals go) doesn't need it -- and it's total crack. It means that you can't order a subquery! (And thus #470258: Groupwise maximum ('representative') relationships becomes impossible.)
Comment #5
merlinofchaos commentedThis was added specifically due to pgsql which does, in fact, appear to need this. Or at least did when this went in.
Comment #6
joachim commentedSee http://www.postgresql.org/docs/8.2/static/sql-select.html#SQL-ORDERBY
Postgresl is st00pid but it's not THAT st00pid ;)
Comment #7
davideads commentedI've given a bit more thought to this after seeing Dereine's example view. That was the use case I was looking for, but couldn't figure out. Because the field is computed, it must be aliased to allow ordering.
I don't believe, based on my research, that a field must be in the select clause to allow ordering, at least in SQL-92 and both Postgres and MySQL's implementations. And, to use a real world example that I've had to work with lately: Django's ORM uses
ORDER BY tablealias.fieldnamesyntax to build sorts with both the MySQL and PostgreSQL backends. Fields are added to the select clause as requested, with no connection between then. Admittedly, this makes the "last comment or last update" sort harder to accomplish than in View's handler-oriented framework. My point is really just that AFAICT (and I'm no expert) and based on other real world implementations, this shouldn't be out of the question.This suggests two other possible approaches:
One approach would be to make a new rule: If a field uses a database function (such as a comparison function or user defined function for IP address manipulation), it must invoke
add_orderbywith a declared alias. At least it would narrow down the list of potentially backward incompatible code to calls toadd_orderbythat use a computed field AND don't specify its own alias.For example,
views_handler_sort_ncs_last_updated.incalready specifies an explicit alias when adding ordering in itsquerymethod:I've attached a patch that implements this behavior. I think there's still room for improvement, but I believe this is fairly backwards compatible, and doesn't fundamentally change the function as described in the doxygen documentation. Personally, I think this is a reasonable approach, and represents a small step towards magic elimination.
Another approach would simply be to add a flag to the function signature which specifies the "build" mode. This would be the safest, most backward compatible approach, and I'd still be quite happy to see it.
Take a look at this (contrived) example view:
With the patch applied, this generates:
In this case, I think the question becomes "what happens when a relationship is specified in a sort or filter handler?"
Comment #8
davideads commentedOh darn, again, here is patch.
Comment #9
davideads commentedHas any thought been given to this patch? I've been successfully using it on a testing/development site for several weeks, with no adverse effects.
Comment #10
joachim commentedIs that a PostgreSQL site though?
Comment #11
davideads commentedNo, it is MySQL.
Comment #12
davideads commentedWant me to test with PostgreSQL?
Comment #13
davideads commentedI'm in the midst of turning my project that depends on this patch into a buildable system with Drush make and realized the prior patch doesn't apply cleanly.
Here's a fixed version. I've been using this patch on a testing site (running MySQL) for the past five months without adverse effects. I tested on PostgreSQL back in August, and everything seemed fine but I haven't done additional testing since.
Comment #14
dawehnerThis seems to be like a reviewable patch.
In general the issue is still valid: http://drupal.org/node/844910#comment-3163046
Comment #15
joachim commentedhttp://en.wikipedia.org/wiki/Order_by says:
> The sort criteria do not have to be included in the result set.
The other problem is that:
> The SQL standard is not freely available. The whole standard may be purchased from the ISO as ISO/IEC 9075(1-4,9-11,13,14):2008. http://en.wikipedia.org/wiki/SQL:2008
I dunno about you but I'm not forking out for it!
Comment #16
joachim commentedPatch works for me in that it fixes the problem. Yay!
Few things though...
That sentence needs fixing, not sure how.
I think the comments could go into a bit more detail about what being done and how. The deep innards of Views get quite complex!
Also I'm curious about this:
when we have this:
$table is not optional -- so surely $table will always be true?
Powered by Dreditor.
Comment #17
davideads commentedJoachim, thanks for the review. I frankly don't remember with respect to the $table variable -- I wrote the patch 11 months ago. But I'll check; most likely it was a brain fart on my part.
In terms of the the errant sentence, it should simply read "If called without an alias, only add the field to the order-by clause." For me, that seems like sufficient commenting within in the context of the function. Should it explain how otherwise the field is added to both the where clause and the order by clause?
Comment #18
joachim commentedI'm not sure what calling without an alias actually entails... though arguably that should be the job of the actual method's documentation to explain! Is it that you can either add a sort order on a field that is already present in the query (and thus has an alias) or a new one?
Comment #19
tim.plunkettThere are still references to this in 8.x-3.x
Comment #20
xjmSo there are references to something that we need to remove but it's still filed as a feature request. Can we update the summary explaining the current status? :)
Comment #21
joachim commentedI understand the problem as it was filed and discussed on 6x3x but I don't know the state of the Views 8 code to write a summary.
This is definitely a bug though, because of #1819656: sort handlers don't know the alias of their query field which is really just another consequence of this.
Comment #22
dawehnerAnd it does :)
There is a public available draft of a previous sql standard, which is helpful: http://www.contrib.andrew.cmu.edu/~shadow/sql/sql1992.txt
Are you sure this is true? Couldn't you just add the subquery to the result and then order by it?
Comment #23
joachim commentedA subquery that you use on a JOIN or a WHERE must select only one column. If you want to order by a different column, and that column must be selected, you're stuffed:
// Works:
SELECT * FROM outer WHERE outer.a = (SELECT inner.b FROM inner ORDER BY inner.c LIMIT 1)
// Fails, because the outer equality expression sees the wrong thing on the right of the =
SELECT * FROM outer WHERE outer.a = (SELECT inner.b, inner.c FROM inner ORDER BY inner.c LIMIT 1)
Comment #24
dawehnerViolating against the sql standard seems to be the wrong way to solve that problem, so i have no idea how to continue on this issue.
Comment #25
joachim commentedUnless I'm misreading it, that does not say the sort column must be in the SELECT part. It just says that the sort column must be a column of the table.
Comment #26
mitokens commentedComment #33
anybodyTLDNR: I want to order entities from an entity reference by the delta of the entity reference field but that leads to duplicate results because a further JOIN is added automatically which should not happen.
--
Hey all, thank you very much for this issue. I just ran into the problem of multiple results despite of active DISTINCT option because views adds a further join when an order by is added on a delta of an entity reference.
I found a module that provides a working workaround (https://www.drupal.org/project/views_order_by_delta) and opened an issue to find related core issues #3020585: Isn't this a core issue? because this should definitely be solved in core, I think.
Could someone from this issue perhaps have a look if this issue handles the same problem? Then we should try to revive it. Or is this a completely different problem? Thank you!
Comment #41
acbramley commentedThis issue came up in BSI triage. There hasn't been a comment here in almost 6 years. Because of this, I'm going to close this as outdated due to the time it has been since the last comment. It does seem like we still add a field in https://git.drupalcode.org/project/drupal/-/blob/11.x/core/modules/views... but only if one is passed in.
If anyone thinks this is still a bug that needs fixing, please feel free to reopen.