Problem/Motivation

Usecase:
There are three nodes with the following titles: foo, bar, drupal. Requirement: Create a view that lists nodes that does not have title foo or bar.

Using views string filter you can't create a SQL expression like "some_field NOT IN ('foo', 'bar')", as a workaround when searching a multiple value string in a string Regular Expression operation can be chosen but it's hard to implement a reverse search for regex because the regex is done on SQL level which is limited according to https://dev.mysql.com/doc/refman/5.7/en/regexp.html

POSIX regexes don't support using the question mark ? as a non-greedy (lazy) modifier to the star and plus quantifiers like PCRE (Perl Compatible Regular Expressions). This means you can't use +? and *?

http://stackoverflow.com/questions/18317183/1139-got-error-repetition-op...

Proposed resolution

Implement negation for REGEXP operation, it can be found in documentation as NOT REGEXP.

NOT REGEXP Negation of REGEXP

https://dev.mysql.com/doc/refman/5.7/en/regexp.html

Remaining tasks

  1. Requeue tests for 8.8.x branch and more DBs.
  2. Final review / RTBC
  3. Commit

User interface changes

Adds a new "Negated Regular expression" option to the choices for the operation to use for Views string and integer filters.

API changes

None.

Data model changes

None.

CommentFileSizeAuthor
#38 2853188-38.patch8.76 KBenyug
#37 2853188-37.patch8.76 KBAnas_maw
#30 2853188-14.patch8.7 KBquietone
#30 2853188-14-fail.patch6.24 KBquietone
#14 interdiff-2853188-12-14.txt711 bytesaerozeppelin
#14 2853188-14.patch8.7 KBaerozeppelin
#12 interdiff-2853188-8-12.txt4.58 KBaerozeppelin
#12 2853188-12.patch8.7 KBaerozeppelin
#10 interdiff8_10.txt1.19 KBMunavijayalakshmi
#10 2853188-10.patch5.67 KBMunavijayalakshmi
#8 2853188-8.patch5.66 KBaerozeppelin
#4 views_string_filter-2853188-4.patch2.54 KBSurfinSpirit
#2 views_string_filter-2853188-2.patch1.29 KBSurfinSpirit

Issue fork drupal-2853188

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

SurfinSpirit created an issue. See original summary.

SurfinSpirit’s picture

Status: Needs work » Needs review
FileSize
1.29 KB
dawehner’s picture

Issue tags: +Needs tests

Do you mind expanding the existing test coverage for it?

SurfinSpirit’s picture

Noticed there is one more filter that has regex option NumericFilter. Added the same functionality there as well.

Note. There is no test case for StringFilter that uses regular_expression operation.

@dawehner do you suggest creating a new test case or extending existing one to include a new filter?

dawehner’s picture

Status: Needs review » Needs work

Good catch. Let's ensure though we don't forget the tests :)

Anonymous’s picture

@SurfinSpirit, cool!

creating a new test case or extending existing one to include a new filter?

We need a new tests for NumericFilter and StringFilter, like #2821112: Views NumericFilter 'regular_expression' operator is broken. Because it new operator.

+++ b/core/modules/views/src/Plugin/views/filter/NumericFilter.php
@@ -277,6 +283,16 @@ protected function opRegex($field) {
+    $this->query->addWhere($this->options['group'], $field, $this->value, 'NOT REGEXP');

It should be $this->value['value'] in NumericFilter.

Also we need check the patch for all supported databases. Now it fails on PostgreSQL. We need fix it via separate issue (with tests), like #2845543: PostgreSQL regular expression match operators works only for text.

But we haven't NOT REGEXP converter. Hense we also needs add it here like
'NOT REGEXP' => ['operator' => '!~*'],
and create test for it too.

SurfinSpirit’s picture

Component: user.module » views.module
aerozeppelin’s picture

Status: Needs work » Needs review
FileSize
5.66 KB

Updates to patch as per #6.

Munavijayalakshmi’s picture

Assigned: Unassigned » Munavijayalakshmi
Status: Needs review » Needs work
  1. +++ b/core/modules/views/tests/src/Kernel/Handler/FilterNumericTest.php
    @@ -244,6 +244,49 @@ public function testFilterNumericRegularExpression() {
    +   * Tests the numeric filter handler with the negated 'regular_expression' operator.
    

    Line exceeding 80 characters.

  2. +++ b/core/modules/views/tests/src/Kernel/Handler/FilterStringTest.php
    @@ -783,6 +783,49 @@ public function testFilterStringGroupedExposedEmpty() {
    +   * Tests the string filter handler with the negated 'regular_expression' operator.
    

    Line exceeding 80 characters.

Munavijayalakshmi’s picture

Assigned: Munavijayalakshmi » Unassigned
Status: Needs work » Needs review
FileSize
5.67 KB
1.19 KB
Anonymous’s picture

Status: Needs review » Needs work

@Munavijayalakshmi, thanks for cleanup.

@aerozeppelin, nice work, it's really almost cover first half of #6. But we also need to add a GroupedExposed* tests for FilterNumericTest and FilterNumericTest. Like with other operators. E.g.:

  public function testFilterNumericRegularExpression()
  
  public function testFilterNumericExposedGroupedRegularExpression()

See getGroupedExposedFilters() for do it.

Also we need open new issue (Feature request) about support NOT REGEXP operator in PostgreSQL. See PostgreSQL fail in #8. Because while we have problem with one of supported database, we can not add code to the core.

aerozeppelin’s picture

Status: Needs work » Needs review
FileSize
8.7 KB
4.58 KB

@vaplas, thank you for the review. I've tried to update the patch with comments in #11. Let's see what the test bot thinks.

Status: Needs review » Needs work

The last submitted patch, 12: 2853188-12.patch, failed testing.

Anonymous’s picture

Title: Views string filter negated regular expression » [PP-1] Views string filter negated regular expression
Status: Needs review » Postponed

@aerozeppelin thank you, last patch looks perfect. I opened #2860644: Add support of NOT REGEXP operator to PostgreSQL + fix testRegexCondition. And we need solve it, before change status of current issue to RTBC.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

dww’s picture

Still applies to 8.7.x(!) and works great. Would love to get this in.

Looks like #2860644: Add support of NOT REGEXP operator to PostgreSQL + fix testRegexCondition got close, but no longer applies. I don't personally care about pgsql, but maybe we can get that in soonish...

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

dww’s picture

Title: [PP-1] Views string filter negated regular expression » Add negated regular expressions for views filters (string and integer)
Status: Postponed » Needs review

a) Yay, #2860644: Add support of NOT REGEXP operator to PostgreSQL + fix testRegexCondition is now in, so this is no longer blocked.

b) More accurate title, since this supports both string and integer filters.

c) I'll requeue tests for modern branches + environments, but I think it's all going to be green.

d) Fairly careful (but not utterly pedantic) review doesn't uncover any problems that need addressing. This is probably RTBC as-is. The actual code changes are very small and simple. There's basic test coverage for everything. The coverage could potentially be improved (along the lines of the tests in #2860644) to have a data provider and cover more possible cases, but we've at least got some coverage of the new option, which should be enough for commit.

Anyway, I'll come back to RTBC once the bot's happy.

Thanks,
-Derek

dww’s picture

Issue summary: View changes

Fixing the summary to be more accurate.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

muaz91’s picture

Hi,

Testing out the patch from #14. it works for me. test with drupal 9.3

+1 for RTBC

muaz91’s picture

Status: Needs review » Reviewed & tested by the community
quietone’s picture

This needs to be on 10.0.x now. Changing version. The patch still applies so I downloaded it and made a fail patch as well.

There are tests so removing tag.

The last submitted patch, 30: 2853188-14-fail.patch, failed testing. View results

dww’s picture

dww’s picture

p.s. confirming that the fail patch is producing the right kinds of failures: Ringo keeps showing up! 😂

Failed asserting that two arrays are identical.
--- Expected
+++ Actual
@@ @@
         'age' => '27'
     )
     2 => Array &3 (
+        'name' => 'Ringo'
+        'age' => '28'
+    )
+    3 => Array &4 (
         'name' => 'Paul'
         'age' => '26'
     )
-    3 => Array &4 (
+    4 => Array &5 (
         'name' => 'Meredith'
         'age' => '30'
     )
 )
Lendude’s picture

Very nitty nits

  1. +++ b/core/modules/views/tests/src/Kernel/Handler/FilterNumericTest.php
    @@ -244,6 +244,49 @@ public function testFilterNumericRegularExpression() {
    +   * Tests the numeric filter handler with the negated 'regular_expression' operator.
    

    > 80 characters

  2. +++ b/core/modules/views/tests/src/Kernel/Handler/FilterStringTest.php
    @@ -420,6 +420,36 @@ public function testFilterStringGroupedExposedStarts() {
    +  public function testFilterStringGroupedNotRegularExpression() {
    

    This doesn't have doc block. I know a lot of the tests in this file don't have any, but I *think* we still need one for all new methods?

Queued on all databases again, cause want to be sure we still have support on all databases for this.

alexpott’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs change record

Let's address the nits in #34 and this one too...

+++ b/core/modules/views/tests/src/Kernel/Handler/FilterNumericTest.php
@@ -272,6 +315,39 @@ public function testFilterNumericExposedGroupedRegularExpression() {
+  /**
+   * Tests the numeric filter handler with the 'not_regular_expression' operator
+   * to grouped exposed filters.
+   */
+  public function testFilterNumericExposedGroupedNotRegularExpression() {

More than a single line.

Plus as something new we should have a change record to announce the new thing we can do... see the "add change notice" link above.

BramDriesen’s picture

Issue tags: -Needs change record

Drafted a change record here: https://www.drupal.org/node/3314465

Still needs work for #34 & #35

Anas_maw’s picture

Reroll patch in #30 to work on the latest 9.5 version

enyug’s picture

BramDriesen’s picture

Shouldn't this issue be targeting 10.1.x or 11.x?

BramDriesen’s picture

Version: 10.0.x-dev » 11.x-dev

BramDriesen’s picture

Status: Needs work » Needs review
smustgrave’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: +Needs Review Queue Initiative

2 nitpicky changes in the MR.

Going to go ahead and mark it if the committer could make that one word change.

Seems points in #34 and #35 have been addressed as well.

BramDriesen’s picture

Fixed one more nit. The dropdown items were showing Negated Regular expression, no need to have Regular with a capital so changed that to lowercase. Also fixed the nits with the suggestion of @smustgrave in #43.

  • longwave committed 4516d625 on 11.x
    Issue #2853188 by BramDriesen, aerozeppelin, SurfinSpirit, quietone,...
longwave’s picture

Status: Reviewed & tested by the community » Fixed

Committed and pushed 4516d62542 to 11.x. Thanks!

Also published the change record.

BramDriesen’s picture

Issue summary: View changes

Thanks @longwave! Also updated the issue summary (remaining tasks).

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.