cant get views php field to work with sarnia.. once a global php field is added.. I get timeouts on the search. I've tried on two different instances with the same results...

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

rulley created an issue. See original summary.

jmdeleon’s picture

Priority: Critical » Major
Status: Active » Needs work

What you're seeing is related to a long-standing issue with Views PHP that Sarnia unfortunately exposes: that Views PHP doesn't scale well to large result sets (like those returned from an arbitrarily large Solr index). These are the relevant issues:

https://www.drupal.org/node/1586234

https://www.drupal.org/node/1600488

You basically run into an out-of-memory condition when the PHP environment is repeatedly spawned to run the code in a Views PHP field, over any large result set returned by Views -- a common occurrence returning data from a Sarnia index.

(I've run into the issue you are describing, as I have built Sarnia server and index configurations over Solr indices containing literally hundreds of thousands of records.)

There is an advisory on the front page of the Views PHP project page, that applies even more to Views that process data from a Sarnia Search API index:

While this module allows you to directly use PHP inside views which may be useful for quick and easy solutions, it is highly advisable to use regular handlers and plugins when available (or even to create one yourself). Take note that filtering and sorting a view using PHP always has a considerable perfomance impact.

The only true solution at this point is to code a Views handler in a Views plugin, with any custom PHP you want to use to process over the result set from a Sarnia data source. Using Views PHP may be a convenience, but it also encourages a poor development practice of keeping relatively complex processing code (and potentially business logic and business rules) in a configuration item like a View.

I can understand that you would consider this a Critical issue, but there isn't a whole lot that the Sarnia module can do to address this, as this really is a limitation of the Views PHP module.

rulley’s picture

Priority: Major » Normal

Ok thank you for the response.. I now remember reading that about views php in the past. Maybe you could help point me in the right direction.. what I need to do is filter out sarnia entities that have been flagged by the current user from the view. Theses would normally be available options via the relationships/contextual filters.. any ideas?

jmdeleon’s picture

I was going to write a longer response to this... Contextual filters seem to work generally well in Sarnia, but relationships do not work at all (a relationship is essentially a database join, which is fundamentally not supported here).

It seems like you are asking for a relationship between a Solr item in your index on a Drupal user entity, which a Solr index most likely has no conception of, unless you somehow build such a relation in your index. You might look into being able to get the user ID as a contextual filter.

rulley’s picture

Ok, was thinking that since I am able to have a drupal flag in place that is able to flag a sarnia entity... that there must be some way to filter on which sarnia entities within views have been flagged by a user... I have come across articles where it appears there are ways to mix local and remote entities within views.. Looking through search api issues - it seems there have been similar questions like this.

jmdeleon’s picture

I've been looking at this a bit (mostly to address a few possibly-unrelated issues in Views PHP), but I found that making a couple of changes in Views PHP code does help get Sarnia Views working better with Views PHP:

  • removing/commenting out the following two lines in the member function pre_execute() in the file views_php/plugins/views/views_php_plugin_pager.inc
        $this->wrapped->view->query->set_limit(0);  
        $this->wrapped->view->query->set_offset(0);
    

    This is in both Views PHP 7.x-1.x and 7.x-2.x

  • For Views PHP 7.x-1.x: removing/commenting out the following following lines in the member function post_execute() in the file views_php/plugins/views/views_php_plugin_pager.inc
        $this->wrapped->total_items = count($this->wrapped->view->result);
        $this->wrapped->update_page_info();
    
        $item_per_page = $this->wrapped->get_items_per_page();
        if ($item_per_page > 0) {
          $offset = $this->wrapped->get_current_page() * $item_per_page + $this->wrapped->get_offset();
          $this->wrapped->view->result = array_slice($this->wrapped->view->result, $offset, $item_per_page);
        }
    
  • For Views PHP 7.x-2.x: removing/commenting out the following line in the member function pre_render() in the file views_php/plugins/views/views_php_plugin_pager.inc
        $this->update_wrapped_pager();
    

The following issues in Views PHP describe these changes, with a series of patches that do implement the above edits:

#2123315: Pager does not appear if fields use Views PHP (7.x-1.x)

#1586234: keywords: views pager query limit

#2276165: Pager disappears when Global: PHP used in Views 3.8

#2123315: Pager does not appear if fields use Views PHP (7.x-1.x)

These changes also have odd effects when PHP code is used as a filter. This is partly because the PHP filters are applied after the result set count is calculated, which may result in a different number of result items being returned from the View. Removing the set_limit setting is also controversial, as it may introduce performance hits.

jmdeleon’s picture

Rolled a patch to the Views PHP module 7.x-1.x (not Sarnia) that incorporates the changes I describe in post #6.

ibakayoko’s picture

I am using views php with sarnia without any issue.
We had some memory issues, due to the fact by default sarnia return ~ 1 millions results and you need to make some query alteration in sarnia
to return only the fields you need in the query.

sarnia usually return all the fields.

We fixed this issue in our implementation by returning only the fields we are using form solr and limit the number of result returned.

jmdeleon’s picture

Rolled a similar patch as #7, but for Views PHP 7.x-2.x-dev.