#1841788: Add project browser requires that d.o run the project_browser_server in order to function. There is an issue for that: #1243332: Deploy Project Browser Server and drupalorg_pbs on d.o.

It appears that wildkatana had previously set up a devdrupal.org site at http://drupal_7.redesign.devdrupal.org and the project_browser module is configured to use that domain in place of d.o. Unfortunately, that site is returning a 404 error and wildkatana has not been active in the above issues for over a month.

I'd like to get the project_browser issue moving again, and a devdrupal.org site would allow that.

Comments

tvn’s picture

Status: Needs review » Postponed

Development site for project_browser is still present: http://pbs-drupal_7.redesign.devdrupal.org/
Wildkatana requested (#1858426: I want my drupal.org development site re-imaged) this site to be re-imaged. After that he probably did not re-configure it as needed.

I suggest you first contact wildkatana and ask him to re-configure the site. Then we can add your SSH key to the dev server so you could do it yourself.

douglasmiller’s picture

That sounds reasonable. I have sent wildkatna a message through the contact form.

douglasmiller’s picture

Status: Postponed » Needs review

I've contacted wildkatana via d.o contact and have had no response since Monday.

Would it be possible to give me access to the server so that I can configure it?

jthorson’s picture

I pulled down drupalorg_pbs and project_browser_server to the pbs site, and got c4doug admin access ... but he could still use individual access to devwww.

tvn’s picture

Status: Needs review » Fixed

c4doug, added your ssh key. Here you can find some tips on how to work on our dev server: https://drupal.org/node/1018084

douglasmiller’s picture

Status: Fixed » Needs review

Argh, I haven't updated my public key since I got a new computer. I've added a new key (Gozer) to my profile. Could you add that key please?

Anonymous’s picture

Hey guys, really sorry about my absence. Between school, work and a sick baby and wife, life has been hectic so far this year! Anyways, I want to make this Project Browser a priority and make sure we get this (in my opinion) indispensable feature in before D8. What can I do to help?

douglasmiller’s picture

@Wildkatana, I figured that you were busy when I saw a 3 month old post saying that you have a new born. Congratulations on that!

Jthorson gave me the user/1 account on this server. I've been trying to get everything back up and running as best as I can.

From what I've ascertained, the project_browser_server and drupalorg_pbs modules need to be installed, access needs to be given to the anonymous user, and solr needs to index the site. From there, the module should just work (tm). Am I correct in understanding what is needed?

So far, we've gotten the project_browser_server and drupalorg_pbs modules installed. I was initially getting 403 forbidden errors, but corrected that by granting the 'access project browser server' permission to the anonymous user.

http://pbs-drupal_7.redesign.devdrupal.org/project_browser/server/catego... is now correctly returning the terms in the 'Module categories' taxonomy vocabulary (vid 3). http://pbs-drupal_7.redesign.devdrupal.org/project_browser/server/query/... is returning an empty set.

The current issue that I'm running up against is getting solr to index the site. I am getting the same errors that you reported in this issue comment. I've read that entire issue, though it was late at night. I think that you were using D6 at that point in time though.

Do you have any suggestions about how to get past this hurdle?

drumm’s picture

Status: Needs review » Fixed

Done

drumm’s picture

(done re #6, added new key for c4doug)

douglasmiller’s picture

Status: Fixed » Needs review

I'm having trouble getting the project browser server to return any results.

The project_browser_server module and it's api function correctly. I tested by disabling the drupalorb_pbs implementation and enabled the project_browser_server_example.

The server example implementation is written to return hard coded static values. If a query is executed against the api, then correct results are returned.

Executing the same query against the drupalorg_pbs implementation returns 0 results, but correctly returns an empty set.

I looked at the apache solr configuration page and noticed that 'Project' is not configured to be indexed. I suspect that this is why 0 results are returned. The solr reports page shows that there are 2831895 documents in the index, so it is certainly not empty.

I found this page (https://drupal.org/node/1334962) while reading the linked document in comment #5. I followed the instruction there, but run into issues.

Attempting to index queued content produces a few log entries full of JasperExceptions:
"500" Status: PWC6344: Unable to create output writer for file /tmp/Jetty_0_0_0_0_8112_solr.war__solr__bq4vnd/jsp/org/apache/jsp/admin/stats_jsp.java org.apache.jasper.Jasper
"500" Status: background merge hit exception: _b45(3.6):Cv462 _b4b(3.6):Cv546 _b4h(3.6):Cv87 _b4n(3.6):Cv239 into _b5h [maxNumSegments=1] java.io.IO

I also run into these errors in the logs, but think that they are unrelated:
Socket error 111: Connection refused in /var/www/dev/pbs-drupal_7.redesign.devdrupal.org/htdocs/sites/all/libraries/pheanstalk/classes/Pheanstalk/Socket/NativeSocket.php on line 36.

I'm not really sure what the problems are, but I'm scrambling to beat the feature freeze date.

drumm’s picture

You hopefully do not need to index content from this site. Unless you run into indexing bugs, please keep this read-only. Under normal circumstances, indexing should only happen on either 7.devdrupal.org or git7site. I'm looking into which it should be and confirming that it is working.

Anonymous’s picture

Yes, the drupalorg_pbs module runs using ApacheSolr, so it needs an index of Projects to work (hopefully the full index like how it is on d.o).

douglasmiller’s picture

@Drumm: Are you saying that I may need to configure Apache Solr to use a different Solr server? One of the ones that you mentioned instead of stagingsolr.drupal.org? If so, let me know what server and if I need to do anything special other than changing the Solr server URL.

@Wildkatana: Is the project_browser_server module ready to roll out to production? I consistently see php notices on the pbs-drupal-7 server and see development notes throughout.

Example notices:

    Notice: Undefined index: categories in project_browser_server_query() (line 62 of /var/www/dev/pbs-drupal_7.redesign.devdrupal.org/htdocs/sites/all/modules/project_browser_server/project_browser_server.module).
    Warning: Illegal offset type in isset or empty in apachesolr_static_response_cache() (line 1496 of /var/www/dev/pbs-drupal_7.redesign.devdrupal.org/htdocs/sites/all/modules/apachesolr/apachesolr.module).
    Warning: Illegal offset type in apachesolr_static_response_cache() (line 1497 of /var/www/dev/pbs-drupal_7.redesign.devdrupal.org/htdocs/sites/all/modules/apachesolr/apachesolr.module).
    Warning: Illegal offset type in apachesolr_static_response_cache() (line 1499 of /var/www/dev/pbs-drupal_7.redesign.devdrupal.org/htdocs/sites/all/modules/apachesolr/apachesolr.module).

I've seen a TODO on the permissions description.

The drupalorg_pbs module makes a call to db_query on line 156, but the result set is never used.

Anonymous’s picture

@c4doug - I was never able to get a good Solr Index to use for testing, so it is not currently ready for deployment, it needs more testing, since at that time the D7 for drupal.org wasn't ready yet, and there were missing pieces like the project_solr module.

douglasmiller’s picture

Any update regarding the Solr server?

douglasmiller’s picture

To allow progress on #1841788: Add project browser, I have re-purposed the original example implementation of the project browser server api to return results for Drupal 8 queries.

I will only be able to re-activate the drupalorg_pbs implementation once can figure out how to access a Solr server that includes Project entities in the index.

drumm’s picture

The solr server is getting to be in better condition, but not ready yet. http://git7site.devdrupal.org/ is the only site which should write to the index. The site is rebuilt daily, bringing in fresh data from Drupal.org, and it had been choosing a new site hash every day, doing a really good job of filling up the disk. The disk space is cleared up and it is catching up indexing. I'm waiting to see how that goes.

#1559946: Port project_solr to D7 may have problems left which affect indexing. If so, those need to be fixed before we'll have a good index.

Anonymous’s picture

@drumm - Thanks for the update. The staging site only needs read access to the solr index. The main thing I am concerned with is making sure it is using the latest modules (project_solr and others) it needs to be able to display the projects so I know it has access to a valid index.

Is Drupal.org already running Drupal 7? http://drupal.org/project/modules works on Drupal.org, but http://pbs-drupal_7.redesign.devdrupal.org/project/modules doesn't... Any thoughts?

drumm’s picture

Drupal.org isn't running D7 yet.

Reindexing http://git7site.devdrupal.org/ is working now. http://git7site.devdrupal.org/download is working and accurate as far as I know; including for project usage, which was the last index that needed work. The project_solr, etc, code should be stable at this point, but we might find problems in QA.

I'd recommend wiping http://pbs-drupal_7.redesign.devdrupal.org/ and making a new dev site with the latest code. That will lose any project browser configuration, but that should be documented as deployment instructions of the issue summary at #1243332: Deploy Project Browser Server and drupalorg_pbs on d.o anyway.

douglasmiller’s picture

Thanks for the update, drumm. I agree that the dev server should be wiped. I reviewed the server configuration and added deployment configuration steps for Drupal 7.

Based on IRC discussion, I added a $conf['apachesolr_site_hash'] value to the settings.local.php file. Will this configuration be necessary after the server is wiped?

It looks like the Drupal.org D7 upgrade is starting to move again!

drumm’s picture

Status: Needs review » Fixed

$conf['apachesolr_site_hash'] is now hard-coded in settings.php, so it should be good.

I started the rebuild, it should be done within 2 hours.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

Component: Operating system » Servers