Currently, the export will hit the disqus servers at least twice for every comment, which can be VERY slow.

Its doing this because its checking, every time, if there is a thread on disqus for the comments parent node. Which is fine if you only have 1 comment per node, but as soon as you get nodes with a few comments (or perhaps 300) it gets really heavy.

To fix this all we need to do is create a NID => Thread mapping array and consult that before we hit disqus's servers.

Optionally, we should also alter the comment query to be orded by n.created (not c.timestamp), so comments created on the same node, are listed together and processed together. As this will reduce the frequency of hits to Disqus's servers and insure nodes are completely migrated before moving on to the next one.

Will upload a patch once i've backed out the other changes.

CommentFileSizeAuthor
#6 disqusing.patch2.51 KBrobloach
#2 disqus_migrate-improvments-987258.patch2.36 KBa_c_m

Comments

a_c_m’s picture

a_c_m commented
Version: 7.x-1.x-dev » 6.x-1.x-dev

whoops, 6.x not 7.x

a_c_m’s picture

a_c_m commented
Status: Active » Needs review
StatusFileSize
new disqus_migrate-improvments-987258.patch2.36 KB

So 3 changes in this patch.

1) Ordering of the comments by n.created instead of c.timestamp, so nodes are fully migrated as a unit
2) NID -> Thread mapping, reducing the number of queries to Disqus
3) 'name' generation from email address and auto-skip if we dont have both (its a requirement by Disqus).

rootwork’s picture

rootwork
he/him
Primary language English
Location 🇺🇸 US-Pacific 🌎 Chinook (Multnomah, Clackamas) & Cowlitz lands 🌹 Portland, OR
commented

This sounds excellent. As soon as I have a moment to test this out, I will. In the meantime: Subscribe!

robloach’s picture

robloach
he/him
commented

Is there anything to note before committing this? Think it's good to go in? I'm going through a issue review making a 1.7 Disqus release.

rootwork’s picture

rootwork
he/him
Primary language English
Location 🇺🇸 US-Pacific 🌎 Chinook (Multnomah, Clackamas) & Cowlitz lands 🌹 Portland, OR
commented
Status: Needs review » Reviewed & tested by the community

Oops -- yes, sorry for not commenting back. I didn't do extensive testing, but as far as I could tell this worked as advertised. RTBC as far as I'm concerned.

robloach’s picture

robloach
he/him
commented
Status: Reviewed & tested by the community » Needs review
StatusFileSize
new disqusing.patch2.51 KB

Updated to latest DRUPAL-6--1.... Mind testing?

davidseth’s picture

davidseth commented

I can confirm that this patch did work for me for exporting from Drupal comments to Disqus comments. It fixes issue: http://drupal.org/node/777842 for me.

robloach’s picture

robloach
he/him
commented
Status: Needs review » Fixed

Thanks! http://drupal.org/cvs?commit=490040

I used the follow as davidseth suggested at http://drupal.org/node/777842#comment-4008022 :

    // Disqus requires both mail and name. Give them an anonymous mail/name if not provided.
    if (empty($mail)) {
      $mail = t('anon@anonymous.com');
    }
    if (empty($name)) {
      $name = t('Anonymous');
    }

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

netikseo’s picture

netikseo commented
Version: 6.x-1.x-dev » 6.x-1.9

Hi, I'm using 6.x-1.9 version of Disqus module and when I'm trying to export my comments I get the error which suppose to be fixed:
Fatal error: Uncaught DisqusException: [0]: There was an error querying the Disqus API. thrown in /var/www/vhosts/-hidden-/httpdocs/sites/all/modules/disqus/disqus.php on line 344

I'm using Drupal 6.27 and it's first time I tried to install Disqus. Anyone else having the same mistake? Any suggestions how to fix it?

Thanks!

netikseo’s picture

netikseo commented
Status: Closed (fixed) » Active

I think I should change status to active as well...

josh@pixael.com’s picture

josh@pixael.com commented

Same problem here...

Tresler’s picture

Tresler commented

I'm hitting the same error while handling an export, but the cause is different.

The Disqus API is intermittently throwing an http 500 error, which will cause this exception (although other things will also cause this exception).

For some reason, I'm having issues catching this exception when encapsulating the calling statement in a try{}catch{} block, and I don't know why that is.

the_g_bomb’s picture

the_g_bomb commented
Issue summary: View changes
Status: Active » Needs review

I have uploaded a patch to #2185007: Error during export: You must be authenticated to perform this action which patches the dev version and solves migration issues for me.