Hi

I have a Flash UI that is connected to Drupal AMFPHP and Services. It is an historic timeline (events plotted). I created a custom content type for this.

You can see the Flash version here:

http://www.mauriciogiraldo.com/vgline/beta/

And the "plain text" Drupal version here:

http://www.mauriciogiraldo.com/vgline/

Right now I have ~400 events that will grow to ~1000 or more. What I do is load the whole bunch of events from the Flash UI. This takes ~30 seconds (there is a distractor so you don't leave in the meantime).

My question:

Would it be better in terms of performance to make, say, requests for 10 events at a time instead of the 400 at once? Or would this overload Drupal (although each request would only be made once the previous was complete).

My main problem is Views being unable to load the whole node data at once as documented in this issue:
http://drupal.org/node/678202

From the Devel module, I am now getting 6-10s for page execution of the Services admin (but was having 30s before making a database upgrade).

Any ideas on which option is better? One big load, multiple "smaller" loads?

Thanks

Comments

carlosg2’s picture

carlosg2 commented

Short anwser: Small lazy loaded requests.

A long time ago i hit the same issue building rich flash clients that loads a render large data sets...

Because the node load that drupal use, you cant request large data sets without hitting performance issues, BUT Drupal is not the problem, working with < 50 nodes works fine.

The solution in your case is to architect your client application in a way that fetch just and only the the data that is needed for your initial screen, and later on request more if is needed. This concept in called Lazy Loading.

Check this timeline example:
http://www.youtube.com/watch?v=aAE8SBpqEsI

In this way you cant scale to infinite, just what timeline needs to be happy!

Later on you probably need to implement virtual list too beacause render performace. (remove and add display objects from your stage and memory)
but that is for another issue.

Best regards

Carlos Garza
PRISMACORE

mga’s picture

mga commented

Thanks for your reply.

So hitting ~100 times Drupal will not be as much a penalty as that one big load (assuming 400 as "big")?

Because I guess this lazy loading would be:

1- load minimal data (node ids) so as to render them
2- FOR EACH NODE: load node full data

Step 2 would be made 400 times on every visit to the timeline. And I plan on putting a couple hundred more events.

Will that be ok for Drupal or the server? Assuming the site gets a few (4-10) simultaneous visitors?

Thanks again.

I will watch the video you sent.

carlosg2’s picture

carlosg2 commented

One simple starting point could be...
1. Request only the records needed to fill the screen using views service, passing the first-last dates of the current timeline screen as arguments to the view.
2. Clear your timeline if already has something, fill with the fetched array and render it.
3. As you scroll/pan, repeat steps 1 and 2.

This is not lazy loading. But may work in your case and later on can be implemented.

Kevin Hoyt explains your problems and solutions in this videos:
http://tv.adobe.com/watch/adc-presents/ria-problems-you-never-expected-w...
http://tv.adobe.com/watch/adc-presents/ria-problems-you-never-expected-l...

Best regards

Carlos Garza
PRISMACORE

mga’s picture

mga commented

Hi

Thanks again.

My problem is not much of a rendering issue as a data transfer issue. Drupal/MySQL pretty much dies from the amount of data requested when I start to get a few visitors (or maybe other sites since it is a shared server). The Flash view can handle the data just fine (unless you have a rather old computer but that I don't care very much since gamers tend to have good computers).

The lazy loading in the video you sent would be right since the whole point of the Flash view is to visualize simultaneous data.

My problem lies in bringing the data to Flash (where the Views module is not working in the most optimal way). The second video talks about scaling the server in case of trouble. The thing is, my trouble is starting with just 400 events (which are cross-referenced I know but it's not THAT big a dataset).

As I understand it, you would recommend then 400 hits instead of one "big" one?

marcingy’s picture

marcingy commented
Status: Active » Closed (fixed)