Media files on multiple server setup

By andershal on 17 Oct 2007 at 08:37 UTC

We are building a high-traffic website with Drupal and can't find any information on media files setup when using several servers. Our problem is that files uploaded to one frontend isn't available on the other frontends.

Possible solutions, but not sure if they will work:

Create hooks on file upload and download, and thereby manage files on a dedicated server.
- Is there any hooks to accomplish this?
- We still need imagecache module to work.
Syncronise uploaded content (with rsync) automatically or invoked by file upload hook ^^.
- Even though there will be many file uploads during the day, it seems expensive to run rsync every 3 minuttes!
Store images in database.
- This will create a performance issue in the future :-(

Any suggestions?

Comments

Distributed File DownLoad

el777 commented 20 October 2007 at 13:06

Hi!

I have the same problem and I want to create module for distributed file download - so you may keep your site in one place and store files on different servers. Even each user will be able to get the link to file placed on the nearest mirror.

I think such solution for this problem - each node contains several files, when node get's published all it's files are going distributed to all servers in your list. Later, whe you create url for the file just call a function which return url with required host.

Unfortunately I have no required expirience.
If you are interested in this module may be we may create it togather?

Yours
Eugene

You can also mount the

deadlyromio commented 22 October 2007 at 05:48

You can also mount the remote drives so they show up as an existing drive on your server...your hosting provider should be able to do this very easily...there's tons of docs on how to do this

Nice idea. But there's some

el777 commented 22 October 2007 at 14:07

Nice idea.
But there's some troubles - how to make several mirrors and not all hosters will give NFS access to the server.

Eugene

He has his own boxes so he

deadlyromio commented 23 October 2007 at 01:30

He has his own boxes so he would have access to root where to change NFS access permissions...

He didn't ask about mirroring but he can always use lvs if he needs to do some clustering

I also have my own box for

el777 commented 26 October 2007 at 20:57

I also have my own box for central server, but I don't want to buy 10 boxes in different countries. I think it's easier to use one central fully customizabe server (to store and convert info) and several satellites to distribute the files for visitors. They must be as simple as possible - best solutions is dummy hosting with ftp upload and http download with flv streaming. These hostings will be very chip and easy to maintain.