fighting download managers

Fix it!!

fighting download managers

Postby nixus » Mon Feb 19, 2007 4:10 pm

Hello everyone!

I am trying to fight download managers for quite a long time but without big success. Let me explain.

I have a site serving large files (approx 400mb each). It is located in shared hosting environment and have quite a big xfer bandwith so it's not a problem in general.

The problem is abusers with download accelerators sending 100-120 requests simultaneously and making server go crazy and hang up finally.

Just one note - I cannot make a PHP download script, since it hangs for a long time and gets killed by process monitor at this shared environment. This process monitor supposes it's a non-friendly php.cgi process and kills it. So the only way is to link directly to a file so it is served by Apache.

There are several things I tried to do:

1) Use mod_rewrite to block bad HTTP_USER_AGENTS. A little help, since most of these abusers nowadays mask as real browsers. Forgot about it.

2) Limit number of simultaneous connections to 1-2 per IP: unforuntately mod_limitipconn cannot be installed in this environment. Forgot about it.

3) Was trying to figure out if it is possible to block HTTP 206 requests (partial data) without modifying Apache (for example via RewriteCond %{HTTP_REQUEST)). It seems it's not possible, or is it?? Anyone can help here??

4) Created a small php script (which can be included in the top of any other php script) that keeps track of recent requests and if someone requesting too often (e.g. once every 3 seconds) blocks the abuser by not proceeding further and issue some "Warning". Unfortunately I cannot link this script to a file i am serving, since it becomes a download script again, however as i said the only way is to link to a binary file directly.


What I though is that maybe there is a way to do a tricky thing with mod_rewrite and still have this small protection script ran before sending a large file to a user but keeping it transparent? E.g. request goes to /downloads/largefile.zip however it's a PHP file which first does some checking and then if it's allright redirects to an actual file but without changing the filename for a browser??


Any advice appreciated!


Thanks,
Nick
nixus
 
Posts: 3
Joined: Mon Feb 19, 2007 3:56 pm

Postby richardk » Tue Feb 20, 2007 10:34 am

3) Was trying to figure out if it is possible to block HTTP 206 requests (partial data) without modifying Apache (for example via RewriteCond %{HTTP_REQUEST)). It seems it's not possible, or is it?? Anyone can help here??

206 is returned be Apache. You might be able to check if one of the range headers (rfc2616 section 14.35) is present. Try
Code: Select all
RewriteCond %{ENV:byte-ranges-specifier} !^$
RewriteRule ^downloads/.+$ - [F,L]


What I though is that maybe there is a way to do a tricky thing with mod_rewrite and still have this small protection script ran before sending a large file to a user but keeping it transparent? E.g. request goes to /downloads/largefile.zip however it's a PHP file which first does some checking and then if it's allright redirects to an actual file but without changing the filename for a browser??

You can pass full control to PHP, and have PHP send the file (don't know how well that would work for big files), but without a RewriteMap (to use one you need access to the httpd.conf file) you cannot run PHP then pass control back to Apache.

Or, you could try sending all requests to PHP, then if you want to allow them, create a file of their IP address then redirect them to the same place. If mod_rewrite finds the file, it won't pass control to PHP. Every time the PHP script is run, check the person hasn't got a file already (that's too young) and delete all old files.
richardk
 
Posts: 8800
Joined: Wed Dec 21, 2005 7:50 am

Postby nixus » Tue Feb 20, 2007 12:51 pm

Thanks for reply!

So I guess the only way (and atually one of the best!) would be to try to block partial data requests.

Unfortunately this

RewriteCond %{ENV:byte-ranges-specifier} !^$
RewriteRule ^downloads/.+$ - [F,L]

did not help, download accelerators are not blocked. How can I see whole ENV variable? Any more hints? I am ready to dig :)


Nick
nixus
 
Posts: 3
Joined: Mon Feb 19, 2007 3:56 pm

Postby richardk » Tue Feb 20, 2007 1:36 pm

Unfortunately this did not help, download accelerators are not blocked.

Try HTTP instead of ENV.

How can I see whole ENV variable?

You can't. You can't see the whole of any of them.

Any more hints?

Find out what's unique about a partial data request (the headers). And use %{HTTP:...} (or other things from the docs) to make something that'll match it.
richardk
 
Posts: 8800
Joined: Wed Dec 21, 2005 7:50 am

Postby nixus » Wed Feb 21, 2007 2:12 am

Thanks alot!

The correct one was

RewriteCond %{HTTP:Range} !^$
nixus
 
Posts: 3
Joined: Mon Feb 19, 2007 3:56 pm


Return to Security with Mod_Rewrite

Who is online

Users browsing this forum: No registered users and 18 guests

cron