Some code to throttle rapid requests to your CF server from one IP address
Note: This blog post is from 2010. Some content may be outdated--though not necessarily. Same with links and subsequent comments from myself or others. Corrections are welcome, in the comments. And I may revise the content as necessary.Some time ago I implemented some code on my own site to throttle when any single IP address (bot, spider, hacker, user) made too many requests at once. I've mentioned it occasionally and people have often asked me to share it, which I've happily done by email. Today with another request I decided to post it and of course seek any feedback.
It's a first cut. While there are couple of concerns that will come to mind for some readers, and I try to address those at the end, it does work for me and has helped improve my server's stability and reliability, and it's been used by many others.
Update in 2020: I have changed the 503 status code below to 429, as that has become the norm for such throttles. I had acknowledged it as an option originally. I just want to change it now, in case someone just grabs the code and doesn't read it all or the comments. Speaking of comments, do see the discussion below with thoughts from others, especially from James Moberg who created his own variant addressing some concerns, as offered on github, and the conversation that followed about that, including yet another later variant.
Update in 2021: Rather than use my code, perhaps you would rather have this throttling done by your web server or another proxy. It is now a feature offered in IIS, Apache, and others. I discuss those in a new section below.
Background: do you need to care about throttling? Perhaps more than you realize