Implement Rate Limiting

[stefankoegl]

nginx provies rate limiting functionality (https://www.nginx.com/blog/rate-limiting-nginx/).

This could be used to reduce the stress on the limited resources.

[SiqingYu]

Let us have discussions? 😊

• What Nginx version are we using on production?
• What are our purposes of rate limiting? Protect against DDoS? Password-guessing attempts? Or reduce network transfer footprint?
• What network resource should we limit? Transfer rate? The number of simultaneous connections?
• Should we analyze Nginx access logs to sort out requests as per usages and then decide where to put the limit_req directive?
• How should we present 503 (Service Temporarily Unavailable) to clients? Are the clients (AntennaPod, etc.) already prepared to handle the issue?
• What strategy of rate limiting should we adopt? e.g. considering bursts.

[stefankoegl]

What Nginx version are we using on production?

$ nginx -v
nginx version: nginx/1.10.3 (Ubuntu)

What are our purposes of rate limiting? Protect against DDoS? Password-guessing attempts? Or reduce network transfer footprint?

I have noticed (from manual inspection, no thorough analysis) that there are some misbehaving clients out there, eg requesting the same resource repeatedly many times (maybe failing to process the response and trying again). Some my main intention was to ensure the best usage of our limited resources for well-behaved clients, and to reduce the impact of misbehaving ones.

What network resource should we limit? Transfer rate? The number of simultaneous connections?

I don't think that we are limited by transfer rate. My concern is load (cpu, memory, disk) caused by "unnecessary" requests. Therefor I'd limit connections.

Should we analyze Nginx access logs to sort out requests as per usages and then decide where to put the limit_req directive?

That'd be great. I don't have much experience in that kind of analysis.

How should we present 503 (Service Temporarily Unavailable) to clients?

I don't know of any best practices here, but 503 sounds ok.

Are the clients (AntennaPod, etc.) already prepared to handle the issue?

I don't know, probably not.

What strategy of rate limiting should we adopt? e.g. considering bursts.

Again, not much experience from my side.

[SiqingYu]

I want to analyze the logs. Could you please email me the log examples? It'd be great if you also highlight the rogue client requests in the emails. Tools like goaccess are at our disposal. 😄

[stefankoegl]

@SiqingYu you should now have access to the logs directly on the servers. One of the "rogue" requests that I have in mind is repeatedly requesting /api/2/episodes/username.json?since=0 over and over again. A similar pattern can probably be found for other resources as well.

[SiqingYu]

Hi Stefan, Thanks for your trust. I'd investigate the repeated request after a while. What is the domain/IP address of the server? Regards, Ssuching Yu

…

On Mon, Feb 10, 2020 at 7:57 PM Stefan Kögl ***@***.***> wrote: @SiqingYu <https://github.com/SiqingYu> you should now have access to the logs directly on the servers. One of the "rogue" requests that I have in mind is repeatedly requesting /api/2/episodes/username.json?since=0 <https://gpoddernet.readthedocs.io/en/latest/api/reference/events.html#get-episode-actions> over and over again. A similar pattern can probably be found for other resources as well. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2?email_source=notifications&email_token=AFKB4YVSXJEZ4YRJCROUKDTRCE6K3A5CNFSM4JQ6KNKKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELIHYMQ#issuecomment-584088626>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFKB4YXNEA26LCUWW6UWFRTRCE6K3ANCNFSM4JQ6KNKA> .

-- Ssuching Yu Software Developer site: sqyu.wordpress.com email: [email protected] address: China <https://twitter.com/siqing_yu/> <https://github.com/SiqingYu> <https://www.linkedin.com/in/siqing-yu/>