DISQUS

tm · 3 years ago

You had better support TTL in rss feeds because mine are only to be loaded once a day.

Nicolas Lehuen · 3 years ago

tm, TTL support is coming up in the next version of our crawler. We are of course interested in reducing our side of the bandwidth as much as your side.

In any case, please note that if your server supports conditional GETs (most of them do if your RSS file is a regularly generated static file), then we should be pretty much painless - and if you don't you'll only get 24 requests a day in the current instance of the crawler, instead of 200 000 requests from our users. Could you please send me in a private e-mail the adress of your RSS feed, so that I can check what we do with it ?

Regards,
Nicolas

Bananeweizen · 3 years ago

Hi. Today your bot crashed our web server by submitting around 5000 requests for the same feed (with different order of parameters and with the parameters encoded multiple times). And it's the recent changes feed from a standard MediaWiki installation, so I hope that you fix this bug really fast.
Additionally I hope that your bot respects the robots.txt, as I don't want to have it index our page again.

Ciao, Michael.

Rakesh Choudhary · 2 years ago

We are impressed with your concept and
wish you with the warmest regards to continue
doing this great service .

Always willing to support your NOBLE WORK .

http://blogs.mindbodynsoul.com

autoverzekering · 2 years ago

Yeah I'm getting more and more impressed as well as I find out more and more each second :D

Adriana · 2 years ago

No more spam, man!

Adrianne · 2 years ago

Spammers suck a lot

Agnes · 2 years ago

Not bad, it really can occur