Nie jesteś zalogowany na forum.
Strony: 1
Over the previous few months, as an alternative of engaged on our priorities at SourceHut, I have spent anywhere from 20-100% of my time in any given week mitigating hyper-aggressive LLM crawlers at scale. 4 to stats, but I want to do this to get that from this man to do this and run over there and do all this shit, it’s the identical precise shit that I’d be doing in my regular life if I was working or writing music. These bots crawl all the things they can discover, robots.txt be damned, together with costly endpoints like git blame, each page of every git log, and each commit in every repo, they usually accomplish that using random Person-Agents that overlap with end-users and are available from tens of hundreds of IP addresses - mostly residential, in unrelated subnets, each making no a couple of HTTP request over any time period we tried to measure - actively and maliciously adapting and mixing in with end-user visitors and avoiding attempts to characterize their habits or block their site visitors.
Here is my blog - big cock
Here is my website ... ebony sex
Offline
Strony: 1