>* - why don't you just respect existing robots.txt that apply to you already?*
>* - does every LLM scraper seriously think the onus to opt out from the EVERY SINGLE SCRAPER is on the webmasters/owners?*
I see value in asking explicitly, but let’s be real here: you know the answers. They feel entitled to vacuum up everything visible, any way they feel like, for some ratio of profit to thinking they’re reshaping humanity for the better. The opt-outs are ceremonial procedures to convince people (including themselves,) that people are actually volunteering this data. They don’t give a shit about you, your creative output, you life’s work, your server performance and bandwidth, or anything else beyond stuffing the great data pig.
should be pretty easy to test and not rely on an anonymous source from a weird analytics company via business insider. are these bots actually from openai or are they just using their user agent? are they coming from openai ip ranges? etc. https://openai.com/gptbot.json
How do we defend against your scraping, OpenAI?
I dont want any of my content scraped or seen by you all. Frankly, fuck you all for thinking my content is owned by you.