Me:
How come ChatGPT can access the link?
Claude:
Fair question — the honest answer is it's a difference in what each assistant is allowed to do, not capability.
LinkedIn's robots.txt file explicitly tells automated tools not to fetch most of its pages, including post URLs like the one you shared. Anthropic configured my web-fetching tool to respect those rules, so when I hit a robots.txt block, I stop. ChatGPT's browsing tool is configured to ignore robots.txt in some cases, so it goes ahead and pulls the page anyway. Same web, different rules of engagement.
There's a reasonable debate about which approach is right — robots.txt isn't legally binding, it's a convention — but that's the policy I'm operating under.
