I don't want that my site get indexed by search engines like google and so on. I already tried whats mentioned in the docs (https://nextjs.org/learn/seo/crawling-and-indexing) and followed along this advice on a reddit post (https://www.reddit.com/r/nextjs/comments/tnbbk0/deploy_to_vercel_but_prevent_search_engine/), but none of this helped. So can any help me this?
#How to prevent search engine indexing?
8 messages · Page 1 of 1 (latest)
The reddit post you linked tells you exactly how. Add a disallow on robots.txt.
However, its not something that happens instantly.
You have to wait for it to propagate and take effect.
You can probably remove it at https://search.google.com/search-console
As well, for it to happen faster.
<meta name="robots" content="noindex, nofollow" /> you can add this as well
So is it correct to put following statement
# Block all crawlers User-agent: * Disallow: /
in my robot.txt file wich lives in the /public directory?