To restrict robot access to non-HTML documents like PDF information, you need to use the x-robots tag in the HTTP Header. I do not think that's something anyone can really even know and is extremely dependent upon the particular use case. https://okko.pixxxel.ru/