How to Configure Joomla's robots.txt File
Find the file at the web root
Joomla includes a standard robots.txt file intended for the root of the domain or subdomain. Crawlers look for /robots.txt at that root; a robots.txt file placed only inside a Joomla subdirectory is not the site's valid robots file. If Joomla itself lives in a subdirectory, the rules must reflect that path after the file is placed at the domain root.
Start from Joomla's standard rules
Review the current Joomla robots.txt supplied with your installation before replacing it with an old copy from another Joomla version. Joomla's documented standard file disallows crawler access to internal paths such as administrator, api, cache, cli, libraries, logs, plugins, and tmp while leaving normal public content crawlable.
Use User-agent, Disallow, and Allow carefully
A User-agent line selects the crawler or crawler group affected by the following rules. Disallow identifies paths that crawler should not request; an empty Disallow does not block the site. Add only rules you understand, keep path spelling accurate, and remember that robots.txt is publicly readable.
Account for a Joomla subdirectory
If the public site is example.com/joomla/, the robots file still belongs at example.com/robots.txt. Joomla documentation notes that paths then need the installation directory prefix, such as Disallow: /joomla/administrator/. Test the public file after moves because an old prefix can block or expose the wrong paths.
Do not use robots.txt for private content
Robots.txt is a crawler instruction, not authentication. A disallowed URL can still be visited directly and may be discovered through links or other sources. Protect confidential Joomla content with access levels, authentication, or server controls rather than relying on a robots rule.
Test the published file
Open https://your-domain.example/robots.txt and confirm the server returns the intended plain-text file without an unexpected redirect, error page, or cached older copy. Review syntax with a current search-engine testing or inspection tool and check that important public URLs are not covered by broad Disallow rules.
Recheck after Joomla or routing changes
Review robots.txt after migrations, changing Joomla's installation path, adding API or custom application paths, changing domains, or replacing the file during an update. Keep a documented copy of intentional custom rules so they can be compared with the current Joomla-distributed defaults.
Need More Help with Joomla?
Still having trouble? Open a support ticket with QuantaCade Support and we'll be happy to help where we can.
Support priority is given to QuantaCade products, services, and customers. However, we're also happy to assist fellow Joomla users with general Joomla questions and troubleshooting when possible.
QuantaCade is an independent Joomla extension developer and is not official Joomla support. Some issues involving third-party extensions, hosting environments, server configurations, or other systems outside our development control may be beyond what we're able to resolve.