UrlMapping for static files in Grails

Question

I want to map the static sitemap.xml and robots.txt files that are in my web application directory. URLs should be as follows:

http://www.mydomain.com/sitemap.xml http://www.mydomain.com/robots.txt 

How do I configure the url so that these routes work?

Answer 1

I use this mapping for robots.txt :

 "/robots.txt" (view: "/robots") 

And then enter grails-app/views/robots.gsp which contains the content for robots.txt . That way I can use <g:if env="..."> to have different content for different environments.

For this to work for the .xml extension, you need to change the Consolidation of Content configuration.

 grails.mime.file.extensions = false // disables the parsing of file extensions from URLs into the request format 

Answer 2

The easiest way is to say that grails ignore them in UrlMappings.groovy :

 class UrlMappings { static excludes = ['/robots.txt', '/sitemap.xml'] static mappings = { // normal mappings here ... } } 

Answer 3

It may also be useful to configure nofollow for your staging environment if you use it. Not sure if there is a use case for indexing an intermediate site .... therefore, if you agree, you can use these steps to block it.

If you use Tomcat, set an environment variable such as NOFOLLOW = true -> see here, for example: TOMCAT_OPTS, environment variable and System.getEnv ()

Next, as @doelleri mentioned, install urlMappings

Urlmappings

 //Robots.txt "/robots.txt"(controller: 'robots', action:'robots') 

Then use robotsController to determine the environment variable that you set in your staging cat.

Robotscontroller

 def robots() { if (System.getenv('NOFOLLOW') == 'true') { def text = "User-agent: *\n" + "Disallow: /cgi-bin/ \n" + "Disallow: /tmp/ \n" + "Disallow: /junk/ \n" + "Disallow: /admin/ \n" + "Crawl-delay: 5 \n" + "Sitemap: https://www.example.com/sitemap.xml" render(text: text, contentType: "text/plain", encoding: "UTF-8") } else { render(status: 404, text: 'Failed to load robots.txt') } } 

robots.gsp

 <%-- Content rendered from controller -> so leave blank :) --%>