Make cached squid mediawiki fill all pages

Question

To speed up the creation of a MediaWiki site that has content that uses many templates but otherwise has pretty much static content when the templates have completed their tasks, I would like to set up a squid server.

https://www.mediawiki.org/wiki/Manual:PurgeList.php

and

https://www.mediawiki.org/wiki/Manual:Squid_caching

and then populate the squid cache server “automatically” using the script to make wget / curl calls that go to all pages of Mediawiki. My guess would be that after this procedure, each individual page is in the squid cache (if I make it big enough), and then every access will be done using squid.

How can I make this work? For instance:.

• How to check the configuration?
• How to find out how much memory is needed?
• How can I check that the pages are in squid3 cache?

What have i tried so far

I started by figuring out how to install squid using:

• https://wiki.ubuntuusers.de/squid

and

• https://www.mediawiki.org/wiki/Manual:Squid_caching

I found out my IP address xx.xxx.xxx.xxx (not disclosed here) via ifconfig eth0

in / etc / squid3 / squid.conf I put

http port xx.xxx.xxx.xxx:80 transparent vhost defaultsite=XXXXXX cache_peer 127.0.0.1 parent 80 3130 originserver acl manager proto cache_object acl localhost src 127.0.0.1/32 # Allow access to the web ports acl web_ports port 80 http_access allow web_ports # Allow cachemgr access from localhost only for maintenance purposes http_access allow manager localhost http_access deny manager # Allow cache purge requests from MediaWiki/localhost only acl purge method PURGE http_access allow purge localhost http_access deny purge # And finally deny all other access to this proxy http_access deny all 

Then I set up apache2 server

 # /etc/apache2/sites-enabled/000-default.conf Listen 127.0.0.1:80 

I added

 $wgUseSquid = true; $wgSquidServers = array('xx.xxx.xxx.xxx'); $wgSquidServersNoPurge = array('127.0.0.1'); 

for my LocalSettings.php

Then I restarted apache2 and started squid3 with

 service squid3 restart 

and made the first attempt to access using

 wget --cache=off -r http://XXXXXX/mediawiki 

result:

 Resolving XXXXXXX (XXXXXXX)... xx.xxx.xxx.xxx Connecting to XXXXXXX (XXXXXXX|xx.xxx.xx.xxx|:80... failed: Connection refused. 

Answer 1

Assuming Apache 2.x.

While Squid is unrelated, you can achieve this using only Apache modules. Check out mod_cache here: https://httpd.apache.org/docs/2.2/mod/mod_cache.html

You can simply add this to your Apache configuration and ask Apache to cache the contents of the displayed content.

You need to make sure that your content has relevant information about the expiration of the cache in the received PHP response, MediaWiki should take care of this for you.

Adding such a cache level may not have the desired result, since this level does not know if the page has changed, managing the cache here is difficult and should only be used for actual static content.

Ubuntu:

 a2enmod cache cache_disk 

Apache configuration:

 CacheRoot /var/cache/apache2/mod_disk_cache CacheEnable disk / 

I would not recommend pre-populating your cache by accessing each page. This will lead to the fact that inactive (not often used) pages will occupy valuable space / memory. If you still want to do this, you can look at wget:

 Description from: http://www.linuxjournal.com/content/downloading-entire-web-site-wget $ wget \ --recursive \ --no-clobber \ --page-requisites \ --html-extension \ --convert-links \ --restrict-file-names=windows \ --domains website.org \ --no-parent \ www.website.org/tutorials/html/ This command downloads the Web site www.website.org/tutorials/html/. The options are: --recursive: download the entire Web site. --domains website.org: don't follow links outside website.org. --no-parent: don't follow links outside the directory tutorials/html/. --page-requisites: get all the elements that compose the page (images, CSS and so on). --html-extension: save files with the .html extension. --convert-links: convert links so that they work locally, off-line. --restrict-file-names=windows: modify filenames so that they will work in Windows as well. --no-clobber: don't overwrite any existing files (used in case the download is interrupted and resumed). 

Best Option: Memcached

MediaWiki also supports the use of Memcached as a very fast in-memory caching service for data and templates only. It's not as cruel as a cache site like Squid or Apache mod_cache. MediaWiki will manage Memcached so that any changes are immediately reflected in the cache storage, which means that your content will always be valid.

Please see MediaWiki installation instructions here: https://www.mediawiki.org/wiki/Memcached

My recommendation is not to use Apache mod_cache or Squid for this task, but instead install Memcached and configure MediaWiki to use it.