Why block libwww-perl?
Any business is concerned with their websites search engine rankings. There are a number of online tools which analyse your site and give hints as to how best to improve your results. I’ve been trying some of these out for a future article and one of them recommended blocking requests to your website with the libwww-perl user-agent. (A user-agent is where a piece of software identifies itself when accessing your website).
The logic is simple enough – botnet scripts often use libwww-perl. And botnet scripts are often looking for ways to exploit your website. Thus it makes sense to block libwww-perl. Unless you host some data on your website for other scripts to collect, you’re not likely to cause any problems for legitimate visitors.
How can I block it?
You need to edit a special file called ‘.htaccess’. This file can usually be found in the root directory of your website – especially if you’ve already installed software such as WordPress or Joomla.
First, you need to make sure the following line already exists in the .htaccess file, if it doesn’t, add it:
RewriteEngine On
Then add these lines below it:
RewriteCond %{HTTP_USER_AGENT} libwww-perl.*
RewriteRule .* ? [F,L]
When someone (or, more accurately, somebot) tries to access your site with libwww-perl now, they will get a 403 Forbidden error.
cPanel Instructions
Log into cPanel and find the ‘File Manager’ option.
If you’ve not used File Manager before, once you’re on the File Manager page you’ll need to click the ‘Settings’ option and put a tick next to ‘Show Hidden Files’, then save.
Now go into the ‘public_html’ directory and you may see a file called ‘.htaccess’. If you do, click it then click ‘Edit’.
Add the following two lines below the ‘RewriteEngine On’ line, then save the file.
RewriteCond %{HTTP_USER_AGENT} libwww-perl.*
RewriteRule .* ? [F,L]
If you don’t already have a .htaccess file, from inside public_html click the ‘+ File’ button and type: .htaccess
Then add the following three lines and save the file.
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} libwww-perl.*
RewriteRule .* ? [F,L]
You can now exit file manager and you’ve got this additional protection in place.