TWEAK: Squid Cache + TreeWalk DNS = Faster Browsing

Having a local caching proxy server has significantly improved browsing speed; and, the local caching DNS server has improved security at the same time minimized dependence with the ISP. Both of these tweaks have boosted browsing speed. The apparent impacts of both are remarkable for sites frequently visited. This can be taken a step further by making the squid caching proxy take advantage of faster lookups by accessing the local DNS table cached by TreeWalk.

If you can recall, during the installation of TreeWalk, one of the prompts on the later parts was: "You are currently using a PROXY. If the proxy is not a local one disable it if possible, otherwise you may not take full advantage from the local DNS resolver." Good! It detected our squid caching proxy.

Since we are using a local caching proxy, we can expect to gain the full benefit of having a local caching DNS server -- just the exact end state we want.

Open the squid configuration file on your favorite editor. On the default installation settings, the config file is in C:\squid\etc\squid.conf. Add this line:

dns_nameservers 127.0.0.1

Where: 127.0.0.1 is the IP address of the TreeWalk DNS cache. If the DNS cache resides on a different machine, replace the 127.0.0.1 with the corresponding IP address of the host. For some reason, squid doesn't like a host name in this directive. So IP address it is.

Save the configuration changes restart squid via Start > Run > C:\squid\sbin\squid -k reconfigure -n Squid.

To do a quick restart of squid, you might want to add this line to squid.conf:

shutdown_lifetime 2 second

With the above tweak you will notice restart or reconfigure is quicker. And finally, for a big speed boost enable pipelining on squid. Add this line to squid.conf:

pipeline_prefetch on

You will need to reconfigure (or restart) squid after making the tweaks above. Browsing will now be even faster given the benefits of both local caches. As always, don't forget to configure your browser to use the proxy. All this will be in vain if the browser will not take advantage of it.

Read More

TWEAK: Peer Squid Caching Proxy Server

Following up with the install and configuration with another squid proxy tweak.. Setting up your own squid caching proxy server on your very own Windows machine at home is very beneficial, but there are more of those that remain unstated. If you happen to have multiple caching proxy servers set-up, you can further tweak them to share their caches. They help each other, so to speak.

That is another benefit of having a caching proxy server. Regardless of whether there are hardware limitations on memory and hard drive space, proxies can be set-up as peers. And this is not limited to just two (2) peers. You can have many peers set-up to help each other out with the caching. Fetching from local cache is faster than fetching the same information from the web, which definitely helps speed-up browsing.

It doesn't take rocket science to set this up. The same procedure in setting up a squid caching proxy server is followed. The configuration file is the pretty much the same, only tweak it a little to let squid know the peer exists. This part of the previous squid.conf is very important.

icp_port 3130
icp_query_timeout 5000

It is required for peer caches to be able to query each other. The directive icp_port 3130 tells squid to listen to port 3130 for peer queries. And icp_query_timeout of 5000 means the timeout of a peer query is 5 seconds, ample enough to wait for timeout between proxies. That is what I have in my configuration, test it out to see which applies best in your environment.

The other directive that is required for a peer cache proxy is cache_peer. It lets squid know who the peers are and which port it listens to. Further information about this directive and its options are on the squid documentation. The code to use is below:

cache_peer [peer_proxy_name or ip_address] http_port icp_port options 

Note that in the above syntax, http_port and icp_port both refer to the ports used on the peer, not on the local proxy server. It helps to simplify things by using the same ports for peer caches.

In order to see if the peer caches really query each other, you may wish to turn on access_logging. Checking the squid.conf.default file you can see that the original directive for this is:

access_log c:/squid/var/logs/access.log squid

Just replace the line "access_log none" with the above code. Then, re-configure squid to reflect the changes.

After browsing for a bit, check the log file and see if the IP address (or hostname) of the peer cache proxy appears on the log. That should be a good indication that the proxies are now helping each other out.

On this part, it is easy to miss the portion where you configure the browser. I have had many experiences of trying to troubleshoot the proxy (and wasting precious time in the process) only to find out the browser was never configured to use the proxy in the first place. Lessons learned here, so the first question I always ask is: Have you configured the browser to use the proxy?

Read More

HOW-TO: Squid Caching Proxy Server in Windows

Having a caching proxy server even at home can have a lot of benefits. For one, having a caching proxy server can make browsing faster. Data frequently used can be served by the proxy instead of fetching them from the webserver itself. This is how a caching proxy server can make browsing seem faster.

You can also save on disk space -- catch all cache for all your browsers. If you use multiple browsers, you will notice that firefox has its own disk cache, internet explorer has its own and so does chrome and all others. While having a proxy, the proxy will cache all the data and you can set each browser's disk cache to zero. Saving you disk space. I know this has little impact due to the price per gigabyte of disks now. But imagine if you have an SSD on your notebook. Would you rather waste its write cycles as browser disk cache?

Another benefit of a local caching proxy server is bandwidth savings. In the Philippines where ISPs are unregulated and there is no committed information rate (or CIR), bandwidth can vary in the so-called "peak" hours and between locations. Getting a bigger/fatter pipeline for more bandwidth doesn't justify the cost. Internet connection in the Philippines is just plain expensive. The caching proxy server can serve the requests rather than fetch the information from the webserver itself, thus saving you bandwitdh.

Given the above reasons, are you now "sold" on the idea of putting up your own caching proxy? Those are just a few benefits to mention, there are more on the list. For now, just know that squid is not just for geeks you can set it up for yourself -- even on your home network. And if you search the internet for squid, there will be a lot of links where Linux is involved. Squid is not just for Linux, it runs in your Windows desktop too.

First, download squid for Windows. You can obtain the stable version of 2.7 or play around with the experimental version 3. For my desktop, I have version 3 and installed version 2.7 on the notebook both running Windows 7.

Unpack the downloaded zip file and copy the directory \squid over to C:. You will then have C:\squid containing all folders and files from the downloaded zip file. All default configurations for squid are on this path, so simplify the install and configuration by putting it on that path.

Next, inside the C:\squid\etc directory are default files (.default extension) for squid configuration. Rename them so the .default extension is eliminated (for example: squid.conf.default --> squid.conf). Do this for the three files: squid.conf.default, mime.conf.default and cachemgr.conf.default.

Create a backup of the file squid.conf (perhaps a file named squid.conf.bak). Using your favorite text editor, open squid.conf and modify the configuration as follows. For detailed explanation on the directives, consult the squid documentation.

##--- ACLs
acl localnet src 127.0.0.1/255.255.255.255
acl localnet src 192.168.0.0/24
acl localnet src 10.0.0.0/24
acl all src all
##--- for Squid3
visible_hostname host  # change this to the hostname of the proxy server
##--- NO SNMP/HTCP
htcp_port 0
htcp_access deny all
snmp_port 0
snmp_access deny all
http_port 3128
http_access allow localnet
http_access deny all
##--- ICP (required for peer siblings)
icp_port 3130
icp_query_timeout 5000  # 5 second timeout
icp_access allow localnet
icp_access deny all
##--- CACHE
cache_mem 128 MB  # use 128MB cache memory, change as appropriate
cache_dir ufs c:/squid/var/cache 4096 12 128
pid_filename c:/temp/squid.pid
##--- NO LOGS
access_log none
cache_log none
cache_store_log none
##--- refresh_patterns
refresh_pattern ^ftp: 1440 25% 100800
refresh_pattern -i \.(gif|png|jpg|jpeg|ico|bmp)$ 1440 300% 260009 override-expire
refresh_pattern -i \.(iso|avi|wav|mp3|mp4|mpeg|swf|flv|x-flv|mpg|wma|ogg|wmv|asx|asf)$ 1440 150% 260009 override-expire
refresh_pattern -i \.(deb|rpm|exe|zip|tar|tgz|ram|rar|bin|ppt|doc|tiff|pdf|uxx)$ 1440 150% 260009 override-expire
refresh_pattern -i \.index.(html|htm)$ 1440 90% 40320 refresh-ims
refresh_pattern -i \.(html|htm|css|js)$ 1440 90% 40320 refresh-ims
refresh_pattern (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 25% 4320
##--- unlinkd.exe
unlinkd_program c:/squid/libexec/unlinkd.exe
##--- MIME
mime_table c:/squid/etc/mime.conf

The configuration above should get you started on your caching proxy server. Should there be troubles on the configuration file, enable the logs by changing the "none" to a full path to the log file. Just keep in mind that squid doesn't like spaces in the filenames. You should wrap those in quotations.

Before running squid for the very first time, create the proxy cache directories via command prompt and using the command "C:\squid\sbin\squid -z". Then install squid as a service so each time the computer reboots, it should also start in the background. Use command "C:\squid\sbin\squid -i" to accomplish this.

If the above commands do not return an error, it is time to start squid. On the same command prompt, use command "net start squid" or via GUI through Start > Control Panel > Administrative Tools > Services select Squid and start the service.

The only other task left is to configure browsers to use the proxy. Just remember that the port used here is directed by the value assigned to http_port.

Read More