Squid supports the concept of a hierarchy of proxies. If your proxy does not have an object on disk, it's default action is to connect to the origin web server and retrieve the page. In a hierarchy, your proxy can communicate with other proxies (in the hope that one of these servers will have the relevant page). You will, obviously, only peer with servers that are 'close' to you, otherwise you would end up slowing down access. If access to the origin server is faster than access to neighboring cache servers it is not a good idea to get the page from the slower link!
Having the ability to treat other caches as siblings is very useful in some interactions. For example: if you often do business with another company, and have a permanent link to their premises, you can configure your cache to communicate with their cache. This will reduce overall latency: it's almost certainly faster to get the page from them than from the other side of the country.
When querying more than one cache, Squid does not query each in turn, and wait for a reply from the first before querying the second (since this would create a linear slowdown as you add more siblings, and if the first server stops responding, you would slow down all incoming requests). Squid thus sends all ICP queries together - without waiting for replies. Squid then puts the client's request on hold until the first positive reply from a sibling cache is received, and will retrieve the object from the fastest-replying cache server. Since the earliest returning reply packet is usually on the fastest link (and from the least loaded sibling server), your server gets the page fast.
Squid will always get the page from the fastest-responding cache - be it a parent or a sibling.
The cache_peer option allows you to specify proxy servers that
your server is to communicate with. The first line of the following
example configures Squid to query the cache machine
cache.myparent.example as a parent. Squid will communicate with
the parent on HTTP port 3128, and will use ICP to query the server using
port 3130. Configuring Squid to query more than one server is easy:
simply add another cache_peer line. The second line configures
cache.sibling.example as a sibling, listening for HTTP request on port
8080 and ICP queries on port 3130.
cache_peer cache.myparent.example parent 3128 3130
cache_peer cache.sibling.example sibling 8080 3130
If you do not wish to query any other caches, simply leave all cache_peer lines commented out: the default is to talk directly to origin servers.
Cache peering and hierarchy interactions are discussed in quite some detail in this book. In some cases hierarchy setups are the most difficult part of your cache setup process (especially in a distributed environment like a nationwide ISP). In depth discussion of hierarchies is beyond the scope of this chapter, so much more information is given in chapter 8. There are cases, where you need at least one hierarchy line to get Squid to work at all. This section covers the basics, just for those setups.
You only need to read this material if one of the following scenarios applies to you:
You have to use your Internet Service Provider's cache.
You have a firewall.
If you have to use your Internet Service Provider's cache, you will have to configure Squid to query that machine as a parent. Configuring their cache as a sibling would probably return error pages for every URL that they do not already have in their cache.
Squid will attempt to contact parent caches with ICP for each request.
This is essentially a ping. If there is no response to this
request, Squid will attempt to go direct to the origin server. since
(in this case, at least) you cannot bypass your ISP's cache, you may
want to reduce the latency added by this extra query. To do this,
place the default and no-query keywords at the end of
your cache_peer line:
cache_peer cache.myisp.example parent 3128 3130 default no-query
The default option essentially tells Squid "Go through this
cache for all requests. If it's down, return an error message to the
client: you cannot go direct".
The no-query option gets Squid to ignore the given ICP port (leaving the port number out will return an error), and never to attempt to query the cache with ICP.
Firewalls can make cache configuration hairy. Inter-cache protocols generally use packets which firewalls inherently distrust. Most caches (Squid included) use ICP, which is a layer on top of UDP. UDP is difficult to make secure, and firewall administrators generally disable it if at all possible.
It's suggested that you place your cache server on your DMZ (if you have one). There are a few advantages to this:
Your cache server is kept secure.
The firewall can be configured to hand off requests to the cache server, assuming it is capable.
You will be able to peer with other, outside, caches (like your ISP's), since DMZ networks generally have less rigid rule sets.
The remainder of this section should help you getting Squid and your firewall to co-operate. A few cases are covered for each type of firewall: the cache inside the firewall; the cache outside the firewall; and, finally, on the DMZ.
The vast majority of firewalls no nothing about ICP. If, on the other hand, your firewall does not support HTTP, it's a good time to have a serious talk to the buyer that had an all-expenses-paid weekend on the firewall supplier. Configuring the firewall to understand ICP is likely to be painful, but HTTP should be easy.
If you are using a proxy-level firewall, your client machines are probably configured to use the firewall's internal IP address as their proxy server. Your firewall could also be running in transparent mode, where it automatically picks up outgoing web requests. If you have a fair number of client machines, you may not relish the idea of reconfiguring all of them. If you fall into this category, you may wish to put your firewall on the outside (or on the DMZ) and configure the firewall to pass requests to the cache, rather than reconfiguring all client machines.
The cache is considered a trusted host, and is protected by the firewall. You will configure client machines to use the cache server in their browser proxy settings, and when a request is made, the cache server will pass the outgoing request to the firewall, treating the firewall as a parent proxy server. The firewall will then, connect to the destination server. If you have a large number of clients configured to use the firewall as their proxy server, you could get the firewall to hand-off incoming HTTP requests back into the network, to the cache server. This is less efficient though, since the cache will then have to re-pass these requests through the firewall to get to the outside, using the parent option to cache_peer. Since the latter involves traffic passing through the firewall twice, your load is very likely to increase. You should also beware of loops, with the cache server parenting to the firewall and the firewall handing-off the cache's request back to the cache!
As described in chapter 1, Squid will also send ICP queries to parents. Firewalls don't care for UDP packets, and normally log (and then discard) such packets.
When Squid does not receive a response from a configured parent, it will mark the parent as down, and proceed to go directly.
Whenever Squid is setup to use a parent that does not support ICP, the cache_peer line should include the "default" and "no-query" options. These options stop Squid from attempting to go direct when all caches are considered down, and specify that Squid is not to send ICP requests to that parent.
Here is an example config entry:
cache_peer inside.fw.address.domain parent 3128 3130 default no-query
There are only two major reasons for you to put your cache outside the firewall:
One: Although squid can be configured to do authentication, this can lead to the duplication of effort (you will encounter the "add new staff to 500 servers" syndrome). If you want to continue to authenticate users on the firewall, you will have to put your cache on the outside or on the DMZ. The firewall will thus accept requests from clients, authenticate them, and then pass them on to the cache server.
Two: Communicating with cache hierarchies is easy. The cache server can communicate with other systems using any protocol. Sibling caches, for example, are difficult to contact through a proxying firewall.
You can only place your cache outside if your firewall supports hand-offs. Browsers inside will connect to the firewall and request a URL, and the firewall will connect to the outside cache and request the page.
If you place your cache outside your firewall, you may find that your client PC's have problems connecting to internal web servers (your intranet, for example, may be unreachable). The problem is that the cache is unable to connect back through to your internal network (which is actually a good thing: don't change that). The best thing to do here is to add exclusions to your browser settings: this is described in Chapter 5 - you should specifically have a look at the section on browser autoconfig. In the meantime, let's just get Squid going, and we will configure browsers once you have a cache to talk to.
Since the cache is not protected by the firewall, it must be very carefully configured - it must only accept requests from the firewall, and must not run any strange services. If possible, you should disable telnet, and use something like SSH (Secure SHell) instead. The access control lists (which you will setup shortly) must only allow the firewall, otherwise people will be able to relay their requests through your cache, using your bandwidth.
If you place the cache outside the firewall, you client PC's will be configured to use the firewall as their proxy server (this is probably the case already). The firewall must be configured to hand-off client HTTP requests to the cache server. The cache must be configured to only allow HTTP requests when from the firewall's outside IP address. If not configured this way, other Internet users could use your cache server as a relay, using your bandwidth and hardware resources for illegitimate (and possibly illegal) purposes.
With your cache server on the outside network, you should treat the machine as a completely untrusted host, lest a cracker find a hole somewhere on the system. It is recommended that you place the cache server on a dedicated firewall network card, or on a switched ethernet port. This way, if your cache server were to be cracked, the cracker would only be able to read passing HTTP data. Since the majority of sensitive information is sent via email, this would reduce the potential for sensitive data loss.
Since your cache server only accepts requests from the firewall, there is no cache_peer line needed in the squid.conf. If you have to talk to your ISP's cache you will, of course, need one: see the section on this a bit further back.
The best place for a cache is your DMZ.
If you are concerned with the security of your cache server, and want to be able to communicate with outside cache servers (using ICP), you may want to put your cache on the DMZ.
With Squid on your DMZ, internal client PCs are setup to proxy to the firewall. The firewall is then responsible for handing-off these HTTP requests to the cache server (so the firewall in fact treats the cache server as a parent).
Since your cache server is (essentially) on the outside of the firewall, the cache doesn't need to treat the firewall as a parent or sibling: it only accepts requests from the firewall: it never passes them to the firewall.
If your cache is outside your firewall, you will need to configure your client PC's not to use the firewall as a proxy server for internal hosts. This is quite easy, and is discussed in the chapter on browser configuration.
Since the firewall is acting as a filter between your cache and the outside world, you are going to have to open up some ports on the firewall. The cache will need to be able to connect to port 80 on any machine on the outside world. Since some valid web servers will run on ports other than 80, you should consider allowing connections to any port from the cache server. In short, allow connections to:
Port 80 (for normal HTTP requests)
Port 443 (for HTTPS requests)
Ports higher than 1024 (site search engines often use high-numbered ports)
If you are going to communicate with a cache server outside the firewall, you will need even more ports opened. If you are going to communicate with ICP, you will need to allow UDP traffic from and to your cache machine on port 3130. You may find that the cache server that you are peering with uses different ports for reply packets. It's probably a bad idea to open all UDP traffic, though.
Squid will normally live on the inside of your packet-filtering firewall. If you have a DMZ, it may be best to put your cache on this network, as you may want to allow UDP traffic to and from the cache server (to communicate with other caches).
To configure your firewall correctly, you should make the minimum number of holes in your filter set. In the remainder of this section we assume that your internal machines can connect to the cache server unimpeded. If your cache is on the DMZ (or outside the firewall altogether) you will need to allow TCP connections from your internal network (on a random source port) to the HTTP port that Squid will be accepting requests on (this is the port that you set a bit earlier, in the "Setting Squid's HTTP Port" section of this chapter.
First, let's consider the firewall setup when you do not query any outside caches. On accepting a request, Squid will attempt to connect to a machine on the Internet at large. Almost always, the destination port will be the default HTTP port, port 80. A few percent of the time, however, the request will be destined for a high-numbered port (any port number higher than 1023 is a high-numbered port). Squid always sources TCP requests from a high-numbered port, so you will thus need to allow TCP requests (all HTTP is TCP-based) from a random high-numbered port to both port 80 and any high-numbered port.
There is another low-numbered port that you will probably need to open. The HTTPS port (used for secure Internet transactions) is normally listening on TCP port 443, so this should also be opened.
In the second situation, let's look at cache-peering. If you are planning to interact with other caches, you will need to open a few more ports. First, let's look at ICP. As mentioned previously, ICP is UDP-based. Almost all ICP-compliant caches listen for ICP requests on UDP port 3130. Squid will always source requests from port 3130 too, though other ICP-compliant caches may source their requests from a different port.
It's probably not a good idea to allow these UDP packets no matter what source address they come from. Your filter should probably specify the IP addresses for each of the caches that you wish to peer from, rather than allowing UDP packets from any source address. That should be it: You should now be able to save the config file, and get ready to start the Squid program.