«

»

Feb 27 2011

Setup Pound, Varnish & Apache w/ Multiple IPs & BackEnds

The following is a tutorial in setting up Pound (SSL Wrapper) that is being proxied to Varnish (Cache) which is finally proxied to Apache. This setup also allows for multiple IP Addresses (for SSL Cert) and multiple Varnish Backends.

This article assumes you have working knowledge of Linux and CentOS distribution in particular. This installation is based upon the “yum” paths for all instances. It also assumes you have apache setup already, serving content with multiple IP Addresses and your server’s external IP Address 1 = 2.3.4.5 & IP Address 2 = 6.7.8.9
———————————————————————————————————

Pound SSL Wrapper

Pound is a reverse proxy, load balancer and SSL wrapper that allows you to listen on a secure port (443), decrypts the SSL Certificate and then sends the decrypted data to a local non-ssl port (80) to be served by the Varnish. First, let’s setup the pound configuration.

yum install pound
edit /etc/pound.cfg
[code]ListenHTTP
Address 2.3.4.5
Port 80
Service
Redirect “https://www.mydomain.com”
End
End
ListenHTTPS
Address 2.3.4.5
Port 443
Cert “/var/certs/mydomain.com.pem”
Service
BackEnd
Address 127.0.0.3
Port 80
End
End
End[/code]

In this example, ListenHTTP, we want to make sure that the entire site is served securely and redirecting 80 traffic to 443. In ListenHTTPS, we pass it the path to the cert in .pem format. Not only is this required for ListenHTTPS, but it MUST be in .pem format.

To convert a crt to a pem, run the following command (replacing files with your own)
cat mydomain.com.key mydomain.com.crt (mydomain.com.ca) > mydomain.com.pem

Varnish Caching

Varnish is a Reverse Proxy (Caching, HTTP Accelerator) that allows for Load Balancing.

yum install varnish
edit /etc/sysconfig/varnish

I used Alternative 2 for the configuration. Set the IP(s) in the -a option. While the IPs are comma separated, make sure there isn’t a space between the IP addreses. It will fail. Pound is listening for the IP Address 2.3.4.5, so we don’t need to have Varnish listen for that IP. We need to have it listen on the port that pound specified as its backend. In this case, 127.0.0.3:80. Since pound is not serving secure content for IP Addresss 6.7.8.9, we can have varnish listen for that IP.
[code]
## Alternative 2, Configuration with VCL
DAEMON_OPTS=”-a \”6.7.8.9:80,127.0.0.3:80\” \
-T localhost:6082 \
-f /etc/varnish/default.vcl \
-u varnish -g varnish \
-s file,/var/lib/varnish/varnish_storage.bin,1G”
[/code]

Next, we need to edit the default vcl file for Varnish which is where all the settings will be added.
/etc/varnish/default.vcl
[code]
# For IP Address 6.7.8.9 and the rest of the domains on the server.
backend default {
.host = “127.0.0.2”;
.port = “80”;
}

# For IP Address 2.3.4.5 listening on Pound, proxied to varnish on 127.0.0.3. Setting Apache Backend to 127.0.0.4
backend mydomain{
.host = “127.0.0.4”;
.port = “80”;
}

sub vcl_fetch {
# Add a unique header containing the cache servers IP address:
remove obj.http.X-Varnish-IP;
set obj.http.X-Varnish-IP = server.ip;
}

sub vcl_recv {
# choose a backend depending on domain
if (req.http.host ~ “www.mydomain.com”) {
set req.backend = mydomain;
}else{
set req.backend = default;
}
# remove and reset the clients IP address
remove req.http.X-Forwarded-For;
set req.http.X-Forwarded-For=client.ip;
}
[/code]

In the sub vcl_recv and sub vcl_fetch, I am doing two things. 1. Setting the backend server depending on which domain it is looking for. 2. Passing along the client’s IP Addresss to reflect in the apache log files.

SSL Conf

edit /etc/httpd/conf.d/ssl.conf
Pound is now listening on port 443, which means you can’t bind the ssl.conf to 443 as well. We need to comment out the listener.
[code]
# Listen 443
[/code]

httpd.conf

/etc/httpd/conf/httpd.conf
We need to set apache to listen and set the NameVirtualHost on the ports that we specified in /etc/varnish/default.vcl
[code]
Listen 127.0.0.2:80
Listen 127.0.0.4:80

NameVirtualHost 127.0.0.2:80
NameVirtualHost 127.0.0.4:80
[/code]

Vhosts File

/etc/httpd/sites-available/mydomain.com.conf
Finally we need to change the virtualhost IP Address in the vhost.conf to listen to the same ports as above depending on domain.
[code]

or

[/code]

If you need the client’s IP Address (I don’t know why you wouldn’t), you need to add the following into the virtualhost above. Replace with your log file paths
[code]
LogFormat “%{X-Forwarded-For}i %l %u %t \”%r\” %>s %b \”%{Referer}i\” \”%{User-Agent}i\”” varnishcombined
CustomLog /www/mydomain.com/logs/access.log varnishcombined
[/code]

Now that we have finished the setup, this will require a restart of all services for it to work. If it was setup properly, they should all say “ok”.
/etc/init.d/pound start (restart)
/etc/init.d/varnish start (restart)
/etc/init.d/httpd restart

Monitoring Varnish Stats, Logs and Histogram

Based on install paths, the following should work. From the prompt: varnishstat, varnishlog, varnishhist
Varnishstat and Varnishlog give detailed information on headers being passed and the status of the cache. Varnishhist is a histogram of the traffic coming to your site. The left side is cached content and the right side is server generated content.

Stress Testing New Setup

If you have very little traffic, you might need to use a stress test tool to find out how the server will handle your new setup. Siege is a tool that forces many concurrent connections. If you run this tool before implementing Varnish, it may but more likely will bring down your server. The load average will shoot up very quickly (to 80.00+) until the server can’t handle it and stops responding. After installing Varnish, you can run up to thousands of concurrent connections and the load average won’t go past 10.00 (depending on many factors).

The parameters being passed to siege below, -c350 (350 is the number of concurrent connections, default is 15), -t1M (1M is the total time that siege will run for, 1 minute)

From command prompt, type:
[code]
siege -c350 -t1M www.mydomain.com
[/code]
* It helps to have multiple shells up and watch “top”, “varnishstat” and “varnishhist”.

Log Files

tail -f /var/log/messages and /var/log/varnish/varnish.log

About the author

Timothy Haroutunian

Timothy Haroutunian is a ServiceNow Cloud Implementation Specialist at Acorio. ServiceNow is an IT Management solution that allows for a complete view of your IT and physical environment.

Permanent link to this article: http://www.armenianeagle.com/2011/02/27/setup-pound-varnish-apache-w-multiple-ips-backends/

2 comments

1 ping

  1. tmmonirMonir

    Can you update the script to varnish 4.0. I have added virtual host post to 8443 but when I put pound https to 443 then log says pound cannot bind 443. can you help?

  2. Ash Glover

    Hi there

    Thanks for this post!

    However, when i try to list 2 ports for varnish startup, it gives the error varnishd[12855]: Error: Unknown protocol for the second port, no matter what that is.

    Any advice?

  1. Akismet False Positives when Behind a Load Balancer | Gabriel's Blog

    […] Setup Pound, Varnish & Apache w/ Multiple IPs & BackEnds (armenianeagle.com) […]

Leave a Reply