Linux System Administrator | Squid Proxy Server- Installation & Configuration

The term proxy is generally combined with two different terms: one is forward proxy and the other is reverse proxy.

When we say proxy, it generally refers to forward proxy. A forward proxy acts as a
gateway between a client’s browser and the Internet, requesting the content on behalf of
the client. This protects intranet clients by exposing the proxy as the only requester. A
proxy can also be used as a filtering agent, imposing organizational policies. As all
Internet requests go through the proxy server, the proxy can cache the response and
return cached content when a similar request is found, thus saving bandwidth and time.

A reverse proxy is the exact opposite of a forward proxy. It protects internal servers
from the outside world. A reverse proxy accepts requests from external clients and
routes them to servers behind the proxy. External clients can see a single entity serving
requests, but internally, it can be multiple servers working behind the proxy and sharing
the load.

In this recipe, we will discuss how to install a squid server. Squid is a well-known
application in the forward proxy world and works well as a caching proxy. It supports
HTTP, HTTPS, FTP, and other popular network protocols.


Benefits of a Web Proxy:

  • It can be used to accelerate the internet as a proxy can build up a cache of frequently used websites, which makes it easier & faster to load up after,
  • Can be used to block/allow websites as required,
  • also can be used to bypass another web proxy . For example in many organizations Social networking websites like Facebook, Twitter , Youtube etc are not allowed. So a web proxy can be used to bypass those restrictions & provide access to restricted websites.

Squid proxy server

Squid is a full-featured web proxy cache server application which provides proxy and cache services for Hyper Text Transport Protocol (HTTP), File Transfer Protocol (FTP), and other popular network protocols. Squid can implement caching and proxying of Secure Sockets Layer (SSL) requests and caching of Domain Name Server (DNS) lookups, and perform transparent caching. Squid also supports a wide variety of caching protocols, such as Internet Cache Protocol (ICP), the Hyper Text Caching Protocol (HTCP), the Cache Array Routing Protocol (CARP), and the Web Cache Coordination Protocol (WCCP).

The Squid proxy cache server is an excellent solution to a variety of proxy and caching server needs, and scales from the branch office to enterprise level networks while providing extensive, granular access control mechanisms, and monitoring of critical parameters via the Simple Network Management Protocol (SNMP). When selecting a computer system for use as a dedicated Squid caching proxy server for many users ensure it is configured with a large amount of physical memory as Squid maintains an in-memory cache for increased performance.


Scenario Setup

Firstly, to test or create a squid proxy setup, we will need a squid server & a client machine.

Squid server                                                              Client’s Machine

OS : Ubuntu  server  (VM)                                      OS: Ubuntu Desktop

Configuration file       /etc/squid/squid.conf

Default port                 3128


Installation

At a terminal prompt, enter the following command to install the Squid server:

sudo apt install squid
squid3 -v

Configuration

ACL rules are need to add in squid configuration file /etc/squid/squid.conf. Remember that squid Squid always applied first matching rules from top to down order and ignore other after matching any rule

At first we will setup with very simple configuration changes and test it.

We need to create an ACL rule (Access Control List), which is the list or rule with list of access control entries.Some acl rules are already written in configuration file by default in the configuration file,

Prior to editing the configuration file, you should make a copy of the original file and protect it from writing so you will have the original settings as a reference, and to re-use as necessary. Make this copy and protect it from writing using the following commands:

sudo cp /etc/squid/squid.conf /etc/squid/squid.conf.original
sudo chmod a-w /etc/squid/squid.conf.original

now edit the config file

sudo vim /etc/squid/squid.conf

uncomment below 2 lines

acl localhost src 127.0.0.1
http_access allow localhost

So, this is what an acl rule look like. Lets see what these means,

firstly,acl this is declaring that a new acl is starting

then,localhost is the name of acl created

src is used in case acl is for local Ipadress , srcdomain is used for declaring Localdomain, dst for public IP & dstdomain for publlic domain name

and lastly,127.0.01 declares the IP Address on which the acl is to be applied, in this case its localhost or 127.0.0.1

Next line i.e. http_access allow localhost, means
http_access will initiate an action based on next word
allow/deny will either allow or deny access
and,localhost again is the name of acl as declared above.

So, basically that how we create a ACL/rule in squid proxy server.

Now, lets restart our server (with default config file) & configure the client machine to see if proxy is working properly.

sudo systemctl restart squid.service

OR

service squid restart

Configuration on Client Side

Before making any change please check what was your default setting

Open Firefox Browser &

  • Open Edit menu —> Preferences —> Advanced —-> Settings
  • Check the box ‘ Manual proxy configuration’ & enter IP Address & Port Number of squid proxy server.
  • And also mark  the Use this proxy server for all protocols option as show in below screenshot.

proxysetting.png

  • Click OK

& that’s all we need to configure on Client’s side.

Then we check out if its works. Open a website (example aol.com), if proxy server is working properly you will be greeted with an error ‘ Access Denied’. That’s because by default internet access is denied for all in server.

block.png

Now, lets check logs in server, to see if a request was received by proxy server or not,

tail -f /var/log/squid/access.log

and it should show you all the received requests from client to server.

Restricting access to websites

In order to restrict access to a website, open configuration file

vim /etc/squid/squid.conf

Block Specific Website with Squid

Let’s start with the additional configuration like blocking any website using squid proxy server. Add below rules to block specific website before any allow all rules. Below example will block yahoo.com and http://www.rediff.com.

acl blocksite1 dstdomain yahoo.com 
acl blocksite2 dstdomain www.rediff.com 
http_access deny blocksite1 
http_access deny blocksite2 
http_access allow all

Note Also set http_access deny all to http_access allow all , otherwise we wont be able to access internet.

Now, restart your squid proxy server to apply changes or we can also use squid -k reconfigure to implement changes to server without restarting the server.

squid -k reconfigure

then, we will access client’s machine and open yahoo.com/rediff.com but you wont be able to access it at all. As for other websites you can access them just fine.

If you have a long list of domain names, Create a file /etc/squid/blockwebsites.lst and put domain names one per line and add below rule in squid configuration file.

acl blocksitelist dstdomain "/etc/squid/blockwebsites.lst"
http_access deny blocksitelist
http_access allow all

blockwebsites.lst file content example:

# cat /etc/squid/blockwebsites.lst
yahoo.com
www.rediff.com

Block Specific Keyword with Squid

Add below rules to block specific website before any allow all rules. Below example will block all pages having keyword yahoo or Gmail.

acl blockkeyword1 url_regex yahoo
acl blockkeyword2 url_regex gmail
http_access deny blockkeyword1
http_access deny blockkeyword2
http_access allow all

If you have a long list of keywords, Create a file /etc/squid/blockkeywords.lst and put keywords one per line and add below rule in squid configuration file.

acl blockkeywordlist url_regex "/etc/squid/blockkeywords.lst"
http_access deny blockkeywordlist
http_access allow all

blockkeywords.lst file content example:

# cat /etc/squid/blockkeywords.lst
yahoo
gmail
facebook

Block All Sites For Single MAC Address

Following configuration will block all the sites to system having MAC address 01:23:45:AB:CD:EF.

Squid ACL Rule:

acl pcmac1 arp 01:23:45:AB:CD:EF
http_access deny pcmac1

Block Single Site for Single MAC Address

Following configuration will block http://www.example.com site to system having MAC address 01:23:45:AB:CD:EF.

Squid ACL Rule:

acl blocksite1 dstdomain www.example.com
acl pcmac1 arp 01:23:45:AB:CD:EF
http_access deny blocksite1 pcmac1

Block All Sites for Multiple MAC Addresses

Following configuration will block all the sites to systems having MAC addresses 01:23:45:AB:CD:EF and AB:CD:EF:01:23:45.

MAC Addresses List

# cat /etc/squid/mac-addrs.lst
01:23:45:AB:CD:EF
AB:CD:EF:01:23:45

Squid ACL Rule:

acl pcmacs arp "/etc/squid/mac-addrs.lst"
http_access deny pcmacs

Block Single Site for Multiple MAC Addresses

Following configuration will block http://www.example.com to systems having MAC addresses 01:23:45:AB:CD:EF and AB:CD:EF:01:23:45.

MAC Addresses List

# cat /etc/squid/mac-addrs.lst
01:23:45:AB:CD:EF
AB:CD:EF:01:23:45

Squid ACL Rule:

acl blocksite1 dstdomain www.example.com
acl pcmacs arp "/etc/squid/mac-addrs.lst"
http_access deny blocksite1 pcmacs

Allow Specific Site for Single MAC Address

Following configuration will allow http://www.example.com to system having MAC address 01:23:45:AB:CD:EF and deny other sites.
Squid ACL Rule:

acl pcmac1 arp 01:23:45:AB:CD:EF
acl allowsite1 dstdomain www.example.in
http_access allow allowsite1 pcmac1
http_access deny pcmac1

Allow Multiple Sites for Single MAC Address

Following configuration will allow all sites added in /etc/squid/allowsites.lst to system having MAC address 01:23:45:AB:CD:EF and deny other sites.

Allowed Sites List

# cat /etc/squid/allowsites.lst
www.google.co.in
yahoo.com
in.yahoo.com

Squid ACL Rule:

acl pcmac1 arp 01:23:45:AB:CD:EF
acl allowsite1 dstdomain "/etc/squid/allowsites.lst"
http_access allow allowsite1 pcmac1
http_access deny pcmac1

Allow Specific Site for Multiple MAC Addresses

Following configuration will allow http://www.example.com to systems having MAC address 01:23:45:AB:CD:EF and and AB:CD:EF:01:23:45 and deny other sites.

MAC Addresses List

# cat /etc/squid/mac-addrs.lst
01:23:45:AB:CD:EF
AB:CD:EF:01:23:45

Squid ACL Rule:

acl blocksite1 dstdomain www.example.com
acl pcmacs arp "/etc/squid/mac-addrs.lst"
http_access allow blocksite1 pcmacs
http_access deny pcmacs

Allow Multiple Sites for Multiple MAC Addresses

Following configuration will allow all the sites listed in /etc/squid/allowsites.lst to all systems having MAC address listed in /etc/squid/mac-addrs.lst and deny other sites.

MAC Addresses List

# cat /etc/squid/mac-addrs.lst
01:23:45:AB:CD:EF
AB:CD:EF:01:23:45

Allowed Sites List

# cat /etc/squid/allowsites.lst
www.google.co.in
yahoo.com
in.yahoo.com

Squid ACL Rule:

acl pcmacs arp "/etc/squid/mac-addrs.lst"
acl allowsites dstdomain "/etc/squid/allowsites.lst"
http_access allow allowsites pcmacs
http_access deny pcmacs

To set your Squid server to listen on TCP port 8888 instead of the default TCP port 3128, change the http_port directive as such:

http_port 8888

Change the visible_hostname directive in order to give the Squid server a specific hostname. This hostname does not necessarily need to be the computer’s hostname. In this example it is set to weezie

visible_hostname  rupin-squid-server

squid-hostname


Using Squid’s access control, you may configure use of Internet services proxied by Squid to be available only users with certain Internet Protocol (IP) addresses. For example, we will illustrate access by users of the 192.168.42.0/24 subnetwork only:

Add the following to the bottom of the ACL section of your /etc/squid/squid.conf file:

acl fortytwo_network src 192.168.42.0/24

Then, add the following to the top of the http_access section of your /etc/squid/squid.conf file:

http_access allow fortytwo_network

Time based acl

Using the excellent access control features of Squid, you may configure use of Internet services proxied by Squid to be available only during normal business hours. For example, we’ll illustrate access by employees of a business which is operating between 9:00AM and 5:00PM, Monday through Friday, and which uses the 10.1.42.0/24 subnetwork:

Add the following to the bottom of the ACL section of your /etc/squid/squid.conf file:

acl biz_network src 10.1.42.0/24
acl biz_hours time M T W T F 9:00-17:00

Then, add the following to the top of the http_access section of your /etc/squid/squid.conf file:

http_access allow biz_network biz_hours

After making changes to the /etc/squid/squid.conf file, save the file and restart the squid server application to effect the changes using the following command entered at a terminal prompt:

sudo systemctl restart squid.service

Enabling cache to speed up browsing

At this point, Squid is running, but it is not actually caching. A google search of “squid not caching” will show that this is not an uncommon problem. The first reason for this is that the cache directory is not defined or built.

To enable cache , open configuration file

vi /etc/squid/squid.conf

and add uncomment below line.

cache_dir ufs /var/cache/squid 2000 16 256

where ufs is squid storage format,

/var/cache/squid is path for cache storage,

2000 is size in MB can be used for cache,and, 16 is number of 1st level sub-directories & 256 is 2nd level sub directories in cache folder.

In /etc/squid3/squid.conf it’s uncomment; (Note that this is for a 100 meg cache.)

cache_dir ufs /var/spool/squid3 100 16 256

In /etc/squid3/squid.conf uncomment; (Note that this is for a 100 meg cache.)

cache_dir ufs /var/spool/squid3 100 16 256

Another problem is that not everything on the web has cache friendly expire tags, and the refresh patterns by default are somewhat conservative. Adding this line to /etc/squid3/squid.conf can help.

refresh_pattern -i \.(gif|png|jpg|jpeg|ico)$ 3600       90%     43200

This will force squid to cache all the images listed above for at least and hour, and up to half a day.

After it has been running for a while you can see if it is cacheing with this command;

#sudo du -sh /var/spool/squid3
19M     /var/spool/squid3

To flush the Squid cache

Squid caches the web pages it serves. If the cache expiration time of the web page is specified, Squid seems to obey it. If you are serving some of the pages, update your page in the Apache directory, Squid won’t fetch the new version, instead relying on its cached version until its default expiration time. To flush the Squid cache

sudo /etc/init.d/squid3 stop
sudo rm -fr /var/spool/squid3/*
sudo squid3 -z
sudo /etc/init.d/squid3 start

SARG ( Squid Analysis Report Generator )- Installation & Configuration

It is an Open-Source tool, which helps us analyze Squid Proxy logs & generates reports in HTML format with all the information from logs presented in nice & easy to understand format  & it gives information about User’s IP addresses , total & individually used bandwidth etc with access to Daily, Weekly & Monthly reports.

 Installing Sarg from Source

On Debian based distributions, sarg package can be easily install from the default repositories using apt-get package manager.

$ sudo apt-get install sarg

Configuring Sarg

root@ubuntu:~# cp /etc/sarg/sarg.conf /etc/sarg/sarg.conf.original 
root@ubuntu:~# chmod a-w /etc/sarg/sarg.conf.original

Now it’s time to edit some parameters in SARG main configuration file. The file contains lots of options to edit, but we will only edit required parameters like:

  1. Access logs path
  2. Output directory
  3. Date Format
  4. Overwrite report for the same date.

Open sarg.conf file with your choice of editor and make changes as shown below.

$ sudo vim /etc/sarg/sarg.conf

Now Uncomment and add the original path to your squid access log file.

access_log /var/log/squid/access.log

Next, add the correct Output directory path to save the generate squid reports in that directory. Please note, under Debian based distributions the Apache web root directory is ‘/var/www‘. So, please be careful while adding correct web root paths under your Linux distributions.

output_dir /var/www/html/squid-reports

Set the correct date format for reports. For example, ‘date_format e‘ will display reports in ‘dd/mm/yy‘ format.

date_format e

Next, uncomment and set Overwrite report to ‘Yes’.

overwrite_report yes

That’s it! Save and close the file.

Generating Sarg Report

Once, you’ve done with the configuration part, it’s time to generate the squid log report using the following command.

root@ubuntu:~# sudo sarg -x   
SARG: Init 
SARG: Loading configuration from /etc/sarg/sarg.conf 
SARG: Unknown option resolve_ip 
SARG: Loading exclude host file from: /etc/sarg/exclude_hosts 
SARG: Loading exclude file from: /etc/sarg/exclude_users 
SARG: Parameters: 
SARG:           Hostname or IP address (-a) =  
SARG:                    Useragent log (-b) =  
SARG:                     Exclude file (-c) = /etc/sarg/exclude_hosts 
SARG:                  Date from-until (-d) =  
SARG:    Email address to send reports (-e) =  
SARG:                      Config file (-f) = /etc/sarg/sarg.conf 
SARG:                      Date format (-g) = Europe (dd/mm/yyyy) 
SARG:                        IP report (-i) = No 
SARG:             Keep temporary files (-k) = No 
SARG:                        Input log (-l) = /var/log/squid/access.log 
SARG:               Resolve IP Address (-n) = No 
SARG:                       Output dir (-o) = /var/www/html/squid-reports/ 
SARG: Use Ip Address instead of userid (-p) = No 
SARG:                    Accessed site (-s) =  
SARG:                             Time (-t) =  
SARG:                             User (-u) =  
SARG:                    Temporary dir (-w) = /tmp/sarg 
SARG:                   Debug messages (-x) = Yes 
SARG:                 Process messages (-z) = No 
SARG:  Previous reports to keep (--lastlog) = 0 
SARG:  
SARG: SARG version: 2.3.10 Apr-12-2015 
SARG: Loading User table: /etc/sarg/usertab 
SARG: Reading access log file: /var/log/squid/access.log 
SARG:    Records read: 1349, written: 1349, excluded: 0 
SARG: Squid log format 
SARG: Period: 02 May 2018-03 May 2018 
SARG: Sorting log /tmp/sarg/192_168_43_114.user_unsort 
SARG: Making file /tmp/sarg/192_168_43_114 
SARG: Sorting file "/tmp/sarg/192_168_43_114.utmp" 
SARG: Making report 192.168.43.114 
SARG: Making index.html 
SARG: Purging temporary file sarg-general 
SARG: End 
root@ubuntu:~#

Note: The ‘sarg -x’ command will read the ‘sarg.conf‘ configuration file and takes the squid ‘access.log‘ path and generates a report in html format.

Assessing Sarg Report

The generated reports placed under ‘/var/www/html/squid-reports/‘ or ‘/var/www/squid-reports/‘ which can be accessed from the web browser using the address.

http://localhost/squid-reports
OR
http://ip-address/squid-reports

Note:- Make sure apache is running.

sarg.png


Automatic Generating Sarg Report

To automate the process of generating sarg report in given span of time via cron jobs. For example, let’s assume you want to generate reports on hourly basis automatically, to do this, you need to configure a Cron job.

# crontab -e

Next, add the following line at the bottom of the file. Save and close it.

* */1 * * * /usr/local/bin/sarg -x

The above Cron rule will generate SARG report every 1 hour.


SETUP SQUID AUTHENTICATION

For fine control you may need to use Squid proxy server authentication. This will only allow authorized users to use proxy server.
You need to use proxy_auth ACLs to configure ncsa_auth module. Browsers send the user’s authentication in the Authorization request header. If Squid gets a request and the http_access rule list gets to a proxy_auth ACL, Squid looks for the Authorization header. If the header is present, Squid decodes it and extracts a username and password.
However squid is not equipped with password authentication. You need to take help of authentication helpers. Following are included by default in most squid and most Linux distros:
=> NCSA: Uses an NCSA-style username and password file.
=> LDAP: Uses the Lightweight Directory Access Protocol
=> MSNT: Uses a Windows NT authentication domain.
=> PAM: Uses the Linux Pluggable Authentication Modules scheme.
=> SMB: Uses a SMB server like Windows NT or Samba.
=> getpwam: Uses the old-fashioned Unix password file.
=> SASL: Uses SALS libraries.
=> NTLM, Negotiate and Digest authentication

CREATE A USERNAME/PASSWORD

First create a NCSA password file using htpasswd command. htpasswd is used to create and update the flat-files used to store usernames and password for basic authentication of squid users.

# htpasswd -c /etc/squid/passwd rupin

Output:

root@ubuntu:~# htpasswd -c /etc/squid/passwd rupin 
New password:  
Re-type new password:  
Adding password for user rupin 
root@ubuntu:~#

Make sure squid can read passwd file:

# chmod o+r /etc/squid/passwd

LOCATE NSCA_AUTH AUTHENTICATION HELPER

Usually nsca_auth is located at /usr/lib/squid/ncsa_auth. You can find out location using rpm (Redhat,CentOS,Fedora) or dpkg (Debian and Ubuntu) command:
# dpkg -L squid | grep ncsa_auth
Output:

/usr/lib/squid/ncsa_auth

CONFIGURE NSCA_AUTH FOR SQUID PROXY AUTHENTICATION

auth_param basic program /usr/lib/squid/basic_ncsa_auth /etc/squid/passwd 
auth_param basic children 5 
auth_param basic realm Squid proxy-caching web server 
auth_param basic credentialsttl 2 hours 
auth_param basic casesensitive on 
acl ncsa_users proxy_auth REQUIRED 
http_access allow ncsa_users

Where,

  • auth_param basic program /usr/lib/squid/ncsa_auth /etc/squid/passwd : Specify squid password file and helper program location
  • auth_param basic children 5 : The number of authenticator processes to spawn.
  • auth_param basic realm Squid proxy-caching web server : Part of the text the user will see when prompted their username and password
  • auth_param basic credentialsttl 2 hours : Specifies how long squid assumes an externally validated username:password pair is valid for – in other words how often the helper program is called for that user with password prompt. It is set to 2 hours.
  • auth_param basic casesensitive off : Specifies if usernames are case sensitive. It can be on or off only
  • acl ncsa_users proxy_auth REQUIRED : The REQURIED term means that any authenticated user will match the ACL named ncsa_users
  • http_access allow ncsa_users : Allow proxy access only if user is successfully authenticated.

Restart squid:
# /etc/init.d/squid restart


 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s