David Alger

Principal Site Reliability Engineer
Technology Enthusiast

Installing pfSense on Packet Host Cloud Servers

Posted on
#pfsense #bare-metal #packet-host

In this post, I’ll be walking you through how to install the pfSense software on bare-metal cloud servers powered by Packet Host. pfSense is a free open source network firewall distribution based on the FreeBSD operating system. Netgate (the company behind the project) offers “Security Gateway Appliances” but you can also build your own system as long as it meets the minium system requirements, or in my case deploy on cloud infrastructure. Either bare-metal servers or virtual machines with good hardware underneath them will work equally well. I’ve seen some rather impressive results running on an f1-micro instance on Google Cloud for example.

As part of my home network setup I have an SG-3100 appliance (with an AirPrime EM7455 modem chipset inside for an always on LTE PPoE connection) as my firewall and router. I’ve also had the chance to use a few of the larger rack-mount appliances as primary firewall routers and VPN gateways in some SMB networks and my experience in all cases has been pretty good, so whenever I need a reliable and rock-solid firewall I usually turn to the Security Gateway Appliances that Netgate provides. But here I need to deploy pfSense for an always on VPN connection providing a static IP address to the home network (regardless of what ISP my traffic is being carried by) and also to provide a road-warrior style VPN to which I can connect from my Mac or iPad Pro when traveling. Seeing as how pfSense supports both OpenVPN and IPSec VPN networks, it fits the bill rather nicely. Another potentially very useful scenario would be using pfSense for a transit network between cloud providers or on-prem gear. The possibilities are endless.

Packet Host has a t1.small.x86 server type that runs a very reasonable $0.07 per hour (as of this writing, with discounts available for committed usage). This baby has a 4 Core CPU (Intel Atom C2550, same one as what’s used in some of the appliances Netgate offers), 8GB RAM, SSD storage and a full 2.5Gbps network. Nice! In my case, since I’ll be routing a significant amount of traffic through this pfSense instance it’s worth noting that there are applicable bandwidth fees, but only for outgoing traffic (meaning I don’t get dinged for the traffic flowing in both directions through the server). The rate (as of this writing) is a very reasonable $0.05/GB, so in a hypothetical scenario transferring an average of 250 GB/mo the total cost for this comes out to just over $60/month. Not bad! Another really cool thing about Packet Host is they offer self-service BGP as well, so if you have them, you can bring your own IP addresses and use them for your cloud workloads (or in my case, dream of having some personal IPv4 addresses to throw around).

Installing pfSense on the Server

Alright, enough with the intro, let’s get to the the fun part. If you don’t already have an account though, please hop on over to Packet and set one up so we can get started!

  1. Go to https://www.pfsense.org/download and lookup the URL for the latest stable pfSense release. For this purpose, choose AMD64 -> USB Memstick Installer -> Serial -> New York City and then right-click the “Download” button to copy the URL of the most current stable release.

  2. Create a t1.small.x86 server using the FreeBSD 11.2 operating system. Include the following setup script in the user data field (replacing the INSTALLER_URL value with the URL obtained in step 1 above) and make sure you have a valid public key selected for use connecting to the server later on.

    This script downloads the install media and expands it onto a memory disk to facilitate setting the comconsole_port value correctly (as despite running on hardware similar to some of the Netgate appliances, pfSense does not correctly identify the hardware and for somewhat obvious reasons will then freeze on boot). Once the setting is configured, the script simply overwrites the first disk on the server followed by a reboot to launch into the pfSense installer.

    #!/usr/bin/env bash
set -ex

INSTALLER_URL="https://nyifiles.pfsense.org/mirror/downloads/pfSense-CE-memstick-serial-2.4.4-RELEASE-amd64.img.gz"

# download install media and write expanded file onto memory disk
curl -sO "${INSTALLER_URL}"
mdconfig -a -t swap -s 1g -u 1
gunzip -c "$(basename ${INSTALLER_URL})" | dd of=/dev/md1 bs=512k

# mount memory disk and configure correct comconsole_port needed for the pfsense installer to boot
mkdir /mnt/memstick
mount -t ufs /dev/md1s2a /mnt/memstick
echo 'comconsole_port="0x2F8"' >> /mnt/memstick/boot/loader.conf.local

# unmount memory disk, disable kernel protection and overwrite first disk on server
umount /mnt/memstick
sysctl kern.geom.debugflags=0x10
dd if=/dev/md1 of=/dev/ada0 bs=512k

# schedule reboot allowing cloud-init a little time to gracefully exit
shutdown -r +10s

  3. Once the server has completed building, connect to the “Out-of-Band Console” (you can find the SSH connection string for this on the server detail page). Wait for the server to completely reboot. You should be presented with a welcome message that looks something like this, followed by a license acceptance prompt.

    Welcome to pfSense!

Please choose the appropriate terminal type for your system.
Common console types are:
   ansi     Standard ANSI terminal
   vt100    VT100 or compatible terminal
   xterm    xterm terminal emulator (or compatible)
   cons25w  cons25w terminal

Console type [vt100]:

  4. Proceed through the installer steps until you get to “Partitioning” and select the “Manual Disk Setup” option. Automatic will not work here since we booted the installer from the same device we will be installing pfsense on. Here’s what you’ll need to do:

    a) Remove the ada0s1 partition (being relatively small, this is the boot loader for the install media and is not needed). All that should remain is a single BSD-type partition group with a freebsd-ufs volume of a little over 700MB of size inside (this one is the install media and should not be deleted just yet).

    b) Add a freebsd partition to ada0 creating the nested partition scheme in which we’ll create the root volume.

    c) Add a freebsd-ufs partition with a mountpoint of / to the ada0s1 (which we just re-created in step B above). This should result in a partition map looking something like this.

    ada0            140 GB  MBR             
  ada0s2        711 MB  BSD             
    ada0s2a     711 MB  freebsd-ufs     
  ada0s1        139 GB  BSD             
    ada0s1a     139 GB  freebsd-ufs    /

    d) Select “Finish” and commit the changes to start the installer copying the software distro to the newly created root volume.

  5. Once the installer has finished it should present a “Manual Configuration” prompt. Choose “Yes” to drop into a shell and set the correct comconsole_port setting for the boot loader on the freshly installed pfSense system before rebooting into pfSense.

    echo 'comconsole_port="0x2F8"' >> /boot/loader.conf.local && reboot

  6. Once the system reboots you should see a prompt such as the following.

    pfSense 2.4.4-RELEASE amd64 Thu Sep 20 09:03:12 EDT 2018
Bootup complete

FreeBSD/amd64 (pfSense.localdomain) (ttyu1)

pfSense - Serial: <SERIAL> - Netgate Device ID: <DEVICE_ID>

*** Welcome to pfSense 2.4.4-RELEASE (amd64) on pfSense ***

 WAN (wan)       -> igb0       -> v4/DHCP4: <PUBLIC_IPV4>/31
 LAN (lan)       -> igb1       -> v4: 192.168.1.1/24

 0) Logout (SSH only)                  9) pfTop
 1) Assign Interfaces                 10) Filter Logs
 2) Set interface(s) IP address       11) Restart webConfigurator
 3) Reset webConfigurator password    12) PHP shell + pfSense tools
 4) Reset to factory defaults         13) Update from console
 5) Reboot system                     14) Enable Secure Shell (sshd)
 6) Halt system                       15) Restore recent configuration
 7) Ping host                         16) Restart PHP-FPM
 8) Shell

Enter an option:

  7. Open a shell (using option 8 in the pfSense CLI menu) and cleanup the install media on ada0s2

    gpart delete -i2 ada0

Initial pfSense Configuration

By this point, pfSense is probably still pretty useless as although it gets a public IPv4 address automatically via DHCP, since there are two interfaces on the t1.small.x86 server type there is a LAN automatically configured resulting in pfSense blocking any access attempts via the WAN interface. This will need to be cleaned up, and some basic configuration changes made to get you started.

  1. Connect to the “Out-of-Band Console” and choose option 1 to “Assign Interfaces”

    a) Assign WAN to igb0 (what it is already assigned to)

    b) Leave LAN unassigned (hit enter when prompted and confirm the request to unload the interface)

    This is what the process should look like:

    Valid interfaces are:

igb0    0c:c4:7a:e5:43:30   (up) Intel(R) PRO/1000 Network Connection, Version 
igb1    0c:c4:7a:e5:43:31 (down) Intel(R) PRO/1000 Network Connection, Version 
...
Should VLANs be set up now [y|n]? n
...
Enter the WAN interface name or 'a' for auto-detection 
(igb0 igb1 or a): igb0
...
Enter the LAN interface name or 'a' for auto-detection 
NOTE: this enables full Firewalling/NAT mode.
(igb1 a or nothing if finished): <ENTER>

The interfaces will be assigned as follows:

WAN  -> igb0

Do you want to proceed [y|n]? y

You have chosen to remove the LAN interface.

Would you like to remove the LAN IP address and 
unload the interface now [y|n]? y

Writing configuration...done.

  2. Next select option 2 “Set interface(s) IP address” and configure the IPv4 address statically, setting the gateway IP for the WAN interface. You can find the gateway IP on the server detail page from your browser. This process should look something like the following:

    Configure IPv4 address WAN interface via DHCP? (y/n) n

Enter the new WAN IPv4 address.  Press <ENTER> for none:
> <PUBLIC_IPV4>

Subnet masks are entered as bit counts (as in CIDR notation) in pfSense.
e.g. 255.255.255.0 = 24
     255.255.0.0   = 16
     255.0.0.0     = 8

Enter the new WAN IPv4 subnet bit count (1 to 31):
> 31

For a WAN, enter the new WAN IPv4 upstream gateway address.
For a LAN, press <ENTER> for none:
> <GATEWAY_IPV4>

Configure IPv6 address WAN interface via DHCP6? (y/n) n

Enter the new WAN IPv6 address.  Press <ENTER> for none:
> <ENTER>

Do you want to enable the DHCP server on WAN? (y/n) n

Do you want to revert to HTTP as the webConfigurator protocol? (y/n) n

Please wait while the changes are saved to WAN...
 Reloading filter...
 Reloading routing configuration...
 DHCPD...

The IPv4 WAN address has been set to <PUBLIC_IPV4>/31
You can now access the webConfigurator by opening the following URL in your web browser:
                https://<PUBLIC_IPV4>/

Press <ENTER> to continue.

  3. Connect to pfSense UI via https using the public IPv4 address (default credentials are admin/pfsense) and run through the wizard to secure the system with non-default access credentials. Once you do this, it wouldn’t hurt to also review the secure log for succesful logins from IPs other than yours to be 100% certain nobody else logged in between opening the WAN interface and locking down access to the webConfigurator (or if you stay connected to the console while doing this, logins will be printed to the console for you to easily observe).

  4. Under System -> Advanced, configure a few important pfSense settings to make sure the system is well protected. This is very important given the firewall will be running on a public IP address.

    • Configure setting Admin Access -> webConfigurator -> Protocol to HTTPS

    • Enable setting Admin Access -> Secure Shell -> Enable Secure Shell

    • Enable setting Admin Access -> Secure Shell -> SSHd Key Only to Public Key Only

    • Enable setting Admin Access -> Console Options -> Password protect the console menu

  5. Under System -> Advanced, configure setting Firewall & NAT -> Firewall Advanced -> Firewall Maximum Table Entries to 1000000 to make sure pfSense has enough space allocated to reload firewall rules.

  6. Under System -> User Manager create an admin user preferably with a unique username other than simply “admin” and make it a member of the “admin” group.

  7. Under System -> User Manager edit the “admin” user and disable it by checking the box “This user cannot login” then logout and re-login with the user created in the previous step. Note: this disables login via the webConfigurator, it will not lock this user out of Console or SSH access.

  8. Under Firewall -> Virtual IPs, optionally add a VIP for the private management address to WAN interface using the IP Alias type. This would allow for blocking non-whitelisted access to the public IP address, but still provide a means to access the pfSense admin from anywhere using the Doorman Customer VPN provided by Packet.

Wrapping Up

Well there you have it, a working pfSense installation on a bare-metal cloud server powered by Packet Host. I’ve had my pfSense based VPN running on Packet Host for well over the past 6 months and it continues to serve me well.

P.S. If you enjoyed this post, feel free to give me a shoutout on Twitter. Cheers!