Https with PerfSonar 4.3.4
- 1 ECDF's Way
- 2 The Dirty Durham Way
Whats this about
Congratulations you have a perfSonar instance setup and running.
If you don't then please read one of the many ancient repositories of sysadmin knowledge distributed across the explored region of the local galactic cluster, and please remember, don't panic.
In Edinburgh at the time of writing we have perfSONAR 4.3.4 running as installed from the official perfSONAR iso images on CentOS7 deployed in February 2020. https://docs.perfsonar.net/install_getting.html
This guide is written to explain/educate so I apologise if some of these steps are overly verbose and/or covering ground well understood by people familiar with most/all of the tools written here.
The https configuration in perfSONAR
perfSONAR supports using https for providing access to several internal pages including secure web-pages for the per-server configuration.The important file covering this configuration is:
As part of the installation perfSONAR generates a self-signed certificate for https connections. This is not best practice but not intrinsically insecure if you chose to trust this certificate properly.
However this certificate (by default) is only valid for 12 months. This means that 1-year after your installation you should generate a new certificate. There is no component within perfSONAR which checks and updates this certificate every year.
Speaking to at least 2 sysadmins and after checking multiple sites it's clear that the use of this limited/short-lived certificate is not well advertised and so I've written this guide to replace it.
Getting a letsencrypt certificate
As of 2021 Python2.x is dead long live Python3. Instructions here are very Python3 dependent. Trying to do the equivalent in Python2.x is just not advised anymore.
First step install the virtualenv tool on your perfsonar box.
# Install epel yum install epel-release # Install python3.6 virtualenv from epel yum install python36-virtualenv
Now build a virtualenv for certbot to run in and install certbot there (NB: yes I am running this as root on purpose!):
# Build a Python3 virtual environment so as to not pollute the host virtualenv-3 /root/certbot_env # Source/Activate this virtual env so that we can install Python packages source /root/certbot_env/bin/activate # Install the most recent certbot client in this env pip install certbot
To check this is all setup correctly:
# which certbot /root/certbot_env/bin/certbot
Now in order to use certbot with letsencrypt you will need access to privileged ports to respond in a secure way to confirm that you are in control of the server which you're wanting a certificate to be issued for (challenge-response).
In order to make proper use of these ports we should first stop the apache service running. Then we can get the certbot tool to do what it does for us:
# Stop apache from perfSONAR (temporary) systemctl stop apache # # Use the certbot tool to request certs be issued by letsencrypt for our sever in an isolated way: # For Edinburgh the 2 hostnames which point to the 1 box are: gridpp-ps-band and gridpp-ps-lat # We want the service to be secure when accessed via either interface. # certbot -d gridpp-ps-lat.ecdf.ed.ac.uk -d gridpp-ps-band.ecdf.ed.ac.uk certonly --standalone
Please follow the appropriate prompts and give a sensible email address so that you get notified when the cert is next due to expire.
You can now restart the perfSONAR apache instance if we want, but it's not any more secure (yet!)If all went well you should now have a folder under
/etc/letsencrypt/live/with the first FQDN you requested above. In the case of Edinburgh this folder was:
This folder should contain something like the following:
# ls /etc/letsencrypt/live/gridpp-ps-lat.ecdf.ed.ac.uk/ cert.pem chain.pem fullchain.pem privkey.pem README
Installing a new certificate for apache
The default perfSONAR configuration makes use of a certificate and private key for the host stored at:
# Certificate /etc/pki/tls/certs/localhost.crt # Private Key /etc/pki/tls/private/localhost.keyThese files are referenced from the apache ssl configuration:
In order to use the files as generated by letsencrypt the (best?) solution is to re-configure apache to point to the new letsencrypt certificates. To do this the ssl.conf file should be edited such that the following section looks similar to below:
... #SSLCertificateFile /etc/pki/tls/certs/localhost.crt SSLCertificateFile /etc/letsencrypt/live/gridpp-ps-lat.ecdf.ed.ac.uk/fullchain.pem ... #SSLCertificateKeyFile /etc/pki/tls/private/localhost.key SSLCertificateKeyFile /etc/letsencrypt/live/gridpp-ps-lat.ecdf.ed.ac.uk/privkey.pem ... #SSLCertificateChainFile /etc/pki/tls/certs/server-chain.crt SSLCertificateChainFile /etc/letsencrypt/live/gridpp-ps-lat.ecdf.ed.ac.uk/chain.pem ...
i.e. localhost.crt == fullchain.pem, localhost.key == privkey.pem and it's good to let apache know about chain.pem
NB: If you want to check file permissions be aware that certbot/letsencrypt use symlinks by default and you will need to follow them to the origin to check the ultimate file permissions which are presented to apache when it reads a given file.
After this you should be free to (re-)start apache again for perfSONAR:
systemctl start httpd
You should now see a secure padlock (or equivalent) when you point your bowser to your perfsonar instance web-interface, e.g. for Edinburgh https://gridpp-ps-band.ecdf.ed.ac.uk/toolkit/
Renewing a letsencrypt certificate
To be written but again can be done using the certbot above. The process is likely the same as above but including a renew command to certbot.Updating the certbot instance (if/when needed) should be as easy as:
pip install --upgrade certbotwithin the virtualenv setup earlier.
The Dirty Durham Way
Working from ECDF's findings above, we've pushed Lets Encrypt to PerfSonar in a very dirty (read unsupported i.e. It's not our fault if it breaks) fashion. This assumes you're running the perfsonar image based on CentOS. This way may break things if an update modifies the ssl.conf. We chose this method because stopping apache to renew the certificate feels like it could go very wrong.
- Install Perfsonar
- Install certbot
- Modify /etc/httpd/conf.d/ssl.conf
- restart apache
- Request LE cert
- Modify /etc/httpd/conf.d/ssl.conf
- Restart apache
Then renewal is simply:
- Renew Certificate
- Restart apache
Assumes you're root (sudo su)
yum install certbot
pip install certbot
yum install snapd ln -s /var/lib/snapd/snap /snap snap install core snap refresh core snap install --classic certbot ln -s /snap/bin/certbot /usr/bin/certbot
Edit SSL Settings
Add this line to your /etc/httpd/conf.d/ssl.conf (you can put it anywhere but this file makes sense to me) just before the closing virtual host tag
AliasMatch "^.well-known" "/var/www/html/.well-known"
then reload/restart apache
systemctl reload httpd
Request a certificate via the certonly and webroot option. such as below, change your hostname to your public hostname
certbot certonly --webroot -w /var/www/html/ -d perfsonar.dur.scotgrid.ac.uk
certbot certonly --webroot -w /var/www/html/ -d $(hostname -f)
Re-Edit SSL Settings
You then need to re-edit the /etc/httpd/conf.d/ssl.conf, you need to edit
- Line 100 - SSLCertificateFile /etc/pki/tls/certs/localhost.crt
- Line 107 - SSLCertificateKeyFile /etc/pki/tls/private/localhost.key
- Line 116 - SSLCertificateChainFile /etc/pki/tls/certs/server-chain.crt
To the following (change the location to your domain):
- SSLCertificateFile /etc/letsencrypt/live/perfsonar.dur.scotgrid.ac.uk/cert.pem
- SSLCertificateKeyFile /etc/letsencrypt/live/perfsonar.dur.scotgrid.ac.uk/privkey.pem
- SSLCertificateChainFile /etc/letsencrypt/live/perfsonar.dur.scotgrid.ac.uk/fullchain.pem
Then simply restart httpd.
Renew & Restart HTTPD
Don't forget to add the following in as a cron job running as root
certbot renew --renew-hook "service httpd restart"
# Attempt a renew every three days at 33 minutes past 3 am and dump the output (plus error) into oblivion. 33 3 */3 * * root certbot renew --renew-hook "service httpd restart" >/dev/null 2>&1