Let's Encrypt scripts

Let's Encrypt scripts for cPanel shared hosting
Login

Let's Encrypt scripts for cPanel shared hosting

Permalink

About

Let's Encrypt is a Certificate Authority (CA), recognized by all modern browsers, which offers free TLS/SSL certificates.
Let's Encrypt offers a mechanism to request, obtain, and renew certificates automatically.
The client (a program normally run on the web server host) and the server (the CA) comminucate using the Automatic Certificate Management Environment (ACME) protocol. [Wikipedia] [specification]

Once the client has obtained the certificate from the CA, installing the certificate depends only on the type of web server.
The certificate consists of 3 files - domain key, domain cert, CA cert.
The term "install the certificate" simply means to copy the 3 files, add a few lines to your web server configuration, and then restart your web server.
That is, if you have root access.

In this article we will focus on certificate installation on a cPanel-enabled shared web hosting account, without root access.

Existing script alternatives

The recommended way to get, install and renew Let's Encrypt certificates is to use the certbot script.
If you have root access to your web server, you should probably stick to certbot, and stop reading here.

At the the time of this writing, certbot supports automatic certificate installation for Apache and nginx.
Be sure to check out the Inst column in this array of certbot plugins for current support.

There are also several certbot alternatives available.
(We will actually use one of these alternatives, Get HTTPS for free!, for one of our task, the First time certificate installation.)
Be sure to check out the section of Projects integrating with Let’s Encrypt.
It mentions few web hosting control panels with included Let's Encrypt support, but unfortunately, cPanel is not one of them.
If one of those scripts cover your needs, you would have another reason for stop reading here.

But... if you are in the same situation as me, which means that you have:

you may probably use certbot (or one of the alternatives) to get and renew a certificate, but not to install it.
In this case, chances are that the install scripts on this page may work for you.
Anyway, read the Beware! section about shared hosting and cPanel issues.

Besides scripts, this page contains several examples to understand each step in the certificate request, install, and renewal process.
Even if cPanel is a Web GUI, we will only use command line tools to run all scripts and examples.

This document digs rather deeps into the Let's Encrypt certificate mechanism.
If you just want to install your certificate ASAP, without Background and Theory, skip to the First time certificate installation section.

Background

The Let's Encrypt philosophy:

Obtaining, installing and renewing Let's Encrypt certificates basically consists of the following steps:

First time certificate installation:

  1. Create an account key and request a new certificate from Let's Encrypt
  2. Install the certificate on your web site

Certificate renewal:

  1. Check if the certificate has expired (cron job)
  2. If expired, request a certificate renewal using an existing Let's Encrypt account key
  3. If expired, install the renewed certificate on your web site
  4. If expired, optionally notify someone (probably yourself) about the renewal

Even if there are good efforts to pack all these steps into a single script, such as the (indeed very complete) certbot, I want to stress that there is no such thing as one magic silver bullet script for all possible scenarios:

This is [an example] [id2] reference-style link.

Then, anywhere in the document, you define your link label like this, on a line by itself:

Beware!

Beware! (1) : SNI

If you are using a shared hosting account, and ask your web hosting service if they "support https", they will most probably try hard to sell you a dedicated IP address "to make your https work correctly".
You may kindly answer them that SNI has been supported by OpenSSL since version 0.9.8f (released 11 Oct 2007), so if OpenSSL packages on their servers are somewhat recently updated, there should be no problem.
You may also point out that your https web site will not stand or fall with the missing end users still using Windows XP/IE, Windows XP/Safari, nor the Nokia Browser for Symbian.
In other words, at the time of this writing (late 2016), all modern web browsers supports SNI.
So, no thanks, no dedicated IP address required, because "your https will work correctly" even when using a shared IP address.

On the other hand, if supporting old browsers (Windows XP/IE, Windows XP/Safari, Nokia Browser for Symbian) is an issue for you, then you have no other option than to go and buy a dedicated IP address.
(Those browsers only perform a DNS lookup of your domain, followed by a reversed DNS lookup, which will return the domain name of your web host's server.
As the domain names don't match, that old browser will complain.)

Here is a command line demo that shows how SNI works.
Example 1 returns the end date of the web hosting server's certificate, not using SNI.
Example 2 returns the end date for the hosted domain's certificate, using SNI (through the -servername option). 

  # Example 1: Get the certificate expiry date for the web hosting server's domain, not kuu.se (no SNI):
  echo -n | openssl s_client -connect kuu.se:443 2>/dev/null | openssl x509 -noout -enddate
  notAfter=May  6 12:00:00 2019 GMT
  
  # Example 2: Get the certificate expiry date for the hosted domain kuu.se (SNI):
  echo -n | openssl s_client -connect kuu.se:443 -servername kuu.se 2>/dev/null | openssl x509 -noout -enddate
  notAfter=Jan  2 11:50:00 2017 GMT

Beware! (2): cPanel API

Now that you have kindly convinced your web hosting server that you will be fine without a dedicated IP, you should ask them if their version of cPanel supports installing your own certificates.
(Tell them you want to upload and test a self-signed certificate, for example.)

The certificate installation is a standard cPanel service, and normally you can check by yourself if this service is enabled.
If your cPanel includes this icon...

cPanel Security SSL/TLS icon

... and this menu...

cPanel Security SSL/TLS menu

... then chances are good that you may install certificates on your account.

Aynyway, beware of hosting services, for example www.hostmonster.com, where this functionality is disabled for shared hosting accounts.
In such a case the certificate install script below will fail (or any way to try to install a certificate).
You are forced to either buy a dedicated IP address from HostMonster, or change to a web hosting provider which supports installing your own certificates.

Prepare!

This section contains some command line tests for SNI and cPanel.
It may be useful to understand some gory details.

cPanel API tests

You can test if your current cPanel account supports certificate installation, from the command line.
Here are some examples that will, step by step, test the cPanel install_ssl functionality.
(The environment variable declarations are repeated for each test, just to be clear that they are required. Obviously, they only need to be declared once in a shell session.)

Test 1: Try to connect to your cPanel host without authentication (should return HTTP Status 401 Access Denied):

export CPANEL_HOST=YOUR-CPANEL-HOSTNAME
printf "GET / HTTP/1.1\r\nHost: ${CPANEL_HOST}\r\n\r\n" | \
     openssl s_client -connect ${CPANEL_HOST}:2083 -ign_eof 2>/dev/null| grep ^HTTP
HTTP/1.1 401 Access Denied

Test 2: Add authentication and the cPanel flag PERL_LWP_SSL_VERIFY_HOSTNAME (should return HTTP Status 302 Moved):

export CPANEL_HOST=YOUR-CPANEL-HOSTNAME
export CPANEL_USERPW=`echo -n YOUR-CPANEL-USERNAME:YOUR-CPANEL-PASSWORD | openssl base64 -A`
export PERL_LWP_SSL_VERIFY_HOSTNAME=0
printf "GET / HTTP/1.1\r\nHost: ${CPANEL_HOST}\r\nAuthorization: Basic ${CPANEL_USERPW}\r\n\r\n" | \
     openssl s_client -connect ${CPANEL_HOST}:2083 -ign_eof 2>/dev/null| grep ^HTTP
HTTP/1.1 302 Moved

Test 3: Access the URL for install_ssl (using GET instead of POST, should return HTTP Status 200 OK):

export CPANEL_HOST=YOUR-CPANEL-HOSTNAME
export CPANEL_USERPW=`echo -n YOUR-CPANEL-USERNAME:YOUR-CPANEL-PASSWORD | openssl base64 -A`
export PERL_LWP_SSL_VERIFY_HOSTNAME=0
printf "GET /execute/SSL/install_ssl HTTP/1.1\r\nHost: ${CPANEL_HOST}\r\nAuthorization: Basic ${CPANEL_USERPW}\r\n\r\n" | \
     openssl s_client -connect ${CPANEL_HOST}:2083 -ign_eof 2>/dev/null| grep ^HTTP
HTTP/1.1 200 OK

Test 4: Access the URL for list_certs to get a list of already installed certificates (should return 200 OK and a JSON-formatted list of certificates, possibly empty ):

export CPANEL_HOST=YOUR-CPANEL-HOSTNAME
export CPANEL_USERPW=`echo -n YOUR-CPANEL-USERNAME:YOUR-CPANEL-PASSWORD | openssl base64 -A`
export PERL_LWP_SSL_VERIFY_HOSTNAME=0
printf "GET /execute/SSL/list_certs HTTP/1.1\r\nHost: ${CPANEL_HOST}\r\nAuthorization: Basic ${CPANEL_USERPW}\r\n\r\n" | \
     openssl s_client -connect ${CPANEL_HOST}:2083 -ign_eof 2>/dev/null > list_certs.response.txt

First check the HTTP status code:

grep ^HTTP list_certs.response.txt
HTTP/1.1 200 OK

Then get the (possibly empty) JSON-formatted list of installed certificates:

Raw JSON:

cat list_certs.response.txt | perl -0777 -pe's/.*Content-Length: \d+.*?\r.*?\r.*?(\S.*)\nclosed$/$1/gms'

Pretty formatted JSON (and suppressing some output):

Using the Perl module JSON (from CPAN):

cat list_certs.response.txt | perl -0777 -pe's/.*Content-Length: \d+.*?\r.*?\r.*?(\S.*)\nclosed$/$1/gms' | perl -MData::Dumper -MJSON=from_json -ne'print Dumper(from_json($_))' | grep -v modulus

Using the Python tool json.tool:

cat list_certs.response.txt | perl -0777 -pe's/.*Content-Length: \d+.*?\r.*?\r.*?(\S.*)\nclosed$/$1/gms' | python -m json.tool | grep -v modulus

Using the C tool jq:

cat list_certs.response.txt | perl -0777 -pe's/.*Content-Length: \d+.*?\r.*?\r.*?(\S.*)\nclosed$/$1/gms' | jq -C '.' | grep -v modulus

Connect, get response, and parse JSON, all-in-one:

printf "GET /execute/SSL/list_certs HTTP/1.1\r\nHost: ${CPANEL_HOST}\r\nAuthorization: Basic ${CPANEL_USERPW}\r\n\r\n" | openssl s_client -connect ${CPANEL_HOST}:2083 -ign_eof 2>/dev/null | perl -0777 -pe's/.*Content-Length: \d+.*?\r.*?\r.*?(\S.*)\nclosed$/$1/gms' | jq -C '.'

Test 5: Other cPanel UAPI Functions related to TLS/SSL

We have already checked our web site for SNI support, but we can also use the cPanel API to check this, with the is_sni_supported method:

printf "GET /execute/SSL/is_sni_supported HTTP/1.1\r\nHost: ${CPANEL_HOST}\r\nAuthorization: Basic ${CPANEL_USERPW}\r\n\r\n" | openssl s_client -connect ${CPANEL_HOST}:2083 -ign_eof 2>/dev/null | perl -0777 -pe's/.*Content-Length: \d+.*?\r.*?\r.*?(\S.*)\nclosed$/$1/gms' | jq -C '.status'

This method returns either 1 or 0, depending on if SNI is supported or not.

Test 6: Install a certificate and key calling install_ssl and using HTTP POST.

This is described further on, in the Install certificate section.
Anyway, first you need a certificate to install, so please read on.

So now that we know almost everything about SNI and the cPanel API, it is time to get the feet wet with the certificate installation.

First time certificate installation

Get certificate

Using Get HTTPS for free!

To get a Let's Encrypt account key and certificate for the first time, we will use the web tool Get HTTPS for free!.
You will need access to a UNIX/Linux shell prompt.
The most convenient way is to run the commands on your shared hosting server.
You may also run the commands on a local computer, but then you have to upload the generated files to the server.

Resumed, the steps to get a new certificate include:

  1. Create your "Let's Encrypt account", which is actually a file you create yourself, account.key.
    This file is created once.
  2. Create the file YOUR-DOMAIN.key, and use it to sign the Certificate Signing Request (CSR), which may be saved to YOUR-DOMAIN.csr.
    The key file is created once per domain, which includes any subdomains such as www.YOUR-DOMAIN, www2.YOUR-DOMAIN etc.
    The request file is created once per request, be it a new certificate, or a certificate renewal.
  3. Sign 4 web requests using account.key before sending them to Let's Encrypt.
    The web response from Let's Encrypt contains information about the file name and content that should be created for each subdomain in Step 4.
  4. Create a file in the directory .well-known/acme-challenge for each subdomain.
    Sign 1 web request per subdomain, using account.key before sending them to Let's Encrypt to confirm that HTTP verification can take place.
    NOTE: Option 2 - file based HTTP verification is used (Option 1 requires a server to be running, which is not always possible on a shared web hosting account).
  5. The web response should contain both your certificate and the CA's (in this case Let's Encrypt). Save them as YOUR-DOMAIN.crt and YOUR-DOMAIN-issuer.crt, respectively.

Now, go and get your Let's Encrypt certificate.
All the steps above are very well explained, with all required commands included, following this link:

https://gethttpsforfree.com/

After these steps you should have the following 4 (optionally 5) files (you may name the files as you want):

account.key
# Your Let's Encrypt private account key. Keep it secret. Not used in the install script.

YOUR-DOMAIN.key
# The private key for your domain. This file will be used in the install script as /PATH/TO/YOUR/DOMAIN/KEY/FILE

YOUR-DOMAIN.crt
# The certificate for your domain. This file will be used in the install script as /PATH/TO/YOUR/CRT/FILE

YOUR-DOMAIN-issuer.crt
# The intermediate certificate for your domain. Not used in the install script, but used by Apache and Nginx web servers, so don't throw it away.
# The intermediate certificate is your issuer's certificate, in this case Let's Encrypt.  
# This certificate has many synonyms:
# - Issuer's certificate
# - Intermediate certificate
# - Certificate chain
# - CA Bundle

YOUR-DOMAIN.csr (optional)
# The certificate request file.
# Only used to request a certificate.
# This file is not used in any other configuration or script, and may be deleted when the certificate has been obtained.

You may now jump to the Install certificate section.
Anyway, here follows a detailed explaination of what really happened during each of the 5 steps when we got our certificate.

Step 1: Account Info

This step includes creating your "account", which is simply a file account.key generated by openssl.
This file only has to be created once, regardless how many domains you have. Keep it secret!

Step 2: Certificate Signing Request

This step includes creating a TLS private key file for your domain, YOUR-DOMAIN.key, generated by openssl.
This file has to be created once per domain (note that YOUR-DOMAIN, www.YOUR-DOMAIN, and whatever.YOUR-DOMAIN is considered one single domain).
Keep it secret!

The certificate signing request (CSR) is what you send to Let's Encrypt in order to issue you a signed certificate.

Step 3: Sign API Requests

TODO: CHECK HOW THESE FOUR HASHES ARE GENERATED, AND HOW THE OUTPUT SIGNATURES ARE SENT/VALIDATED

PRIV_KEY=./account.key; echo -n LONG_HASH_1      | openssl dgst -sha256 -hex -sign $PRIV_KEY

PRIV_KEY=./account.key; echo -n LONG_HASH_2      | openssl dgst -sha256 -hex -sign $PRIV_KEY

PRIV_KEY=./account.key; echo -n LONG_HASH_3      | openssl dgst -sha256 -hex -sign $PRIV_KEY

PRIV_KEY=./account.key; echo -n VERY_LONG_HASH_4 | openssl dgst -sha256 -hex -sign $PRIV_KEY

Step 4: Verify Ownership

TODO: CHECK HOW THESE HASHES ARE GENERATED, AND HOW THE OUTPUT SIGNATURES ARE SENT/VALIDATED

Create one signature per subdomain:

PRIV_KEY=./account.key; echo -n LONG_HASH_FOR_kuu.se     | openssl dgst -sha256 -hex -sign $PRIV_KEY

PRIV_KEY=./account.key; echo -n LONG_HASH_FOR_www.kuu.se | openssl dgst -sha256 -hex -sign $PRIV_KEY

Choose Option 2 - file-based

Log into your server, and at the web root directory for your domain, run:

mkdir -p .well-known/acme-challenge

File to validate domain kuu.se

echo A8_I4Xmb2ubsKKkMAVuHpSAzQqhTLoNLRGBzQAXOjfc.VDUu_Bt8ofHUCPXz4Y6O49r6n6miGk3O6nCSANBFeH4 > .well-known/acme-challenge/A8_I4Xmb2ubsKKkMAVuHpSAzQqhTLoNLRGBzQAXOjfc

If you included subdomains with different web root directories than the main domain, you have to switch to that directory:

mkdir -p .well-known/acme-challenge

File to validate domain www.kuu.se

echo SOPGl_gNA1zeuaaJI0AGVlYLf5wE6dIZxcOCSqUt5Cw.VDUu_Bt8ofHUCPXz4Y6O49r6n6miGk3O6nCSANBFeH4 > .well-known/acme-challenge/SOPGl_gNA1zeuaaJI0AGVlYLf5wE6dIZxcOCSqUt5Cw

Install certificate

As we mentioned before, this document is focused on certificates for a cPanel environment.
Anyhow, here goes some brief instructions for Apache config and Nginx config.

Installing your certificate on a cPanel-enabled host is just a matter of doing one single HTTPS POST operation.
The POST request must contain:

The HTTPS response contains status information about the installation result (success or failure).

While it is possible to execute a HTTP/HTTPS POST request from the command line, it is highly recommended to run such a task from a script.
For the command-line curious, read POST-HTTPS-HOWTO.txt. (Be prepared for a lot of typing!)

install-ssl-example.pl:

This script is similar to an example in the cPanel SDK documentation
As the script uses a HTTPS POST request to install the certificate, it is possible to run the script from any computer with an internet connection.
Anyhow, to run the script during a renewal, from a cron job (see below), you will need a 24/7 up-and-running computer, which probably makes your shared hosting account more suitable than your Desktop computer.

TODO: SHELL SCRIPT ALTERNATIVE TO PERL SCRIPT, CHECK letsencrypt.sh FOR HTTPS POST

The script requires the following CPAN Perl modules to be installed on your shared account:
(Normally cPanel comes with an option to install Perl modules using the web GUI.)

LWP::UserAgent;
LWP::Protocol::https;
JSON;

Download install-ssl-example.pl 

#!/usr/bin/env perl
 
# https://documentation.cpanel.net/display/SDK/Tutorial+-+Call+UAPI%27s+SSL%3A%3Ainstall_ssl+Function+in+Custom+Code
 
# Return errors if Perl experiences problems.
use strict;
use warnings;
# Allow my code to perform web requests.
use LWP::UserAgent;
use LWP::Protocol::https;
# Use the correct encoding to prevent wide character warnings.
use Encode;
use utf8;
# Properly decode JSON.
use JSON;
# Function properly with Base64 authentication headers.
use MIME::Base64;

### ----- CHANGEME - BEGIN ----
 
# Authentication information.
my $username   = 'YOUR-USERNAME-HERE';
my $password   = 'YOUR-PASSWORD-HERE';
my $domain     = 'YOUR-DOMAIN-HERE';
my $cpanel_host= 'YOUR-CPANEL-HOST-HERE'; # NOTE: Your cPanel host name may, or may not, be the same as your domain name.

# Files for certificate and domain key
my $cert_file=  '/PATH/TO/YOUR/CRT/FILE';        # YOUR-DOMAIN.crt
my $key_file =  '/PATH/TO/YOUR/DOMAIN/KEY/FILE'; # YOUR-DOMAIN.key

### ----- CHANGEME - END ------

# The URL for the SSL::install_ssl UAPI function.
# Cpanel secure port number is always 2083.
my $request= 'https://'. $cpanel_host . ':2083/execute/SSL/install_ssl';

# Required to allow HTTPS connections to unsigned services.
# Services on localhost are always unsigned.
$ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} = 0;
# Create a useragent object.
my $ua = LWP::UserAgent->new();
# Add authentication headers.
$ua->default_header(
  'Authorization' => 'Basic ' . MIME::Base64::encode("$username:$password"),
);
# Read in the SSL certificate and key file.
my ( $cert, $key );
{
    local $/;
    open ( my $fh, '<', $cert_file ) or die "Cannot open file $cert_file: $!n";
    $cert = <$fh>;
    close $fh;
    open ( $fh, '<', $key_file ) or die "Cannot open file $key_file: $!n";;
    $key = <$fh>;
    close $key;
  }
# Make the HTTPS POST call.
my $response = $ua->post($request,
    Content_Type => 'form-data',
    Content => [
        domain => $domain,
        cert   => $cert,
        key    => $key,
    ],
);

# The installation has finished. The result message is JSON-formatted to be easily parsed.

# Create an object to decode the JSON.
# Sorted by keys and pretty-printed.
my $json_printer = JSON->new->pretty->canonical(1);
# UTF-8 encode before decoding to avoid wide character warnings.
my $content = JSON::decode_json(Encode::encode_utf8($response->decoded_content));
# Print output, UTF-8 encoded to avoid wide character warnings.
print Encode::encode_utf8($json_printer->encode($content));

Edit the CHANGEME section to fit your environment, then run the script:

chmod 755 install-ssl-example.pl
./install-ssl-example.pl

If there were no errors in the script output, you should now be the proud owner of a secure website. Congratulations!
Test https://YOUR-DOMAIN in your browser. The locker symbol should indicate that your certificate is valid, and issued by Let's Encrypt.
If you click your way to the certificate details, the expiration date should be 90 days from the day you requested the certificate.

Now you can take a break for about a little less than 90 days.
Then it's time to think of the certificate renewal.

Certificate renewal

If you registered your Let's Encrypt account with a valid email, you will receive a reminder a couple of weeks before the certificate expires.
You could just repeat the steps First time install certificate section, but we would like the renewal to be automatic.

To get the expiry date and time, use the openssl x509 command with the -enddate option:

echo -n | openssl s_client -connect kuu.se:443 -servername kuu.se 2>/dev/null | openssl x509 -noout -enddate | sed 's/notAfter=//'

The output should be similar to:

Jan  2 11:50:00 2017 GMT

That date format is what openssl x509 offers. It's nice and readable to us humans, but harder (though not impossible) for a script to parse.
If you want to get the expiry time in seconds, for example, download and and play around with openssl-expiredate.sh.
The script basically obtains a certificate, gets the expiry date, and converts the date to a timestamp.

TODO: JSON,AJAX

Content-Type: application/json;charset=UTF-8
Access-Control-Allow-Origin: *

To achieve this, we start with an expiry check for the certificate, using the openssl x509 command with the -checkend <SECONDS> option:

echo -n | openssl s_client -connect kuu.se:443 -servername kuu.se 2>/dev/null | openssl x509 -noout -checkend 86400; echo $?

If the command above returns 0, the certificate has not expired, and will not expire within a day (86400 seconds).
If the command above returns 1, the certificate will expire within a day (86400 seconds), or has already expired.

This command is perfectly sufficient for a cron job to check if the certificate has to be renewed.

Check if the certificate has expired (cron job)

It is actually enough to run this line as a daily cron job:

    echo -n | openssl s_client -connect ${domain}:443 -servername ${domain} 2>/dev/null | openssl x509 -noout -checkend 86400; return $?

TODO: EXPLAIN SOME MORE

If expired, request a certificate renewal using an existing Let's Encrypt account key

If expired, install the renewed certificate on your web site

If expired and/or renewed, optionally notify someone (probably yourself) about the renewal

Links

Extensive test of your website SSL/TLS certificate online (change URL to point to your site):
https://www.ssllabs.com/ssltest/analyze.html?d=www.kuu.se&latest

The following link is a cPanel example for dedicated hosts, but includes a script that may be used to access the cPanel API:

https://forums.cpanel.net/threads/how-to-installing-ssl-from-lets-encrypt.513621/