Overview

The LEEC library is the Ceylan fork of the original and much appreciated letsencrypt-erlang, which is a Let's Encrypt client library for Erlang, whose author is Guillaume Bour.

LEEC's purpose is to obtain proper X.509 security certificates from Let's Encrypt - or more generally from any ACME server (version 2) - to perform Domain Validation.

Such certificates are typically used in order to secure one's webserver so that it can offer a solid HTTPS connectivity, which is pretty much standard nowadays; yet they may also used for mail servers, FTP servers, and many more [1].

Three levels of certificates can be considered:

• pure single-domain certificates; for example a certificate applying only to foobar.org
• single-domain certificates with SANs (Subject Alternative Name); for example a certificate applying only to a domain (e.g. foobar.org) and related extra, predetermined DNS names (like alpha.foobar.org, beta.foobar.org, etc.)
• wildcard certificates; for example a certificate applying to a domain (e.g. foobar.org) and to any subdomain thereof (e.g. *.foobar.org)

Subdomains may be managed by different hosts or may correspond to virtual hosts, i.e. be managed by a single host that is able to multiplex the accesses that are made to them, based on the pseudo-host that the requester specified (then for example serving different web contents based on it).

Each certificate is obtained by proving to its ACME server of the issuer the ownership/control of a webserver or of a DNS entry.

At least currently, no wildcard certificate is managed as in-depth as it is done for per-domain certificates (more complex challenges would have to be implemented), yet it is nevertheless supported.

To ensure freshness, these certificates have intentionally a short lifespan (e.g. they last for 90 days); they shall therefore be renewed regularly and automatically from the certificate authority.

LEEC is notably used in the context of US-Web.

Differences Introduced by this Fork

Compared to the original letsencrypt-erlang library, the main differences introduced by LEEC are:

• it is more specialised, in the sense that LEEC focuses on the "slave" use case (i.e. to be directly integrated within an Erlang webserver), as opposed to the "webroot" one (a third-party webserver is running separately with little possibilities of direct interactions) or the "standalone" one (where no specific prior webserver would be running, the certificate agent operating then its own one)
• more comments, more spell-checking, much clarification
• more typing, more runtime checking, extended traces supported
• security increased (notably using 4096-bit RSA keys)
• dependency onto Ceylan-Myriad added, to benefit from its facilities
• JSON parser can be JSX (the default), or Jiffy (refer to the Switching JSON Parsers section)
• HTTP client can be either Shotgun or the Erlang-native httpc client, to avoid any extra dependencies on Gun and Cowlib (whose versions could potentially clash with the ones required by any Cowboy-based integrating webserver)
• porting done from gen_fsm (soon to be deprecated) to the newer gen_statem
• minor API changes and additions, for a clearer and more flexible mode of operation
• fixed the compilation with Erlang version 24.0 and higher (e.g. w.r.t. to http_uri/uri_string, to updated dependencies such as Jiffy, and newer Cowboy for the examples)
• allow for concurrent certificate requests (e.g. if managing multiple domains with different keys, new certificates being requested for all of them at webserver start-up); so LEEC generates certificates in parallel and does not rely on a registered FSM (Finite State Machine) anymore
• global, ETS-based TCP connection pool replaced by an (optional) per-FSM internal cache (if relying on Shotgun)
• support for SAN (Subject Alternative Name) certificates, an extension to X.509 enabling a certificate to include a subjectAltName field to list, here, extra DNS names that are covered by this certificate
• basic support for the management of:
  • Ephemeral Diffie-Helman key, to ensure Forward Secrecy by relying on a set of keys that are never communicated
  • Intermediate Let's Encrypt Certificates
• SSL-based verification of the ACME peer, to avoid any man-in-the-middle attack (quite relevant when fetching security elements)
• first basic support of wildcard certificates

So, even if LEEC can be seen mostly as a "reckless" fork (in the sense that it became quickly obvious that retaining upstream compatibility could hardly be achieved) - with so many source-level differences (in terms of conventions, Myriad integration, whitespace cleanup) that a pull request can difficultly be considered - yet, in spite of the appearances, it remained quite close to the original (mainly differences of form) and followed the same structure.

By some ways, this LEEC fork is safer and more robust than the original, by others not (e.g. test coverage, autonomous use, continuous integration). A key goal was to make it easier to understand and maintain.

Most of the elements of this pull request from Marc Worrell have also been integrated.

Prerequisites

Building

Two build procedures can be used (from the root of LEEC), and are now mostly the same:.

• either a rebar3-based one; then run make all-rebar3, simply corresponding to:

$ rebar3 upgrade --all
$ rebar3 compile

• or one relying on Ceylan's native build system; once the relevant prerequisites have been setup (selected, downloaded, built), just run make all

This last procedure is the one that we prefer and use routinely (see the US-Web native deployment script as an example thereof).

Usage Example

The main example of LEEC in action can be found in link with US-Web, whose sources can be found here; see notably class_USCertificateManager.erl and us_web_letsencrypt_handler.erl [2].

# Never recording credentials in the shell history of this (UNIX) user:
$ unset HISTFILE

# Application Key:
$ AK="3a2b42bc0caeb7d4"

# Application Secret:
$ AS="fb7472abcd8f9980210571ca502d793c"

# Consumer Key (attached to one's account):
$ CK="5b1226df0d5ea12192ae2c9642cf7b60"

# As a one-liner, to ensure that timestamp is fresh enough (otherwise: "Query
# out of time"):
#
$ METHOD=GET; QUERY="https://eu.api.ovh.com/1.0/me/api/application"; \
 BODY=""; TIMESTAMP="$(date +%s)"; \
 SHA="$(echo -n ${AS}+${CK}+${METHOD}+${QUERY}+${BODY}+${TIMESTAMP} \
 | shasum | cut -d ' ' -f 1)"; SIGNATURE="\$1\$${SHA}"; \
 curl -X ${METHOD} -H "Content-type: application/json" \
 -H "X-Ovh-Application: ${AK}" \
 -H "X-Ovh-Consumer: ${CK}" -H "X-Ovh-Signature: ${SIGNATURE}" \
 -H "X-Ovh-Timestamp: ${TIMESTAMP}" ${QUERY}
[212174,212168]

# Presumably the oldest one:
$ TOKEN_TO_DELETE=212168

# Checking the target token (validity not returned by this call):
$ METHOD=GET; \
QUERY="https://eu.api.ovh.com/1.0/me/api/application/${TOKEN_TO_DELETE}"; \
BODY=""; TIMESTAMP="$(date +%s)"; \
SHA="$(echo -n ${AS}+${CK}+${METHOD}+${QUERY}+${BODY}+${TIMESTAMP} \
| shasum | cut -d ' ' -f 1)"; \
SIGNATURE="\$1\$${SHA}"; curl -X ${METHOD} -H "Content-type: application/json" \
-H "X-Ovh-Application: ${AK}" \
-H "X-Ovh-Consumer: ${CK}" -H "X-Ovh-Signature: ${SIGNATURE}" \
-H "X-Ovh-Timestamp: ${TIMESTAMP}" ${QUERY}
{"applicationId":212168,"name":"LEEC-for foobar.org",
 "description":"LEEC foobar.org updater","status":"active",
 "applicationKey":"3a2b42bc0caeb7d4"}

# Deletion:
$ METHOD=DELETE; \
QUERY="https://eu.api.ovh.com/1.0/me/api/application/${TOKEN_TO_DELETE}"; \
BODY=""; TIMESTAMP="$(date +%s)"; \
SHA="$(echo -n ${AS}+${CK}+${METHOD}+${QUERY}+${BODY}+${TIMESTAMP} | shasum \
| cut -d ' ' -f 1)"; SIGNATURE="\$1\$${SHA}"; \
curl -X ${METHOD} -H "Content-type: application/json" \
-H "X-Ovh-Application: ${AK}" -H "X-Ovh-Consumer: ${CK}" \
-H "X-Ovh-Signature: ${SIGNATURE}" -H "X-Ovh-Timestamp: ${TIMESTAMP}" ${QUERY}
null

# Checking the deletion:
$ METHOD=GET; QUERY="https://eu.api.ovh.com/1.0/me/api/application"; \
BODY=""; TIMESTAMP="$(date +%s)"; \
SHA="$(echo -n ${AS}+${CK}+${METHOD}+${QUERY}+${BODY}+${TIMESTAMP} | shasum \
| cut -d ' ' -f 1)"; SIGNATURE="\$1\$${SHA}"; \
curl -X ${METHOD} -H "Content-type: application/json" \
-H "X-Ovh-Application: ${AK}" -H "X-Ovh-Consumer: ${CK}" \
-H "X-Ovh-Signature: ${SIGNATURE}" -H "X-Ovh-Timestamp: ${TIMESTAMP}" ${QUERY}
[213174]

dns_ovh_endpoint = ovh-eu
dns_ovh_application_key = 3a2b42bc0caeb7d4
dns_ovh_application_secret = fb7472abcd8f9980210571ca502d793c
dns_ovh_consumer_key = 5b1226df0d5ea12192ae2c9642cf7b60

$ chmod 400 leec-ovh-credentials-for-foobar.org.txt
$ chown web-srv:us-srv leec-ovh-credentials-for-foobar.org.txt
$ chattr +i leec-ovh-credentials-for-foobar.org.txt

$ certbot certonly --dns-ovh --dns-ovh-credentials \
/etc/xdg/universal-server/leec-ovh-credentials-for-foobar.org.txt \
-d foobar.org -d *.foobar.org

$ sudo -u web-srv certbot certonly --config-dir . --work-dir . \
--logs-dir . --non-interactive --agree-tos --email you@foobar.org \
--dns-ovh --dns-ovh-credentials \
/etc/xdg/universal-server/leec-ovh-credentials-for-foobar.org.txt \
-d foobar.org -d *.foobar.org

Design Notes

Getting Information about the Generated Certificates

If using LEEC to generate a certificate for a baz.foobar.org host, the following three files shall be obtained from the Let's Encrypt ACME server:

• baz.foobar.org.csr: the PEM certificate request, sent to the ACME server (~980 bytes)
• baz.foobar.org.key: the TLS private key regular file, generated on the server host, and kept there (~1675 bytes)
• baz.foobar.org.crt: the PEM certificate itself of interest (~3450 bytes), to be used by the webserver afterwards

To get information about this certificate:

$ openssl x509 -text -noout -in baz.foobar.org.crt

Certificate:
   Data:
       Version: 3 (0x2)
       Serial Number:
           04:34:17:fd:ee:9b:bd:6b:c2:02:b1:c0:84:62:ed:a6:88:5c
       Signature Algorithm: sha256WithRSAEncryption
       Issuer: C = US, O = Let's Encrypt, CN = R3
       Validity
           Not Before: Dec 27 08:21:38 2020 GMT
           Not After : Mar 27 08:21:38 2021 GMT
       Subject: CN = baz.foobar.org
       Subject Public Key Info:
           Public Key Algorithm: rsaEncryption
               RSA Public-Key: (2048 bit)

              Modulus:
                   [...]
               Exponent: 65537 (0x10001)
       X509v3 extensions:
           X509v3 Key Usage: critical
               Digital Signature, Key Encipherment
           X509v3 Extended Key Usage:
               TLS Web Server Authentication, TLS Web Client Authentication
           X509v3 Basic Constraints: critical
               CA:FALSE
           X509v3 Subject Key Identifier:
               [...]
           X509v3 Authority Key Identifier:
               keyid:C0:CC:03:46:B9:58:20:CC:5C:72:70:F3:E1:2E:CB:20:B6:F5:68:3A

           Authority Information Access:
               OCSP - URI:http://ocsp.stg-int-x1.letsencrypt.org
               CA Issuers - URI:http://cert.stg-int-x1.letsencrypt.org/

           X509v3 Subject Alternative Name:
               DNS:hello.baz.foobar.org.crt, DNS:world.foobar.org.crt, DNS:somesite.foobar.org.crt
           X509v3 Certificate Policies:
               Policy: 2.23.140.1.2.1
               Policy: 1.3.6.1.4.1.44947.1.1.1
                 CPS: http://cps.letsencrypt.org

           CT Precertificate SCTs:
               Signed Certificate Timestamp:
                   Version   : v1 (0x0)
                   Log ID    : [...]
                   Timestamp : Jan  2 09:23:20.310 2021 GMT
                   Extensions: none
                   Signature : ecdsa-with-SHA256
               Signed Certificate Timestamp:
                   Version   : v1 (0x0)
                   Log ID    : [...]
                   Timestamp : Jan  2 09:23:20.320 2021 GMT
                   Extensions: none
                   Signature : ecdsa-with-SHA256
                               [...]
   Signature Algorithm: sha256WithRSAEncryption
   [...]

Other Files of Interest

A *.key (e.g. my-foobar-leec-agent-private.key) file is a (PEM, strong enough) RSA private key generated by LEEC so that its agent can safely authenticate to the ACME servers it is interacting with.

lets-encrypt-r3-cross-signed.pem is the (PEM) certificate associated to the Certificate Authority (Let's Encrypt here). It is automatically downloaded by LEEC if not already available.

The dh-params.pem file contains the parameters generated by LEEC in order to allow for a safer Ephemeral Diffie-Helman key exchange, which is used to provide Forward Secrecy with TLS (see this article for further information). Its generation may take quite some time (but is to happen once).

Troubleshooting HTTPS Certificate-related Issues

In order to understand why a given host (typically a webserver) does not seem to handle properly certificates, one may experiment with these commands from a client computer:

$ curl -vvv -I https://foobar.org
$ wget -v https://foobar.org -O -
$ openssl s_client -connect foobar.org:443

From the server itself:

$ iptables -nL
$ lsof -i:443
$ netstat -ltpn | grep ':443'

Third-party solutions might also be used, like testing your server with SSL Labs; thanks to LEEC, US-Web can be ranked "grade A" there.

Licence

Ceylan-LEEC is distributed under the APACHE 2.0 licence, like the original work that it derives from.

Support

Bugs, questions, remarks, patches, requests for enhancements, etc. are to be sent through the project interface (typically issues), or directly at the email address mentioned at the beginning of this document.

Possible Enhancements

• re-using ACME accounts: not creating throwaway, anonymous accounts but (possibly) reusing them by registering the ACME client with its email, etc.
• supporting certificate revocation
• supporting Elliptic Curve cryptography
• reintroducing elements brought by the upstream project yet not updated by the current fork: unit testing, standalone testing, hex package, various escripts and yml files involved
• besides the slave mode (main use case of interest with LEEC), better integrating/testing the other modes (webroot and standalone)
• supporting extra validation challenges, besides http-01 and dns-01 (necessary to obtain wildcard certificates, i.e. applying to all subdomains of a given domain without leaking their names through SANs), like proof-of-possession-01
• supporting directly other ACME services besides Let's Encrypt (like ZeroSSL)
• implementing for good dns-01 rather than using certbot for that

Please React!

If you have information more detailed or more recent than those presented in this document, if you noticed errors, neglects or points insufficiently discussed, drop us a line! (for that, follow the Support guidelines).

One may also be interested in our mini-HOWTO regarding cybersecurity.

Ending Word

Have fun with Ceylan-LEEC! (not supposed to involve any memory leak)