So let's say you have a SMTP mail server that supports STARTTLS, awesome! Most mail delivery agents (MDA) that support TLS will automatically negotiate some form of encryption. However, it turns out that STARTTLS is especially vulnerable to Man-in-the-middle attacks due to its plain-text handshake, allowing adversaries such as your ISP, the NSA, or even Chinese hackers to read your private emails.

On the browser side, we've solved the problem by creating the "HTTP Strict Transport Security" (HSTS) Preload List, a set of websites that are hardcoded as HTTPS only. Recently, in 2018, the IETF created the "SMTP MTA Strict Transport Security" (MTA-STS) specification in a similar vein. This guarantees that mail delivery attempts are encrypted, which fail otherwise. This is how to get it working:

1. The MDA checks for the existence of a DNS TXT Record under _mta-sts:

   v=STSv1; id=20160831085700Z;
   

   Add the TXT record using your DNS provider. You can change id to any string, the only purpose is to signal a change in the MTA-STS policy, to invalidate cached policies.

2. Next, the MDA fetches the MTA-STS policy from the following URL:

   https://mta-sts.<email-recipient.com>/.well-known/mta-sts.txt
   

   Here is an example, showing the authorized mail servers:

   version: STSv1
   mode: enforce
   mx: mail.example.com
   mx: *.example.net
   max_age: 604800
   

   Change the mx: lines to point to your MX server(s) that support TLS 1.2+. Keep in mind that these mail servers must present a publically-trusted TLS certificate (not self-signed). You cannot add IP addresses here, because IP addresses cannot be assigned TLS certificates. max_age: represents the maximum number of seconds you would like the MTA-STS policy to be potentially cached for.

   When you are finished, replace every newline character from the MTA-STS policy with \r\n, so that it looks like this:

   version: STSv1\r\nmode: enforce\r\nmx: *.naut.ca\r\nmax_age: 604800\r\n
   

   Finally, add the following block to your nginx configuration. Change the server_name to reflect your domain, fix the SSL parameters, and paste in your new MTA-STS policy on the line that starts with return 200.

   server {
     server_name mta-sts.naut.ca;
     listen 443 ssl http2;
     listen [::]:443 ssl http2;
     ssl_protocols TLSv1.2 TLSv1.3;
     ssl_certificate /etc/ssl/certs/naut.ca.pem;
     ssl_certificate_key /etc/ssl/private/naut.ca.key;
     location = /.well-known/mta-sts.txt {
       default_type text/plain;
       return 200 "version: STSv1\r\nmode: enforce\r\nmx: *.naut.ca\r\nmax_age: 604800\r\n";
     }
   }
   

3. Restart NGINX, and point the mta-sts subdomain to the NGINX server through an A or AAAA record!

Update 2021/03/30: Added default_type to NGINX configuration