Rick van Rein

Tue 18 July 2017


Using SASL with HTTP, Mail and LDAP in Nginx

All our work on identity must somehow end up benefiting applications. One of most interesting bits of software to do this is a frontend proxy. As so often, we find a few parts missing to complete our vision of a better-unified Internet.

Our ideas about identity our nicely shaping up, and prove to bring great benefits of control to users, such as aliases and groups and Bring Your Own IDentity through our work on Realm Crossover. But in the end, our architecture needs services to actually be beneficial to users. So what does it take to get there?

HTTP in need of SASL Auth

Most applications these days are web applications. Its popularity has led to many innovations, but surprisingly authentication mechanisms have been notoriously weak. So weak even, that we end up putting it into the website itself, as part of an application. Even if application programmers are not the least bit interested in authentication issues, and can think of little more than a username/password scheme. With credentials stored in the same unprotected database as their application data.

But wait — these applications tend to use JavaScript. Its code is downloaded from all over the place, and often includes behavioural trackers and advertisements with dubious intentions. How can we mix that with something as vital to protect as authentication?

The best approach is to perform authentication in a level below that which is controlled by the application. In web applications, that means either HTTP or TLS.

The standard answer is to use TLS with an X.509 client certificate. This introduces a lot of confusing work to the client, and is therefore only used in practice where it can be enforced — in corporate environments.

There is an option at the HTTP level as well, namely Kerberos authentication. That does require user settings and, again, being part of an infrastructure that is common in commercial environments but does not include regular users.

Most HTTP authentication applications come down to password authentication, simply because it is the lowest common denominator that is accessible to all. Even though the InternetWide Architecture aims to make both forms more readily available, it is unfair to assume that everyone will adapt. And so, it is not reasonable to expect web site builders to restrict their access to only those who comply with the X.509 or Kerberos profiles. But an important choice has been missing all along in HTTP; one that email tools have adopted, but that is somehow escaping the web, and that is to offer a choice by using a generic protocol that allows the user to select a form of authentication. Once embedded in the HTTP layer, this should work out to support both private and corporate usage patterns, especially if there already is software handling all this.

Such a protocol exists, and it is known as SASL. In fact, this is not a protocol in itself, but merely a description of a request/response interaction between a client and server to authenticate the client. This general pattern can be embedded in most protocols, and this is actually done all over the place: email protocols SMTP, POP3 and IMAP have it, data exchange protocol LDAP has it, chat protocol XMPP does it, and we could go on with a long list of useful protocols that you may not even have heard of. But HTTP, somehow, is still lacking SASL.

And there's so much to be had — most importantly, off-loading of authentication to SASL backends, possibly the same for all front-end protocols. It may include X.509 certificates, OpenPGP keys and Secure Remote Passwords through the EXTERNAL method; it may include GSS-API and so Kerberos5 single-signon through the GSSAPI method. And in all these cases the application programmer does not need skills; it is up to the operator to relay the SASL messages to a backend.

Adding SASL to HTTP should not be very difficult though. The HTTP Auth workgroup is in favour, as it is just another plugin for the general methods for authentication in the HTTP protocol. It just needs to be defined. And to do that, it needs to be built.

Expanding Nginx with SASL and LDAP

We believe that Nginx can benefit greatly from the inclusion of SASL support, first in a generic sense and then applied to HTTP. So it looks like the perfect engine to test these ideas on. This is a block diagram (that is not as daunting as it may look):

Nginx support for SASL and LDAP

In the middle, we find a generic SASL block as a central switchpoint. This is an internal API over which the general SASL protocol would be switched.

Then there are backends. HTTP SASL is one over which the generic requests could be forwarded to a backend; a common alternative is LDAP, which wraps it into its authentication request. Plus, there might be a database with the various credentials, pretty much like a .htpasswd file; but that might be less in line with the Nginx style of being a proxy, rather than an endpoint.

An interesting backend type that could be supported, is to run SASL over EAP, which in turn travels over Diameter or its less-aligned predecessor RADIUS. Although EAP-SASL has not been defined yet, the recent uptake of network access as a user service (through VPN technology) indicates that reuse of user credentials for such "lower" uses is of more use to end users than it originally was when EAP was primarily the mechanism for network bootstrapping.

To the front, we see many ways of using the generic SASL block. The proposed HTTP SASL Auth extension is one, but did you know that Nginx also supports mail through proxies for SMTP, POP3 and IMAP? These stand to gain a lot from this backend, because the current implementation runs challenge/response interactions of SASL with a fixed challenge — which is a technical way of saying that the security is reduced to the level of a fixed password. In fact, sysops are aware that challenge/response cannot be replayed, unlike passwords, and many systems have default configurations to reject password authentication over unencrypted links, but not challenge/response — which demonstrates a true problem with current Nginx. But all that is needed to make Nginx as secure as its mechanisms claim it to be is to make hooks in the mail proxies linking them to the generic SASL block.

Nginx should support LDAP

We also believe that Nginx is a perfect proxy for use with LDAP. We believe that this protocol is a perfect carrier for public credentials, as well as contact data and even documents. We rely on it for X.509 certificates, PGP key service, and for publishing data including as the index for our Reservoir component.

We don't rely on LDAP for light-headed reasons. Its standardisation is of extremely high quality, including many standard attributes but also an ability for independent parties to add their own, without ever clashing into others, and when published it is open for anyone to adopt, without risk of misinterpretation. Wow.

On top of that, it is an efficient protocol for searching and storing small pieces of data, such as metadata to the Reservoir pointing into its object store; such as certificates and public keys for identity purposes. And, perhaps most importantly, it is a domain service, which means that you can run your own and represent everything you intend to share as structured data, rather than the loose, manually retrieved information found on most of the web. Although HTTP is attempting to define its way into structured data as well, it is inherently less suitable, and will therefore never achieve the same quality level as LDAP. In practice, the web world has a habit of inventing proprietary protocols for searching and passing data, but rarely with the broad and refined semantics that LDAP offers — on top of it being an open standard that does not demand code from the same vendor on both ends.

So, with that out of the way, why is Nginx such a good idea for LDAP? Well, as with HTTP you can have clients lurking for a long time, so pooling backend connections is a clever idea. And the model of "stripping off" the protective outer layers of TLS and SASL make Nginx into a gateway into a protected backend network, it becomes possible to mix connections by including a proxied authorisation control in forwarded LDAP request messages.

The code to handle LDAP for this purpose has already been made in our project, and is called LillyDAP. This is an efficient, flexible, pluggable framework for procesing and sending LDAP and making small changes to it while in transit. It was coded in the style of Nginx and we believe that integrating it into Nginx can greatly benefit the power of this lovely proxy.

Coding it as Open Source

We want to make this an open source project, as always. And we aim for integration of this work into the Nginx mainstream distribution. Given the many advantages that it has for Nginx, we believe it to be a valuable contribution to this wonderful proxy.

Are you interested in extending our team with your experience of Nginx internals? Then please contact us!

Go Top