Say "little" with a thick tongue, and it sounds like "lill'". That is
how we constructed the name LillyDAP, as a nickname for our Little LDAP
library. A completely new approach to the protocol, approaching it
as a semantically rich data access protocol, rather than a database
language.
This is the second article in an article series On Display in which we report on the output from projects that fall under the InternetWide Umbrella of projects.
What is it?
LDAP is a protocol for access to a directory. Directories are hierarchically organised collections of objects, where each object can be of a different class and have a variety of attributes to hold data fragments. LDAP moves in this world, and operates on objects and their directory locations as well as on attributes and their relationship to the classes of the containing objects.
One might say that directories are semi-structured in comparison to the tightly organised structure of an SQL database. Indeed, where it takes an extra table (and an extra lookup) to allow multiple values for one attribute in LDAP, it is simply added to the pile in LDAP. In comparison to many no-SQL designs however, directories have a little more structure.
But that may not even be the most important aspect to LDAP. What we are incredibly pleased about, is that it is a standardised protocol. And not just a writeup of how software works, but the design of LDAP is elegant, extensible and semantically very, very rich. If that term does not appeal to you, then let me rephrase: for most of the things you might care to do to your data, LDAP has you covered. You can add your own object classes and attribute types, for example. And the way in which it is done precludes clashes with someone else's extensions. In many cases however, you will be able to simply apply the mountain of standard types; and if not, chances are that you can simply reuse the extension work done before by others.
And even though HTTP could do many of the things that LDAP does, it usually comes down to a very generic service (namely pushing around MIME-typed data blobs) and leaving all the interpretation of that data to software. As a result, you will often find that an application is delivered to you as a tighly coupled pair of client and server software; the ability to do this is a big part of the success of JavaScript. But it has nothing to do with open protocols; the only open part is the very generic service level of HTTP. LDAP has us covered at a much more refined level of detail, which is why you can find completely different client and server implementations talking freely.
Design Criteria
We designed LillyDAP to be a very small library in support of LDAP. It is in fact a wrapper around Quick DER, which itself is a very small implementation of DER. LDAP uses BER, but that is okay to Quick DER. Quick DER writes a more constrained format than LDAP requires, and it shrugs over the frivolous extensions made by BER.
Most importantly, we wanted to make LillyDAP a library that makes LDAP easy to use as a protocol in its own right. We have seen too many situations where people would have benefited from LDAP but decided to turn elsewhere because they could not find a simple enough implementation. Indeed, the OpenLDAP server has a steep learning curve. In general, the complexity of setting up a complete directory, so a data store driven by LDAP, may be avoidable for the kind of applications that we are aiming for with LillyDAP.
You can use LillyDAP as a client or as a server; or you can make it an intermediate to filter LDAP messages, and perhaps split off its SASL authentication of TLS flow. To that end, you are given fine control over the level at which LillyDAP's structural parsers invoke callbacks.
We wanted this to be a highly efficient component. In fact, we have our mind set to integration with the Nginx proxy server as a protocol next to HTTP, and have adopted several of the excellent programming techniques used in Nginx, most notably a compatible pooled memory allocation technique.
The library can work quite well in an asynchronous environment, where only one thread is active and midway data is stored away for future continuation. But we do support threading, and in the best possible manner, namely a lock-free implementation. This means that multiple threads may be running, and even when they get together on a critical piece of code they will not end up locking each other out of the criticial code; instead, each does its own bit in such a way that all can continue without being stopped. This is the modern way in which high-performance server-side code is designed, for optimum performance on multitasking CPUs. At the same time, for LillyDAP it happens to be the simplest solution to make the threads work together.
With the resulting library, it is possible to cover a large range of use
cases; multi-tenant hosting should be as straightforward as a simple tool
that runs from your user account to exchange contact details of PGP keys.
In spite of the large range, the code is still very compact; so compact
in fact, that we've been wondering if it could run on a ZX Spectrum with
48 kB memory, with the tape beeping interface for exchange of LDAP messages!
(And if it hadn't been for the rejection by sdcc of certain coding
constructs we might actually have done it.)
Manipulation of Data
LDAP can be viewed as an RPC protocol that operates on directory data. To implement an application, you can have callbacks for operations sent to you, or you can send out operations and receive callbacks when the results come in. There is a peer-to-peer extension that easily makes LDAP bidirectional, so it can also be used for the exchange of data in a standardised format in P2P networks.
The kinds of operations you can create include addition and removal of objects, of attributes within an object, but also things like locking and even complete ACID transactions are defined. Very important is searching for data, which traditional LDAP servers do with high efficiency due to proper indexing and mostly readonly operation, but there is no reason why a dedicated application server could not do better, possibly even on already-existing indexes and storage structures.
It is common for LDAP to frown upon delivered data, and check if it would break structural integrity. For example, removing a directory node that has children nodes underneath, or removing an attribute that is required by, or adding one that is not permitted by an object's class. Standardised error codes explain such things to the client.
For heavy-duty operations, there are replication and backup protocols, also usable as bulk update protocols and for subscribing to data changes. That is a common pattern: One party sends a new value for an attribute and a lot of lurking clients all receive an update.
Usage Ideas
LillyDAP can be used for domain publication of data, as well as for internal affairs. Furthermore, we designed LillyDAP because we need a tool to modify LDAP messages in transit.
LDAP defines a so-called global directory, which is a fancy term for the definition of an SRV record that appoints a domain's LDAP server. That server is the place to turn to for your queries. Since common uses of LDAP include contact information and configuration data, that's a good start for such publications that may be retrieved at will by others.
A few concrete ideas for a global directory:
-
Publish contact data of your employees. Include those things that are considered truly public; perhaps a phone number but not their email address. Since LDAP is highly suitable for automation, it may be an idea to use revolving temporary contact information. This can help to protect sensible access methods such as highly interruptive SIP phone addresses by making them somehow dynamically changing.
-
Since user identities can be easily searched in LDAP, use it to publish PKIX certificates and/or OpenPGP public keys to the World. LDAP happens to be the standard protocol for PKIX certificates; the LDAP global directory also supported by default in the popular GnuPG tool for OpenPGP, so outgoing mail servers might retrieve keys automatically.
A few concrete ideas for an internal directory, to which access may be protected with SASL authentication:
-
Publish internal contact data. This time, a lot more information can be made available.
-
Publish account configuration information, including groups and roles. This can be picked up by various servers; it is our intended use in the IdentityHub.
-
Publish contact information for external contacts such as clients. This allows all employees to access information entered centrally by one of them.
-
Publish public key information for external contacts, such as their PKIX certificates and OpenPGP public keys. Again, the data can be used by all employees but also by servers that access this automated resource. And again, GnuPG can be configured to use your LDAP server as its key server.
-
Publish PKCS #11 URIs along with public keys for local users. This is not a standard use, but it is very useful in our IdentityHub, which automatically creates, refreshes and destroys keys and certificates. Being able to search public key material in LDAP should yield great speedups in comparison to doing the same over PKCS #11.
A few ideas for user-level applications:
-
Publish your personal contact information to others on your LAN. You could even include dynamic data, such as your presence.
-
Publish your GnuPG key ring to your other locations, so the keys you collected (and signed, perhaps locally) are available everywhere.
Getting Started
If you feel like diving in, please head for these documents first:
The interface to C is the most developed, but the Python wrapper is currently incomplete (meaning, open to pull requests). The same holds for the projected utilities and controls; we add those aspects as soon as we need them, but if you are more in a hurry we welcome your pull requests. Note that neither of these is required for general use of the library.
Go Top