Configuring security in NiFi can be a bit tricky. NiFi's base security mechanism, mutual authentication with X.509 (a.k.a. TLS/SSL) certificates provides great security. But great security comes with a price, and that price is the complexity of configuring X.509 mutual authentication. There are a log of settings, and it may not be clear up front what all needs to be done, in what order. The error messages can be cryptic and frustrating. In this post, we'll start with an overview of the security situation in NiFi using X.509 certificates, focusing on human users rather than system-to-system access.
NiFi's default security mechanism uses X.509 certificates, also known as SSL or TLS certificates, for mutual authentication and encryption. This means that both servers and clients have certificates, and they need to be properly configured before the whole thing works. Lets look at a picture:
The diagram above shows three machines, a NiFi server and two client machines, one for each of two users. Each has their own pair of public and private keys. The public keys are shown as "certificates", meaning the public key plus metadata and verification information. And each certificate and key needs to be installed in the right store. Let's start by looking at the keys and their holders:
Each user of a NiFi system has their own pair of public and private encryption keys. The public key, sent as a certificate with metadata, performs many functions
- Uniquely identify the user in the form of a "Distinguished Name"
- Verify the user's identity through a mutually-trusted Certificate Authority
- Encrypt communications with the user The private key must remain securely held by the client, and is used to for encyrpting and decrypting data. A user's key pair can be long-lived, used across multiple NiFi instances, and even used for completely separate applications.
The NiFi server has a private key and public key certificate securely held in the KeyStore. These keys identify one particular NiFi server, and should not be shared across servers. The public key certificate is exchanged with clients as they contact the server, it does not need to be exchanged in advance. A server's keys may also be used for site-to-site communication with other servers, although that is not our focus here.
And the containers that store keys also deserve explanation:
KeyStore is the Java-platform term for an archive file containing the NiFi server's public and private key,
plus optional certificates establishing a chain of trust from a Certificate Authority.
NiFi supports the Java KeyStore (.JKS) and PKCS-12 file formats.
The KeyStore file may have a passphrase of its own to protect from meddling with the file.
It is fairly easy to make a KeyStore file using the
keytool utility distributed with the Java Development Kit (JDK), or OpenSSL.
TrustStore is the Java-platform term an archive file containing public keys for all of the users recognized by the NiFi server in any way. However, the existence of a key in the store does not authorize the user to do anything in NiFi yet. TrustStores may also be configured with certificates of Certificate Authorities verifying individual users. The TrustStore file may have a passphrase of its own to protect from meddling with the file. NiFi's supported TrustStore file formats are the same as for the KeyStore, JKS or PKCS-12.
NiFi stores authorization -- who has permissions to do what -- separately from the authentication provided by the Trustore. Authorization is typically stored in a separate config file, authorized-users.xml, which is a simple XML file detailing the roles granted to each user. Before a user can actually do anything in NiFi, they must have a matching authorization record. Authorization records are matched to users via the user certificate subject distinguished name.
Before a user can log in to a secure NiFi server, the user's public and private key pair must be installed locally. The local storage can be particular to a web browser or shared by browsers with the operating system, it varies by client.
Certificates and Trust
While mutual authentication uses the same public key infrastructure that you see in more typical SSL security, there are some important differences in who you trust, and how. For typical web site SSL, the server's certificate must be trusted by a well-known Certificate Authority (CA), which is defined by a CA whose certificate has already been installed in the client's browser trust chain. The certificate trust chain is very public.
NiFi security is very private by contrast. Your NiFi server is likely to be in contact with a relatively tight circle of users, web service clients, and flow file exchange partners. It is undesirable to extend trust any farther than you need to.
Do the certificates need to be signed by a Certificate Authority (CA)?
No. You can use self-signed certificates for the server, clients, or both. You can create your own certificate authority key to sign client and server certificates to establish mutual trust, but this is not required. However, you will enjoy a more secure and smooth user experience if your certificates are signed by a CA mutually trusted by cients and server. The ideal case is that your organization can provided trusted certificates for internal use on both client and server. For short-lived NiFi servers, self-signed certificates may be a practical solution.
Some tools you will want to have for configuring and troubleshooting NiFi security:
- OpenSSL - OpenSSL is a many-functioned tool for working with keys and certificates, and troubleshooting key issues.
- keytool - distributed with the JDK, keytool provides commands for creating Java KeyStore files, importing, exporting, and creating keys.
It can help to read up on the basics of public key encryption, certificates, and certificate authorities.
- Wikipedia: Public Key Certificate - Explains what a certificate is, with a helpful diagram explaining the sequence of requesting, signing, and using certificates.
- Wikipedia: X.509 - Overview of the X.509 public key infrastructure standard.
Get Hands On
Enough with the overview. If you want to understand how NiFi security really works, it is very helpful to walk through configuring a NiFi server and at least one client. Please read our walkthrough, Configuring Apache NiFi SSL Authentication, for detailed steps.