Kafka ACLs
In this article I will talk about how to enable and manage ACLs in KAFKA and some common ACL commands.
Almost every organisation makes use of Apache Kafka to exchange sensitive information between various systems with in the organisation and also between organisations. So securing the Kafka cluster is necessary!
Kafka provides the ability to exercise granular control over access to objects and operations within the cluster through the use of Kafka Access Control Lists (ACLs) and through several interfaces (command line, API, etc.)
Kafka ACL’s are defined in the general format of:
“Principal P is [Allowed/Denied] Operation O from Host H on Resource R”
Principal — A principal is the user id of the client that can be authenticated by the authoriser. Clients of a Kafka broker identify themselves as a particular principal using various security protocols. The way a principal is identified depends upon which security protocol it uses to connect to the Kafka broker.
Operation — An operation is an action performed on a resource. In addition to identifying the resources to which users or groups have access, ACLs also identify the operations those users or groups are authorised to perform. Below are the various operations for the resources.
Host — The IP address of the server from where the client is trying to access the Kafka cluster and its’ resources. If host ip address is not provided then by default all host ip address are allowed. This is a more restrictive access for a production environment in which we can whitelist at the host IP level. So applications only from those whitelisted hosts will be allowed access even if you have access to the user id who is authorised.
Resources —There are various resources in KAFKA that can be wrapped with authorisation like a cluster, group, topic, transactional ID, or Delegation token. ACLs specify which users can access a specified resource, and the operations they are permitted to run against that resource.
In any environment following basic authorisations would be required:
- The producer(client) needs permission to write and describe a topic.
- The consumer(client) needs permission to read and describe a topic.
- The Kafka brokers needs permission to access the KAFKA cluster and all the topics.
How to enable Kafka ACL ?
Kafka manages and enforces ACLs through an authoriser. Kafka provides a default authoriser implementation SimpleAclAuthorizer that stores ACLs in Zookeeper. If no such configuration exists, then everyone is authorised to access any resource.
The authoriser class name is provided in the server.properties of the broker as shown below:
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
We can also have our own implementation of the Authoriser. Once the authoriser property is enabled in the server properties, every request to the broker will be authorised.
Super Users
If the ACLs is enabled in a broker and a resource has no associated ACLs yet, then no one is allowed to access that resource except “Super Users”. This behaviour can also be controlled by including the following in server.properties:
allow.everyone.if.no.acl.found=true.
If a super user is trying to access a resource, then no authorisation check will be performed for that user.
To enable super user provide the user name/id in the brokers server.property file as below:
superusers=User:<userid1>; User:<userid2>
As practice all the KAFKA broker users should be given superuser access to ensure that brokers have all access to all resources across the KAFKA cluster.
Important ACL Commands
Liked it? Clap👏🏻 , Share , Comment!!!
