Streaming Keycloak events

reactive IAM with Keycloak event listeners

Streaming data is a commodity. Thanks to all sorts of streaming data sources we can build reactive systems whereby an event occuring in one corner of the system triggers events somewhere else. Streaming data speeds up processes because businesses can react to events instead of proactively having to ask for “what’s new”. IAM is no exception. Actually, an IAM system should be a source of critical information shared with other systems because IAM events are core to everything else happening within a modern, data-driven organization.

Why? Here are some reasons:

  • A sign-up triggers a welcome email and a follow-up email campaign.
  • A customer modifying their profile propagates new data to a CRM or an HR system.
  • A large number of login attempts triggers alerts or flag an account with suspicious activity.
  • A membership change reconfigures external systems, for example, database permissions or Kafka ACLs.
  • It’s a general auditing mechanism.
  • Data from other systems can be actively fused directly back to IAM in response to selected events.

You can surely come up with other reasons within your business domain.

Various IAM solutions offer different notification methods, webhooks are the most common. Systems like Auth0 or Ory offer webhook extension points. An action, for example, a sign in triggers a preconfigured URI with some payload. When triggered, the endpoint can further pass the received information, and sometimes modify it.

The problem with this approach is that we are often faced with a biased choice of events we can subscribe to. This choice might be for several less and more nefarious reasons. Maybe the original architect of the IAM solution didn’t think of a particular use case? Or specific events would enable users to implement what the provider offers in a higher-paid plan. Sometimes we are lucky and can contribute additional notifications, but there’s often friction. Open-source is excellent, but many maintainers suffer feature creep and hesitate to take in one-off features serving a limited number of users because every additional feature comes with a maintenance burden.

An event bus is a better solution. All events flow through the event bus, there is a universal way to subscribe to the event bus. Events contain raw information, they can be observed and reacted on. Many modern infrastructure components come with an event bus. Docker and Kubernetes come to mind. Surprisingly, Kafka doesn’t have one.

§Keycloak Event Listener SPI

Keycloak has one, too, and it comes in the form of an Event Listener SPI. An implementation of an Event Listener SPI plugs in directly into the core of Keycloak and receives all events happening inside of Keycloak deployment. Keycloak comes with a default implementation - its logging mechanism. There are one hundred and four events in Keycloak 18.0.1 we can act on. That’s pretty significant. There are some very useful event types defined in there. User-related, for example:

  • Login, logout.
  • OAuth client login and logout.
  • Account linking.
  • Consent grant, update, and rejection.
  • Various account information changes.

But also internal Keycloak changes, the most significant one being CLIENT_UPDATE because it carries all information modified in OAuth clients, even event authorization services!

Have a look yourself[1].

§but I said streaming

Let’s face it. Tailing and parsing logs isn’t the most convenient way of working with streaming data. The best would be to have a method to consume structured data. Like JSON. Or protocol buffers. Well, the good news is that Event Listener is a Keycloak SPI. We can write one. More than 2.5 years ago, I wrote one and put it somewhere on GitHub, then forgot about it. I rediscovered it recently while cleaning up old GitHub repositories. I have updated it for Keycloak 18 and made it available on GitHub.

What is it?

This SPI shows how to turn Keycloak into a streaming IAM source. I have chosen gRPC, I can’t remember the exact reason, but nothing is preventing us from streaming those events directly to Apache Kafka or any other system.

§how: try it out

If you have Docker and Docker Compose available, you can try it now, and it shouldn’t take more than 10 minutes of your time. Simply follow the instructions from the Keycloak protobuf SPI compose4 repository readme.

The Docker Compose configuration in that example builds on my previous articles showing how to run local Keycloak with TLS behind an Envoy proxy, so there is a little friction related to setting up TLS. The following sections of one of the previous articles about Keycloak contain relevant details:

Once running and configured, you can see what’s going on by looking at the logs of the gRPC server:

docker logs -f $(docker ps | grep keycloak-protobuf-event-server | awk '{print $1}')

Open the browser and go to, click on the Administration Console link to navigate to Keycloak login screen. The username is admin, password is admin. In the left menu, click on Events.

realm events

Next, click on the Events Config tab and focus the Event Listeners field, you will see a drop-down with the new SPI listed:

realm events config

Select the new SPI, switch all possible options to On, and click Save.

Look at the log of the event server:

2022-06-18T11:00:33.359Z [INFO]  keycloak-protobuf-event-server: OnAdminEvent: admin-event="time:1655550032713  realmId:{value:\"a3149aad-f5bf-4213-b771-004c95322ce9\"}  authDetails:{realmId:{value:\"a3149aad-f5bf-4213-b771-004c95322ce9\"}  clientId:{value:\"1d0bdadd-b82b-49dd-9193-f18a2e9e461c\"}  userId:{value:\"cff2f608-293b-4cac-b640-fb399009274c\"}  ipAddress:{value:\"\"}}  resourceType:{value:\"REALM\"}  operationType:UPDATE  resourcePath:{value:\"events/config\"}  representation:{value:\"{\\\"eventsEnabled\\\":true,\\\"eventsListeners\\\":[\\\"jboss-logging\\\",\\\"keycloak-protobuf-spi-event-listener\\\"],\\\"enabledEventTypes\\\":[\\\"LOGIN\\\",\\\"LOGIN_ERROR\\\",\\\"REGISTER\\\",\\\"REGISTER_ERROR\\\",\\\"LOGOUT\\\",\\\"LOGOUT_ERROR\\\",\\\"CODE_TO_TOKEN\\\",\\\"CODE_TO_TOKEN_ERROR\\\",\\\"CLIENT_LOGIN\\\",\\\"CLIENT_LOGIN_ERROR\\\",\\\"FEDERATED_IDENTITY_LINK\\\",\\\"FEDERATED_IDENTITY_LINK_ERROR\\\",\\\"REMOVE_FEDERATED_IDENTITY\\\",\\\"REMOVE_FEDERATED_IDENTITY_ERROR\\\",\\\"UPDATE_EMAIL\\\",\\\"UPDATE_EMAIL_ERROR\\\",\\\"UPDATE_PROFILE\\\",\\\"UPDATE_PROFILE_ERROR\\\",\\\"UPDATE_PASSWORD\\\",\\\"UPDATE_PASSWORD_ERROR\\\",\\\"UPDATE_TOTP\\\",\\\"UPDATE_TOTP_ERROR\\\",\\\"VERIFY_EMAIL\\\",\\\"VERIFY_EMAIL_ERROR\\\",\\\"VERIFY_PROFILE\\\",\\\"VERIFY_PROFILE_ERROR\\\",\\\"REMOVE_TOTP\\\",\\\"REMOVE_TOTP_ERROR\\\",\\\"GRANT_CONSENT\\\",\\\"GRANT_CONSENT_ERROR\\\",\\\"UPDATE_CONSENT\\\",\\\"UPDATE_CONSENT_ERROR\\\",\\\"REVOKE_GRANT\\\",\\\"REVOKE_GRANT_ERROR\\\",\\\"SEND_VERIFY_EMAIL\\\",\\\"SEND_VERIFY_EMAIL_ERROR\\\",\\\"SEND_RESET_PASSWORD\\\",\\\"SEND_RESET_PASSWORD_ERROR\\\",\\\"SEND_IDENTITY_PROVIDER_LINK\\\",\\\"SEND_IDENTITY_PROVIDER_LINK_ERROR\\\",\\\"RESET_PASSWORD\\\",\\\"RESET_PASSWORD_ERROR\\\",\\\"RESTART_AUTHENTICATION\\\",\\\"RESTART_AUTHENTICATION_ERROR\\\",\\\"IDENTITY_PROVIDER_LINK_ACCOUNT\\\",\\\"IDENTITY_PROVIDER_LINK_ACCOUNT_ERROR\\\",\\\"IDENTITY_PROVIDER_FIRST_LOGIN\\\",\\\"IDENTITY_PROVIDER_FIRST_LOGIN_ERROR\\\",\\\"IDENTITY_PROVIDER_POST_LOGIN\\\",\\\"IDENTITY_PROVIDER_POST_LOGIN_ERROR\\\",\\\"IMPERSONATE\\\",\\\"IMPERSONATE_ERROR\\\",\\\"CUSTOM_REQUIRED_ACTION\\\",\\\"CUSTOM_REQUIRED_ACTION_ERROR\\\",\\\"EXECUTE_ACTIONS\\\",\\\"EXECUTE_ACTIONS_ERROR\\\",\\\"EXECUTE_ACTION_TOKEN\\\",\\\"EXECUTE_ACTION_TOKEN_ERROR\\\",\\\"CLIENT_REGISTER\\\",\\\"CLIENT_REGISTER_ERROR\\\",\\\"CLIENT_UPDATE\\\",\\\"CLIENT_UPDATE_ERROR\\\",\\\"CLIENT_DELETE\\\",\\\"CLIENT_DELETE_ERROR\\\",\\\"CLIENT_INITIATED_ACCOUNT_LINKING\\\",\\\"CLIENT_INITIATED_ACCOUNT_LINKING_ERROR\\\",\\\"TOKEN_EXCHANGE\\\",\\\"TOKEN_EXCHANGE_ERROR\\\",\\\"OAUTH2_DEVICE_AUTH\\\",\\\"OAUTH2_DEVICE_AUTH_ERROR\\\",\\\"OAUTH2_DEVICE_VERIFY_USER_CODE\\\",\\\"OAUTH2_DEVICE_VERIFY_USER_CODE_ERROR\\\",\\\"OAUTH2_DEVICE_CODE_TO_TOKEN\\\",\\\"OAUTH2_DEVICE_CODE_TO_TOKEN_ERROR\\\",\\\"AUTHREQID_TO_TOKEN\\\",\\\"AUTHREQID_TO_TOKEN_ERROR\\\",\\\"PERMISSION_TOKEN\\\",\\\"DELETE_ACCOUNT\\\",\\\"DELETE_ACCOUNT_ERROR\\\"],\\\"adminEventsEnabled\\\":true,\\\"adminEventsDetailsEnabled\\\":true}\"}  error:{noValue:{}}"

This is a string-serialized protobuf object printed to the log. What can we infer from this event?

  • It’s an admin event: OnAdminEvent: admin-event="time:1655549196131.
  • Modified entity was a realm: resourceType:{value:\"REALM\"}.
  • We have opted-in for admin events: \\\"adminEventsEnabled\\\":true.
  • We hvae opted-in for regular events: \\\"eventsEnabled\\\":true.

Any action executed in the Master realm will now be logged. For example, log out from the realm and observe the log:

2022-06-18T11:03:16.875Z [INFO]  keycloak-protobuf-event-server: OnEvent: event="time:1655550196794  type:LOGOUT  realmId:{value:\"a3149aad-f5bf-4213-b771-004c95322ce9\"}  clientId:{noValue:{}}  userId:{value:\"cff2f608-293b-4cac-b640-fb399009274c\"}  sessionId:{value:\"241a5172-107d-48e0-b5c2-d6b83966648f\"}  ipAddress:{value:\"\"}  error:{noValue:{}}  details:{key:\"redirect_uri\"  value:\"\"}"

Log back in:

2022-06-18T11:03:34.235Z [INFO]  keycloak-protobuf-event-server: OnEvent: event="time:1655550214221  realmId:{value:\"a3149aad-f5bf-4213-b771-004c95322ce9\"}  clientId:{value:\"security-admin-console\"}  userId:{value:\"cff2f608-293b-4cac-b640-fb399009274c\"}  sessionId:{value:\"5ccb5161-c3b1-4dee-935a-6c67bb8400da\"}  ipAddress:{value:\"\"}  error:{noValue:{}}  details:{key:\"auth_method\"  value:\"openid-connect\"}  details:{key:\"auth_type\"  value:\"code\"}  details:{key:\"code_id\"  value:\"5ccb5161-c3b1-4dee-935a-6c67bb8400da\"}  details:{key:\"consent\"  value:\"no_consent_required\"}  details:{key:\"redirect_uri\"  value:\"\"}  details:{key:\"username\"  value:\"admin\"}"
2022-06-18T11:03:34.741Z [INFO]  keycloak-protobuf-event-server: OnEvent: event="time:1655550214737  type:CODE_TO_TOKEN  realmId:{value:\"a3149aad-f5bf-4213-b771-004c95322ce9\"}  clientId:{value:\"security-admin-console\"}  userId:{value:\"cff2f608-293b-4cac-b640-fb399009274c\"}  sessionId:{value:\"5ccb5161-c3b1-4dee-935a-6c67bb8400da\"}  ipAddress:{value:\"\"}  error:{noValue:{}}  details:{key:\"client_auth_method\"  value:\"client-secret\"}  details:{key:\"code_id\"  value:\"5ccb5161-c3b1-4dee-935a-6c67bb8400da\"}  details:{key:\"grant_type\"  value:\"authorization_code\"}  details:{key:\"refresh_token_id\"  value:\"2a19e586-bf5c-417f-b314-75df22335b33\"}  details:{key:\"refresh_token_type\"  value:\"Refresh\"}  details:{key:\"scope\"  value:\"openid profile email\"}  details:{key:\"token_id\"  value:\"ce75b467-73e3-46b7-b97b-a259780b12ce\"}"

And so on.

Of course, in a real implementation, you’d be streaming those events directly to something like Kafka instead of printing them to the screen, so other apps in your company could react to those.

Event listeners are registered on a per-realm basis. To observe events in another realm, you need to repeat the installation steps on every realm for which you want to observe events.

§get in touch

Are you integrating Keycloak into your company and need a partner who can help you take it beyond where you thought it was possible? Reach out to me at