How to implement Event Registry in Event Driven Architecture

Image Source: pixabay

Events are lifeline of an Event Driven Architecture. All microservices rely on the data in these events to be accurate and complete. One of the many ways to implement Event Driven Architecture is through Amazon Simple Notification Service (SNS) and Amazon Simple Queue Service (SQS).

Event Driven Architecture with SNS and SQS

Example of a user created event.

{
  domain: User,
  eventName: UserCreatedSuccess,
  eventData: {
    username: 'newUserName',
    email: 'test@test.com',
    originalImageUrl: <some url> ,
  }
}

Now imageService can listen to UserCreatedSuccess event and start working on the image. UserJourney service can start working on welcome emails and so on. If any information in the event is missing or incorrect, it would lead to suboptimal user experience.

What is Event Registry

Event Registry is a central service where all the events and event definitions lives. Event Registry should answer the following questions:

• What are the valid events in the system?
• What is the schema/definition of every event in the system?

What are the valid events in the system?

You can save a list which contains all the events grouped by service/domain. If services are defined with Domain Driven Design in mind, each service will raise an event in a single domain. A sample for the schema list is defined below

list of supported events

The above list implies that there are two events in the User domain.

Now before publishing an event, validator will check if the event exists in the list above.

What is the schema/definition of every event in the system?

You can define your schema and its validation manually or use existing packages. In the JavaScript world, Joi is commonly used for schema description and data validation.

For example, in the current context, you can define the schema for the UserCreatedSuccess event:

Joi schema for the UserCreatedSuccess Event

Here is a snapshot of the validator code

https://gist.github.com/abhinavdhasmana/af617d24dc26599b0345489959668312

Event Registry workflow

You now have individual pieces of the puzzle. You need to fit the above code into the workflow

Design 1: Central validator and publisher

You call the Event Registry with the payload when you want to publish an event. The Event Registry will validate the event permission and payload. If the payload validates, the event will be published otherwise an error will be thrown.

Event registry workflow with central publisher

Current workflow has following advantages:

• All the logic of validation and raising the event lives in a single service.

Current workflow has following disadvantages:

• Event Registry is a single point of failure in the system.
• Tight coupling exists between event validation and raising an event
• One api call is required before an event can be raised. This will add few ms of delay in the whole workflow.
• Event Registry needs to scale with the number of events generated by the system

Design 2: Distributed validator and publisher

In this design, event registry only contains the definition and permissions of events and event schemas. The schema is requested by the service before raising an event. Validation happens at the service and an event is raised if it’s successful.

Distributed Event Registry workflow with API calls

Current workflow has following advantages:

• Decentralised event validator and publisher
• Easy to scale as each service will validate and publish its own set of events.
• A single overwhelmed service will not affect other services in the system.

Current workflow has following disadvantages:

• One api call is required before an event can be raised. This will add few ms of delay in the whole workflow.
• Even if the service caches the data, it can lead to cache stampede.

Design 3: Optimised Distributed validator and publisher

In this design, the service on its boot up, makes an API call to event registry and gets all the relevant event schemas and event definition. Going forward, all the events and event schema is present locally and validated at each service.

Distributed Event Registry workflow with one API calls

In the current design, you have all the advantages of Design 2 plus

• The service can decide to throw error and become unhealthy if Event Registry fails to respond.
• No cache stampede

If you found this story interesting or useful, please support it by clapping it👏 .