Enterprises rely heavily on technology to build scalable, agile and highly available businesses. Event-Driven Architecture is emerging as a key cornerstone enabling modern enterprises to operate in real-time, adapt to changes quickly, and make intelligent business decisions.
“Event-driven architecture (EDA) is a design paradigm in which a software component executes in response to receiving one or more event notifications.”
Combining agile methodologies with a platform that can respond rapidly to real-time events empowers businesses to make manual or automated intelligent decisions and achieve true enterprise agility. Events from a variety of sources from Servers to Wearables can then be harnessed to support a number of industrial use cases.
Large organizations that adopt such architectures need to treat Events as first-class citizens of their Platform, along with their APIs and Business Services.
This guide describes how Event-Driven Architecture (EDA) can benefit organizations, a step-by-step approach to adopt EDA based on OpenSource technologies, agile practices that can support rapid innovation, and a few real-world use cases. It will also explore how Wipro’s Digital Integration as-a-Service (DIaaS) and Red Hat Integration can support an organization’s adoption of event-driven architecture.
Event-Driven Architecture Adoption Patterns
Enterprises that embrace event-driven architecture can capitalize on the benefits of scalable and reliable real-time communication. This can help drive many strategic initiatives across IoT, eCommerce, Data Integration across systems, data and applications, Edge and Financial Fraud Detection, among others.
Companies that add event-driven architecture to their systems and applications can improve the scalability and responsiveness of applications. They can also access the data and context needed for better business decisions, be that stream processing, data integrations or website activity tracking.
Events are captured as they occur from a spectrum of event sources. On a high-level, these events are categorized into Business/Functional or Technical.
A Guide to Deploying an Event-Driven Reference Architecture
All businesses produce events in one form or the other. They just might not have a system which can make use of these events to produce tangible business value. While businesses start adopting EDA, it is important for them to identify their specific requirements while building the right architecture. These events may span a variety of patterns such as real-time events and decisions, streams processing, and complex event processing.
Setting up such an event platform to start reaping the benefits of event-driven architecture can be simple, but like every journey, it helps to have a map to reach the destination. The following guide outlines what an organization should consider and deliberate, the actions it should take to set up a solid EDA which can cater to a plethora of patterns and business requirements.
Consider this a reference architecture and pick the right elements and patterns based on every organization’s specific needs which can help build new use cases and realize great business value.
1. DEFINE: ARCHITECTURE AND TECHNOLOGY
The first step addresses the core technology components that form the runtime for the event mesh. This includes the technologies that ingest, queue, process, store and serve Events. A typical large enterprise may have one or more Pub-Sub Message Broker platforms, an event streaming platform, cloud messaging platforms, and event consumers and producers spread across a wide variety of technologies and locations. These platforms tend to reside in silos and are not considered part of a unified event mesh. To address this, organizations should:
- Identify the requirements around Events Driven architectures that might exist across Integration, Application Architecture and Analytics groups
- Create a vision of a unified events mesh with all of the existing components
- Put in place a plan to fill the gaps that might exist in the technology capabilities required to meet the organization’s Event Processing requirements.
- Setup reference architectures and define reference implementations for patterns such as those to manage logical transactions and data integrity across multiple databases, Event Sourcing and CQRS.
- The technology toolkit could be a mix of multiple components across message streaming using Kafka, a JMS broker or a message broker, or AMQ Interconnect to build a large-scale messaging network.
Apache Kafka remains the default choice for all streaming use cases, with a rock-solid and super-fast event-streaming backbone. It’s an enabler for many uses, including activity tracking, log aggregation, stream processing, change-data capture, Internet of Things (IoT) telemetry and more. Red Hat AMQ Streams makes it easy to run and manage Apache Kafka natively on Red Hat OpenShift via the upstream project Strimzi.
Event Streams Processing and Complex Events Processing
The Events platform also needs Processing frameworks that provide for processing styles such as Complex Events Processing (CEP), the more common Event Stream Processing styles, and multiple levels of programming abstractions including SQL to query the fast-moving data. The processing framework should also be able to integrate with Machine Learning frameworks to score models. Most of the stream processing frameworks, including AMQ Streams, provide support for all of these programming models. Red Hat Decision Manager provides a solid foundation for CEP and for making intelligent decisions.
Real-Time Insights for Business Users
Most Events map directly to business concepts and, if placed in the hands of business users, will be powerful tools for business agility. A user interface that allows users to define processing rules based on the occurrence of an Event or a combination of Events within a time-window would be a great lever for business agility. Wipro DIaaS Business Insights brings precisely this to help deliver real-time insights.
Microservices as Events Consumers
Once the events are received, they need to be consumed and processed by microservices to bring business value to the organization. Quarkus, which is supersonic, subatomic Java, is one such runtime allows for unification of both imperative and reactive (events) programming paradigms, has great built-in support for messaging and is perfect for building EDA microservices which need high performance runtimes.
Event Driven Serverless Architecture
Event driven architecture also plays a huge part in serverless environments. Serverless apps respond to demand and automatically scale up and down as needed and can be through an event-driven execution model. Red Hat OpenShift Serverless provides an enterprise-grade serverless platform which brings portability and consistency across hybrid and multi-cloud environments.
2. SETUP: GOVERNANCE PRACTICES
As the adoption of event-driven architectures reaches a scale where there are dozens of Events being published and consumed, it becomes critical to have formal practices and tools to govern the Definition, Design, Creation and Consumption of events. This is analogous to the capabilities that API Management platforms provide for APIs.
These are a few considerations while setting up governance and practices to ensure success.
- An interface definition language that provides a readable definition of an Event’s data structure and mechanisms, protocols and location URLs to consume the Event. Emerging standards such as AsyncAPI provide this capability. There is also a related effort to extend Open API to add support for Event-Driven APIs as well. An Event’s structure would refer to and reuse Enterprise Data Definition.
- A searchable Registry of all Events. Just as with an API, the Events Registry provides a searchable store for metadata for Events. The registry should be able to categorize and organize Events by business domains and allow Event owners to set visibility rules and access restrictions on their Events.
- An Events Developer Portal that includes the aforementioned Registry as well as tools that provide collaboration between the developers and Event owners, provide access to documentation, best practices, templates, client libraries etc.
- Analytics dashboards, that provide runtime metrics related to the publication and consumption of Events.
Wipro DIaaS Events Hub provides the capabilities listed above and integrates with Red Hat Service Registry and Developer Portal to provide a unified view of both events and API Management platforms. It also provides support for associating AsyncAPI specifications for each Event.
3. ENABLE: UNIFIED OPERATIONS AND MANAGEMENT
The next step is to set up a unified monitoring capability that provides a view of all the Events published and consumed across the landscape. This unified console should provide support to operational personnel to (a) effectively monitor the platform, (b) monitor failed Events, and (c) edit the payloads and re-publish them, again, across all Message Brokers from a unified console.
Another critical capability is a view of Event Lineage graphs that show the connection between an event, its parent and its child events. The graph gives a complete view of the overall transaction spanning multiple services and systems that coordinate their work using Event driven interactions. This feature is also very useful when an event needs to be re-published and to check the downstream side-effects of this action.
Wipro DIaaS Events Hub provides the monitoring and operations capabilities described above. It integrates with Red Hat AMQ Streams and other popular JMS-based messaging platforms, and enables unified monitoring and management of all event brokers in the organization.
Red Hat AMQ Streams also supports automated installation and configuration on Kubernetes platforms via a set of Red Hat Operators framework for AMQ Streams. These operators are responsible for deploying and managing Apache Kafka clusters, Kafka topics and Kafka users.
Adopt Event-Driven Architecture to Derive Business Value
Traditionally, customers rely heavily on APIs as they have a standard interface for data exchange and HTTP is popular among developers. To derive benefits from EDA, it is important to provide the right tooling and a unified developer platform which helps overcome existing technical restrictions and win over developers.
Let’s discuss a few recommendations for faster event adoption within legacy and greenfield architecture.
- Identify business-driven use cases to create pilots that will define suitable patterns such as Simple Events, Events Streaming, Complex Event Processing.
- Pick the right components from the reference architecture discussed in the previous section so as to build the suitable patterns based on the business requirements
- Make developer adoption easy by providing access to the Architecture, Technologies and Processes through internal enablement and deep-skilled training.
- Enable EDA in legacy applications to derive the maximum benefit from the Events Platform.
- Start capturing events through Change Data Capture (CDC) to create database-driven events
- Use enhancements and CRs within existing applications as an enabler to create required events
- Synthesize events from existing legacy applications
- Build new use cases with the tech toolkit available in the reference architecture to further accelerate business transformation and derive benefits.
Wipro DIaaS with Red Hat AMQ on Red Hat OpenShift Container Platform provides a solid platform for enterprises to build their agile, scalable and distributed event-driven applications to enable rapid Digital Transformation. Red Hat AMQ is part of Red Hat Integration which brings a comprehensive open source cloud native platform for digital integration to support capabilities like API Management, Camel connectors for integration, Data streaming and change data capture. When organizations use this in conjunction with Red Hat Process Automation, they can better craft and control event management to build efficient and resilient EDAs.
EDA Drives Business Value: True Stories
1. Create a new business model through new channels
Emirates NBD uses Red Hat Fuse and Apache Kafka on Red Hat OpenShift to support real-time banking and partner interactions.
2. Incredible improvement in customer experience
A global Financial Services leader implements a real-time campaign solution that delivered a ~60% increase in campaign conversion rates
3. Measurable and Effective Operational improvement
A global logistics provider uses an events fabric to provide real-time visibility to shipments across its wide global network
4. In the field of IoT
A global electronic devices manufacturer used proactive event-based monitoring to reduce the number of service maintenance calls by 45%. Event streaming analytics helped to recognize patterns and prioritize responses to alerts.
5. EDA for Inter-microservices communication
A European bank used an event-driven architecture to improve its relationship with customers. A lightweight event bus-based solution helped keep applications in sync and provided a single view of customer data that enabled in streamlining the customer experience.
Learn more about how Wipro and Red Hat are helping customers in their Digital Modernization journey here.
The authors wish to thank Wipro’s Danesh Hussain Zaki for his meaningful contributions to this article.