Having read Opher’s excellent blog posting on describing CEP maturity models, I found that while I agreed with Opher’s descriptions of differences between messaging and events, I disagreed with describing phase 3 of CEP as “towards looking at ‘event clouds’ instead of events one-by-one”.
The glossary notes on the definition of an event cloud says that “CEP usually refers to event processing that assumes an event cloud as input, and thereby can make no assumptions about the arrival order of events”. This implies that events “arrive” – just not necessarily in a defined order such as creation time, etc. It also implies that events may arrive one-by-one. It certainly does not preclude one-by-one processing.
The other implication of Opher’s posting is that the cloud may somehow be processed as a whole. Looking at the definition of a cloud, it is made up of many events of differing types where each event may have been created at a different time, and may have a different time-to-live value within the cloud. But in order to make the entire cloud accessible to an event processing agent as a whole, a mechanism must exist that persists the events within the event cloud and manages the cloud events according to their time-to-live values.
(An easy parallel to this view is “ordinary” data processing where sets of persisted data (i.e. events) are made available for queries. Data/Events are stored in tables and keyed by their time-to-live values. Obviously, given a large enough quantity of events, the storage and processing requirement may be considerable.)
But I disagree that this is the only way to define CEP. Indeed it has long been a fiery debate among the CEP community on how, exactly, an event cloud may be practically processed without creating a partially ordered set of events (which may be regarded as a stream of one-by-one events). I would argue that persisting an entire event cloud is fine for ad-hoc processing and analysis, but that the vast majority of CEP involves detection of predefined patterns and is efficiently performed as a form of one-by-one processing.
In Operational Intelligence, applying CEP to the enterprise event cloud is a practical application whereby predefined patterns are detected and acted-upon in real-time. It would be impractical and practically impossible to persist the entire event cloud as the volumes of events are considerable, and the rate of events would require a lot of expensive equipment to provide the required processing power. Yet Vantify Experience Center uses CEP to process an enterprise’s event cloud, and provides real-time intelligence to operational staff to meet challenges and opportunities for maximum business benefit with relatively inexpensive equipment. The events are processed one-at-a-time rather than as a single cloud for efficiency, and the value to customers has been demonstrated many times. To imply that CEP excludes one-by-one processing is inaccurate and wrong – rather it is an important and critical subset of CEP.
Hi Brian. The phrase “event cloud” may have been somewhat misleading, since people have different associations about it – my meaning is that if an event is processed without any relation to other event it is not CEP, since by definition CEP function looks at multiple events. So the better phrase may be that in phase III we don’t look at events in isolation but look at multiple event. When looking at multiple events – then events may be processed in single event mode – where each event is checked to see if it completes a pattern, or in a “batch” mode (like some of the stream processing products do).
cheers,
Opher