Project: Dynamic Complex Event Processing for Hybrid Telecommunication Networks and Smart Grids

Explosively growing data and control networks, such as encountered by data and mobile network operators in current deployment and imminently by energy operators in Smart Grid efforts, present monitoring solutions with a new problem category. The ability to monitor large arrays of devices and gather data from them in real time is a challenge in itself. At the same time, deluges of data are meaningless without the capacity to identify critical signals from background noise. Considering that even a medium national telecom network can have hundreds of thousands of measurement targets producing fault and performance information, refining the data becomes a crucial task that no current commercial software solves._x000D_To this end, DYNE applies semantic technologies and complex event processing to an existing large scale monitoring system to provide higher level situational views, offering customers the capability to see not only single faults, but aggregated information, analysis of root causes of multiple dispersed problems, and preventive fault management which not only analyses measurements, but also their trends and changes in behavior. By considering the overall state of all measurements and identifying the important hierarchies, the system can point the user directly to the ultimate causes of problems and help them ignore lower level simple events that arise as by-products._x000D_The existing monitoring platform used as the basis for DYNE is provided by BaseN. The platform is a cloud-based system that collects, stores, analyzes and visualizes measurement data in a scalable manner. It is built around a dynamic service discovery system; the complex event processing system developed in DYNE will be implemented as a new service that subscribes to data feeds from existing simple event sources and provides refined events to existing export services. It will thus be a separate component that is loosely coupled to BaseN’s platform and can easily be reused in other projects (Fig. A3.1)_x000D_The new processing component will be based on the well-proven concepts of complex event processing but extended by the ability to aggregate and abstract information with respect to time, space and meaning (semantics) dynamically. The resulting Dynamic Complex Event Processing (DCEP) will be able to cope with the massive amounts of sensor data that needs to be processed in near real-time and allows dynamic rearrangements of parts of the event stream in order to investigate a specific matter in more detail. _x000D_DYNE proposes to apply the DCEP engine to measurement data to research the following issues:_x000D_• Normal state definition (what is the system normal state and how much variability is normal) and anomaly detection_x000D_• Event aggregation: distinguishing truly single events from multiple related events _x000D_• Root cause analysis: tracing multiple events to a single cause_x000D_• Emerging faults: the system is still in the normal state but heading toward problems_x000D_• Key indicators: identifying the measurements that are sufficient for tracking normal state_x000D_• Dynamic provisioning: identifying measurements to provision when particular circumstances arise, in order to perform additional event processing_x000D__x000D_The DCEP concepts can be applied in a wide variety of fields, but DYNE concentrates on two specific and related use cases: telecommunications network measurement and Smart Grid measurement, presented in the use case description in the Appendix. Both share the qualities of massive incoming data feeds, hierarchical structure of measurements, and networks where usage patterns are normally predictable. Furthermore, both scenarios require flexible organization of the analysis processes to dynamically focus on specific sets of information sources to provide detailed views of relevant developments or situations that would be unavailable under the normal organization of the processing system. Specifically, with the capability for a dynamic reorganization of the analysis process, DYNE will extend the concepts of event processing to be able to bring sufficient flexibility to these high volume, low latency use cases._x000D_From the business point of view, current BaseN telecom and utility customers have voiced a clear demand for DCEP type of complicated analysis requiring high processing power and capability to handle Big Data. _x000D_DYNE will be developed by a small consortium, in which BaseN will act as the corporate P and FHNW as the research P. BaseN will provide the doCO knowledge for each use case, the well proven computation platform for measurements and analysis, software development resources and all required test data. Additional real world data will be provided by the Finnish telecommunications company DNA (www.dna.fi). FHNW provides expertise and software development resources in the field of complex event processing, cloud based dynamic scaling of event processing systems, semantic technologies and special correlation of event stream data._x000D_

Acronym	DYNE (Reference Number: 7377)
Duration	03/09/2012 - 31/05/2015
Project Topic	DYNE brings novel dynamic complex event processing research into doCO independent real-time data and control network management, allowing for predictive fault prevention and analysis of the underlying causes of individual events.
Project Results (after finalisation)	Project was used to test and pilot various event processing and semantic technologies to produce a next generation real time event processing module for BaseN platform. Project also produced new visualization and data analysis technologies for BaseN._x000D_The pilot architecture developed in Dyne will form the basis of the actual production system offered for the customer.
Network	Eurostars
Call	Eurostars Cut-Off 8

Project partner

Number	Name	Role	Country
2	BaseN oy	Coordinator	Finland
2	University of Applied Sciences and Arts Northwestern Switzerland	Partner	Switzerland