Taming and Composing high performance Stream-Oriented Processing Systems for Big Data

location_city Bengaluru schedule Nov 18th 03:00 - 03:45 PM place Regal people 13 Interested

Real time applications are dominating the industry! Data is the main ingredient in Internet-based, social media and Internet of things (IoT) systems, which generate continuous streams of events used for real time analytics. This poses a tremendous challenge due to the massive volume of data collected and processed. These event-based Real-time analysis systems can easily process millions of messages per second through new generation solutions by simply defining small flows and then combining them together to create processing graphs. In this talk, will cover the concepts behind high-performance streamed-oriented big data processing systems. We will explore messaging queue systems like Kafka and Akka Streams which let developers define their process workflows at a higher level to define a graph system enabling a high throughput. You will learn how to integrate high performance stream message queues and how to define process workflows in C# and F#.


Outline/Structure of the Experience Report

The problem of realtime analysis for BigData

The solution : Intro and concept about Event-Streaming

Use case of analyzing 6 millions messages per second using Kafka and Akka Stream

Pro and Cons

Learning Outcome

Patterns and designs to handle millions of event messages per seconds.

Composing and event stream analysis

Target Audience

Developers that are interested in Real-Time event analysis and high performance systems

schedule Submitted 3 years ago