Software Engineering for Big Data by the University of Waterloo, 2016

The main goal of this course is to explore the forefront of research involving the interaction between software engineering and Big Data. The course will examine topics such as requirements, models, designs, architectures, and testing of applications based on big data techniques such as analytics, mining, clustering, recommendation, streams and machine learning, in a variety of domains.

https://cs.uwaterloo.ca/~palencar/cs846/fall-2016/cs846.html

Do you know what is powerful real-time analytics?

In the Digital age today, world has become smaller and faster. 

 

Global audio & video calls which were available only in corporate offices, are now available to common man on the smartphone.

 

Consumers have more information of the products and comparison than the manufactures at any time, any place, and any device.

 

Gone are the days, when organizations used to load data in their data warehouse overnight and take decision based on BI, next day. Today organizations need actionable insights faster than ever before to stay competitive, reduce risks, meet customer expectations, and capitalize on time-sensitive opportunities – Real-time, near real-time.

 

Real-time is often defined in microseconds, milliseconds, or seconds, while near real-time in seconds, minutes.

 

With real-time analytics, the main goal is to solve problems quickly as they happen, or even better, before they happen. Real-time recommendations create a hyper-personal shopping experience for each and every customer.

 

The  Internet of Things(IoT) is revolutionizing real-time analytics. Now, with sensor devices and the data streams they generate, companies have more insight into their assets than ever before.

 

Several industries are using this streaming data & putting real-time analytics. 

 

·        Churn prediction in Telecom

·        Intelligent traffic management in smart cities

·        Real-time surveillance analytics to reduce crime

·        Impact of weather and other external factors on stock markets to take trading decisions

·        Real-time staff optimization in Hospitals based on patients 

·        Energy generation and distribution based on smart grids

·        Credit scoring and fraud detection in financial & medical sector

 

Here are some real world examples of real-time analytics:

 

·        City of Chicago collects data from 911 calls, bus & train locations, 311 complaint calls & tweets to create a real-time geospatial map to cut crimes and respond to emergencies

·        The New York Times pays attention to their reader behavior using real-time analytics so they know what’s being read at any time. This helps them decide which position a story is placed and for how long it’s placed there

·        Telefonica the largest telecommunications company in Spain can now make split-second recommendations to television viewers and can create audience segments for new campaigns in real-time

·        Invoca, the call intelligence company, is embedding IBM Watson cognitive computing technology into its Voice Marketing Cloud to help marketers analyze and act on voice data in real-time.

·        Verizon now enables artificial intelligence and machine learning, predicting the customer intent by mining unstructured data and correlations

·        Ferrari, Honda & Red Bull use data generated by over 100 sensors in their Formula 

One cars and apply real-time analytics, giving drivers and their crews the information they need to make better decisions about pit stops, tire pressures, speed adjustments and fuel efficiency.

 

Real-Time analytics helps getting the right products in front of the people looking for them, or offering the right promotions to the people most likely to buy. For gaming companies, it helps in understanding which types of individuals are playing which game, and crafting an individualized approach to reach them.

 

As the pace of data generation and the value of analytics accelerate, real-time analytics is the top most choice to ride on this tsunami of information.

 

More and more tools such as Cloudera Impala, AWS, Spark, Storm, offer the possibility of real-time processing of Big Data and provide analytics,

Now is the time to move beyond just collecting, storing & managing the data to take rapid actions on the continuous streaming data – Real-Time!!