V For Velocity

Generate, Distribute, Collect. Three key elements of data velocity. Data velocity is the speed at which data are processed. This includes inputs such as processing of social media posts, financial transactions, and outputs such as the processing requirements to take a critical decision or execute a process.

There are different types and techniques to process data. Real-time where data are processed really closed, in few milliseconds, to the time that they are input. An example of real-time processing is bank ATMs, where immediate processing is crucial to make the system work properly. Near real-time where data are processed as events after their input just like an IT system monitoring tool. Batch, where data are processed when computing resources are available. Examples of uses for batch processing include payroll and billing activities, which usually occur on monthly cycles, and deep analytics that are not essential for fast intelligence necessary for immediate decision making. Custom job, batch processing executed once. Analytical processing, tied to decision-making and real-time processes. Real-time data is only as helpful as your ability to execute it quickly. While high-velocity decision-making is important, fast execution is equally critical. Any delay can cost when downstream processes are slow and inflexible.

High-velocity data are generated at such a pace that it requires distinct (distributed) processing techniques. Nowadays we do not only need to acquire data as quickly as possible, but also process and use them at a faster rate. Many data types have limited life, in terms of value, and in some cases, their value lasts only for a few seconds. Analyzing data quickly can alert businesses to stocking issues fast so the problem can be solved before it gets worse. Data velocity can also speed up the decision-making process to keep up with market changes.

In the year 2000, Google was receiving 32.8 million searches per day, social media and web 2.0 were just meaningless words. As for now, Google is receiving more than 3.5 billion searches per day and Facebook currently has 2.7 billion monthly active users. The hidden data behind these numbers are beyond imagination. And where is the value to all that? Imagine that you are running a marketing campaign for a newly released product and you want to know how the consumers "out there" are feeling about it right now. Then think about the customer data collected today, online purchase data, click-through rates, browsing behavior, social media interactions, mobile device usage, geolocation data, etc. They can be valuable if and only if we get them today. They will have no value at all in a week or more. After all, time is money. 

Charalampos Grigoriadis, IT Systems Administrator, WITSIDE