Google Cloud Dataflow

Google Cloud Dataflow has recently entered beta.  It provides a powerful big data processing platform in the cloud.  It’s an end to end solution that comprises compute resources, big data storage, data pipeline development and execution.  Google has open sourced the Cloud Dataflow SDKs that allows users to define data processing jobs.  Here’s a high level overview of dataflow.

For those users and customers who have already invested in Google cloud platform for application development, dataflow offers an easy way to perform analytics.  However we at springML think this platform can be extended to other use cases where data from various systems can be consumed in the Cloudflow platform to run analytics and the resulting summarized and enriched data can then be ingested into a visualization framework e.g. Salesforce Wave.

Here are a few enterprise use cases that can leverage the power of Dataflow.

  1. Consuming journal lines from a Financial system. Dataflow can consume the large number of journal lines from an enterprise Financial system.  The journal lines are high volume and can easily run into 100’s of millions of lines.  Once ingested into Dataflow, the journal line data can be enriched with additional data such as billing or customer data, for example, and various transformations and analytics can be run.
  2. Customer segmentation via RFM analysis. RFM (recency, frequency, monetary) analysis is a marketing technique used to determine quantitatively which customers are the best ones by examining how recently a customer has purchased (recency), how often they purchase (frequency), and how much the customer spends (monetary). RFM analysis is based on the marketing axiom that “80% of your business comes from 20% of your customers.  For retail or ecommerce customers, where the number of customers and transactions can be large Dataflow provides a scalable architecture to perform analysis.

There are of course many other use cases to leverage Dataflow in the enterprise.  This is truly a game changing platform that Google has now released in Beta phase.