Apache Spark

SFTP Package for Apache Spark

According to Gartner, By 2018 spark processing will dominate over hadoop based processing. We are a big fan of Apache Spark and started building our framework using Spark as the data processing layer. Along the way we contributed to the spark community by releasing Salesforce connector. Now we are excited to announce our next package SFTP …

SFTP Package for Apache Spark Read More »

Apache Spark Wave Connector

We at springML have published a Spark package that connects to Salesforce Wave to push data.  This package is available here.  Code is published on GitHub here. The advantage of using this package is that once data is analyzed in the Spark environment, the resulting results dataset can be written into Wave for further visualization on …

Apache Spark Wave Connector Read More »

SparkR and machine learning

Iteration and convergence is a key requirement for machine learning and Spark does this well and fast because it can load data in memory and do in memory computation.  In addition its support for languages like Python and R helps data scientists who are at home with these languages. SparkR is an R package that provides …

SparkR and machine learning Read More »

Apache Spark and Databricks

Databricks is a company founded by the creators of Apache Spark.  As you can tell from our other blogs, we believe Apache Spark is a revolutionary big data technology and springML provides expert consulting services in Spark implementations.  We have been working on the Databricks platform and love the great work they’ve done to simplify Spark …

Apache Spark and Databricks Read More »