CATEGORY: apache-spark

  • SFTP Package for Apache Spark

    January 12, 2016 | Author Prabhu Palanisamy

    According to Gartner, By 2018 spark processing will dominate over hadoop based processing. We are a big fan of Apache Spark and started building our framework using Spark as the data processing layer. Along the way we contributed to the spark community by releasing Salesforce connector. Now we are excited… [Read More.]

  • Apache Spark Wave Connector

    August 16, 2015 | Author SpringML

    We at springML have published a Spark package that connects to Salesforce Wave to push data.  This package is available here.  Code is published on GitHub here. The advantage of using this package is that once data is analyzed in the Spark environment, the resulting results dataset can be written… [Read More.]

  • SparkR and machine learning

    August 16, 2015 | Author SpringML

    Iteration and convergence is a key requirement for machine learning and Spark does this well and fast because it can load data in memory and do in memory computation.  In addition its support for languages like Python and R helps data scientists who are at home with these languages. SparkR… [Read More.]

  • Apache Spark and Databricks

    August 16, 2015 | Author SpringML

    Databricks is a company founded by the creators of Apache Spark.  As you can tell from our other blogs, we believe Apache Spark is a revolutionary big data technology and springML provides expert consulting services in Spark implementations.  We have been working on the Databricks platform and love the great… [Read More.]

  • Using Apache Spark for big data processing

    August 16, 2015 | Author SpringML

    Introduction to Spark Over the last few years, Hadoop has emerged as the standard platform for big data processing.  At a high level, Hadoop consists of distributed storage (HDFS) and distributed computing (MapReduce).  However MapReduce has a couple of limitations.  Programming in MapReduce is difficult and needs chaining multiple MapReduce… [Read More.]

  • Spark connector for Salesforce Wave!

    August 4, 2015 | Author Prabhu Palanisamy

    We are delighted and excited to share that we have published the Spark connector for Salesforce Wave! Using this connector makes it really easy to run machine learning algorithm in spark and push the results to Salesforce Wave. Highlights of the connector: Packaged library that you can deploy in your spark clusters.… [Read More.]