Microsoft fires up its enthusiasm for Apache Spark with new "extensive commitment"

Last year, Microsoft announced Spark for Azure HDInsight, promising "the best environment to run Apache Spark" with a managed service in the cloud.

Today, as the Spark Summit kicks off in San Francisco, Microsoft has announced an "extensive commitment for Spark to power Microsoft's big data and analytics offerings, including Cortana Intelligence Suite, Power BI and Microsoft R Server".

After launching Spark for Azure HDInsight as a public preview last July, it's become generally available today, and Microsoft also highlighted a "fully managed Spark service from Hortonworks that has been hardened for the enterprise and made simpler for you to use."

Tiffany Wissner, Microsoft's Senior Director of Data Platform Marketing also drew attention to the other components of the company's "major commitment to Apache Spark":

R Server for HDInsight in the cloud powered by Spark, previously announced as public preview, R Server for HDInsight will be generally available in the summer making the Spark integration available both on-premises and in the cloud. This makes it easy to move code and projects to the cloud with a few clicks and within a few minutes without buying hardware or hiring specialized operations teams typically associated with big data infrastructure.

R Server for Hadoop on-premises now powered by Spark, as the leading solution in the world to run R at scale,R Server for Hadoop will support both Microsoft R and native Spark execution frameworks available in June. Combining R Server with Spark gives users the ability to run R functions over thousands of Spark nodes letting you train your models on data 1000x larger and 100x faster than was possible with open source R and nearly 2x faster than Spark’s own MLLib.

Power BI support for Spark Streaming, previously announced with Power BI General Availability, Spark support in Power BI is now expanded with new support for Spark Streaming scenarios. This allows you to publish real-time events from Spark Streaming directly into one of the fastest growing visualization tools in the market today.

Wissner added that Microsoft will share more on its plans at the Spark Summit this week, and further details can be found on its Apache Spark and R Server sites.

Source: Microsoft SQL Server Blog