Languages like Scala are making it easier to implement systems with distributed domains and distributed computation. Deployment, monitoring and operation of such systems is often neglected, left to the last moment. Jan will show how architect and implement your system with DevOps in mind right from the start.
This talk presents Apache Spark, Spark Streaming, Apache Kafka, Apache Cassandra and Akka as supporting Lambda architecture in the context of a fault tolerant, streaming big data pipeline. We will walk through the Fault Tolerance story with these technologies to build applications, and how to easily implement and integrate them in a Scala Akka application for real-time delivery of meaning at high velocity, in highly distributed and concurrent environments.
Podcast: Play in new window | Download (Duration: 47:47 — 250.8MB) | Embed
From the abstract: Spark is an open-source computation platform for Big Data. Leaders in the Hadoop community, such as Cloudera, have embraced Spark as a replacement for MapReduce, the venerable standard for writing Hadoop jobs. This talk explores why this change is needed. Spark provides two important benefits compared to MapReduce. First, its performance is … Read More
Spark provides two important benefits compared to MapReduce. First, its performance is significantly better than MapReduce. We’ll discuss why. Second, because Spark is implemented in Scala and rooted in the world of functional programming, it provides better, more composable primitives that make it easier for developers to create a wide variety of high-performance applications. We’ll discuss these primitives and look at some example applications.
Podcast: Play in new window | Download () | Embed
Our old pal Jamie Allen was in the neighborhood talking this-and-that about Scala, Akka and other sundry Typesafe products, so we roped him in to do the DevNews.
Jamie is currently working with Roland Kuhn on Reactive Design Patterns.