RT @datanami: Cloudera Picks Iceberg, Touts 10x Boost in Impala

Arnon Rotem-Gal-Oz (@arnonrgo) retweeted:Cloudera Picks Iceberg, Touts 10x Boost in Impala ow.ly/mlRZ50JMWo9- Datanami (@datanami)12:25 - Jul 01, 2022
Purpose-Driven Microservice Design

Creating purpose-driven microservices should always be a goal. Find out how Render Blueprints can offer a reproducible strategy.Buzzwords aren’t something I expected when I started my career. In those days, most of the technology news arrived in paper-based weekly publications like InformationWeek and Network World. I remember thinking to myself, “Man, they are using these same words over and over...
RT @myfear: Cloud Native AppDev without Tears - Using #odo to Boost your Workflow – @DBrintzinger #Kubernetes https…

Arnon Rotem-Gal-Oz (@arnonrgo) retweeted:Cloud Native AppDev without Tears - Using #odo to Boost your Workflow – @DBrintzinger bit.ly/3bqOZ71 #Kubernetes pic.twitter.com/lSRbS6WeFe- Markus Eisele (@myfear)06:00 - Jun 30, 2022
Why Spark chooses Volcano as built-in batch scheduler on Kubernetes?

Guest post by Volcano Maintainers On June 16, 2022, Apache Spark released its new version, v3.3. The highlight of this version is that it provides framework support for customized Kubernetes schedulers and, for the first time, uses Volcano as the default batch scheduler. Spark users can now easily move from Hadoop to Kubernetes and achieve high performance on large-scale data...
What's new in Apache Spark 3.3 - new functions

New Apache SQL functions are a regular position in my "What's new in Apache Spark..." series. Let's see what has changed in the most recent (3.3.0) release!
Why the Open Sourcing of Databricks Delta Lake Table Format Is a Big Deal - Datanami

Why the Open Sourcing of Databricks Delta Lake Table Format Is a Big Deal DatanamiDatabricks open sources its Delta Lake data lakehouse InfoWorldSnowflake, Databricks Rival in Open Source With Delta Lake and Iceberg Business InsiderDatabricks strikes back VentureBeatView Full Coverage on Google News

Category: Big Data

Intro to Apache Spark (slides)

Published by Arnon Rotem-Gal-Oz on December 16, 2020

I gave a general overview of Apache Spark to our R&D teams. You can find the slides below

Where is Apache Spark heading?

Where is Apache Spark heading?

Published by Arnon Rotem-Gal-Oz on December 4, 2020

I watched (COVID19-era version of “attended”) the latest spark Summit and in one of the keynotes Reynold Xin from Databricks, presented the following two images…

Big data isn’t – well, almost

Big data isn’t – well, almost

Published by Arnon Rotem-Gal-Oz on March 23, 2019

Back in ancient history (2004) Google’s Jeff Dean & Sanjay Ghemawat presented their innovative idea for dealing with huge data sets – a novel idea…

Big data in the cloud – welcome to cost oriented design

Big data in the cloud – welcome to cost oriented design

Published by Arnon Rotem-Gal-Oz on April 1, 2016

A couple of weeks ago I presented @ BDX2016 The slides are available on Slideshare Big data in the cloud – welcome to cost oriented…

Spark, Parquet and S3 – It’s complicated.

Spark, Parquet and S3 – It’s complicated.

Published by Arnon Rotem-Gal-Oz on August 10, 2015

(A version of this post was originally posted in AppsFlyer’s blog. Also special thanks to Morri Feldman and Michael Spector from AppsFlyer data team that…