Skip to content

Are You a Stable or Volatile?

I've written before on this blog about the different personality traits of software engineers, and how different traits balance each other on a team.Some of the spectra I came up with were:Dreamers ↔ PragmatistsBig Picture ↔ Detail-OrientedMove Fast & Break Things ↔ Slow & MethodicalOptimists ↔ PessimistsAnswerers ↔ QuestionersRecently I read a comment on Hacker News (which I can't find...
Meta Uses LLMs to Improve Incident Response

In June, Meta released an article titled Leveraging AI for efficient incident response on their engineering blog. In this article, engineers outline how they leveraged large language models to improve Meta's incident response capabilities. The headline metric from this report: Meta was able to use LLMs to successfully root cause incidents with 42% accuracy in their web monorepo. This means...
Understanding the BM25 full text search algorithm

submitted by /u/emschwartz [link] [comments]
Offset Considered Harmful or: The Surprising Complexity of Pagination in SQL

submitted by /u/TheCrush0r [link] [comments]
Building SQL-based Observability With ClickHouse and Grafana

submitted by /u/mmaksimovic [link] [comments]
The Root Cause of Why Organizations Fail With Data & AI

Wave by CLEANPNGDEMYSTIFY DATA STRATEGYA guide to be successful with the strategic groundwork requiredBusiness value through Data and Artificial Intelligence. Everybody talks about it, yet most companies are struggling to monetize their data. I claim that in most cases, this is due to a lack of effective business strategy. This article shows what strategic groundwork is necessary to activate your company’s data assets.This...

Tag: Hadoop

Intro to Apache Spark (slides)

Intro to Apache Spark (slides)

Published by Arnon Rotem-Gal-Oz on December 16, 2020

I gave a general overview of Apache Spark to our R&D teams. You can find the slides below

Big data isn’t – well, almost

Big data isn’t – well, almost

Published by Arnon Rotem-Gal-Oz on March 23, 2019

Back in ancient history (2004) Google’s Jeff Dean & Sanjay Ghemawat presented their innovative idea for dealing with huge data sets – a novel idea…

Continue reading

Hadoop and the OpenDataPlatform

Hadoop and the OpenDataPlatform

Published by Arnon Rotem-Gal-Oz on February 17, 2015

Pivotal, IBM and Hortonworks announced today the “Open Data Platform” (ODP) – an attempt to standardize Hadoop. This move seems to be backed up by…

Continue reading

Random thoughts on big data

Random thoughts on big data

Published by Arnon Rotem-Gal-Oz on February 10, 2015

I began blogging in 2005, back then I managed to post something new almost everyday. Now, 10 years after, I hardly post anything. I was…

Continue reading

Apache Spark, ETL and Parquet

Apache Spark, ETL and Parquet

Published by Arnon Rotem-Gal-Oz on September 14, 2014

(Edit 10/8/2015 : A lot has changed in the last few months – you may want to check out my new post on Spark, Parquet…

Continue reading