It's Time to Stop Building KV Databases

March 24, 2025 It's Time to Stop Building KV Databases I'm so sick of Key-Value. The most unopinionated, uninspired data model that is a pain to use for anything. They're very popular! Among database vendors. Or, specifically, storage engine vendors. They love to offer you a byte array to byte array map, and if they are feeling particularly generous, it might even be...
The New Look and Feel of Apache Kafka 4.0

The recently introduced Apache Kafka 4.0 comes with a number of upgrades across nearly every aspect of the open source distributed event streaming platform. The recent release has numerous Kafka Improvement Proposals (KIPs) — which provide new functionality from the open source community — across Kafka Streams, Kafka Connect, and Kafka brokers, consumers, producers, and more. Nevertheless, the most significant part...
"Data" Sucks

'Data' sucksMarch 23, 2025One of the two hard things about programming is naming things, and as programmers we spend a fair bit of time thinking about what we name different 'things'. Having a consistently good naming scheme across a large codebase can significantly speed up development, and having confusing names will slow it down. Let's say the "badness" of a variable...
A rust based programming assistant

Working on a rust based coding assistant with strong agentic capabilities! Looking for contributors, welcome to the future of programming. submitted by /u/amritk110 [link] [comments]
Harmel-Law's architecture advice process and fast flow

I’m a fan of Andrew Harmel-Law’s book Facilitating Software Architecture. It’s a must-read for anyone who wants to improve their architecture skills. One of the many gems in the book is the ‘Architecture Advice Process’. It describes how an organization should make architectural decisions. The architecture advice process is a refreshing and superior alternative to traditional architecture practices. Most notably, it promotes team autonomy and...
Applying the Fundamental Axioms to Reduce Uncertainty

The Fundamental Axioms of Project Planning introduced the two fundamental axioms: The Axioms of Project Management: Starting is definite, finishing less so. Divide and conquer to reduce uncertainty. So now let’s apply that to an ambiguous task and see how we can break it down into more manageable chunks. Let’s start with the classic example of building a house: Now that’s pretty ambiguous, we have really no idea how...

Category: Blog

Microservices Granularity Tradeoffs

Published by Arnon Rotem-Gal-Oz on March 17, 2025

When architects and developers embrace microservices, one of the most challenging questions is: “How big should each service be?” While true, the obvious answer –…

Software architecture workshop (slides)

Software architecture workshop (slides)

Published by Arnon Rotem-Gal-Oz on November 29, 2023

The title says it all – These are slides from a session I was working on to explain the basics of software architecture based on…

pandas on spark apply_batch/transform_batch broken? (tl;dr; No – but it isn’t well documented)

pandas on spark apply_batch/transform_batch broken? (tl;dr; No – but it isn’t well documented)

Published by Arnon Rotem-Gal-Oz on October 16, 2022

Using pypark’s pandas integration via apply_batch and transform_batch is very powerful but lacking documentation can cause hard to trace bugs – hopefully my experience (below)…

Replacing Docker Desktop with hyperkit + minikube

Replacing Docker Desktop with hyperkit + minikube

Published by Arnon Rotem-Gal-Oz on September 2, 2021

Edit June 2023: Added a section on Colima MacOS is a Unix but it isn’t a Linux so, unfortunately, if/when we need to use linux-y…

Intro to Apache Spark (slides)

Published by Arnon Rotem-Gal-Oz on December 16, 2020

I gave a general overview of Apache Spark to our R&D teams. You can find the slides below