Cirrus Minor - The nexus of technology, business & people

Skip to content

EP62: Why Does Google Use Monorepo?

This week’s system design refresher:Top 6 most popular API architecture styles (Youtube video)Monorepo vs. MicrorepoInside the HTTP HeaderDeveloper RoadmapsBytebytego CollectiveTop 6 most popular API architecture stylesDo you believe that Google, Meta, Uber, and Airbnb put almost all of their code in one repository? This practice is called a monorepo. Guest post by Xiong Wang.Monorepo vs. Microrepo. Which is the best?...
Multi-Agent Reinforcement Learning: Foundations and Modern Approaches

Download Note: this is the pre-print, non-final version of the book. If you spot any issues in the book (typos, errors in definitions, inaccurate descriptions, unclear figures/images, etc.), please report the issue via e-mail to: issues ~at~ marl-book ~dot~ com Download pre-print PDF (draft date: 29 May 2023) To receive updates about new versions, follow us on Twitter: Stefano, Filippos, Lukas Citation Stefano V. Albrecht,...
SaaS observability done right

Guest post originally published on Logz.io’s blog by Dotan Horovits SaaS (software as a service) is the common model for many businesses today. Even longstanding behemoths such as Cisco and Microsoft have been strategically shifting their software products to SaaS and recurring revenue models (just think Office365 shift from licensed Office). These SaaS businesses need agility to move fast and remain competitive. This...
Adaptive Query Execution in Structured Streaming

In Databricks Runtime, Adaptive Query Execution (AQE) is a performance feature that continuously re-optimizes batch queries using runtime statistics during query execution. Starting...
Active Active

For many types of services and systems it is important to provide low latency as well as high availability for business domain entities. Achieving this requires being able to both read and update state for the same entity in multiple physical locations, data centers or cloud regions. Interactions for a user can be directed to the closest geographic location to get...
Iceberg Community News • Tabular

Design and implement Tabular’s user experience, where people will create, monitor, and manage their data platform

pandas on spark apply_batch/transform_batch broken? (tl;dr; No – but it isn’t well documented)

pandas on spark apply_batch/transform_batch broken? (tl;dr; No – but it isn’t well documented)

Published by Arnon Rotem-Gal-Oz on October 16, 2022

Using pypark’s pandas integration via apply_batch and transform_batch is very powerful but lacking documentation can cause hard to trace bugs – hopefully my experience (below)…

Continue reading

Replacing Docker Desktop with hyperkit + minikube

Replacing Docker Desktop with hyperkit + minikube

Published by Arnon Rotem-Gal-Oz on September 2, 2021

Edit June 2023: Added a section on Colima MacOS is a Unix but it isn’t a Linux so, unfortunately, if/when we need to use linux-y…

Continue reading

Intro to Apache Spark (slides)

Intro to Apache Spark (slides)

Published by Arnon Rotem-Gal-Oz on December 16, 2020

I gave a general overview of Apache Spark to our R&D teams. You can find the slides below

Where is Apache Spark heading?

Where is Apache Spark heading?

Published by Arnon Rotem-Gal-Oz on December 4, 2020

I watched (COVID19-era version of “attended”) the latest spark Summit and in one of the keynotes Reynold Xin from Databricks, presented the following two images…

Continue reading

Intro to Docker & Kubernetes (slides)

Intro to Docker & Kubernetes (slides)

Published by Arnon Rotem-Gal-Oz on May 25, 2020

Just had a 2 hours session with my team introducing Docker and Kubernetes (presentation and hands-on). I thought the slides might be useful to others: