Skip to content

Pragmatic Category Theory | Part 3: Associativity

submitted by /u/ketralnis [link] [comments]
A Replacement for BERT

Finally, a Replacement for BERT TL;DR This blog post introduces ModernBERT, a family of state-of-the-art encoder-only models representing improvements over older generation encoders across the board, with a 8192 sequence length, better downstream performance and much faster processing. ModernBERT is available as a slot-in replacement for any BERT-like models, with both a base (139M params) and large (395M params) model size. ModernBERT will...
BlackSheep: Fast ASGI web framework for Python

BlackSheep is an asynchronous web framework to build event based web applications with Python. It is inspired by Flask, ASP.NET Core, and the work by Yury Selivanov. from datetime import datetime from blacksheep import Application, get app = Application() @get("/") async def home(): return f"Hello, World! {datetime.utcnow().isoformat()}" Getting started using the CLI ✨ BlackSheep offers a CLI to bootstrap new projects rapidly. To try it, first install the blacksheep-cli...
Comparing Rust Database Crates

This page aims to compare Diesel with various other crates that allow to connect to relational databases. We will cover the following crates: As part of this comparison we will try to cover the following topics: Stability, which covers what stability guarantees are provided by the corresponding crates Safety guarantees, which covers what is checked by a specific crate at compile time Flexibility, which covers...
Query 1B Rows in PostgreSQL >25x Faster with Squirrels!

The One Billion Row Challenge has been making waves in the data engineering community lately. Originally created to test CSV parsing performance, the challenge involves processing a file containing 1 billion weather measurements to calculate basic temperature statistics for each city. In this post, I'll tackle a variation of this challenge using PostgreSQL and demonstrate how to achieve dramatic performance...
Optimize Amazon RDS performance with io2 Block Express storage for production workloads

In March 2024, Amazon Relational Database Service (Amazon RDS) announced the availability of Provisioned IOPS (input/output operations per second) io2 Block Express volumes for database engines. Choosing the right storage configuration that meets performance requirements is a common challenge when creating and managing database instances. With io2 Block Express volumes, your database workloads benefit from consistent sub-millisecond latency, enhanced durability...

Tag: Spark

pandas on spark apply_batch/transform_batch broken? (tl;dr; No – but it isn’t well documented)

pandas on spark apply_batch/transform_batch broken? (tl;dr; No – but it isn’t well documented)

Published by Arnon Rotem-Gal-Oz on October 16, 2022

Using pypark’s pandas integration via apply_batch and transform_batch is very powerful but lacking documentation can cause hard to trace bugs – hopefully my experience (below)…

Continue reading

Where is Apache Spark heading?

Where is Apache Spark heading?

Published by Arnon Rotem-Gal-Oz on December 4, 2020

I watched (COVID19-era version of “attended”) the latest spark Summit and in one of the keynotes Reynold Xin from Databricks, presented the following two images…

Continue reading

Spark, Parquet and S3 – It’s complicated.

Spark, Parquet and S3 – It’s complicated.

Published by Arnon Rotem-Gal-Oz on August 10, 2015

(A version of this post was originally posted in AppsFlyer’s blog. Also special thanks to Morri Feldman and Michael Spector from AppsFlyer data team that…

Continue reading

Apache Spark, ETL and Parquet

Apache Spark, ETL and Parquet

Published by Arnon Rotem-Gal-Oz on September 14, 2014

(Edit 10/8/2015 : A lot has changed in the last few months – you may want to check out my new post on Spark, Parquet…

Continue reading

Is there a future for Map/Reduce?

Is there a future for Map/Reduce?

Published by Arnon Rotem-Gal-Oz on June 3, 2014

Google’s Jeffrey Dean and Sanjay Ghemawat filed the patent request and published the map/reduce paper 10 year ago (2004). According to WikiPedia Doug Cutting and Mike Cafarella created Hadoop,…

Continue reading