Implementing a Database System with DuckDB for Local Processing and MotherDuck for Scalable Cloud…

OverviewContinue reading on T3CH »
Scaling Large Language Models for e-Commerce: The Development of a Llama-Based Customized LLM

Third-party LLMs like Llama 3.1 allow us to adapt powerful tools for the e-commerce domain with a mix of eBay and general data to enable our magical AI experiences.
Thoughts on a Month with Devin

In March 2024, a new AI company burst onto the scene with impressive backing: a $21 million Series A led by Founders Fund, with support from industry leaders including the Collison brothers, Elad Gil, and other tech luminaries. The team behind it? IOI gold medalists - the kind of people that solve programming problems most of us can’t even understand....
Nepenthes is a tarpit to catch AI web crawlers

Nepenthes This is a tarpit intended to catch web crawlers. Specifically, it's targetting crawlers that scrape data for LLM's - but really, like the plants it is named after, it'll eat just about anything that finds it's way inside. It works by generating an endless sequences of pages, each of which with dozens of links, that simply go back into a the tarpit. Pages are...
Scalable Automated Config-Driven Data Validation with ValiData

As the world's largest professional network company, LinkedIn processes and stores massive volumes of data daily. To keep up with the fast-changing business requirements, these datasets often evolve continuously over time due to changes in business strategy, tech stack migrations of underlying platforms, addition or removal of a set of features, changes to specific field definitions, changes to filter conditions...
Scaling Strategies: Why Scaling Up Isn’t Always a Bad Idea

Image Generated with AIWhen you think about scaling, do you think of scaling up or scaling out? What is your go-to strategy?🎞️ Scaling Recap:As a system’s usage grows, additional demand begins to create resource constraints.Resource constraints could be CPU, Memory, or something internal to the application. Either way, if more resources are not made available, the system will experience issues at some point.Scaling...

Service Watchdog

Problem

How can you increase availability by identifying and resolving problems and failures that are service-specific?

Solution

Implement the Service Watchdog pattern, where the service actively monitors its internal state, acts on potential trouble, tries to heal itself, and continuously publishes its status.