<!DOCTYPE html><html lang="en"><head><meta http-equiv="Content-Type" content="text/html charset=UTF-8"><meta charset="UTF-8"><meta name="viewport" content="width=device-width"><meta name="x-apple-disable-message-reformatting"><title>TLDR Data</title><meta name="color-scheme" content="light dark"><meta name="supported-color-schemes" content="light dark"><style type="text/css">
:root {
color-scheme: light dark; supported-color-schemes: light dark;
}
*,
*:after,
*:before {
-webkit-box-sizing: border-box; -moz-box-sizing: border-box; box-sizing: border-box;
}
* {
-ms-text-size-adjust: 100%; -webkit-text-size-adjust: 100%;
}
html,
body,
.document {
width: 100% !important; height: 100% !important; margin: 0; padding: 0;
}
body {
-webkit-font-smoothing: antialiased; -moz-osx-font-smoothing: grayscale; text-rendering: optimizeLegibility;
}
div[style*="margin: 16px 0"] {
margin: 0 !important;
}
table,
td {
mso-table-lspace: 0pt; mso-table-rspace: 0pt;
}
table {
border-spacing: 0; border-collapse: collapse; table-layout: fixed; margin: 0 auto;
}
img {
-ms-interpolation-mode: bicubic; max-width: 100%; border: 0;
}
*[x-apple-data-detectors] {
color: inherit !important; text-decoration: none !important;
}
.x-gmail-data-detectors,
.x-gmail-data-detectors *,
.aBn {
border-bottom: 0 !important; cursor: default !important;
}
.btn {
-webkit-transition: all 200ms ease; transition: all 200ms ease;
}
.btn:hover {
background-color: #f67575; border-color: #f67575;
}
* {
font-family: Arial, Helvetica, sans-serif; font-size: 18px;
}
@media screen and (max-width: 600px) {
.container {
width: 100%; margin: auto;
}
.stack {
display: block!important; width: 100%!important; max-width: 100%!important;
}
.btn {
display: block; width: 100%; text-align: center;
}
}
body,
p,
td,
tr,
.body,
table,
h1,
h2,
h3,
h4,
h5,
h6,
div,
span {
background-color: #FEFEFE !important; color: #010101 !important;
}
@media (prefers-color-scheme: dark) {
body,
p,
td,
tr,
.body,
table,
h1,
h2,
h3,
h4,
h5,
h6,
div,
span {
background-color: #27292D !important; color: #FEFEFE !important;
}
}
a {
color: inherit !important; text-decoration: underline !important;
}
</style><!--[if mso | ie]>
<style type="text/css">
a {
background-color: #FEFEFE !important; color: #010101 !important;
}
@media (prefers-color-scheme: dark) {
a {
background-color: #27292D !important; color: #FEFEFE !important;
}
}
</style>
<![endif]--></head><body class="">
<div style="display: none; max-height: 0px; overflow: hidden;">DuckDB 2.0 LTS improves client-server support, faster file reads, improved semi-structured data handling, an AI agent mode, and SQL triggers. β β β β β β β β β β β β β β β β β β β β β β β β β β β β β β β β β β β β β β β β β β β β β β β β β β β β </div>
<div style="display: none; max-height: 0px; overflow: hidden;">
<br>
</div>
<table align="center" class="document"><tbody><tr><td valign="top">
<table align="center" border="0" cellpadding="0" cellspacing="0" class="container" width="600"><tbody><tr class="inner-body"><td>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr class="header"><td bgcolor="" class="container">
<table width="100%"><tbody><tr><td class="container">
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" style="margin-top: 0px;" width="100%"><tbody><tr><td style="padding: 0px;">
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div style="text-align: center;">
<span style="margin-right: 0px;"><a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Ftldr.tech%2Fdata%3Futm_source=tldrdata/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/2oOO1AmPmWaytBRMlP9DZ3hIye0DcxtfIz42Kp1qXMY=452" rel="noopener noreferrer" target="_blank"><span>Sign Up</span></a>
|<span style="margin-right: 2px; margin-left: 2px;"><a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fadvertise.tldr.tech%3Futm_source=tldrdata%26utm_medium=newsletter%26utm_campaign=advertisetopnav/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/PXYn5wDfyIdi2HIwgFfARLbRfzZOhHjoDHbxjIW7s9o=452" rel="noopener noreferrer" target="_blank"><span>Advertise</span></a></span>|<span style="margin-left: 2px;"><a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fa.tldrnewsletter.com%2Fweb-version%3Fep=1%26lc=aa6c57ca-be20-11f0-8e04-b5caa3a65826%26p=fc35c15c-cdda-11f1-a553-cb7122dc6967%26pt=campaign%26t=1792664616%26s=4f1919768f61848d0a4ecfb096c83116180b5a26f089f965c3cba96b098c07ea/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/LLUVg0yZKaZdZ-sA3k5pPB2btTGFtsjQN7yel8QBjJY=452"><span>View Online</span></a></span>
<br>
</span></div>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="text-align: center;"><span data-darkreader-inline-color="" style="--darkreader-inline-color:#3db3ff; color: rgb(51, 175, 255) !important; font-size: 30px;">T</span><span style="font-size: 30px;"><span data-darkreader-inline-color="" style="color: rgb(232, 192, 96) !important; --darkreader-inline-color:#e8c163; font-size:30px;">L</span><span data-darkreader-inline-color="" style="color: rgb(101, 195, 173) !important; --darkreader-inline-color:#6ec7b2; font-size:30px;">D</span></span><span data-darkreader-inline-color="" style="--darkreader-inline-color:#dd6e6e; color: rgb(220, 107, 107) !important; font-size: 30px;">R</span>
<br>
</td></tr></tbody></table>
<br>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr id="together-with"><td align="center" height="20" style="vertical-align:middle !important;" valign="middle" width="100%"><strong style="vertical-align:middle !important; height: 100%;">Together With </strong>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Faws.amazon.com%2Fmarketplace%2Fbuild-learn%2Fai-agent-learning-series%2Fdata-pipelines-and-lineage%3Ftrk=821f0c1f-c303-460c-bb91-ad4a8daced4c%26sc_channel=el%26utm_source=TLDR%26utm_medium=newsletter%26utm_campaign=2026-10-22_tldr_data_primary_aws%26utm_content=header_real_time_pipelines%26utm_term=2026-10-22_tldr_data/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/uAv_QLhRvFiE0bolB4Vmc34fVUh2VZRgrNW4zQ5r7xE=452"><img src="https://images.tldr.tech/awsmarketplace.png" valign="middle" style="vertical-align: middle !important; height: 100%;" alt="AWS"></a></td></tr></tbody></table>
<table style="table-layout: fixed; width:100%;" width="100%"><tbody><tr><td style="padding:0;border-collapse:collapse;border-spacing:0;margin:0;">
<div style="text-align: center;">
<h1><strong>TLDR Data <span id="date">2026-10-22</span></strong></h1>
</div>
</td></tr></tbody></table>
<table style="table-layout: fixed; width:100%;" width="100%"><tbody><tr id="sponsy-copy"><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Faws.amazon.com%2Fmarketplace%2Fbuild-learn%2Fai-agent-learning-series%2Fdata-pipelines-and-lineage%3Ftrk=821f0c1f-c303-460c-bb91-ad4a8daced4c%26sc_channel=el%26utm_source=TLDR%26utm_medium=newsletter%26utm_campaign=2026-10-22_tldr_data_primary_aws%26utm_content=header_real_time_pipelines%26utm_term=2026-10-22_tldr_data/2/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/liXvs1qp7UiYVNPDZJ-qYSn4SIuIlQk0PxSAFvhavn8=452">
<span>
<strong>Real-time pipelines for clean data (Sponsor)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
Stakeholders often spot broken data before the data team does.<p></p><p>According to a Monte Carlo/Wakefield survey, 74% of organizations now report it. The failure is usually the ingestion path, not the dashboard. βBuilding a real-time data transport layer with Confluent Cloud and AWS' demonstrates the change-data-capture transport pattern. <a class="ProsemirrorEditor-link" href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Faws.amazon.com%2Fmarketplace%2Fbuild-learn%2Fdata-analytics%2Fbuilding-a-real-time-data-transport-layer-with-confluent-cloud-and-aws%3Ftrk=095d76cb-2f2a-4c77-9d64-7414ca1355cb%26sc_channel=el%26utm_source=TLDR%26utm_medium=newsletter%26utm_campaign=2026-10-22_tldr_data_primary_aws%26utm_content=body_tutorial_aws_marketplace%26utm_term=2026-10-22_tldr_data/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/QHypMqk54CuJ3NWWPK1yC5hNXpWCKulSzpdokxc95Fg=452" rel="noopener noreferrer nofollow" target="_blank"><span>Read the tutorial in AWS Marketplace</span></a>. The Data Pipelines and Lineage guide goes a level up from the tutorial: lineage from source to AI agent response, data quality, pipeline observability.</p>
<p><a class="ProsemirrorEditor-link" href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Faws.amazon.com%2Fmarketplace%2Fbuild-learn%2Fai-agent-learning-series%2Fdata-pipelines-and-lineage%3Ftrk=821f0c1f-c303-460c-bb91-ad4a8daced4c%26sc_channel=el%26utm_source=TLDR%26utm_medium=newsletter%26utm_campaign=2026-10-22_tldr_data_primary_aws%26utm_content=cta_data_pipeline_patterns%26utm_term=2026-10-22_tldr_data/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/knH-CnuXQcFBhXUaE9do6j6INAPV0htprIW4J1Xtvmk=452" rel="noopener noreferrer nofollow" target="_blank"><span>See the data pipeline patterns</span></a>.
</p>
</span></span></div>
</td></tr></tbody></table>
</td></tr></tbody></table>
</td></tr></tbody></table>
</td></tr>
<tr bgcolor=""><td class="container">
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td style="padding: 0px;">
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;"><span style="font-size: 36px;">π±</span></div></div>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;">
<h1><strong>Deep Dives</strong></h1>
</div>
</div>
</td></tr></tbody></table>
<table style="table-layout: fixed; width: 100%;" width="100%"><tbody><tr><td style="padding:0;border-collapse:collapse;border-spacing:0;margin:0;" valign="top">
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fshopify.engineering%2Fopen-model-task-experts%3Futm_source=tldrdata/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/_BtoDlq0Cp9IsXZGu8wPw_8zPuYiAv9LI6fcKVjP1F4=452">
<span>
<strong>How Shopify turns open models into task experts (5 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
Shopify trains and hosts specialist open models for Sidekick's focused tasks, using production failures and human feedback to improve them. Its GraphQL specialist slightly outperformed the frontier model on Shopify's evaluation while responding 7x faster and costing about 90% less to serve.
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Flinks.tldrnewsletter.com%2FkLhP8u/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/JGsEWqiGblCKVYwfLqYGVtolpwjPnw11_ixuw5JEgrQ=452">
<span>
<strong>Governing Data and AI Agents in Snowflake: A Data Engineer's Field Notes (25 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
Governing AI agents in Snowflake starts with strong data controls: identify sensitive information, apply access and masking policies, and monitor usage. Agent-specific permissions, masking, and prompt injection checks add protection by limiting what agents can access, see, and change.
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Flinks.tldrnewsletter.com%2F9ejxUP/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/QA6B5UkkGQ2D30j1jGrjVi3KdYURcp9-7qmS6KZqKsE=452">
<span>
<strong>From Disk Pages to Columnar Memory: Why Apache Arrow? (13 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
Apache Arrow addresses a different problem than Parquet: fast in-memory analytics rather than compact storage. By keeping data in contiguous typed buffers instead of heap objects, Arrow preserves column locality for cache-friendly scans and SIMD execution, and it also reduces repeated serialization between engines like Spark, pandas, and DuckDB.
</span>
</span>
</div>
</td></tr></tbody></table>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;"><span style="font-size: 36px;">π</span></div>
</div>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;">
<h1><strong>Opinions & Advice</strong></h1>
</div>
</div>
</td></tr></tbody></table>
<table style="table-layout: fixed; width: 100%;" width="100%"><tbody><tr><td style="padding:0;border-collapse:collapse;border-spacing:0;margin:0;" valign="top">
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Flinks.tldrnewsletter.com%2Fp6mVTh/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/2ee-b7eJOQc_dQDo5uEmgalxyrRP9QRRJdd1OrU_Ecc=452">
<span>
<strong>Why Most Analytics Teams Won't Survive the Next 5 Years (23 minute video)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
AI is putting pressure on analytics teams that simply fulfil requests, as businesses increasingly believe they can get answers without them. To stay valuable, data professionals must understand the business, challenge assumptions, and turn reliable analysis into clear decisions.
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Finferenceengineer.substack.com%2Fp%2Fthe-application-is-scaffolding%3Futm_source=tldrdata/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/1jEN3K-8C6NSp5JGMFxqdhuzNYqdwJ235FfH_cZiNYI=452">
<span>
<strong>The Application Is Scaffolding (12 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
As software assembly gets cheaper, the durable asset shifts from application code to a governed historical record of entities, state changes, and provenance. The key lesson is to let AI generate queries and transformations, while append-only history and deterministic validation preserve meaning, permissions, and auditable βas-ofβ views.
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Flinks.tldrnewsletter.com%2FyYZZZE/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/kgtNKdjaouTeyu1KQqYMQcZNv-Zzm3uD8uepSZ-kV1c=452">
<span>
<strong>Scaling AI-Native Analytics Patterns at Meta (6 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
Meta has reduced repeated product-analytics work to seven parameterized measurement patterns and exposes their governed outputs through agent-readable models and a deterministic query generator. The library now powers more than 9,000 production pipelines, including 3,250 metric-materialization pipelines. Agents choose inputs and recipes, while code, SQL, and validation remain controlled.
</span>
</span>
</div>
</td></tr></tbody></table>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;"><span style="font-size: 36px;">π»</span></div>
</div>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;">
<h1><strong>Launches & Tools</strong></h1>
</div>
</div>
</td></tr></tbody></table>
<table style="table-layout: fixed; width: 100%;" width="100%"><tbody><tr><td style="padding:0;border-collapse:collapse;border-spacing:0;margin:0;" valign="top">
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fadvertise.tldr.tech%2F%3Futm_source=tldrdata%26utm_medium=newsletter%26utm_campaign=secondary10222026%26sp=E622CpJabL-s8hUboW3G-nlD/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/O3TbuUO9a7pb5jgMKSUJxPqpzPowsZ2rGkxh2jaW52c=452">
<span>
<strong>Reach 8 million tech professionals reading TLDR (Sponsor)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
Ads get ignored on social media but not in TLDR! Reach data engineers, analysts and data leaders where they actually pay attention. <a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fadvertise.tldr.tech%2F%3Futm_source=tldrdata%26utm_medium=newsletter%26utm_campaign=secondary10222026%26sp=E622CpJabL-s8hUboW3G-nlD/2/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/iZ82oiPGFAylYBTo0y2sxTD6DeHeQRTHWEI6qhTOscw=452" rel="noopener noreferrer nofollow" target="_blank"><span>Learn more about sponsorship opportunities</span></a>.
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fduckdb.org%2F2026%2F10%2F21%2Fannouncing-duckdb-200%3Futm_source=tldrdata/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/sN0bmVznCiuCr4ILeyBumNklkWlWZOvVjKrE7et065I=452">
<span>
<strong>Announcing DuckDB 2.0.0 LTS βCinnamon Tealβ (14 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
DuckDB 2.0 LTS improves client-server support, faster file reads, improved semi-structured data handling, an AI agent mode, and SQL triggers. It is supported until at least October 2027, but database files created in 2.0 cannot be read by older versions.
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fgithub.com%2Fdebezium%2Fdebezium-ai-python%3Futm_source=tldrdata/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/olpSKhFapDqfXOalzJAh7b34-ZiOeTKDfBZC9Do9J1U=452">
<span>
<strong>PyDebeziumAI (GitHub Repo)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
PyDebeziumAI turns database changes into deterministic vector-store upserts and deletes by embedding Debezium's CDC engine inside a Python process. It removes Kafka and Connect from small RAG deployments, compacts bursts of mutations, and supports Chroma, PGVector, and Milvus. The contribution is treating retrieval freshness as a CDC problem, with idempotency and a dead-letter queue rather than periodic re-indexing.
</span>
</span>
</div>
</td></tr></tbody></table>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;"><span style="font-size: 36px;">π</span></div></div>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;"><strong><h1>Miscellaneous</h1></strong></div>
</div>
</td></tr></tbody></table>
<table bgcolor="" style="table-layout: fixed; width: 100%;" width="100%"><tbody><tr><td style="padding:0;border-collapse:collapse;border-spacing:0;margin:0;" valign="top">
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fengineering.fb.com%2F2026%2F10%2F19%2Fdata-center-engineering%2Finside-prometheus-meta-gigawatt-scale-supercluster%3Futm_source=tldrdata/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/fMXLKjGpJz1SqhYGi9ElCqM5IEPU_qAH25PfYiHRnx8=452">
<span>
<strong>Inside Prometheus: Meta's Journey to a Gigawatt-Scale Supercluster (12 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
Meta's Prometheus is built to keep giant distributed training runs productive at extreme scale. The supercluster is growing toward roughly 1 million GPUs, with hundreds of thousands already deployed, while its networking, resilience, and power design helped push training goodput above 85% and cut interruptions by 99% since mid-2025.
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fblog.gopenai.com%2Fone-engineer-trained-a-3-8b-parameter-llm-from-scratch-for-998-7ef5cd7809b7%3Futm_source=tldrdata/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/O1XB88grwgaxNap5tevGNmps4YCB9uN1i1bH1Szg4kk=452">
<span>
<strong>One Engineer Trained a 3.8B Parameter LLM from Scratch for $998 (12 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
A solo engineer trained a 3.848B-parameter LLM from scratch for a reported $998 by pairing a lightweight stack with a few high-leverage efficiency choices. The run finished in 43 hours on eight rented B200s and reached 0.384 CORE on DCLM, roughly 50% above GPT-2's reported score.
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fwww.architecture-weekly.com%2Fp%2Fthoughts-on-dynamic-consistency-boundary%3Futm_source=tldrdata/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/kA6L1cknGu7xj3wu3QI3SWKyyoAHfT9BmltWcWmI2_g=452">
<span>
<strong>Thoughts on Dynamic Consistency Boundary (31 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
Dynamic Consistency Boundaries let each event-sourced command protect only the events and types it reads, avoiding false conflicts from unrelated changes. The counterargument is that tag-based implementations merely relocate fixed boundaries into high-cardinality indexes, while schema changes, retagging, archiving, and query evolution become harder. Prefer multi-stream transactions and workflow streams unless narrower contention is proven.
</span>
</span>
</div>
</td></tr></tbody></table>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;"><span style="font-size: 36px;">β‘</span></div></div>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;">
<h1><strong>Quick Links</strong></h1>
</div>
</div>
</td></tr></tbody></table>
<table bgcolor="" style="table-layout: fixed; width: 100%;" width="100%"><tbody><tr><td style="padding:0;border-collapse:collapse;border-spacing:0;margin:0;" valign="top">
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fblog.dataengineerthings.org%2Fdbt-snapshots-cant-handle-late-data-so-we-built-one-that-can-ef107cc9e02e%3Futm_source=tldrdata/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/0_-KhqCwnrGXhVupN_84fjp04zaS5pTUAMsuwKkO1RE=452">
<span>
<strong>dbt Snapshots Can't Handle Late Data. So We Built One That Can (6 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
Mercedes-Benz's open-source dbt_iterative_snapshot package for Databricks processes batches chronologically and automatically repairs historical records when late data arrives.
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fwww.mercor.com%2Fresearch%2Flabor-index%2Fbenchmark-map%2F%3Futm_source=tldrdata/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/X4x9T36dTCOrWzoUHc4433826MqKvC-dXoC-AUAwIeI=452">
<span>
<strong>A benchmark map of the US economy (17 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
Mercor's analysis of 29 AI benchmarks estimates that tasks accounting for 90% of US working hours have little or no coverage, exposing major gaps in how we measure AI's workplace capabilities.
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fcameronrwolfe.substack.com%2Fp%2Fsubjective-evals%3Futm_source=tldrdata/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/03djOsFjuL9mU04oa5nI2w9RNJw5mq5L4esDcmdkqVU=452">
<span>
<strong>Evaluating Agents on Highly Subjective Tasks (20 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
Reliable evaluation of AI agents on subjective tasks combines calibrated human reviews, automated judges, and production monitoring with statistical checks to distinguish real improvements from noise.
</span>
</span>
</div>
</td></tr></tbody></table>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td align="left" style="word-break: break-word; vertical-align: top; padding: 5px 10px;">
<p style="padding: 0; margin: 0; font-size: 22px; color: #000000; line-height: 1.6; font-weight: bold;">
Want to advertise in TLDR? π°
</p>
<div class="text-block" style="margin-top: 10px;">
If your company is interested in reaching an audience of data engineering professionals and decision makers, you may want to <a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fadvertise.tldr.tech%2F%3Futm_source=tldrdata%26utm_medium=newsletter%26utm_campaign=advertisecta/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/EVVoAKk9mnkgD4UNZXRIZW8rSeuISK4CEeM6ohMwfyA=452"><strong><span>advertise with us</span></strong></a>.
</div>
<br>
<!-- New "Want to work at TLDR?" section -->
<p style="padding: 0; margin: 0; font-size: 22px; color: #000000; line-height: 1.6; font-weight: bold;">
Want to work at TLDR? πΌ
</p>
<div class="text-block" style="margin-top: 10px;">
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fjobs.ashbyhq.com%2Ftldr.tech/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/XIaqA7beJS_laOiSE77BwpPirp8tS4sucqysmzHN7NA=452" rel="noopener noreferrer" style="color: #0000EE; text-decoration: underline;" target="_blank"><strong>Apply here</strong></a>,
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fjobs.ashbyhq.com%2Ftldr.tech%2Fc227b917-a6a4-40ce-8950-d3e165357871/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/1B4WX20e402Ar6-HUCqOznr__19cWNlAwHvztVQJqgs=452" rel="noopener noreferrer" style="color: #0000EE; text-decoration: underline;" target="_blank"><strong>create your own role</strong></a> or send a friend's resume to <a href="mailto:jobs@tldr.tech" style="color: #0000EE; text-decoration: underline;">jobs@tldr.tech</a> and get $1k if we hire them! TLDR is one of <a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fwww.linkedin.com%2Ffeed%2Fupdate%2Furn:li:activity:7401699691039830016%2F/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/ctMxcAWYGQieDvzBdj-_k6m4BbZuQZi95MefIGHHFkY=452" rel="noopener noreferrer" style="color: #0000EE; text-decoration: underline;" target="_blank"><strong>Inc.'s Best Bootstrapped businesses</strong></a> of 2025.
</div>
<br>
<div class="text-block">
If you have any comments or feedback, just respond to this email!
<br>
<br> Thanks for reading,
<br>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fwww.linkedin.com%2Fin%2Fjoelvanveluwen%2F/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/ETVc6h_zyOZhyCzVs-6xCPQ7MSXLc0sWeL4tGEUzbYU=452"><span>Joel Van Veluwen</span></a>, <a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fwww.linkedin.com%2Fin%2Fjennytzurueyching%2F/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/KIUCFBsjz323OR-4OOCnIYPk05bfFOKJKTCMf-pQs_8=452"><span>Tzu-Ruey Ching</span></a> & <a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fwww.linkedin.com%2Fin%2Fremi-turpaud%2F/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/3TsKBU4pVp-OlmkOf7SrISv5TlJhrJbfCKat5wf8YnY=452"><span>Remi Turpaud</span></a>
<br>
<br>
</div>
<br>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block" id="testing-id">
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Ftldr.tech%2Fdata%2Fmanage%3Femail=donut.sweater.23%2540rss.andrealin.it/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/_whT5_HlXnpHdUJ0Rb9G2eYt_KP5EY1kKhblaGzZj8c=452">Manage your subscriptions</a> to our other newsletters on tech, startups, and programming. Or if TLDR Data isn't for you, please <a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fa.tldrnewsletter.com%2Funsubscribe%3Fep=1%26l=037ede50-92cc-11ee-b0f2-b761aa2217ad%26lc=aa6c57ca-be20-11f0-8e04-b5caa3a65826%26p=fc35c15c-cdda-11f1-a553-cb7122dc6967%26pt=campaign%26pv=5%26spa=1792663278%26t=1792664616%26s=0d8e30c254fbe3f76e7be486cf027c6c3052e5dcae13863ae83e3c9c7ade4154/1/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/39OxzGNtUpJ7YscHdcv-K3Kw62ujEsBYzKImr9WOD3A=452">unsubscribe</a>.
<br>
</div>
</td></tr></tbody></table>
</td></tr></tbody></table>
</td></tr></tbody></table>
</td></tr></tbody></table>
</td></tr></tbody></table>
<img alt="" src="http://tracking.tldrnewsletter.com/CI0/010001a163233d88-fda417c1-4eed-4759-9946-5307382958fc-000000/Z53aAZUxysIbeA9WAUrXq90fD6agbwnhPUVl16qBNlA=452" style="display: none; width: 1px; height: 1px;">
</body></html>