Open in app
Home
Notifications
Lists
Stories

Write
Christine Betadam
Christine Betadam

Home

Published in EatCodePlay

·Pinned

A Simple DAG to Quickly Purge Old Airflow Logs

Earlier on, we had little automation around the maintenance of our Airflow cluster. Those of you familiar with Airflow may be aware of the bloat accumulated over time from DAG runs. …

Airflow

4 min read

A Simple DAG to Quickly Purge Old Airflow Logs
A Simple DAG to Quickly Purge Old Airflow Logs

Published in EatCodePlay

·Pinned

Upgrading to Airflow 2.0: Massive Performance Wins and Lessons Learned

At RealScout, we use Apache Airflow to orchestrate our crucial workflows such as data ingestions, health checks, and reconciliation processes. Our team recently upgraded our stack from Airflow 1.10.10 to the long awaited 2.0. We are now about two weeks into the new Airflow deployment in production and are already…

Airflow

11 min read

Upgrading to Airflow 2.0: Massive Performance Wins and Lessons Learned
Upgrading to Airflow 2.0: Massive Performance Wins and Lessons Learned

Published in EatCodePlay

·Pinned

Zero-Downtime ECS Cluster Upgrades

Update July 2021: We recently switched to the “Amazon ECS-optimized Amazon Linux 2 AMI” since the older “ECS-optimized Amazon Linux AMI” ended its standard support phase and entered maintenance phase in April of 2021. …

AWS

7 min read

Zero-Downtime ECS Cluster Upgrades
Zero-Downtime ECS Cluster Upgrades
Christine Betadam

Christine Betadam

Senior Data Engineer. Avid learner and problem solver.

Following
  • Jyoti Dhiman

    Jyoti Dhiman

  • Jarek Potiuk

    Jarek Potiuk

  • Anna Geller

    Anna Geller

  • James Samuel

    James Samuel

  • Rob Gruhl

    Rob Gruhl

Help

Status

Writers

Blog

Careers

Privacy

Terms

About

Knowable