PinnedChristine BetadaminEatCodePlayA Simple DAG to Quickly Purge Old Airflow LogsEarlier on, we had little automation around the maintenance of our Airflow cluster. Those of you familiar with Airflow may be aware of the…4 min read·Aug 16, 2021--3--3
PinnedChristine BetadaminEatCodePlayUpgrading to Airflow 2.0: Massive Performance Wins and Lessons LearnedAt RealScout, we use Apache Airflow to orchestrate our crucial workflows such as data ingestions, health checks, and reconciliation…11 min read·Apr 30, 2021----
PinnedChristine BetadaminEatCodePlayZero-Downtime ECS Cluster UpgradesAs part of the DataOps team at RealScout, I recently worked on a project to tackle infrastructure issues that plagued us for some time. We…7 min read·Dec 18, 2020----