Apache Foundation Announces Apache Airflow Becomes Top Project
The Apache Software Foundation announced that Apache Airflow has successfully graduated from incubation and became a new top-level project for the Foundation.
Apache Airflow is a flexible, scalable workflow automation and scheduling system that compiles and manages hundreds of petabytes of data. Projects can easily orchestrate complex computational workflows, and through intelligent scheduling, database and dependency management, error handling, and logging, Airflow automates resource management from a single server to a large cluster. The project is written in Python and is highly extensible, capable of running tasks written in other languages and allowing integration with common architectures and projects such as AWS S3, Docker, Kubernetes, MySQL, Postgres, and more. Apache Airflow was launched by Airbnb in 2014 and submitted to Apache Incubator in March 2016.
It is reported that Apache Airflow is currently being used by more than 200 organizations, including Adobe, Airbnb, Astronomer, Etsy, Google, ING, Lyft, NYC City Planning, Paypal, Polidea, Qubole, Quizlet, Reddit, Reply, Solita, Square, Twitter, etc…
