Automate infrastructure with Cloud Composer

This trial describes a way to automate a cloud infrastructure using Cloud Composer. The example shows how to schedule automated backups of Compute Engine virtual machine (VM) instances.

Cloud Composer is a fully managed workflow orchestration service on Google Cloud. Cloud Composer allows you to create workflows using a Python API, schedule them for automatic execution or start them manually, and monitor task execution in real time via a graphical interface.

Cloud Composer is based on Apache Airflow. Google runs this open source orchestration platform on a Google Kubernetes Engine (GKE) cluster. This cluster manages Airflow compute nodes and offers many integration possibilities with other Google Cloud products.

This is for operators, IT administrators, and developers who want to automate their infrastructures and deepen their technical knowledge of Cloud Composer’s key features. This is not intended to be an enterprise-level disaster recovery guide or a collection of backup best practices.

Architecture

Cloud Composer workflows are made up of Directed Acyclic Graphs, or DAGs. From the Airflow perspective, a DAG is a set of tasks, organized to reflect their directional interdependencies. In this trial, you will learn how to define an Airflow workflow that is executed periodically to perform a backup of a Compute Engine virtual machine instance using Persistent Disk snapshots.