menu
arrow_back

Cloud Composer: Qwik Start - Console

Cloud Composer: Qwik Start - Console

1 hour 1 Credit

GSP261

Google Cloud Self-Paced Labs

Overview

Workflows are a common theme in data analytics - they involve ingesting, transforming, and analyzing data to figure out the meaningful information within. In Google Cloud Platform (GCP), the tool for hosting workflows is Cloud Composer which is a hosted version of the popular open source workflow tool Apache Airflow.

In this lab, you use the GCP Console to set up a Cloud Composer environment. You then use Cloud Composer to go through a simple workflow that verifies the existence of a data file, creates a Cloud Dataproc cluster, runs an Apache Hadoop wordcount job on the Cloud Dataproc cluster, and deletes the Cloud Dataproc cluster afterwards.

What you'll do

  • Use GCP Console to create the Cloud Composer environment

  • View and run the DAG (Directed Acyclic Graph) in the Airflow web interface

  • View the results of the wordcount job in storage.

Join Qwiklabs to read the rest of this lab...and more!

  • Get temporary access to the Google Cloud Console.
  • Over 200 labs from beginner to advanced levels.
  • Bite-sized so you can learn at your own pace.
Join to Start This Lab
Score

—/100

Create a Cloud Storage bucket

Run Step

/ 30

Create Cloud Composer environment

Run Step

/ 30

Uploading the DAG to Cloud Storage

Run Step

/ 40