Distributed Image Processing in Cloud Dataproc




Create a development machine in Compute Engine

Install Software in the development machine

Create a GCS bucket

Download some sample images into your bucket

Create a Cloud Dataproc cluster

Submit your job to Cloud Dataproc

Distributed Image Processing in Cloud Dataproc

1 hour 7 Credits


Google Cloud Self-Paced Labs


In this hands-on lab, you will learn how to use Apache Spark on Cloud Dataproc to distribute a computationally intensive image processing task onto a cluster of machines. This lab is part of a series of labs on processing scientific data.

What you'll learn

  • How to create a managed Cloud Dataproc cluster with Apache Spark pre-installed.

  • How to build and run jobs that use external packages that aren't already installed on your cluster.

  • How to shut down your cluster.


This is an advanced level lab. Familiarity with Cloud Dataproc and Apache Spark is recommended, but not required. If you're looking to get up to speed in these services, be sure to check out the following labs:

Once you're ready, scroll down to learn more about the services that you'll be using in this lab.

Join Qwiklabs to read the rest of this lab...and more!

  • Get temporary access to the Google Cloud Console.
  • Over 200 labs from beginner to advanced levels.
  • Bite-sized so you can learn at your own pace.
Join to Start This Lab