menu
arrow_back

Online Data Migration to Cloud Spanner using Striim

—/100

Checkpoints

arrow_forward

Set up the Striim software

Set up Connector/J

Create a Cloud Spanner access key

Online Data Migration to Cloud Spanner using Striim

1 hora 7 créditos

GSP703

Google Cloud Self-Paced Labs

Overview

This lab demonstrates how to migrate a Cloud SQL for MySQL database to Cloud Spanner using Google Cloud Platform's (GCP) Data Migration Partner, Striim.

Striim is a comprehensive streaming ETL platform that enables online data migrations and continuous streaming replication from on-premises and cloud data sources to GCP data services through a graphical drag and drop interface. The following figure shows Striim's high level architecture.

23578ad3db1d8fff.png

This lab will focus on the implementation of a continuous migration from MySQL to Cloud Spanner, and is not an explanation of database migration or database replication, or why you might want to migrate your underlying database.

ee37d9c44e583b3c.png

The figure shows the various services that you will use and create as well as the data flow from Cloud SQL for MySQL to Cloud Spanner via Striim that you will implement.

What you will learn

In this hands-on-lab you will learn to:

  • Access and use an existing Cloud SQL for MySQL database

  • Access and use an existing Cloud Spanner database

  • Deploy Striim through the GCP Marketplace and set it up

  • Use Striim to read from a source Cloud SQL for MySQL database

  • Use Striim to continuously replicate from Cloud SQL for MySQL to Cloud Spanner.

Setup and Requirements

What you'll need

Before you click the Start Lab button

Read these instructions. Labs are timed and you cannot pause them. The timer, which starts when you click Start Lab, shows how long Google Cloud resources will be made available to you.

This Qwiklabs hands-on lab lets you do the lab activities yourself in a real cloud environment, not in a simulation or demo environment. It does so by giving you new, temporary credentials that you use to sign in and access Google Cloud for the duration of the lab.

What you need

To complete this lab, you need:

  • Access to a standard internet browser (Chrome browser recommended).
  • Time to complete the lab.

Note: If you already have your own personal Google Cloud account or project, do not use it for this lab.

Note: If you are using a Pixelbook, open an Incognito window to run this lab.

How to start your lab and sign in to the Google Cloud Console

  1. Click the Start Lab button. If you need to pay for the lab, a pop-up opens for you to select your payment method. On the left is a panel populated with the temporary credentials that you must use for this lab.

    Open Google Console

  2. Copy the username, and then click Open Google Console. The lab spins up resources, and then opens another tab that shows the Sign in page.

    Sign in

    Tip: Open the tabs in separate windows, side-by-side.

  3. In the Sign in page, paste the username that you copied from the Connection Details panel. Then copy and paste the password.

    Important: You must use the credentials from the Connection Details panel. Do not use your Qwiklabs credentials. If you have your own Google Cloud account, do not use it for this lab (avoids incurring charges).

  4. Click through the subsequent pages:

    • Accept the terms and conditions.
    • Do not add recovery options or two-factor authentication (because this is a temporary account).
    • Do not sign up for free trials.

After a few moments, the Cloud Console opens in this tab.

Activate Cloud Shell

Cloud Shell is a virtual machine that is loaded with development tools. It offers a persistent 5GB home directory and runs on the Google Cloud. Cloud Shell provides command-line access to your Google Cloud resources.

In the Cloud Console, in the top right toolbar, click the Activate Cloud Shell button.

Cloud Shell icon

Click Continue.

cloudshell_continue.png

It takes a few moments to provision and connect to the environment. When you are connected, you are already authenticated, and the project is set to your PROJECT_ID. For example:

Cloud Shell Terminal

gcloud is the command-line tool for Google Cloud. It comes pre-installed on Cloud Shell and supports tab-completion.

You can list the active account name with this command:

gcloud auth list

(Output)

Credentialed accounts:
 - <myaccount>@<mydomain>.com (active)

(Example output)

Credentialed accounts:
 - google1623327_student@qwiklabs.net

You can list the project ID with this command:

gcloud config list project

(Output)

[core]
project = <project_ID>

(Example output)

[core]
project = qwiklabs-gcp-44776a13dea667a6

Before you begin

In order to work on the database migration some initial configuration is required:

  1. In the Cloud Shell, set the default compute zone you wish to deploy your database and compute resources in. For more information, please see Geography and Regions. For the purposes of this lab we will use us-central1-a, but change here if you wish to use a different zone.

gcloud config set compute/zone us-central1-a
export COMPUTE_ZONE=us-central1-a
  1. Verify that MySQL has been setup. Execute the following commands:

CSQL_NAME=striim-sql-src
CSQL_USER_NAME=striim-user
CSQL_USER_PWD=striim-usr-password
CSQL_ROOT_PWD=striim-root-password
  1. Next, with the local variables you just made, connect via executing the following command. It will prompt you for a password, which is the CSQL_USER_PWD you just created, striim-usr-password.

gcloud sql connect $CSQL_NAME --user=$CSQL_USER_NAME
  1. Once you're in, start with these commands.

USE striimdemo;
SELECT 1 FROM DUAL;
  1. Then, load some transactions into it at the mysql> prompt.

USE striimdemo;

CREATE TABLE ORDERS (ORDER_ID Integer, ORDER_DATE VARCHAR(50), ORDER_MODE VARCHAR(8), CUSTOMER_ID Integer, ORDER_STATUS Integer, ORDER_TOTAL Float, SALES_REP_ID Integer, PROMOTION_ID Integer, PRIMARY KEY (ORDER_ID));

INSERT INTO ORDERS (ORDER_ID, ORDER_DATE, ORDER_MODE, CUSTOMER_ID, ORDER_STATUS, ORDER_TOTAL, SALES_REP_ID, PROMOTION_ID) VALUES (1001, 1568927976017, 'In-Store', 1001, 9, 34672.59, 331, 9404);

INSERT INTO ORDERS (ORDER_ID, ORDER_DATE, ORDER_MODE, CUSTOMER_ID, ORDER_STATUS, ORDER_TOTAL, SALES_REP_ID, PROMOTION_ID) VALUES (1002, 1568928036017, 'In-Store', 1002, 1, 28133.14, 619, 2689);

INSERT INTO ORDERS (ORDER_ID, ORDER_DATE, ORDER_MODE, CUSTOMER_ID, ORDER_STATUS, ORDER_TOTAL, SALES_REP_ID, PROMOTION_ID) VALUES (1003, 1568928096017, 'CompanyB', 1003, 1, 37367.95, 160, 30888);

INSERT INTO ORDERS (ORDER_ID, ORDER_DATE, ORDER_MODE, CUSTOMER_ID, ORDER_STATUS, ORDER_TOTAL, SALES_REP_ID, PROMOTION_ID) VALUES (1004, 1568928156017, 'CompanyA', 1004, 1, 7737.02, 362, 89488);

INSERT INTO ORDERS (ORDER_ID, ORDER_DATE, ORDER_MODE, CUSTOMER_ID, ORDER_STATUS, ORDER_TOTAL, SALES_REP_ID, PROMOTION_ID) VALUES (1005, 1568928216017, 'CompanyA', 1005, 9, 15959.91, 497, 78454);

INSERT INTO ORDERS (ORDER_ID, ORDER_DATE, ORDER_MODE, CUSTOMER_ID, ORDER_STATUS, ORDER_TOTAL, SALES_REP_ID, PROMOTION_ID) VALUES (1006, 1568928276017, 'In-Store', 1006, 1, 82531.55, 399, 22488);

INSERT INTO ORDERS (ORDER_ID, ORDER_DATE, ORDER_MODE, CUSTOMER_ID, ORDER_STATUS, ORDER_TOTAL, SALES_REP_ID, PROMOTION_ID) VALUES (1007, 1568928336017, 'CompanyA', 1007, 7, 52929.61, 420, 66256);

INSERT INTO ORDERS (ORDER_ID, ORDER_DATE, ORDER_MODE, CUSTOMER_ID, ORDER_STATUS, ORDER_TOTAL, SALES_REP_ID, PROMOTION_ID) VALUES (1008, 1568928396017, 'Online', 1008, 1, 26912.56, 832, 7262);

INSERT INTO ORDERS (ORDER_ID, ORDER_DATE, ORDER_MODE, CUSTOMER_ID, ORDER_STATUS, ORDER_TOTAL, SALES_REP_ID, PROMOTION_ID) VALUES (1009, 1568928456017, 'CompanyA', 1009, 1, 97706.08, 124, 12185);

INSERT INTO ORDERS (ORDER_ID, ORDER_DATE, ORDER_MODE, CUSTOMER_ID, ORDER_STATUS, ORDER_TOTAL, SALES_REP_ID, PROMOTION_ID) VALUES (1010, 1568928516017, 'CompanyB', 1010, 1, 47539.16, 105, 17868);
  1. Verify that Cloud Spanner instance has been setup with a database and table defined. In the Cloud Console, select Navigation menu > Spanner from the left pane.

  2. Click on the instance striim-spanner-demo, and then click on the database striimdemo. Check that the table "orders" is present.

  3. Click on the table "orders", and then the tab "DATA". The table should be empty.

c225a7167838c0cf.png

  1. Click on the "SCHEMA" and you should see that the schema has been defined.

3acedddf683a7c38.png

Únase a Qwiklabs para leer este lab completo… y mucho más.

  • Obtenga acceso temporal a Google Cloud Console.
  • Más de 200 labs para principiantes y niveles avanzados.
  • El contenido se presenta de a poco para que pueda aprender a su propio ritmo.
Únase para comenzar este lab