Running BERT on Google Cloud Platform With TPU

Use Google Cloud and TPU to Build a Deep Learning Environment

Photo by Rajeshwar Bachu on Unsplash

By 2020, various deep learning models such as Transformer, BERT, and GPT-series make AI become smarter and smarter, using them can achieve many incredible tasks. On the hardware, Google launched Cloud TPU allows programmers to train these “Big” models at a reasonable cost.

In this article, I share how I built a Deep learning environment using Google Cloud Platform and Cloud TPU. And I also demonstrate running a BERT model task with this environment.

There are four parts in this article:

Part 1. Create a Deep Learning VM using the Google Cloud Console

Part 2. Create a Deep Learning VM using the command-line tools

Part 3. Run the BERT Task

Part 4. Clean up

Part 1. Create a Deep Learning VM using the Google Cloud Console

First, we use Google Cloud Console to create a deep learning environment. There are three steps:

1. Create a VM(virtual machine) instance
2. Create a Cloud TPU node
3. Create a Bucket