Cloud Spanner — Almost all you need to know

Find out why Google runs on Spanner

Things to consider — TrueTime, Schema design, Multi-tenancy, Secondary indexes, Best practices, Data Encryption, Audit logging, Access control (IAM), Backup and Restore, Import and Export, Pricing


Spanner helps you trust time in distributed systems using TrueTime.

Cloud Spanner is the first scalable, enterprise-grade, globally-distributed, and strongly consistent database service built for the cloud specifically to combine the benefits of relational database structure with non-relational horizontal scale.

Image for post
Image for post
© Google Cloud


  • Custom manufactured hardware
  • Synchronous replication
  • Data encryption at rest and transit
  • Granular access control (IAM)
  • On demand backup and restore (For whatever reason, this feature had been missing for a long time since the product launch!)
  • Allowed data types

The following diagram illustrates how read and strong consistent write happen in Spanner. The diagram is self evident, where Span Frontend / Server refer to a Spanner server and frontend, whereas Colossus is their distributed file system where the writes are conducted. This way Spanner separates Compute (Spanner Server ) and Storage (Colossus), similar to Cloud BigTable.

Image for post
Image for post
© Google Cloud

You can’t write about Spanner without mentioning TrueTime. In simple words, TrueTime provides an accurate time signal with bounds (Tmin, Tmax). In other words, TrueTime is a highly available, distributed clock that is provided to applications on all Google servers. In distributed systems, a point-in-time can’t be considered reliable or accurate because of clock skews. Thus TrueTime provides a reliable and accurate time interval instead.

Image for post
Image for post
© Google Cloud

TrueTime enables applications to generate monotonically increasing timestamps: an application can compute a timestamp T that is guaranteed to be greater than any timestamp T’ if T’ finished being generated before T started being generated. This guarantee holds across all servers and all timestamps.

This feature of TrueTime is used by Cloud Spanner to assign timestamps to transactions. Specifically, every transaction is assigned a timestamp that reflects the instant at which Cloud Spanner considers it to have occurred. Because Cloud Spanner uses multi-version concurrency control, the ordering guarantee on timestamps enables clients of Cloud Spanner to perform consistent reads across an entire database (even across multiple Cloud regions) without blocking writes.

TrueTime helps Cloud Spanner with external consistency.

Extract from Spanner, TrueTime & The CAP Theorem — Spanner reasonably claims to be an “effectively CA” system despite operating over a wide area, as it is always consistent and achieves greater than five 9s (99.999%) availability. As with Chubby, this combination is possible in practice if you control the whole network, which is rare over the wide area. Even then, it requires significant redundancy of network paths, architectural planning to manage correlated failures, and very careful operations, especially for upgrades. Even then outages will occur, in which case Spanner chooses consistency over availability. Spanner uses two-phase commit to achieve serializability, but it uses TrueTime for external consistency, consistent reads without locking, and consistent snapshots.

Schema Design

There are two ways to define parent-child relationships in Cloud Spanner, table interleaving and foreign keys. Table interleaving implies co-location of the tables in the storage layer while foreign keys don’t. The co-location of child rows with their parent rows can significantly improve performance. In table interleaving, the child table’s primary key includes the parent table’s primary key columns.

If you declare a table to be a child of another table, the primary key column(s) of the parent table must be the prefix of the primary key of the child table. Cloud Spanner stores rows in sorted order by primary key values, with child rows inserted between parent rows that share the same primary key prefix. This insertion of child rows between parent rows along the primary key dimension is called interleaving, and child tables are also called interleaved tables.

You can define hierarchies of parent-child relationships between tables up to seven layers deep, which means you can co-locate rows of seven logically independent tables.

SingerId INT64 NOT NULL,
FirstName STRING(1024),
LastName STRING(1024),
SingerInfo BYTES(MAX),
) PRIMARY KEY (SingerId);

SingerId INT64 NOT NULL,
AlbumTitle STRING(MAX),
) PRIMARY KEY (SingerId, AlbumId),

Cloud Spanner divides data among servers by key ranges, which means your inserts will be directed at a single server if you have a monotonically increasing integer as the key, creating a hotspot. There are techniques that can spread the load across multiple servers and avoid hotspots:

  • Hash the unique key and spread the writes across logical shards, where ShardId = hash(LastAccess and UserId) % N and N is the number of shards.
Image for post
Image for post
  • Swap the order of the columns in the primary key in case the first one is monotonically increasing or decreasing.
  • Bit-reverse sequential values.
  • Use UUID

Cloud Spanner is a distributed database, which means that as your database grows, Cloud Spanner divides your data into chunks called “splits”, where individual splits can move independently from each other and get assigned to different servers, which can be in different physical locations. Cloud Spanner splits data based on load: it adds split boundaries automatically when it detects high read or write load spread among many keys in a split.Thus, the parent-child table relationships that you define, along with the primary key values that you set for rows of related tables, give you control over how data is split under the hood.

You can find a good schema example here.


You might want to provide multi-tenancy if you are storing data that belongs to different customers. For example, a music service might want to store each individual record label’s separately.

The classic way to design for multi-tenancy is to create a separate database for each customer. In this example, each database has its own Singers table:

Image for post
Image for post
© Google Cloud

The recommended way to design for multi-tenancy in Cloud Spanner is to use a different primary key value for each customer in the same database. There are limits on the number of databases per instance and tables per database. Depending on the number of customers, it might not be possible to have separate databases or tables.

Image for post
Image for post
© Google Cloud

If you want to create separate databases, you might have more success if you distribute your tables across databases in such a way that each database has a low number of schema changes per week.

  • Do not choose a column whose value monotonically increases or decreases (timestamp etc.) as the first key part.
  • Limit the row size — The size of a row should be less than 4 GB for best performance. The size of a row includes the top-level row and all of its interleaved child and index rows.
  • Design interleaved tables properly to prevent hotspots — Cloud Spanner can only create splits along top-level rows. Consider the following example with three interleaved tables. Here in the left one (with Songs as a top-level table), Cloud Spanner can split on Songs, whereas on the right one it can’t.
Image for post
Image for post
Image for post
Image for post
  • Use descending order for timestamp-based keys, especially if you’re using an interleaved table for the history, and you’ll be reading the parent row as well.
  • Do not create a non-interleaved index on a column whose value monotonically increases or decreases.

SQL Best practices — Similar to other Database systems, use parametric queries, secondary indexes, write efficient joins. More on this here.

Secondary Indexes

Cloud Spanner automatically creates an index for each table’s primary key column. You can also create secondary indexes for other columns. Adding a secondary index on a column makes it more efficient to look up data in that column.

Cloud Spanner stores the following data in each secondary index:

  • All key columns from the base table
  • All columns that are included in the index
  • All columns specified in the optional STORING clause of the index definition

Other features

  • Cloud Spanner can be used with Cloud Functions, Cloud Dataflow and other GCP Services.
  • You can export data from Spanner into avro and import from csv, avro and other databases.
  • Cloud Spanner Dialect is compatible with Hibernate ORM 5.4 and Spring data.

Choose between Backup and Restore or Import and Export

Cloud Spanner Import and Export serve similar use cases as Backup and Restore. The following table describes similarities and differences between them to help you decide which one to use.

Image for post
Image for post
© Google Cloud

Pricing — as of June 2020

Image for post
Image for post
© Google Cloud

That’s about it for a condensed introduction. For more detailed information, there is no better place than the official documentation. Happy Learning!

A Certified Multi-Cloud Architect/Big Data/ML Specialist and Quantum Computing Enthusiast

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store