Migrating ScaleGrid for Redis®* data from one server to another is a common requirement that we hear from our customers. Two of the main reasons we hear are often due to migration of hardware, or the need to split data between servers.
Typically, you want to migrate with minimal downtime while using the standard Redis tools. In this blog post, we will guide you through the process using the open-source tool Redis-Shake. Developed and maintained by the NoSQL Team in the Alibaba-Cloud Database department, Redis-Shake allows you to easily migrate Redis data between Redis clusters. While this guide is more aimed towards ScaleGrid for Redis®* deployments, this guide can be used for other types of Redis deployments as well with some slight modifications.
Without further ado, let’s get started with the migration guide!
Prerequisites
You’ll need a Linux or Windows machine to perform the migration. (If needed, this can be either the source or the target machines).
An important thing to make sure of before you continue is that Redis-Shake does not support SSL-enabled clusters as of now. Please make sure that both the source and the target clusters are not SSL-enabled.
Firewall Rule
Make sure you have created a firewall rule in order to allow your Linux/Windows machine to connect to both your source and target clusters. Refer to this link for more info on how to set this up on ScaleGrid.
Download Redis-Shake
Download and decompress Redis-Shake on the Linux/Windows machine after choosing the latest/stable version from this link. You can also use the following commands below to download the latest version as of today (v2.1.1).
wget https://github.com/alibaba/RedisShake/releases/download/release-v2.1.1-20210903/release-v2.1.1-20210903.tar.gz tar -xvzf release-v2.1.1-20210903.tar.gz cd release-v2.1.1-20210903/
Next up, we will show you two different use cases for Redis-Shake.
Migrating Data – Standalone and Master-Slave
In this section, we will show you how to migrate your data in these different setups:
- Standalone to Standalone
- Master-Slave to Master-Slave
- Standalone to Master-Slave (and vice-versa)
1. Gather info
Note down the following info from the Overview tab in the cluster details page from both your source and target cluster:
- Hostname: The Connection String is in the format [hostname:port]. It should look something like this: EX-redms-00-redis-master.example.domain.io
- (In the case of a Master-Slave cluster, we’ll connect to the master).
- Port: 6379
- Password: You can find your password and reset it from the console under Credentials
2. Edit Config File
Open your “redis-shake.conf” file in the main folder and edit the following variables:
source.type = standalone
source.address = <source_hostname>:6379
source.password_raw = <source_password>
target.type = standalone
target.address = <target_hostname>:6379
target.password_raw = <target_password>
3. Sync Data
Run the following command to synchronize data between the source and target cluster:
$ ./redis-shake -type sync -conf redis-shake.conf
4. Wait For Logs
Wait until you see the following information in the logs:
sync rdb done.
This means that the full synchronization has been completed and incremental synchronization begins.
5. Stop Incremental Sync
If +writeBytes=0 is seen for a long time, then that means no new data is incremented. You can stop the incremental synchronization by pressing Ctrl+C. It should look like something like this:
sync: +forwardCommands=0 +filterCommands=0 +writeBytes=0
6. Match Number of Keys
Now follow this link to our help doc which helps you connect to both clusters using redis-cli and then match the number of keys between your clusters using the info keyspace command on both clusters.
And that’s it! You’ve now successfully migrated data from one cluster to another using Redis-Shake. If you have clusters in Cluster Mode, you might want to have a look at our next section in this guide to see what you need to do if you’re running that type of cluster.
Migrating Data – Cluster Mode
In this section, we will show you how to migrate data from one Redis deployment in cluster mode to another cluster.
1. Gather info
Note down the following info from the Overview tab in the cluster details page from both your source and target cluster:
- Hostname: The Connection String is in the format [hostname1:port], [hostname2:port], [hostname3:port]. It should look something like this: EX-redms-03-redis-master.example.domain.io, EX-redms-06-redis-master.example.domain.io, EX-redms-09-redis-master.example.domain.io
- Port: 6379
- Password: You can find your password and reset it from the console under Credentials
2. Edit Config File
Open your “redis-shake.conf” file in the main folder and edit the following variables:
source.type = cluster
source.address = <source_hostname1>:6379;<source_hostname2>:6379;<source_hostname3>:6379
source.password_raw = <source_password>
target.type = standalone
target.address = <target_hostname1>:6379;<target_hostname2>:6379;<target_hostname3>:6379
target.password_raw = <target_password>
3. Sync Data
Run the following command to synchronize data between the source and target cluster:
$ ./redis-shake -type sync -conf redis-shake.conf
4. Wait For Logs
Wait until you see the following information in the logs:
sync rdb done.
This means that the full synchronization has been completed and incremental synchronization begins.
5. Stop Incremental Sync
If +writeBytes=0 is seen for a long time, then that means no new data is incremented. You can stop the incremental synchronization by pressing Ctrl+C. It should look like something like this:
sync: +forwardCommands=0 +filterCommands=0 +writeBytes=0
6. Match Number of Keys
Now follow this link to our help doc which helps you connect to both clusters using redis-cli and then match the number of keys between your clusters using the info keyspace command on both clusters.
And that’s it! You’ve now successfully migrated data from one cluster in Cluster Mode to another using Redis-Shake. For more info on Redis-Shake and its capabilities, visit their GitHub page to find out more and get the latest version.
*Redis is a trademark of Redis Labs Ltd. Any rights therein are reserved to Redis Labs Ltd. Any use by ScaleGrid is for referential purposes only and does not indicate any sponsorship, endorsement or affiliation between Redis and ScaleGrid.
