Many teams host their relational data storage on Heroku. However, over time, a significant number find themselves migrating their Postgres workloads over to AWS. In this article, I'll discuss why many teams are making this jump. I'll also detail the steps to migrate from Heroku Postgres to AWS, some obstacles you might face, and some tricks for easing your migration path.
Why migrate from Heroku Postgres to AWS?
Heroku is the go-to platform for many development teams just beginning their journey into the cloud. I touched on some of the reasons why in my last post on Heroku.
First, Heroku is simple to use and understand. AWS tends to overwhelm users with features. Heroku's feature set is clean and often much easier to use than their AWS counterparts.
Second, Heroku's pricing model is straightforward. AWS pricing for individual products often involves multiple factors like usage and data transfer rates. This complexity makes it hard for teams to predict exactly how much they'll spend month over month. By contrast, Heroku offers all-in-one pricing packages that make cloud spend predictable and easy to estimate.
Heroku brings these upsides to its Postgres hosting offering as well. So it's not surprising that many teams go with Heroku for their initial launch.
But, as I also discussed last time, using Heroku has significant downsides as well. When it comes to Postgres, Heroku falls short on several fronts:
Speed. As I discussed previously, Heroku's pre-packaged offerings generally don't provide as much compute capacity as is available on a cloud provider such as AWS.
Pricing. While Heroku's pricing is easier to calculate, it tends to be more expensive than what you can get with a carefully managed offering on a cloud provider. (That's not surprising - Heroku is using Amazon RDS itself and simply charging an overhead for its ease of use!)
Security. Heroku Postgres instances are publicly reachable on the Internet. You can create much more secure architectures directly on AWS. At TinyStacks, for example, we create all Amazon RDS instances for our customer's stacks in a private subnet for enhanced security.
Regional availability. AWS supports spinning up compute in 20+ regions around the world. Heroku offers two "regions": The US and Europe. That makes it less attractive for teams servicing a global user base.
Data residency. Dozens of countries globally have enacted laws that require some or all data on its citizens be kept within a country's borders. With its limited regional reach and services (like Heroku Postgres) that rely on storage in US data centers, Heroku can't meet many - if any - of those requirements.
Given these drawbacks, it's little wonder some teams who started small on Heroku would be looking to jump ship to AWS.
How to migrate from Heroku Postgres to AWS
Fortunately, migrating from Heroku Postgres to AWS is a fairly painless process, although it does involve some coordination.
Install Postgres tools
To migrate data from one Postgres instance to another, you'll need access to the Postgres command line tools. These tools should be automatically available if you are on a Linux Ubuntu system. For other operations systems, see the instructions for your OS on the PostgreSQL Web site.
Create your AWS Postgres database using Amazon RDS
The first step is creating your target database. This means creating an Amazon RDS database instance.
AWS has step by step instructions on creating an RDS database on their Web site. If you're keeping your app on Heroku, these instructions will probably suffice to get you started. Since you'll need to call your database from your Heroku-hosted app over the public Internet, your security setup will be simple for now.
(Note that you can get a more inherently secure setup with your app and Postgres database by migrating it from Heroku to TinyStacks. I talk about that a little bit more below.)
Stop writes to your Heroku Postgres database
You can't migrate a database if your applications are writing data to it. You will need either to take your application offline briefly or put your app into read-only mode while you cut over to your Amazon RDS database.
Heroku users can do this easily by switching their app to maintenance mode. In maintenance mode, users who visit your app will see a message saying that the application is temporarily offline.
Export your data from Heroku Postgres
Next, you'll need to export your data from the Heroku Postgres instance.
There are several ways to do this. Heroku themselves advise one of two methods:
- For data sets under 20GB, users can utilize the backup feature of Postgres to generate a backup file.
- For larger data sets, Heroku recommends creating a short-lived fork of your database and generate a dump file using
Import your data into Amazon RDS
Next, you'll need to import your data into Amazon RDS. Before you do, make sure your Postgres database is configured correctly. AWS has some great documentation on how to configure your Amazon RDS Postgres instance to ensure import occurs smoothly.
After that, import your data. For a backup, you can do this using the Postgres
pg_restore command. For a dump created using
pg_dump, you can just redirect the dump file into the database from the command line. See the Postgres site for more details.
Redirect Heroku Postgres calls to your Amazon RDS instance
Finally, you'll need to tell your app running on Heroku to connect to the database hosted on AWS rather than the Heroku Postgres endpoint. This requires a small configuration that Heroku documents on their Web site.
Challenges of migrating to AWS
Diving straight into using AWS can be a challenge for teams that have been using Heroku exclusively. As I discussed, AWS, with its multitude of features, can be intimidating for users who are new to the cloud. Even creating an Amazon RDS instance presents you with a dizzying array of options and technical terms you may not be familiar with.
Many people who migrate to AWS also often struggle to contain costs. Hidden charges, such as data transfer rates, can cause your AWS bill to bloat quickly.
How TinyStacks can help
There are many benefits to moving from Heroku to AWS. These include speed, scalability, security, features, global reach, and cost.
The problem is that a large gulf exists between the two services. Jumping from Heroku to AWS requires expertise that many teams don't have. And in many cases, their existing staff members don't have the time to develop that expertise. They already have their hands full creating new features for end users!
We created TinyStacks to address this gulf. Using TinyStacks, you can deploy your app just like you would on Heroku. Our dashboard makes it easy to create and manage a full publishing pipeline for your applications.
Even better? We host your app on an AWS account that you own. And we constantly optimize our system to host our customer's code at the lowest possible cost. You get all of the scalability, security, and flexibility of running on AWS without having to become an AWS expert.
If you're ready to make the leap from Heroku to AWS, contact us today!