Faster transferring db from one heroku application to another

Question

Is there a faster way to port my production database to a test application?

I am heroku db:pull doing heroku db:pull on my local machine, then heroku db:push --app testapp , but this is getting time consuming. I have some seed data, but it’s not as accurate as just testing with my real data. And since they are both stored on a neighboring AWS cloud, should there be a faster way to move data?

I thought about using the heroku package but noticed that the animate command is missing?

 bundles:animate <bundle> # animate a bundle into a new app 

Answer 1

This is fairly common for migrating databases between Rails Apps intermediate, test, and production environments. And heroku db:pull/push painfully slow. The best way I've found so far is to use Heroku PG Backups and free . I followed these steps to migrate the production database for the staging server:

1) Back up the db production application

 heroku pg:backups capture --app production-app 

This will create a backup file b001 from the main database (usually produce db in database.yml)

2) To view all backups (OPTIONAL)

 heroku pg:backups --app production-app 

3) Now use the pg: backups restore command to populate the staging server database from the last backup file on the production server

 heroku pg:backups restore $(heroku pg:backups public-url --app production-app) DATABASE_URL --app staging-app 

Remember that recovery is a destructive operation; it deletes existing data before replacing it with the contents of the backup file.

Answer 2

So now everything is easier. Check the transfer command as part of pgbackups

 heroku pgbackups:transfer HEROKU_POSTGRESQL_PINK sushi-staging::HEROKU_POSTGRESQL_OLIVE -a sushi 

https://devcenter.heroku.com/articles/upgrading-heroku-postgres-databases#4b-alternative-transfer-data-between-applications

This worked perfectly for me, taking the production code back to my website.

Answer 3

The correct answer again changed as of March 11, 2015.

 heroku pg:backups restore $(heroku pg:backups public-url --app myapp-production) DATABASE_URL --app myapp-staging 

Note that now the argument is public-url .

https://blog.heroku.com/archives/2015/3/11/pgbackups-levels-up

Answer 4

Update for mid-2015 ...

The pgbackups supplement is deprecated. More pgbackups:transfer . pg:copy perfect for this scenario.

To copy a database from yourapp (db name: HEROKU_POSTGRESQL_PINK_URL example to yourapp_staging (db name: HEROKU_POSTGRESQL_WHITE_URL example)

 # turn off the web dynos in staging heroku maintenance:on -a yourapp-staging # if you have non-web-dynos, do them too heroku ps:scale worker=0 -a yourapp-staging # backup the staging database if you are paranoid like me (optional) heroku pg:backups capture -a yourapp-staging # execute the copy to splat over the top of the staging database heroku pg:copy yourapp::HEROKU_POSTGRESQL_PINK_URL HEROKU_POSTGRESQL_WHITE_URL -a yourapp-staging 

Then, when it is complete, turn it on again:

 # this is if you have workers, change '1' to whatever heroku ps:scale worker=1 -a yourapp-staging heroku maintenance:off -a yourapp-staging 

Reminder: you can use heroku pg:info -a yourapp-staging (and yourapp) to get the database constants.

(source: https://devcenter.heroku.com/articles/upgrading-heroku-postgres-databases#upgrade-with-pg-copy-default )

Answer 5

 psql -h test_host -c 'drop database test_db_name; create database test_db_name;' pg_dump -h production_host production_db_name | psql -h test_host test_db_name` 

This can be done on production_host or test_host - it will work in both directions.

Answer 6

Did not check it, but it can work.

Do this to get the URL of the source database:

 heroku console "ENV['DATABASE_URL']" --app mysourceapp 

Then try db:push with this.

 heroku db:push database_url_from_before --app mytargetapp 

This may not work if Heroku does not allow access to the database machines from outside their network, which is probably the case. You could perhaps try using taps (gems ​​that use the db hero commands inside) from your application code somewhere (maybe a rake task). This will be even faster than the above approach, because everything stays completely within AWS.

Edit:

Here's a (admittedly hacky) way to do what I described above:

Take the URL of the database, as in the first code snippet above. Then from the rake task (you can do it on the console, but you run the risk of running a 30 second timeout on console commands), run the shell command for taps (it's not easy to determine if taps can be used directly from Ruby; docs show the use of the CLI):

 `taps pull database_url_from_source_app #{ENV['DATABASE_URL']}` 

Reverse steps are important; this is how Ruby stands for a shell command that has taps in it. We hope that the crane team is available from the application. This avoids the problem of accessing the database machine from outside Heroku, since you are using this command from your application.

Answer 7

Heroku lets you plug existing applications into production. Use the heroku fork to copy an existing application, including add-ons, configuration bars, and Heroku Postgres data.

Follow Heroku instructions: https://devcenter.heroku.com/articles/fork-app

Answer 8

Update for mid 2016 ...

When creating forks, Heroku now has the --fast flag, but they will expire before 30 hours.

$ heroku addons:create heroku-postgresql:standard-4 --fork HEROKU_POSTGRESQL_CHARCOAL --fast --app sushi

https://devcenter.heroku.com/articles/heroku-postgres-fork#fork-fast-option