Can I transfer a large file with a bazaar, or is there a better approach to version control of a database dump?

Question

Bazaar limits the size of the file that it can commit based on the available virtual memory (in accordance with the opening error ).

I would like to put the database (as a mysqldump text file) under version control. The database is 3 GB and I work on a server with 64 GB of memory. I do not understand why this would be a problem. When I try to commit, I get an error message in error:

bzr: ERROR: exceptions.OverflowError: the requested number of bytes is greater than the Python string may contain

Is there any way to get this file under the control of the bazaar version?

My preference for the bazaar is that I am familiar with it, but I plan to automate the dump and register as a cron job to any suitable version control system.

---

Two options I came across until a better solution appears. I am currently saving a copy of each weekly dump backup; for now, storing data is not a problem. Otherwise, I could save the first dump, distinguish between the original and the new version of the dump and version control. This will keep a record of the changes, but returning to an earlier state is not possible. This is not convenient for me if there is no direct way to return.

mysqldump mydb > mydb_base.sql touch mydb_diff bzr add mydb_diff bzr commit -m 'first commit' 

then in cron script

 mysqldump mydb > mydb.sql diff mydb_base.sql mydb.sql > mydb_diff bzr commit -m "`date +%Y.%m.%d-%H.%M` mydb diff" mydb_diff 

Answer 1

Although I love Bazaar, I would not recommend using it for this purpose unless you definitely need to store each version of your data indefinitely. If instead you only need a fixed number of past versions (e.g. 10) or only for a fixed period (e.g. 2 years), I would suggest using an incremental backup tool like rdiff-backup .

Answer 2

This is a bug in bzr, fixed in recent versions. See https://bugs.launchpad.net/bzr/+bug/683191

Answer 3

If the Foreign VCS clauses apply, I have to say

• Mercurial 2.0 with LargeFiles Extension

Answer 4

First, you want b * ackup things * with the bazaar more than with the version system. This is different! The bazaar stores the changeet — the differences between the files — and tracks them and their merging.
Even if I can understand your question, I think that you are not using the right tool .

In any case, to answer your question, a simple workaround is to reduce the file size . Since mysqldump is a text file, its size can be significantly reduced by compression.

So a possible workaround is to compress it using 7zip (or whatever you want) before sending it to the bazaar.

This method does not allow you to perform a real merger or difference from the bazaar, but I understand that this is not your priority.