Why does Mercurial think my SQL files are binary?

Question

I just wrote scripts stored in SQL Server, processed procs, table definitions, etc. using SQL Server Management Studio, and tried to add them to my Mercurial resource management repository. They added just fine, but now that I am changing them and changing them, Mercurial calls them “binaries” and does not give me the proper unified diff.

I thought coding could be a problem, so I tried to restore the scripts and tell ANSI to output the text file, but I get the same behavior. I can view them just fine in notepad without the appearance of any strange characters. Why does Mercurial consider these files to be binary?

Otherwise, if someone could recommend a good tool for writing a SQL Server database that might not cause this problem, this might work as well.

Answer 1

I ran into this problem because SQL Server Management Studio saves files in Unicode format. The first two bytes (in most cases) of a Unicode text file determine the encoding. Most new text editors (like Notepad) handle this transparently.

The first two bytes are probably where your problem is. They may look like ÿþ. Or FF FE in hexadecimal format.

In the Save dialog box, the Save dialog box displays a selection list. Select "Save with Encoding ..." and select "US-ASCII-Codepage20127." I believe that this parameter is sticky and will remain in the future.

Answer 2

According to docs , it is considered binary if there are zero bytes in the file. SQL files should not have zero bytes, so first I have to check this out (try looking in a hex editor). I assume you know you can get diff to treat it as text

Answer 3

Andrew is right; it's somewhere in the NUL byte (my guess would be the Byte Order Mark at the beginning, introduced by the rude editor tool). Do not worry about this, although, unlike SVN or CVS, Mercurial does not handle binary files differently. It displays them different when you do "hg log", but they are not processed in a completely different way.

The upcoming mercury releases of the special case specifications do not allow them to run "the user probably does not want to see how this happens on the console."

Answer 4

I came across this while editing a stored procedure file from SQL Server on linux and using git. Git thought it was a binary file because the SQL Server file was UTF-16 and therefore contained NUL. My fix for this was emacs, which allows you to change the encoding to UTF-8.

Answer 5

I know this a bit later, but I came up with a script to batch save * .sql files to UTF-8.

The full answer has been sent to another thread on StackOverflow, so I will just post the link here - stack overflow.site/questions/115869 / ....

Answer 6

I had a similar problem and decided to use the tool found at http://www.devio.at/index.php/smoscript to help me solve the problem. I wrote SMOscript by putting the following in a cmd file.

 rd /s /q [the scripts folder] "C:\Program Files\devio IT Services\SMOscript\smoscript.exe" -s [server] -d [database] -F [the scripts folder] -U 

The idea is to delete the old folder so that any objects deleted from the database are deleted from the original control. It also saves files as UTF8 without date and time titration, so they work great in version control.