While writing a post for my blog I noticed that on Windows some of the filenames on-disk showed encoding problems. I have this stored in Mercurial, so somewhere from Mercurial to the checkout on Windows something goes wrong where it concerns character encoding.

After some research and conversations with people on the #mercurial IRC channel, I understand that Mercurial stores everything internally in Python's byte encoding. On Windows it will then convert this to its native ANSI codepage, in my case codepage 1252.

Thankfully Windows 10 has a wonderful option nowadays to fix this issue. If you go to Control Panel, click Clock and Region, click Region, click Administrative, and under Language for non-Unicode programs click Change system locale. In the window that pops up tick the checkbox in front of Beta: Use Unicode UTF-8 for worldwide language support. Maybe by the time you are reading this the beta label has already been removed. Click OK and the system needs to restart.

You will need to clone the repository again since Mercurial (TortoiseHg) will need to properly generate the filenames.

In a previous post I talked about setting up an encrypted file based keyring store for Mercurial. With some recent updates of the Python keyring modules, the setup changed a little bit again.

The file-backed keyrings got moved out to the keyrings.alt package. $HOME/.local/share/python_keyring/keyringrc.cfg needs to be adjusted as follows:

[backend]
default-keyring=keyrings.alt.file.EncryptedKeyring

In an earlier post I documented how to set up an encrypted file store for your keyring. With recent versions of Python keyring (at least 3 and up) the CryptedFileKeyring backend got removed and replaced by EncryptedKeyring. So in your $HOME/.local/share/python_keyring/keyringrc.cfg you need to now have the following:

[backend]
default-keyring=keyring.backends.file.EncryptedKeyring

If PyCharm complains that it Can't start Mercurial: /usr/bin/hg Probably the path to hg executable is not valid, then check if running hg from the command line triggers a problem running a certain extension. In my case I had a version of keyring and mercurial_keyring that did not play nice with each other. After upgrading these to 3.0.5 and 0.6.0 respectively, the problem went away. I guess PyCharm tests the run of the hg binary and if the shell return code (echo $?) is something other than 0 will show this warning.

Mercurial allows for tying in keyring configuration for those of us who do not want to store passwords in plain-text in our .hgrc files or constantly using SSH.

First install the Python keyring library by running pip install keyring. After that is installed, checkout Mercurial keyring repository and add to $HOME/.hgrc the following:

[extensions]
mercurial_keyring = ~/path/to/mercurial_keyring/mercurial_keyring.py

Next up, configure your repositories, e.g. in the case of Bitbucket I use:

[auth]
bitbucket.prefix = bitbucket.org/asmodai
bitbucket.username = asmodai
bitbucket.schemes = https

Mercurial keyring will automatically decide on the best keyring to use. On a FreeBSD system with no Gnome or other systems providing a keyring, if you do not specify a specific keyring, the system will use the file ~/.local/share/python_keyring/keyring_pass.cfg. This keyring file stores the passwords encoded in Base64 in plain-text. This is not quite what you would want from a security point of view. You can configure which backend store to use by editing $HOME/.local/share/python-keyring/keyringrc.cfg. To get a plain-text file with encrypted keys use the following configuration:

[backend]
default-keyring=keyring.backend.CryptedFileKeyring

This will create the file ~/.local/share/python-keyring/crypted_pass.cfg after initializing the backend store with a password. Look at the documentation for keyring on what other configuration options are available.

Note: make sure the PyCrypto dependency is installed with the _fastmath module. This in turn depends on the gmp library.

If you have a Subversion repository setup with multiple top-level projects and their typical branches/tags/trunk setup and want to migrate these to individual Mercurial (Hg) repositories, you can do this with the convert extension.

First you need to enable convert in your .hgrc by adding a section like the following:

[extensions]
convert =

Next, if needed, create a plain-text file, e.g. author-map.txt, containing SVN username to Hg author mappings, e.g. asmodai=Jeroen Ruigrok van der Werven <email@address.tld>.

Next run Hg as follows:

% hg --authors author-map.txt --config convert.svn.branches=project/branches
--config convert.svn.tags=project/tags --config
convert.svn.trunk=project/trunk path/to/svn/repository
path/to/destination/hg/repository

This will start a SVN to Hg conversion, picking up only the changes and commit messages applicable for the various paths you gave for the branches, tags, and trunk, effectively splitting off this project from the main SVN tree into its own Hg repository.

Do note that for large SVN repositories this might not be the most efficient conversion way forward. In that case converting once from SVN to Hg and then split off Hg into many Hg repositories might be faster. Will adjust this post when I write that up.

For my own development I use Mercurial and TortoiseHG for my version control system. I also use, at the moment, a CAcert certificate to use HTTPS with my repositories. I am not sure what changed when, but apparently the certificates now get verified. So this causes obvious problems trying to push or pull due to "SSL: Server certificate verify failed" errors.

To make this work on a Windows 7 machine with TortoiseHG in stalled, first download the CAcert root PEM certificate and place it some permanent directory. Next open the TortoiseHG global settings (right click somewhere in Explorer and select TortoiseHG » Global Settings). In the window that opens click the Edit File button. If it does not exist yet create a section similar to this:

[web]
cacerts = C:\path\to\cacert-root.pem

Press Save and OK and any push and pull action with HTTPS URLs should work as they ought to.

So with recent Mercurial 1.7 releases HTTPS support was tightened, so you are bound to encounter a warning in the form of: warning: bitbucket.org certificate not verified (check web.cacerts config setting).

Now, http://mercurial.selenic.com/wiki/CACertificates there are details on what to configure for certain operating systems. Given I use FreeBSD, I altered my $HOME/.hgrc as follows:

[web]
cacerts = /etc/ssl/cert.pem

For OpenBSD this should be in the same place since release 3.8. But apparently NetBSD does not have such a file in base.

For my own sanity:

In order to change the parent repository URL of a Mercurial clone (as svn switch does) one simply edits .hg/hgrc and adjusts the default under [paths].