Installing Galera MySQL clustering on Ubuntu 10

Galera is a patch and a library for MySQL that work together at the engine level to provide simple, multi-master replication between MySQL instances. It promises to be easier to set up than ordinary MySQL replication, and allow you to write to more than one machine, a feature currently unavailable from vanilla MySQL.

The documentation is laughably sparse, but I found a helpful post on the mailing list which gave me the clues I needed to get it up and running on a trio of local virtual machines. Once I’ve run some more tests I’ll give a more thorough review of the tech, but the smoke-test confirms: yes, you can get replication between 3 masters that survives the loss of any one of the machines, and even supports auto-incrementing indexes! If this tech turns out to be stable, it’s pretty much the holy grail of MySQL.

Installing Galera

You need to install both the mySQL deb with the wsrep patch, and the galera library itself:

wget http://launchpad.net/codership-mysql/5.1/5.1.59-22.2/+download/mysql-server-wsrep-5.1.59-22.2-i386.deb
wget http://launchpad.net/galera/1.x/22.1.1/+download/galera-22.1.1-i386.deb


Install the packages. dpkg will complain that things are broken, so get apt-get to install the dependencies for you:
dpkg -i galera-22.1.1-i386.deb mysql-server-wsrep-5.1.59-22.2-i386.deb
apt-get -f install


Now you need to temporarily start the server so you can configure it (you’ve missed the usual mySQL setup wizard):
/usr/bin/mysqld_safe &


Run the secure installation setup to configure your root password, etc.:
/usr/bin/mysql_secure_installation


Now, set MySQL to bind to the external adapter instead of localhost (127.0.0.1), for example:

nano /etc/mysql/my.cnf
bind: 10.0.1.109


Now that’s done, kill your temporary server. I’m dumb, so I did it this messy way:
ps axf | grep mysqld
kill -9 any pids from above


Now start mySQL properly:
service mysql restart

Now you need to configure galera itself. Galera config is under mySQL’s:
nano /etc/mysql/conf.d/wsrep.cnf

First, tell it where you’ve installed the galera lib:
wsrep_provider=/usr/lib/galera/libgalera_smm.so

Now give it the cluster address. For the very first node, that address is just:
wsrep_cluster_address="gcomm://"

For any subsequent nodes, set it to point at the first node:
wsrep_cluster_address="gcomm://10.0.1.109"

At the moment it’s not clear to me why this works this way, since if the first node falls over the other two nodes seem to remain capable of syncing with each other, i.e. there is no “master” node. I’ll understand better once I’ve done more reading, I guess.

You will need to provide a replication username/password between the nodes:
wsrep_sst_auth=username:password

And restart mysql again.

Finally, connect to MySQL and grant permissions to your replication user (NB: you need to run these two commands as a single line — galera seems to do weird things with connections to clients).

set wsrep_on = OFF; grant all on *.* to 'username'@'%' identified by 'password';

Done! Repeat these steps for as many nodes as you want, changing only the gcomm address, and you’ll get that many replicated nodes. Amazing!

Fun facts

I’m just starting to get to grips with the features of Galera, but so far the cool things I’ve noticed are:

1. New nodes can sync from empty, i.e. no need to take a mysqldump snapshot beforehand
2. Database and table creates (and drops!) are respected and synced to all boxes
3. Tables with auto_increment indexes create non-conflicting IDs (although they are no longer strictly sequential, so you will get gaps in your numerical sequence).
4. Any of the nodes can drop out temporarily and will catch back up
5. Restarting MySQL seems to fix most syncing problems

I’ll write more about pros and cons once I’ve had a chance to play with Galera some more.

You can host static websites on Amazon S3

Hosting websites on Amazon Simple Storage Service is just a matter of uploading HTML files to an S3 bucket and then pointing a domain at it. Great if you have a static website with enormous traffic, but who has a completely static website?

Introducing PHP 5’s Standard Library

Introducing PHP 5’s Standard Library.

A pretty good overview of how to create iterators, pseudo arrays, etc. in PHP’s class structures.

Rails Flash.now – Blog – Tim Harding – g0

Rails’ flash.now is what to use if you want a flash message to appear only once, on the page you’re currently rendering.

Before PostgreSQL, Bruce Momjian wrote an SQL in Shell

Before PostgreSQL, Bruce Momjian wrote an SQL in Shell – Andrew Cholakian’s Blog.

Generating a UUID in PHP

We need to generate UUIDs in PHP. This is actually extremely easy: there is a PHP UUID package in PECL, but you’d never know, because there’s no documentation, a problem that’s apparently been the case forever — a documentation bug was opened in 2006, but never resolved, even though there is some documentation in the source code.

Even more simply, in ubuntu there is a package called php5-uuid. It can be installed just with

apt-get install php5-uuid

However, documentation is even more sparse for this one — the best that exists appears to be a comment on the PHP manual page for uniqid() (of course, if all you need is a unique ID, that function is probably easier still, but we needed a UUID specifically). Anyway, the magic code to create a UUID is simply this:


uuid_create(&$v4);
uuid_make($v4, UUID_MAKE_V4);
uuid_export($v4, UUID_FMT_STR, &$v4String);
echo "My UUID is now in $v4string";


This is obviously a version 4 UUID; you can trivially make the others just by changing the arguments appropriately. Wikipedia has a good page explaining the difference between UUID versions.

Python errors running s3cmd on Amazon AWS

Whenever I ran s3cmd on my AWS instance, I got the following errors:


/usr/lib/python2.6/dist-packages/S3/S3.py:9: DeprecationWarning: the md5 module is deprecated; use hashlib instead
/usr/lib/python2.6/dist-packages/S3/S3.py:10: DeprecationWarning: the sha module is deprecated; use the hashlib module instead


The problem is that this is an old, crappy version of s3cmd, 0.9.8.4. You can get version 0.9.9 using the instructions in the linked blog post. It’s just a matter of adding a new source to your apt-get sources list and then running apt-get update.

ssh_exchange_identification: read: Connection reset by peer

I got this while attempting to ssh into a VM on ubuntu. When I pinged the box, I got this:

PING 192.168.1.86 (192.168.1.86) 56(84) bytes of data.
64 bytes from 192.168.1.86: icmp_seq=1 ttl=64 time=0.614 ms
64 bytes from 192.168.1.86: icmp_seq=1 ttl=64 time=1.89 ms (DUP!)
64 bytes from 192.168.1.86: icmp_seq=2 ttl=64 time=0.602 ms
64 bytes from 192.168.1.86: icmp_seq=2 ttl=64 time=0.925 ms (DUP!)

Whoops! The VM was already running and had been granted the exact same IP. It was responding from two places at the same time, which was confusing the hell out of SSH. Just kill one of the boxes and you’re fine (DHCP release/refresh doesn’t work, because both boxes have the same MAC address).

Thanks to this thread for the hint.

no such file to load — mkmf

I got this when trying to install the ruby thrift client. It’s trying to compile a native extension, which needs ruby 1.8 dev version. Simple fix:


sudo apt-get install ruby1.8-dev