Yes, there is something about fatal X errors in the syslog: http://www.pastebin.ca/1435822
The Xorg.log contains a bracktrace, see here: http://www.pastebin.ca/1435823
My xorg.conf is still at http://www.pastebin.ca/1433605
It doesn't always crash like this, sometimes I can get a console and go back to X, sometimes I can't. Whether I have windows open when I press CTRL-ALT-F1 or not, doesn't seem to matter.
Just now I had this odd thing again where mouse actions switch to the second monitor while the pointer is still on the first. Xorg.log now contains many of the following lines:
miPointerConstrainCursor: Invalid input device pointer
miPointerWarpCursor: Invalid input device pointer
miPointerUpdateSprite: Invalid input device pointer