If you break your ubuntu 12.04 install because nvidia driver 304 breaks, here's how you fix it. Hopefully you made backups of your x config files (it's a selectable option when you install new nvidia drivers). If it's x is truly broken, then you won't see any windows or GUI. Hit ctrl-alt-F1 and login to your user, then do this
Yeah, I managed to kill X by installing 310 after having installed 304. It was managing to put 304 in the kernel while using 310's interface to it. Removing the packages for both and then installing just 310 fixed it.