By default the cuda is not working on Ubuntu. You may not want to crash your system. Please do exactly follow the instruction step by step. If you are brave enough like me or you are still OK to handle the problem properly even if only the Linux kernel and SSH are working fine, there is no need to backup your system at first.

1. Go to the CUDA official website and download the Getting started guide for your system
1. https://developer.nvidia.com/cuda-downloads
2. Read the installation guide from Cuda official website
1. install all the prerequisites for your system 
3. Download Cuda 5 installation
1. typically it’s a cuda5.run file
4. Remove the existing nvidia driver from Ubuntu official reposistory
1. $ sudo apt-get remove –purge nvidia
5. Change tty to anyone without gui:
1. eg. ctrl + alt + F1
6. Stop the default dm:
1. eg. $ sudo service lightdm stop
7. Install gcc and change the default gcc to gcc-4.4:
1. sudo apt-get install gcc gcc-4.4
2. sudo update-alternatives –remove-all gcc ( if you have previous configuration )
3. sudo update-alternatives –config gcc ( if you have previous configuration )
4. sudo update-alternatives –install /usr/bin/gcc gcc /usr/bin/gcc-4.4 20
5. gcc –version
1. gcc 4.4.* or something like that should be printed out
8. Run installation:
1. chmod +x <cuda>.run
2. sudo ./<cuda>.run