Don't run Ubuntu myself, but in CentOS-ese -> Ubuntu-ese yum is = to apt-get. In your checklist you show you've already used apt-get so don't stop now! Installing a server is one thing ... maintaining one is the part of the iceburg not seen. The question becomes what are the same packages called in Ubuntu?
One could use Google or learn how to find things via apt-get. Here's a cheat sheet for apt-get:
http://www.cyberciti.biz/tips/linux-debian-package-management-cheat-sheet.html
In order for the math symbols to appear correctly, they must be converted to an image ... that's what those other command line only apps do.
Have you turned on the math filters? Set the paths to those apps? and tested using:
http://[your_moodle]/[moodle_dir]/filter/tex/texdebug.php
Not trying to be cruel here ... time to learn how to fish! ;)
'spirit of sharing', Ken