In the interest of having this information somewhere handy: Firstly, to successfully install VirtualBox Guest Additions within a server based image:
sudo apt-get install dkms build-essential linux-headers-`uname -r`
I discovered that if you clone an ubuntu server based image, networking stops functioning in the clone. The reason for this is that the new Machine assigns a new MAC address to the NIC. So the Udev rules think it's a new card, and assign it a new device id, like eth1, or eth2, etc.
To prevent this from happening, in your base image, edit
/lib/udev/rules.d/75-persistent-net-generator.rules
Search for 'xen', and you'll see a block about ignoring the Xen virtual interfaces. Add the following below it:
# ignore VirtualBox virtual interfaces
ATTR{address}=="08:00:27:*", GOTO="persistent_net_generator_end"
Save it, and then remove the file /etc/udev/rules.d/70-persistent-net.rules
.
Do the same thing in any cloned images with broken networking, and reboot the VMs.