Proxmox, LXC and Terraform
Even before Broadcom appeared to be doing it's best to kill off VMWare, I've been looking to migrate from VMWare to Proxmox. I already have 2 Proxmox nodes, and just have one last node to migrate, along with 1 VM. I have been migrating for a while and the import tool seems to work very well.
In preparation for the final node to be converted, I've been looking at how I can do the install. I have previously used an sd card, via a USB to SD card converter and that works well. However, that does mean a dedicated SD card to the job and the original one I used is now out of date. That got me thinking about booting over the internet.
After watching TechnoTim's video on netboot.xyz, I had initially opted for that. However, the issue here is that works well for a first option - installation, but what about the rest of the process? What about the configuration? TechnoTim also has a video about the 11 things he does when starting off with Proxmox and I wanted to do those to my proxmox install as well. Hopefully in an automated way, which is when I came access TechnoTim's video on MaaS.
MaaS, or Metal as a Service, is a canonical product for deploying operating systems to bear metal, VMs, etc as well as supporting ansible and host of other automation tools for post configuration. This sounded like what I was after so I decided to try it out. As you can probably tell from the title of this post, I chose to deploy MaaS on to a LXC container hosted on my Proxmox install. For the deployment, I chose to use terraform and a template from Turnkey Linux - I have yet to play about with Hashicorp's Packer for my own images. MaaS also has some specific settings for the LXC container.
NB. Hashicorp has changed the license on Terraform recently and I am looking to move to openTofu. However, at this stage there should be no difference between the two at the moment. When I do move over, expect a post.
Just to be difficult with this, I also wanted to throw DNS (via PowerDNS) and Asset management (Via Netbox) also in to the mix. These weren't helped by the fact that Proxmox had them set up as part of the SDN, which created duplicate entries for me. If you are going to follow along, I do suggest removing those from your zone. I had chosen to do this via terraform, so I could add in all the detail to netbox as I went along, preferably without typing it all out twice.
As I progress through, laying out the files, it soon became clear that the project was going to have some serious issues. For a start, proxmox only supports the provision of capabilities to LXC containers from the 'root@pve' account. I don't want to use that for provisioning, so that was my first issue. Terraform can check the capabilities are correct, however.
The next issue was that in order to get this to order I needed to set some mount options. I searched in the documentation, but I couldn't find these. I will keep looking. So, not to be disheartened by it, I set the options by hand.
That lead me on to the last issue. None of the options seemed to work for MaaS. To top it off, it also appears that proxmox is not directly supported from MaaS, so netboot.xyz might be the best option after all. I will continue to look at MaaS, as I do want to get it working, and proxmox is just a variant on Debian, which does have packer support for MaaS.
Recently, I found a cheap (sub £100) N100 from Ali-express that I have added to the cluster. In order to get it set up, I did use netboot.xyz and my ansible script to help get proxmox in to the state that I wanted.
Time to look in to Packer a bit more closely, while I change tack and use a Virtual machine.