Thursday 13 August 2020

Raspberry Pi 4 Juju Controller

I have a MAAS cluster at home to test out Juju, OpenStack, Kubernetes etc. The cluster is really a collection of machines that have been carefully selected for their cheapness and ability to fit in a small space under a spiral staircase. Since its inception I have used an old laptop with a broken screen as a Juju controller. Unfortunately the laptop did not take kindly to having tea spilt on it a few months ago and so I forced to replace it. I was feeling flush at the time to decided to go for a Raspberry Pi 4. Unfortunately I failed to spot that the Raspberry Pi 4 does not actually network boot like a conventional machine and so I wouldn't be able to enlist it in MAAS. Fortunately while working on something else I spotted that Juju has introduced the idea of Managing multiple clouds with one controller This gave me the idea of setting up a controller on the Pi and then adding MAAS as an additional cloud to it.

Prepare the Pi

Burn an Ubuntu image onto the SD card, I went for the Ubuntu 20.04.1 64-bit server OS for arm64. Give the Pi a static IP address. I haven't tested it but I'm sure the Pi needs to have a wired network connection rather than wireless, otherwise access to containers via the macvlan interface will break.

I believe it would be possible to set the controller up using the Manual cloud type but I prefer the idea of the controller being in its own LXD container so I went for that approach. To configure LXD initialise it and then disable ipv6 as it doesn't currently work with Juju.
ubuntu@pi-controller-host:~$ lxd init --auto
ubuntu@pi-controller-host:~$ lxc network set lxdbr0 ipv6.address none   
The controller needs to have access to the network the MAAS nodes are going to be deployed on. The primary interface on the LXD containers could be switched to be macvlan rather than bridged but unfortunately for reasons the LXD host would loose access to the containers. A neat solution is to give the LXD container two nics, one on the bridged network which will allow the host (pi-controller-host in my case) access to the containers and one macvlan interface that will give the wider network access to the containers.

This Pi is only going to be used as a controller so I make these network changes in the default lxd profile.
ubuntu@pi-controller-host:~$ cat two-nics.yaml
config:
  user.network-config: |
    version: 1
    config:
      - type: physical
        name: eth0
        subnets:
          - type: dhcp
            ipv4: true
      - type: physical
        name: eth1
        subnets:
          - type: dhcp
            ipv4: true
description: Default LXD profile
devices:
  eth0:
    name: eth0
    network: lxdbr0
    type: nic
  eth1:
    name: eth1
    nictype: macvlan
    parent: eth0
    type: nic
  root:
    path: /
    pool: default
    type: disk
name: default
ubuntu@pi-controller-host:~$ lxc profile edit default < two-nics.yaml

Install Juju Controller

Install the juju client. In this case I needed to do some testing with Juju 2.8.2 so I  went for the candidate channel but normally I'd go for the latest/stable channel.
ubuntu@pi-controller-host:~$ sudo snap install --classic  --channel=2.8/candidate juju                                                                                                                                                                                               
juju (2.8/candidate) 2.8.2 from Canonical installed     
Next create the Juju controller.
ubuntu@pi-controller-host:~$ juju bootstrap localhost pi-lxd-controller
Creating Juju controller "pi-lxd-controller" on localhost/localhost
Looking for packaged Juju agent version 2.8.2 for arm64                                                                              
No packaged binary found, preparing local Juju agent binary                                      
To configure your system to better support LXD containers, please see: https://github.com/lxc/lxd/blob/master/doc/production-setup.md
Launching controller instance(s) on localhost/localhost...                                         
 - juju-b84c1e-0 (arch=arm64)                                                                     
Installing Juju agent on bootstrap instance                       
Fetching Juju Dashboard 0.2.0                                                                  
Waiting for address                                                                              
Attempting to connect to 10.119.95.251:22                                                          
Attempting to connect to 172.16.0.20:22                                                         
Connected to 10.119.95.251                                                                                                
Running machine configuration script...                                                                             
Bootstrap agent now started                                                                         
Contacting Juju controller at 10.119.95.251 to verify accessibility...                          
                                                                                                 
Bootstrap complete, controller "pi-lxd-controller" is now available                             
Controller machines are in the "controller" model                                               
Initial model "default" added 
Now the Juju controller is setup for using Juju to deploy applications into local LXD containers. But the idea here is to use this controller for deployments to MAAS. To achieve this we need to add the MAAS cloud to the controller:
ubuntu@pi-controller-host:~$ juju add-cloud --controller pi-lxd-controller     
Cloud Types                                                                   
  lxd                                                                              
  maas                                                                             
  manual                                                                            
  openstack                                                                         
  vsphere                                                                             
                                                                                                       
Select cloud type: maas                                                                    
                                                                                             
Enter a name for your maas cloud: homemaas                                                                                     

Enter the API endpoint url: http://172.16.0.2:5240/MAAS

Cloud "" added to controller "pi-lxd-controller".
WARNING loading credentials: credentials for cloud homemaas not found
To upload a credential to the controller for cloud "homemaas", use
* 'add-model' with --credential option or
* 'add-credential -c homemaas'.
Giving the controller an api key for the MAAS environment is a separate step:
ubuntu@pi-controller-host:~$ juju add-credential --controller pi-lxd-controller homemaas
Using cloud "homemaas" from the controller to verify credentials.
Enter credential name: admin

Regions
  default

Select region [any region, credential is not region specific]:

Using auth-type "oauth1".

Enter maas-oauth:

Controller credential "admin" for user "admin" for cloud "homemaas" on controller "pi-lxd-controller" added.
For more information, see ‘juju show-credential homemaas admin’.
Since I opted for a candidate version of Juju I need to tell Juju where to look for the agent code:
ubuntu@pi-controller-host:~$ juju model-defaults homemaas agent-stream=devel

Deploying to MAAS

A Juju model is specific to a cloud and at this point the controller can deploy to two different clouds: LXD and MAAS. To differentiate between the two when adding a model the cloud needs to be specified:
ubuntu@pi-controller-host:~$ juju add-model maas-model homemaas    
Added 'maas-model' model on homemaas/default with credential 'admin' for user 'admin'
I can now deploy applications using MAAS via my LXD controller on the Pi:
ubuntu@pi-controller-host:~$ juju deploy ubuntu
Located charm "cs:ubuntu-15".
Deploying charm "cs:ubuntu-15".
ubuntu@pi-controller-host:~$ juju status
Model       Controller         Cloud/Region      Version  SLA          Timestamp
maas-model  pi-lxd-controller  homemaas/default  2.8.2    unsupported  09:05:39Z

App     Version  Status  Scale  Charm   Store       Rev  OS      Notes
ubuntu  18.04    active      1  ubuntu  jujucharms   15  ubuntu

Unit       Workload  Agent  Machine  Public address  Ports  Message
ubuntu/0*  active    idle   0        172.16.0.104           ready

Machine  State    DNS           Inst id   Series  AZ       Message
0        started  172.16.0.104  warm-dog  bionic  default  Deployed

Accessing Controller Remotely

I tend to keep all my deployment config on the MAAS server so it makes sense to be able to administer the  Juju deployment from there. To set this up I needed to create a new user with Juju and then use that use on the MAAS server.
ubuntu@pi-controller-host:~$ juju add-user fawkesadmin                
User "fawkesadmin" added                                                                 
Please send this command to fawkesadmin:                                        
    juju register MGwTC2Zhd2tlc2FkbWluMCgTEzVERYLONGSTRING
                                                                                 
"fawkesadmin" has not been granted access to any models. You can use "juju grant" to grant access.
Next I need to grant that user superuser perms on the controller
ubuntu@pi-controller-host:~$ juju grant fawkesadmin superuser -c pi-lxd-controller
Finally on the maas server I can register that controller
1 liam@fawkes:~$ juju register MGwTC2Zhd2tlc2FkbWluMCgTEzVERYLONGSTRING
Enter a new password:                                
Confirm password:                                                                                
Enter a name for this controller [pi-lxd-controller]:                                                                                                                    
Initial password successfully set for fawkesadmin.         
                                                                                                                            
Welcome, fawkesadmin. You are now logged into "pi-lxd-controller".                                  
                                                                                                                                
There are 3 models available. Use "juju switch" to select                                                                        
one of them:                                                                                                                
  - juju switch admin/controller                                              
  - juju switch admin/default                                                                                                    
  - juju switch admin/maas-model   
I now need to add the MAAS creds the fawkesadmin user is going to use (I used the same API key as before)
liam@fawkes:~$ juju add-credential -c pi-lxd-controller homemaas                   
Using cloud "homemaas" from the controller to verify credentials.
Enter credential name: admin                                                                                                                           
                                                 
Regions                                                                   
  default                                                                          
                                                                       
Select region [any region, credential is not region specific]:                   
                                                                                                                            
Using auth-type "oauth1".                                                                          
                                                                   
Enter maas-oauth:                                                             
                                                                          
Controller credential "admin" for user "fawkesadmin" for cloud "homemaas" on controller "pi-lxd-controller" added.
For more information, see ‘juju show-credential homemaas admin’.