Using VRRP inside a lipkg zone on OmniOS
I’m running a HA ldap setup however I keep hitting illumos issue #13326, hoping to mitigate this a tiny bit by using a VIP instead of multiple A records behind a ldap.example.org host name. (Sadly that doesn’t help as the connection state itself ends up in the bad state, but it did send me down the VRRP on illumos rabbit hole.)
Solaris 10 and therefore also the illumos decedents support VRRP for both IPv4 and IPv6, documentation on this is a bit sparse so hopefully this note dump is useful to someone, most likely myself in a few years. On OmniOS it’s pretty easy to get the required utilities.
root@jupiter:~# pkg install system/network/routing/vrrp
If you plan on using the VIP only in a (l)ipkg
zone, you don’t even need to install this in the global zone! So just install it where you will be using the VIP. In my case this is the ldap1 and ldap2, these zones cannot be on the same host!
Sadly there is no support in the brand scripts yet, so we will need to create the special VNIC in the global zone manually:
dladm create-vnic -m vrrp -l ixgbe0 -V 98 -A inet -v 110 ldap1vip0
dladm create-vnic -m vrrp -l ixgbe0 -V 98 -A inet6 -v 110 ldap1vip1
Notice that I create two special VNICs, one for IPv4 and one for IPv6. There are some things to keep in mind there, the VNICs needs to be created on top of the same link and with the same vlan-id as the regular vnic!
root@jupiter:~# dladm show-vnic
LINK OVER SPEED MACADDRESS MACADDRTYPE VID ZONE
ldap10 ixgbe0 10000 0:22:6:fd:d4:db fixed 110 ldap1
ldap1vip0 ixgbe0 10000 0:0:5e:0:1:62 vrrp, 98/inet 110 --
ldap1vip1 ixgbe0 10000 0:0:5e:0:2:62 vrrp, 98/inet6 110 --
In my case the ldap10 VNIC is created automatically by the brand scripts, so my final net config for the ldap1 zone looks like this:
root@jupiter:~# zadm show ldap1 | jq .net
[
{
"global-nic": "ixgbe0",
"mac-addr": "00:22:06:fd:d4:db",
"physical": "ldap10",
"vlan-id": "110"
},
{
"physical": "ldap1vip0"
},
{
"physical": "ldap1vip1"
}
]
Inside the ldap1 zone we now need to configure the VIP and also create the vrrp router.
root@ldap1:~# ipadm create-addr -T static -d -a 172.16.10.89/20 ldap1vip0/v4
root@ldap1:~# vrrpadm create-router -V 98 -l ldap20 -A inet ldap1vip0
It is important to use the same VRID (98) and address type we use in the global zone.
root@ldap1:~# vrrpadm show-router
NAME VRID LINK AF PRIO ADV_INTV MODE STATE VNIC
ldap1vip0 98 ldap20 IPv4 255 1000 eopa- MASTER ldap1vip0
ldap1vip1 98 ldap20 IPv6 255 1000 eopa- INIT ldap1vip1
root@ldap1:~# vrrpadm show-router -x
NAME STATE PRV_STAT STAT_LAST VNIC PRIMARY_IP VIRTUAL_IPS
ldap1vip0 MASTER INIT 15.246s ldap1vip0 172.16.10.89 172.16.11.89
ldap1vip1 INIT NONE 9.371s ldap1vip1 -- --
Of course you need to repeat these steps on the 2nd host + zone.