I previously blogged that you can install and enable FRR on a UniFi UDM (or UDM SE) running UniFi OS 3.0.x. which gives you the ability to run BGP; in my case I was using it to talk to my NSX Tier-0 Edge. I noticed I started getting some weird issues after the first 3.1.x was installed (3.1.6 I think) and after a few posts on the UniFi forums a member of staff confirmed that the reason I was having issues is that I had manually installed FRR and UniFi also now bundle FRR to support their Magic VPN functionality. It would have been great if this was called out in the release notes and I may not have been as quick with upgrading!
While it may break some things, with the knowledge that FRR is now bundled into the OS, it makes it a little easier to configure BGP because there’s no requirement to install it.
This has been tested with UniFi OS 3.1.6, 3.2.5, 3.2.6, and 3.2.7.
Disclaimer: modifying your UniFi OS is unsupported and may break/brick your device. If you do go ahead, make sure you have a recent (and locally saved) backup and be prepared that you may have to factory reset and restore. This is very early stages and I have not had chance to test this much.
If we head into the FRR directory, we see there are 4 files present (daemons, frr.conf, support_bundle_commands.conf, and vtysh.conf). What I did is make backups of all of the files to allow quick rollback. The FRR service is not enabled if you don’t have multi-site VPN set up via the UniFi dashboard, if you do have this enabled then the following steps may not work.
Step 1 – Take a UniFi backup, and back up the original FRR files
Head to your UniFi console web UI and take a backup, I always prefer storing them locally so ensure you do that too.
Then back up all files in /etc/frr either by appending .original or similar to the file names, or tar them up. Whatever works for you, as long as they are backed up for quick recovery.
Step 2 – enable BGP daemon in daemons
You need to edit /etc/frr/daemons and find the bgpd line and enable it, use sed to keep things simple (or use vi and do it by hand, whatever works for you):
sed -i 's/bgpd=no/bgpd=yes/g' /etc/frr/daemonsStep 3 – modify frr.conf
As mentioned, I didn’t want to sway too much away from the default UniFi configuration, in the hope that any modifications I do make are survived during upgrades. Let us hope that they finally introduce BGP into the GUI – that would be cool. Here is my very basic frr.conf with BGP specific config below the default content in the file:
root@UDM-SE:/etc/frr# cat frr.conf
# default to using syslog. /etc/rsyslog.d/45-frr.conf places the log in
# /var/log/frr/frr.log
#
# Note:
# FRR's configuration shell, vtysh, dynamically edits the live, in-memory
# configuration while FRR is running. When instructed, vtysh will persist the
# live configuration to this file, overwriting its contents. If you want to
# avoid this, you can edit this file manually before starting FRR, or instruct
# vtysh to write configuration to a different file.
#log syslog informational
hostname UDM-SE
frr defaults datacenter
log file stdout
service integrated-vtysh-config
!
!
router bgp 65001
bgp router-id 172.16.72.254
neighbor 172.16.72.1 remote-as 65000
neighbor 172.16.72.1 default-originate
!
address-family ipv4 unicast
redistribute connected
redistribute kernel
neighbor V4 soft-reconfiguration inbound
neighbor V4 route-map ALLOW-ALL in
neighbor V4 route-map ALLOW-ALL out
exit-address-family
!
route-map ALLOW-ALL permit 10
!
line vty
!Modify the above to suit your environment/lab.
Step 4 – enable frr service
With the configuration complete, enable the frr service:
systemctl enable frr.service && service frr startIf all is well, you should see your config:
vtysh -c 'show ip bgp'Here’s mine and you can see some BGP routes appearing including the single segment I currently have:
It remains to be seen how this persists during upgrades, or if UniFi finally decide to bring this into the UI which would be really cool.
Let me know how this works for you and if you have any issues or questions. Thanks for reading.
I was kind of curious about attempting to use this to connect an Azure express route just as a test which requires bgp.
With your setup and configuration does that advertise all of your networks on your udmse to the router on the other end?
Hi Zach, I haven’t tested ExpressRoute. You’d most likely have to edit frr.conf and put in the networks and once done I don’t see why it wouldn’t work. My config is set up to advertise a default route to my NSX Edge.
Do you know if this supports multipath or do they just get rejected like in older firmwares?
Also – Is the original UDM-Pro supported, I notice you have the SE?
Ta
Jon
Hi Jon, I haven’t tested multi path, and I don’t have a non-SE to test, sorry.
This is an excellent guide, thanks for sharing this, I followed the guide and successfully peered into my CP Firewall, however, on the CP firewall I can see the connection from UDM is still NAT’d (I’m using the WAN port to peer with the CP), do you have any guide remove the IP masquerading. I prefer to keep my VLANs on the UDM rather than moving to another router.
Hi UdS, no problem. Sorry, I’m not sure if I can help, I peer the UDM-SE to NSX using the LAN port, I haven’t done any WAN side testing.
Hi! Thanks for great guide! Is it possible to use BGP when remote network is connected via S2S tunnel?
I think there is a traffic restriction to remote network connected via S2S tunnel. I can’t ping or curl anything from UDM itself, Ubiquiti support says it’s by design. Is it possible that BGP communication is also restricted?
Hi Yaroslav, you’re welcome. I’m not sure it’s possible and I haven’t tested it, however UniFi are bringing native BGP support in the UI in an upcoming release, once that is out it might be possible.
Hey Yaroslav, just saw your comments and I want to confirm you that it works.
Configured an IPSec IKEv2 with routed virtual tunnel interface (VTI) and dynamic routing check box enabled.
Once tunnel was established and that the VTI on each side were pinging, I went to setup frr like in the example above.
Using the IP of the VTI of the remote site as the neighbor and I needed to adjust the policies (route-map) from the example for routes to be announced and accepted. (you can use the `no bgp ebgp-requires-policy` option to disable that behavior, see: https://docs.frrouting.org/en/latest/bgp.html#clicmd-bgp-ebgp-requires-policy)
I had nothing else to do other than setting up S2S in the UI config.
This was an awesome find thank you. By default it looks like UDM doesn’t support max path so if you have multiple peers advertising the same networks they will be rejected.
I would recommend adding a `maximum-paths 1`
B>r 198.18.0.0/32 [20/0] via 192.168.0.101, br0, weight 1, 00:07:08
r via 192.168.0.102, br0, weight 1, 00:07:08
B>r 198.18.0.1/32 [20/0] via 192.168.0.101, br0, weight 1, 00:07:08
r via 192.168.0.102, br0, weight 1, 00:07:08
B>r 198.18.1.0/32 [20/0] via 192.168.10.101, br0, weight 1, 00:07:08
r via 192.168.0.102, br0, weight 1, 00:07:08
2024/06/21 15:57:24 ZEBRA: [HSYZM-HV7HF] Extended Error: Multipath support not enabled in kernel
2024/06/21 15:57:24 ZEBRA: [WVJCK-PPMGD][EC 4043309093] netlink-dp (NS 0) error: Invalid argument, type=RTM_NEWROUTE(24), seq=133, pid=3279315533
2024/06/21 15:57:24 ZEBRA: [QTT8V-3ZQ34] nl_batch_read_resp: netlink error message seq=133
2024/06/21 15:57:24 ZEBRA: [TX7F6-F3TSG] default(0:254):198.18.0.0/32: Route install failed
After setting `maximum-paths 1`
B>* 198.18.0.0/32 [20/0] via 192.168.0.101, br0, weight 1, 00:00:03
B>* 198.18.0.1/32 [20/0] via 192.168.0.101, br0, weight 1, 00:00:03
B>* 198.18.1.0/32 [20/0] via 192.168.0.101, br0, weight 1, 00:00:03