We are running MPLS on RouterOS for over one year and still I’m getting new errors in MPLS implementation on RouterOS. Below is a list of all issues, I’ve found (versions < 6.33) :
1. The main problem is separation of VRFs. RouterOS is a linux based system and all VRF tables are linux routing tables, where policy based routing (ip rule), RT local, arp/nd and firewall table is common for all tables. In practice it means that if you decided to use 192.168.0.0/24 in VRF1 but your customer got the same range in VRF. You will be able to ping each other and routing will be unstable because of duplicate IPs.
Solution: we have VRFs only for separation of Voice, Internet and Mgnt traffic and to prevent user access to our L3 network.
2. BGP attributes. Second important bug is that inside VRF RouterOS there are problems with supporting of BGP attributes. I’ve got 2 routes – one with Local preference 200, second with 100, first one should be active. But active is the route which was installed to VRF as first. This makes usage of BGP attributes for routing decision useless.
Solution: I have a community + distance based affecting of routes inside VRF.
3. CE to PE dynamic routing. If you have a customer in VRF and you want to connect PE to his CE with BGP – forget about it. RouterOS will not build BGP session on VRF interface with external router.
Solution: Run RIPv2 between CE and PE or static routing
4. Redistribution of static routes. In case of redistribution of static routes to VRF sometimes happens that inactive static route was not deleted from VRF. (redistribution issue)
Solution: Be careful with redistribution to VRF
5. Traffic engineering – on labs it works fine. In real practice deleting of TE tunnel completely broke OSPF process on the router. Because it was a router from core and it happen always (I tried it on 3 live routers), I decided to not use TE in the network. How can I trust technology which has such behavior on regular basis ?
Also one interesting thing is all VRFs are using the tunnel by default if default gateway IP = destination IP of the tunnel. I can understand this in VPLS, but what if I have 3 VRFs and I need different paths to default gateway for each VRF?
Solution: unfortunately don’t use TE
We decided to upgrade main routers to cisco ISR 4400 routers or 3600ME switches.
If we count also that every week one or two RouterOS routers in the core network hang without reason, our decision makes sense.