Skip to content

Fabric Topology

Introduction

Fabric Topology defines how devices are physically and logically connected in your network. AVD uses a hierarchical data model to describe the relationships between spines, leafs, and other network devices. This guide explains how to define your fabric topology, configure uplinks, and leverage automatic interface allocation.

Key Concepts

  • Node Types: Define the role of each device (spine, l3leaf, l2leaf, etc.)
  • Uplinks: Connections from leafs to spines or from L2 leafs to L3 leafs
  • MLAG: Multi-Chassis Link Aggregation for leaf redundancy
  • Default Interfaces: Automatic interface assignment based on node type and platform

Topology Building Blocks

Inventory Structure

The inventory below is an example of an Ansible inventory used to define a fabric hierarchy. Devices are organized into groups that map to their roles:

inventory.yml
---
all:
  children:
    FABRIC:
      children:
        HTFT:
          children:
            HTFT_SPINES:
              hosts:
                htft-spine1:
                  ansible_host: 172.16.2.11
                htft-spine2:
                  ansible_host: 172.16.2.12
            HTFT_LEAFS:
              hosts:
                topo-leaf1a:
                  ansible_host: 172.16.2.101
                topo-leaf1b:
                  ansible_host: 172.16.2.102

Fabric-Wide Settings

Define global settings that apply to all devices in the fabric:

fabric.yml
---
# Fabric Topology How-To Guide
# Fabric-wide settings that apply to all devices

fabric_name: FABRIC # (1)!

# Define underlay and overlay routing protocols
underlay_routing_protocol: ebgp # (2)!
overlay_routing_protocol: ebgp # (3)!

# P2P uplinks MTU - use 1500 for vEOS-lab/cEOS, 9214 for production
p2p_uplinks_mtu: 1500 # (4)!
  1. fabric_name must be defined and match group name covering all devices in scope of the fabric.
  2. Underlay routing protocol - eBGP is common for EVPN/VXLAN fabrics
  3. Overlay routing protocol for EVPN peering
  4. MTU for point-to-point uplinks (1500 for virtual, 9214 for physical)

Default Interfaces

Automatically assign interfaces based on node type and platform, eliminating repetitive per-node definitions:

default_interfaces.yml
---
# Fabric Topology How-To Guide
# Default interfaces - automatically assign interfaces based on node type and platform
# This eliminates the need to define uplink/downlink interfaces on each node

default_interfaces: # (1)!
  - types: [spine]
    platforms: [default]
    downlink_interfaces: [Ethernet1-8] # (2)!
  - types: [l3leaf]
    platforms: [7280R3] # (3)!
    uplink_interfaces: [Ethernet49-52/1] # (4)!
    mlag_interfaces: [Ethernet53-54/1] # (5)!
  - types: [l3leaf]
    platforms: [default] # Fallback for unmatched platforms
    uplink_interfaces: [Ethernet1-2]
    mlag_interfaces: [Ethernet3-4]
    downlink_interfaces: [Ethernet8]
  - types: [l2leaf]
    platforms: [default]
    uplink_interfaces: [Ethernet1-2]
  1. Define interface mappings per node type and platform
  2. Spine downlink interfaces connect to leafs
  3. You can define different default interfaces for different platforms
  4. Leaf uplink interfaces connect to spines
  5. MLAG peer-link interfaces for leaf redundancy

Spine Configuration

Spines are the core of the fabric, providing connectivity between all leafs:

spines.yml
---
# Fabric Topology How-To Guide
# Spine configuration

spine:
  defaults:
    platform: vEOS-lab # (1)!
    loopback_ipv4_pool: 10.255.0.0/27 # (2)!
    bgp_as: 65100 # (3)!

  nodes: # (4)!
    - name: htft-spine1
      id: 1 # (5)!
      mgmt_ip: 172.16.2.11/24 # (6)!
    - name: htft-spine2
      id: 2
      mgmt_ip: 172.16.2.12/24
  1. Platform determines default settings and validation rules
  2. IP pool for Loopback0 interfaces (used for BGP router-id and EVPN peering)
  3. BGP AS number for the spine layer
  4. List of spine nodes
  5. Unique identifier for IP address allocation
  6. Management IP address

L3 Leaf Configuration

L3 leafs provide network services (VLANs, VRFs, SVIs) and connect to endpoints:

l3_leas.yml
---
# Fabric Topology How-To Guide
# L3 Leaf configuration

l3leaf:
  defaults:
    platform: vEOS-lab
    loopback_ipv4_pool: 10.255.0.0/27 # (1)!
    loopback_ipv4_offset: 2 # (2)!
    vtep_loopback_ipv4_pool: 10.255.1.0/27 # (3)!
    uplink_switches: [htft-spine1, htft-spine2] # (4)!
    uplink_ipv4_pool: 10.255.255.0/26 # (5)!
    mlag_peer_ipv4_pool: 10.255.1.64/27 # (6)!
    mlag_peer_l3_ipv4_pool: 10.255.1.96/27 # (7)!
    virtual_router_mac_address: 00:1c:73:00:00:99 # (8)!
    spanning_tree_priority: 4096
    spanning_tree_mode: mstp

  node_groups: # (9)!
    - group: HTFT_LEAF1 # (10)!
      bgp_as: 65101 # (11)!
      nodes:
        - name: htft-leaf1a
          id: 1
          mgmt_ip: 172.16.2.101/24
          uplink_switch_interfaces: [Ethernet1, Ethernet1] # (12)!
        - name: htft-leaf1b
          id: 2
          mgmt_ip: 172.16.2.102/24
          uplink_switch_interfaces: [Ethernet2, Ethernet2]
  1. Loopback IP pool (shared with spines in this example)
  2. Offset to avoid IP conflicts with spines
  3. VTEP loopback pool for VXLAN tunnel endpoints
  4. List of uplink switches (spines)
  5. IP pool for point-to-point uplinks to spines
  6. IP pool for MLAG peer-link (VLAN 4094)
  7. IP pool for MLAG iBGP peering (VLAN 4093)
  8. Virtual MAC for anycast gateway on SVIs
  9. Node groups allow shared configuration and automatic MLAG pairing
  10. Group name for documentation and identification
  11. BGP AS for this leaf pair (unique per MLAG pair in eBGP designs)
  12. Spine interfaces this leaf connects to

MLAG Pairing

When exactly two nodes are in the same node_group, AVD automatically configures them as an MLAG pair:

  • Allocates MLAG peer-link interfaces from mlag_interfaces or default_interfaces
  • Configures VLAN 4094 for MLAG control plane
  • Configures VLAN 4093 for iBGP peering between MLAG peers
  • Assigns matching virtual MAC addresses for anycast gateway

Generated Configuration

AVD generates complete device configurations based on your topology definitions.

Spine Configuration

interface Ethernet1
   description P2P_htft-leaf1a_Ethernet1
   no shutdown
   mtu 1500
   no switchport
   ip address 10.255.255.0/31
!
interface Ethernet2
   description P2P_htft-leaf1b_Ethernet1
   no shutdown
   mtu 1500
   no switchport
   ip address 10.255.255.4/31
!
interface Loopback0
   description ROUTER_ID
   no shutdown
   ip address 10.255.0.1/32
!

!
no enable password
no aaa root
!
username admin privilege 15 role network-admin nopassword
username arista privilege 15 role network-admin secret sha512 $6$Enl0WfE32FthwyiJ$yTyGaEJ2uPKLU.F7314YtB7J1jrzrMi7ogXIRTEHQfLdLgKWWmr1UvNlZLN6AyuxET7G5aH3AI9OYRzxVTkB1.
!
daemon TerminAttr
   exec /usr/bin/TerminAttr -cvaddr=192.168.1.12:9910 -cvauth=token,/tmp/token -cvvrf=MGMT -disableaaa -smashexcludes=ale,flexCounter,hardware,kni,pulse,strata -taillogs -cvsourceintf=Management1
   no shutdown
!
vlan internal order ascending range 1006 1199
!
transceiver qsfp default-mode 4x10G
!
service routing protocols model multi-agent
!
hostname htft-spine1
ip domain lookup vrf MGMT source-interface Management1
ip name-server vrf MGMT 192.168.1.1
!
spanning-tree mode none
!
vrf instance MGMT
!
interface Ethernet1
   description P2P_htft-leaf1a_Ethernet1
   no shutdown
   mtu 1500
   no switchport
   ip address 10.255.255.0/31
!
interface Ethernet2
   description P2P_htft-leaf1b_Ethernet1
   no shutdown
   mtu 1500
   no switchport
   ip address 10.255.255.4/31
!
interface Loopback0
   description ROUTER_ID
   no shutdown
   ip address 10.255.0.1/32
!
interface Management1
   description OOB_MANAGEMENT
   no shutdown
   vrf MGMT
   ip address 172.16.2.11/24
!
ip routing
no ip routing vrf MGMT
!
ip prefix-list PL-LOOPBACKS-EVPN-OVERLAY
   seq 10 permit 10.255.0.0/27 eq 32
!
ntp local-interface vrf MGMT Management1
ntp server vrf MGMT 0.pool.ntp.org prefer
!
route-map RM-CONN-2-BGP permit 10
   match ip address prefix-list PL-LOOPBACKS-EVPN-OVERLAY
!
router bfd
   multihop interval 300 min-rx 300 multiplier 3
!
router bgp 65100
   router-id 10.255.0.1
   no bgp default ipv4-unicast
   maximum-paths 4
   neighbor EVPN-OVERLAY-PEERS peer group
   neighbor EVPN-OVERLAY-PEERS next-hop-unchanged
   neighbor EVPN-OVERLAY-PEERS update-source Loopback0
   neighbor EVPN-OVERLAY-PEERS bfd
   neighbor EVPN-OVERLAY-PEERS ebgp-multihop 3
   neighbor EVPN-OVERLAY-PEERS password 7 Q4fqtbqcZ7oQuKfuWtNGRQ==
   neighbor EVPN-OVERLAY-PEERS send-community
   neighbor EVPN-OVERLAY-PEERS maximum-routes 0
   neighbor IPv4-UNDERLAY-PEERS peer group
   neighbor IPv4-UNDERLAY-PEERS password 7 7x4B4rnJhZB438m9+BrBfQ==
   neighbor IPv4-UNDERLAY-PEERS send-community
   neighbor IPv4-UNDERLAY-PEERS maximum-routes 256000
   neighbor 10.255.0.3 peer group EVPN-OVERLAY-PEERS
   neighbor 10.255.0.3 remote-as 65101
   neighbor 10.255.0.3 description htft-leaf1a_Loopback0
   neighbor 10.255.0.4 peer group EVPN-OVERLAY-PEERS
   neighbor 10.255.0.4 remote-as 65101
   neighbor 10.255.0.4 description htft-leaf1b_Loopback0
   neighbor 10.255.255.1 peer group IPv4-UNDERLAY-PEERS
   neighbor 10.255.255.1 remote-as 65101
   neighbor 10.255.255.1 description htft-leaf1a_Ethernet1
   neighbor 10.255.255.5 peer group IPv4-UNDERLAY-PEERS
   neighbor 10.255.255.5 remote-as 65101
   neighbor 10.255.255.5 description htft-leaf1b_Ethernet1
   redistribute connected route-map RM-CONN-2-BGP
   !
   address-family evpn
      neighbor EVPN-OVERLAY-PEERS activate
   !
   address-family ipv4
      no neighbor EVPN-OVERLAY-PEERS activate
      neighbor IPv4-UNDERLAY-PEERS activate
!
end

L3 Leaf Configuration

interface Port-Channel3
   description MLAG_htft-leaf1b_Port-Channel3
   no shutdown
   switchport mode trunk
   switchport trunk group MLAG
   switchport
!
interface Ethernet1
   description P2P_htft-spine1_Ethernet1
   no shutdown
   mtu 1500
   no switchport
   ip address 10.255.255.1/31
!
interface Ethernet2
   description P2P_htft-spine2_Ethernet1
   no shutdown
   mtu 1500
   no switchport
   ip address 10.255.255.3/31
!
interface Ethernet3
   description MLAG_htft-leaf1b_Ethernet3
   no shutdown
   channel-group 3 mode active
!
interface Ethernet4
   description MLAG_htft-leaf1b_Ethernet4
   no shutdown
   channel-group 3 mode active
!
interface Loopback0
   description ROUTER_ID
   no shutdown
   ip address 10.255.0.3/32
!
interface Loopback1
   description VXLAN_TUNNEL_SOURCE
   no shutdown
   ip address 10.255.1.3/32
!

!
no enable password
no aaa root
!
username admin privilege 15 role network-admin nopassword
username arista privilege 15 role network-admin secret sha512 $6$Enl0WfE32FthwyiJ$yTyGaEJ2uPKLU.F7314YtB7J1jrzrMi7ogXIRTEHQfLdLgKWWmr1UvNlZLN6AyuxET7G5aH3AI9OYRzxVTkB1.
!
daemon TerminAttr
   exec /usr/bin/TerminAttr -cvaddr=192.168.1.12:9910 -cvauth=token,/tmp/token -cvvrf=MGMT -disableaaa -smashexcludes=ale,flexCounter,hardware,kni,pulse,strata -taillogs -cvsourceintf=Management1
   no shutdown
!
vlan internal order ascending range 1006 1199
!
transceiver qsfp default-mode 4x10G
!
service routing protocols model multi-agent
!
hostname htft-leaf1a
ip domain lookup vrf MGMT source-interface Management1
ip name-server vrf MGMT 192.168.1.1
!
spanning-tree mode mstp
no spanning-tree vlan-id 4093-4094
spanning-tree mst 0 priority 4096
!
vlan 4093
   name MLAG_L3
   trunk group MLAG
!
vlan 4094
   name MLAG
   trunk group MLAG
!
vrf instance MGMT
!
interface Port-Channel3
   description MLAG_htft-leaf1b_Port-Channel3
   no shutdown
   switchport mode trunk
   switchport trunk group MLAG
   switchport
!
interface Ethernet1
   description P2P_htft-spine1_Ethernet1
   no shutdown
   mtu 1500
   no switchport
   ip address 10.255.255.1/31
!
interface Ethernet2
   description P2P_htft-spine2_Ethernet1
   no shutdown
   mtu 1500
   no switchport
   ip address 10.255.255.3/31
!
interface Ethernet3
   description MLAG_htft-leaf1b_Ethernet3
   no shutdown
   channel-group 3 mode active
!
interface Ethernet4
   description MLAG_htft-leaf1b_Ethernet4
   no shutdown
   channel-group 3 mode active
!
interface Loopback0
   description ROUTER_ID
   no shutdown
   ip address 10.255.0.3/32
!
interface Loopback1
   description VXLAN_TUNNEL_SOURCE
   no shutdown
   ip address 10.255.1.3/32
!
interface Management1
   description OOB_MANAGEMENT
   no shutdown
   vrf MGMT
   ip address 172.16.2.101/24
!
interface Vlan4093
   description MLAG_L3
   no shutdown
   mtu 1500
   ip address 10.255.1.96/31
!
interface Vlan4094
   description MLAG
   no shutdown
   mtu 1500
   no autostate
   ip address 10.255.1.64/31
!
interface Vxlan1
   description htft-leaf1a_VTEP
   vxlan source-interface Loopback1
   vxlan virtual-router encapsulation mac-address mlag-system-id
   vxlan udp-port 4789
!
ip virtual-router mac-address 00:1c:73:00:00:99
!
ip routing
no ip routing vrf MGMT
!
ip prefix-list PL-LOOPBACKS-EVPN-OVERLAY
   seq 10 permit 10.255.0.0/27 eq 32
   seq 20 permit 10.255.1.0/27 eq 32
!
mlag configuration
   domain-id HTFT_LEAF1
   local-interface Vlan4094
   peer-address 10.255.1.65
   peer-link Port-Channel3
   reload-delay mlag 300
   reload-delay non-mlag 330
!
ntp local-interface vrf MGMT Management1
ntp server vrf MGMT 0.pool.ntp.org prefer
!
route-map RM-CONN-2-BGP permit 10
   match ip address prefix-list PL-LOOPBACKS-EVPN-OVERLAY
!
route-map RM-MLAG-PEER-IN permit 10
   description Make routes learned over MLAG Peer-link less preferred on spines to ensure optimal routing
   set origin incomplete
!
router bfd
   multihop interval 300 min-rx 300 multiplier 3
!
router bgp 65101
   router-id 10.255.0.3
   no bgp default ipv4-unicast
   maximum-paths 4
   neighbor EVPN-OVERLAY-PEERS peer group
   neighbor EVPN-OVERLAY-PEERS update-source Loopback0
   neighbor EVPN-OVERLAY-PEERS bfd
   neighbor EVPN-OVERLAY-PEERS ebgp-multihop 3
   neighbor EVPN-OVERLAY-PEERS password 7 Q4fqtbqcZ7oQuKfuWtNGRQ==
   neighbor EVPN-OVERLAY-PEERS send-community
   neighbor EVPN-OVERLAY-PEERS maximum-routes 0
   neighbor IPv4-UNDERLAY-PEERS peer group
   neighbor IPv4-UNDERLAY-PEERS password 7 7x4B4rnJhZB438m9+BrBfQ==
   neighbor IPv4-UNDERLAY-PEERS send-community
   neighbor IPv4-UNDERLAY-PEERS maximum-routes 256000
   neighbor MLAG-IPv4-UNDERLAY-PEER peer group
   neighbor MLAG-IPv4-UNDERLAY-PEER remote-as 65101
   neighbor MLAG-IPv4-UNDERLAY-PEER next-hop-self
   neighbor MLAG-IPv4-UNDERLAY-PEER description htft-leaf1b
   neighbor MLAG-IPv4-UNDERLAY-PEER route-map RM-MLAG-PEER-IN in
   neighbor MLAG-IPv4-UNDERLAY-PEER password 7 4b21pAdCvWeAqpcKDFMdWw==
   neighbor MLAG-IPv4-UNDERLAY-PEER send-community
   neighbor MLAG-IPv4-UNDERLAY-PEER maximum-routes 256000
   neighbor 10.255.0.1 peer group EVPN-OVERLAY-PEERS
   neighbor 10.255.0.1 remote-as 65100
   neighbor 10.255.0.1 description htft-spine1_Loopback0
   neighbor 10.255.0.2 peer group EVPN-OVERLAY-PEERS
   neighbor 10.255.0.2 remote-as 65100
   neighbor 10.255.0.2 description htft-spine2_Loopback0
   neighbor 10.255.1.97 peer group MLAG-IPv4-UNDERLAY-PEER
   neighbor 10.255.1.97 description htft-leaf1b_Vlan4093
   neighbor 10.255.255.0 peer group IPv4-UNDERLAY-PEERS
   neighbor 10.255.255.0 remote-as 65100
   neighbor 10.255.255.0 description htft-spine1_Ethernet1
   neighbor 10.255.255.2 peer group IPv4-UNDERLAY-PEERS
   neighbor 10.255.255.2 remote-as 65100
   neighbor 10.255.255.2 description htft-spine2_Ethernet1
   redistribute connected route-map RM-CONN-2-BGP
   !
   address-family evpn
      neighbor EVPN-OVERLAY-PEERS activate
   !
   address-family ipv4
      no neighbor EVPN-OVERLAY-PEERS activate
      neighbor IPv4-UNDERLAY-PEERS activate
      neighbor MLAG-IPv4-UNDERLAY-PEER activate
!
end

Default Interfaces

The default_interfaces feature automatically assigns interfaces based on node type and platform, eliminating repetitive configuration:

Node Type Uplink Interfaces MLAG Interfaces Downlink Interfaces
spine - - Ethernet1-8
l3leaf Ethernet1-2 Ethernet3-4 Ethernet8
l2leaf Ethernet1-2 - -

Interface Range Syntax

AVD supports flexible interface range notation:

  • Ethernet1-4 expands to Ethernet1, Ethernet2, Ethernet3, Ethernet4
  • Ethernet49-52/1 expands to Ethernet49/1, Ethernet50/1, Ethernet51/1, Ethernet52/1
  • [Ethernet1, Ethernet2] explicit list of interfaces

Uplinks connect lower-tier (L3 Leafs) devices to higher-tier devices (Spines):

  • L3 Leafs uplink to Spines using routed point-to-point links
  • L2 Leafs uplink to L3 Leafs using port-channels
Variable Description
uplink_switches List of switches this node connects to
uplink_switch_interfaces Interfaces on the uplink switches
uplink_interfaces Local interfaces for uplinks (or use default_interfaces)
uplink_ipv4_pool IP pool for point-to-point uplinks

AVD automatically allocates IPs from uplink_ipv4_pool using a deterministic algorithm based on node id:

Spine interface: uplink_ipv4_pool + (node_id * 2 * uplink_count) + (uplink_index * 2)
Leaf interface:  uplink_ipv4_pool + (node_id * 2 * uplink_count) + (uplink_index * 2) + 1

Best Practices

  1. Consistent naming conventions: Use predictable hostname patterns that work with default_node_types.
  2. Leverage node_groups: Group related nodes together to share configuration and enable automatic MLAG pairing.
  3. Plan your IP pools: Ensure IP pools are large enough for your fabric size with room for growth.
  4. Use loopback offsets: When sharing a loopback pool between node types, use loopback_ipv4_offset to prevent IP conflicts.
  5. Use default_interfaces: Define interface mappings once at the fabric level instead of on every node.

Troubleshooting

BGP Peering Not Establishing

Issue: Underlay or overlay BGP sessions not coming up.

Solution:

  • Verify uplink_switches and uplink_switch_interfaces are correctly defined
  • Check that the spine’s downlink_interfaces in default_interfaces includes the interfaces used by leafs
  • Ensure uplink_ipv4_pool has sufficient addresses

MLAG Not Forming

Issue: MLAG peer-link not establishing between leaf pairs.

Solution:

  • Verify exactly two nodes are in the same node_group
  • Check that mlag_interfaces are defined (directly or via default_interfaces)
  • Ensure mlag_peer_ipv4_pool is defined for the leaf defaults

IP Address Conflicts

Issue: Multiple devices assigned the same IP address.

Solution:

  • Ensure each node has a unique id within its node type
  • When sharing loopback pools between node types, use appropriate loopback_ipv4_offset
  • Verify IP pools are large enough for all nodes

Reference

For complete details on all available topology properties, see: