Skip to content

AVD example for CV Pathfinder

Introduction

Important

  • CVaaS is required to run this example. Without it, only configuration generation is possible.
  • Minimum EOS version 4.32.2F is required.
  • The devices must be able to reach CVaaS via their Management Interface.
  • Proper licenses are required for all nodes (IPSec and throughput).

This example aims to present the basic configuration blocks required to deploy CV Pathfinder using AVD but does not cover all CV Pathfinder features. In particular, it does not cover:

  • Internet Exits
  • WAN routers behind NAT
  • Multiple inventories

The WAN how-to document provides more information on how these features are supported in AVD.

This example will go over the following:

  • Build the intended configurations and documentation
  • Deploy the configuration via CloudVision as a Service (CVaaS)

Installation

Requirements to use this example:

  • Follow the installation guide for AVD
  • Run the following playbook to copy the AVD examples to your current working directory, for example ansible-avd-examples:
ansible-playbook arista.avd.install_examples

This will show the following:

 ~/ansible-avd-examples# ansible-playbook arista.avd.install_examples

PLAY [Install Examples]**********************************************************************************************

TASK [Copy all examples to ~/ansible-avd-examples]*******************************************************************
changed: [localhost]

PLAY RECAP
*********************************************************************************************************************
localhost                  : ok=1    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Info

If the content of any file is modified and the playbook is rerun, the file will not be overwritten. However, if any file in the example is deleted and the playbook is rerun, Ansible will re-create the file.

After the playbook has run successfully, the directory structure of the example should look like below, the contents of which will be covered in later sections:

ansible-avd-examples/ (or wherever the playbook was run)
  ├── cv-pathfinder
    ├── ansible.cfg
    ├── build.yml
    ├── deploy.yml
    ├── documentation
    ├── group_vars
    ├── host_vars
    ├── images
    ├── intended
    ├── inventory.yml
    ├── README.md
    └── switch-basic-configurations

Overall design overview

Physical topology

The target topology comprises two Pathfinder nodes and three sites distributed in two regions.

The drawing below shows the physical topology used in this example.

Figure: Arista CV Pathfinder topology

  • The example considers two path groups: MPLS and INTERNET
  • Pathfinders pf1 and pf2 are connected to both to the INTERNET and the MPLS path groups.
  • inet-cloud and mpls-cloud are used to mimic Service Providers.

The following table describes the characteristics of each site:

Site Name Region Role (Transit/Edge) Number of routers Path groups HA configuration LAN configuration
Site 1 Region 1 Transt 2 routers INTERNET and MPLS on both routers Via the LAN eBGP
Site 2 Region 2 Transit 2 routers MPLS on router 1 and INTERNET on router 2 Direct HA eBGP
Site 3 Region 2 Edge 1 router INTERNET - Subinterfaces facing L2Leaf

IP ranges used

Out-of-band management IP allocation

Subnet: 192.168.17.0/24

Description IP Address
Default gateway 192.168.17.1
Pathfinders
pf1 192.168.17.10
pf2 192.168.17.11
Site 1
site1-wan1 192.168.17.12
site1-wan2 192.168.17.13
site1-border1 192.168.17.14
site1-border2 192.168.17.15
Site 2
site2-wan1 192.168.17.16
site2-wan2 192.168.17.17
site2-leaf1 192.168.17.18
site2-leaf2 192.168.17.19
Site 3
site3-wan1 192.168.17.20
site3-leaf1 192.168.17.21
Clouds
mpls-cloud 192.168.17.30
inet-cloud 192.168.17.31

Other subnet IP allocations

Description Subnet
Loopback 0 interfaces 192.168.255.0/24
DPS/VTEP interfaces 192.168.42.0/24
MLAG peer-link (interface VLAN 4094) 10.255.252.0/24
MLAG iBGP peering (interface VLAN 4093) 10.255.251.0/24
Site1 uplink between WANs and border 10.0.1.0/24
Site2 uplink between WANs and leafs 10.0.2.0/24
pf1 to mpls-cloud 172.18.100.0/24
pf1 to inet-cloud 100.64.100.0/24
pf2 to mpls-cloud 172.18.200.0/24
pf2 to inet-cloud 100.64.200.0/24
site1-wan1 to mpls-cloud 172.18.10.0/24
site1-wan1 to inet-cloud 172.18.10.0/24
site1-wan2 to mpls-cloud 172.18.11.0/24
site1-wan2 to inet-cloud 172.18.11.0/24
site2-wan1 to mpls-cloud 172.18.20.0/24
site2-wan2 to inet-cloud 172.18.21.0/24
site3-wan1 to inet-cloud 172.18.30.0/24

For every connection to inet-cloud or mpls-cloud, the cloud router is allocated .1 and the site / pf router is allocated .2.

VRFs/SVIs used on border routers and leafs for testing

Site Router VRF IP address
Site 1 site1-border1 BLUE 10.66.10.1/24
Site 1 site1-border1 RED 10.42.10.1/24
Site 1 site1-border2 BLUE 10.66.11.1/24
Site 1 site1-border2 RED 10.42.11.1/24
Site 2 site1-leaf1 BLUE 10.66.20.1/24
Site 2 site1-leaf1 RED 10.42.20.1/24
Site 2 site1-leaf2 BLUE 10.66.21.1/24
Site 2 site1-leaf2 RED 10.42.21.1/24
Site 3 site1-wan3 BLUE 10.66.30.1/24
Site 3 site1-wan3 RED 10.42.30.1/24

Note

For site 3, the IP addresses are configured on the WAN router as site3-leaf1 is an l2leaf.

Ansible inventory, group vars, and naming scheme

The following drawing shows a graphic overview of the Ansible inventory, group variables, and naming scheme used in this example:

Figure: Ansible inventory and vars

The following pattern is used:

  • Group names use uppercase and underscore
  • All hostnames use lowercase and dashes

The drawing also shows the relationships between groups and their children. Be aware that all declarations on a higher level are inherited by children automatically.

Content of the inventory.yml file

This section describes the entire ansible-avd-examples/cv-pathfinder/inventory.yml file used to represent the above topology.

In this example, we consider that no DNS entry is available to reach the devices and define the IPs the Ansible host has to reach per device.

CVaaS configuration

  • The example is targeting cv-staging. Please adjust to the correct CVaaS region as described in the cv_deploy role documentation
  • Additionally follow the guide to create the cv_token
  • the cv_token should then be loaded as an ENV variable CV_TOKEN using export CV_TOKEN=<token> for the deploy.yml playbook to work toward CVaaS.
inventory.yml
---
all:
  hosts:
    cloudvision:
      ansible_host: www.cv-staging.corp.arista.io
      # cv_token is coming from an ENV variable
      ansible_password: "{{ lookup('ansible.builtin.env', 'CV_TOKEN') }}"
  children:
    WAN:
      children:
        PATHFINDERS:
          hosts:
            pf1:
              ansible_host: 192.168.17.10
            pf2:
              ansible_host: 192.168.17.11
        SITE1:
          hosts:
            site1-wan1:
              ansible_host: 192.168.17.12
            site1-wan2:
              ansible_host: 192.168.17.13
            site1-border1:
              ansible_host: 192.168.17.14
            site1-border2:
              ansible_host: 192.168.17.15
        SITE2:
          hosts:
            site2-wan1:
              ansible_host: 192.168.17.16
            site2-wan2:
              ansible_host: 192.168.17.17
            site2-leaf1:
              ansible_host: 192.168.17.18
            site2-leaf2:
              ansible_host: 192.168.17.19
        SITE3:
          hosts:
            site3-wan1:
              ansible_host: 192.168.17.20
            site3-leaf1:
              ansible_host: 192.168.17.21
        TRANSPORTS:
          hosts:
            mpls-cloud:
              ansible_host: 192.168.17.30
            inet-cloud:
              ansible_host: 192.168.17.31

This example demonstrates the use of Ansible Vault to keep variables secure. ansible.cfg is configured to use a given file (.vault) as the vault password when required to decrypt files or inline variables.

ansible.cfg
[defaults]
inventory=inventory.yml
jinja2_extensions = jinja2.ext.loopcontrols,jinja2.ext.do,jinja2.ext.i18n

# vault usage example
# In production, the .vault file *must* be excluded from your git repository
vault_password_file=.vault

Danger

The .vault file is included in the example in order to be able to run it. It must never be pushed to any public repository as it allows anyone with read access to decrypt all the secrets.

Basic EOS config

As discussed in the single DC example, basic connectivity between the Ansible host and the switches must be established before Ansible can be used to push configurations. Remember, you must configure the following on all switches:

  • A hostname configured purely for ease of understanding.
  • An IP enabled interface - in this example, the dedicated out-of-band management interface is used.
  • A username and password with the proper access privileges.
site1-border1-basic-configuration.txt
! Basic EOS config
!
! Hostname of the device
hostname site1-border1
!
! Configures username and password for the ansible user
username ansible privilege 15 role network-admin secret sha512 $6$7u4j1rkb3VELgcZE$EJt2Qff8kd/TapRoci0XaIZsL4tFzgq1YZBLD9c6f/knXzvcYY0NcMKndZeCv0T268knGKhOEwZAxqKjlMm920
!
! Defines the VRF for MGMT
vrf instance MGMT
!
! Defines the settings for the Management1 interface through which Ansible reaches the device
interface Management1
   description OOB_MANAGEMENT
   no shutdown
   vrf MGMT
   ! IP address - must be set uniquely per device
   ip address 192.168.17.14/24
!
! Static default route for VRF MGMT
ip route vrf MGMT 0.0.0.0/0 192.168.17.1
!
! Enables API access in VRF MGMT
management api http-commands
   protocol https
   no shutdown
   !
   vrf MGMT
      no shutdown
!
end
!
! Save configuration to flash
copy running-config startup-config

Note

The folder cv-pathfinder/switch-basic-configurations/ contains a file per device for the initial configurations.

Defining device types

To define device types, required by AVD, this example leverages the default_node_types key:

groups_vars/all.yml
# define default node types based on hostnames
default_node_types:
  - node_type: wan_rr
    match_hostnames:
      - pf.*
  - node_type: wan_router
    match_hostnames:
      - site.*-wan.*
  - node_type: l2leaf
    match_hostnames:
      - site3-leaf.*
  - node_type: l3leaf
    match_hostnames:
      - site.*-border.*
      - site.*-leaf.*
  # Transport routers
  - node_type: spine
    match_hostnames:
      - .*-cloud
  1. Using node type spine for transport routers.

Pathfinder nodes use the node_type wan_rr, and all WAN routers (edge and transit) use the node_type wan_router.

Global settings for WAN

The following table lists the eos_designs top-level keys used for WAN and how they should be set:

Key Must be the same for all the WAN routers Comment
wan_mode Two possible modes, autovpn and cv-pathfinder (default).
cv_pathfinder_regions Defines the Region/Zone/Site hierarchy, not required for AutoVPN.
wan_route_servers Indicate to which WAN route servers the WAN router should connect to. This key is also used to tell every WAN Route Reflectors with which other RRs it should peer with.
wan_ipsec_profiles Defines the shared key for the Control Plane and Data Plane IPSec profiles.
wan_stun_dtls_disable Disable dTLS for STUN, for instance, for the lab. (NOT recommended in production).
wan_carriers Defines the list of carriers in the network; each carrier is assigned to a path-group.
wan_path_groups Defines the list of path-groups in the network.
wan_virtual_topologies Defines the Policies and the VRF to policy mappings.
tenants The default tenant key from network_services or any other key for tenant that would hold some WAN VRF information.
application_classification Defines the specific traffic classification required for the WAN, if any.
ipv4_acls List of IPv4 access-lists to be assigned to WAN interfaces.
bgp_peer_groups.wan_overlay_peers.listen_range_prefixes Must be set for the pathfinders for the connectivity to work. Sets the ranges of IP addresses from which to expect BGP peerings for the WAN. Include the VTEP ranges for all WAN routers connecting to this pathfinder.

In this example, the settings are set under the group WAN. To help logically separate the variables in meaningful categories, the group variables for WAN are created in several YAML files under ansible-avd-examples/cv-pathfinder/group_vars/WAN/:

cv-pathfinder/group_vars/WAN
├── cv_pathfinder_settings.yml
├── l3_interface_profiles.yml
├── management.yml
└── tenants.yml

Management

The management.yml file contains the configuration for:

  • The management gateway
  • NTP
  • Terminattr to configure the connection to CVaaS.
  • Local users (ansible/ansible, arista/arista and cvpadmin/cvpadmin)
  • ipv4_acls: a list of ACLs used for Internet-facing WAN interfaces
  • DNS
  • AAA
  • Disabling LLDP on management interface - Only needed for lab environments.
group_vars/WAN/management.yml
---
# WAN Management settings
mgmt_gateway: 192.168.17.1

# NTP
ntp_settings:
  servers:
    - name: 0.pool.ntp.org

# CloudVision/TerminAttr
# **edit the cvp_instance_ips to match your CVaaS region**
cvp_instance_ips: [www.cv-staging.corp.arista.io]
terminattr_smashexcludes: "ale,flexCounter,hardware,kni,pulse,strata"
terminattr_ingestexclude: "/Sysdb/cell/1/agent,/Sysdb/cell/2/agent"

# For each local user in this example, password == username
local_users:
  - name: arista
    privilege: 15
    role: network-admin
    sha512_password: $6$Enl0WfE32FthwyiJ$yTyGaEJ2uPKLU.F7314YtB7J1jrzrMi7ogXIRTEHQfLdLgKWWmr1UvNlZLN6AyuxET7G5aH3AI9OYRzxVTkB1.
  - name: ansible
    privilege: 15
    role: network-admin
    sha512_password: $6$7u4j1rkb3VELgcZE$EJt2Qff8kd/TapRoci0XaIZsL4tFzgq1YZBLD9c6f/knXzvcYY0NcMKndZeCv0T268knGKhOEwZAxqKjlMm920
  - name: cvpadmin
    privilege: 15
    role: network-admin
    sha512_password: $6$a7LdQWHxWzYHpvVt$n62q.1mbm4kzQ5oBr0lhXCE9ntnTn.SNa16DovZHahFQLH.iPcPMZa5JUSFtncrDW4EDQ3oSWgP8G0S4FtOFx1

# Internet ACL
ipv4_acls:
  - name: ACL-PF-INTERNET-IN
    entries:
      - sequence: 1
        remark: "Not for PRODUCTION: This ACL is built this way because the lab has an out-of-band interface"
      - sequence: 10
        # IPSec traffic from anywhere
        action: permit
        protocol: udp
        source: any
        destination: interface_ip
        destination_ports_match: eq
        destination_ports: [isakmp, non500-isakmp]
      - sequence: 20
        # STUN traffic from anywhere
        action: permit
        protocol: udp
        source: any
        destination: interface_ip
        destination_ports_match: eq
        destination_ports: [3478]
      - sequence: 30
        # Troubleshooting
        action: permit
        protocol: icmp
        source: any
        destination: interface_ip
      - action: deny
        protocol: ip
        source: any
        destination: any
  - name: ACL-INTERNET-IN
    entries:
      - sequence: 1
        remark: "Not for PRODUCTION: This ACL is built this way because the lab has an out-of-band interface"
      - sequence: 10
        # IPSec traffic from anywhere
        action: permit
        protocol: udp
        source: any
        destination: interface_ip
        destination_ports_match: eq
        destination_ports: [isakmp, non500-isakmp]
      - sequence: 30
        # Troubleshooting
        action: permit
        protocol: icmp
        source: any
        destination: interface_ip
      - action: deny
        protocol: ip
        source: any
        destination: any

# DNS
name_servers:
  - 192.168.17.1

custom_structured_configuration_dns_domain: wan.example.local

# AAA
custom_structured_configuration_aaa_authorization:
  exec:
    default: local

# disabling LLDP on management interface
custom_structured_configuration_management_interfaces:
  - name: Management1
    lldp:
      transmit: false
      receive: false

CV Pathinfder settings

The ansible-avd-examples/cv-pathfinder/group_vars/WAN/cv_pathfinder_settings.yml file defines the global WAN settings for all the hosts of the WAN group in the inventory.

group_vars/WAN/cv_pathfinder_settings.yml
---
# WAN CV Pathfinder settings

# cv-pathfinder is default
wan_mode: cv-pathfinder

# WAN hierarchy
cv_pathfinder_regions:  # (1)!
  - name: REGION1
    id: 1
    sites:
      - name: SITE1
        id: 101
        location: Copenhagen, Denmark
  - name: REGION2
    id: 2
    sites:
      - name: SITE2
        id: 202
        location: Ottawa, Canada
      - name: SITE3
        id: 203
        location: Milan, Italy

cv_pathfinder_global_sites:  # (2)!
  - name: PF1-GLOBAL
    location: Santa Clara, CA, USA
  - name: PF2-GLOBAL
    location: Coulomiers, France

wan_route_servers: # (3)!
  - hostname: pf1
  - hostname: pf2

# IPSec configuration
wan_ipsec_profiles: # (4)!
  control_plane:
    shared_key: 045A190F1C354D
  data_plane:
    shared_key: 141600021F102B

# DTLS for STUN is enabled by default
# It requires certificates to be generated and distributed on the participating
# devices. To disable it for a lab, uncomment the following line.
# wan_stun_dtls_disable: true

# Fabric Flow tracking - by default only enabled for the Dps1 interface
fabric_flow_tracking: # (5)!
  uplinks:
    enabled: true
  downlinks:
    enabled: true
# Overwrite the default settings
flow_tracking_settings:
  trackers:
    - name: FLOW-TRACKER
      record_export:
        on_inactive_timeout: 70000
        # Small export interval for example
        on_interval: 5000
      exporters:
        - name: CV-TELEMETRY
          collector:
            host: 127.0.0.1
          local_interface: Loopback0

# Vaulting bgp_password variable as an example. It is encrypted with the
# password in .vault
bgp_password: !vault |
 $ANSIBLE_VAULT;1.1;AES256
 33383161313537343432366430633733356331616665323132303263316531373135383439383730
 3539343961636531643631343561356337656439366433320a366364653135376338313731323035
 30633238643661626239656365613535393066666436633535636436633633303962393363343738
 6262343237396166620a386465373366376235356438356139323832383131366462646263303037
 6430

# BGP settings
bgp_peer_groups: # (6)!
  wan_overlay_peers:
    password: "{{ bgp_password | arista.avd.encrypt(passwd_type='bgp', key='WAN-OVERLAY-PEERS') }}"
    listen_range_prefixes:
      - 192.168.42.0/24
  wan_rr_overlay_peers:
    password: "{{ bgp_password | arista.avd.encrypt(passwd_type='bgp', key='WAN-RR-OVERLAY-PEERS') }}"

# WAN path groups
wan_carriers: # (7)!
  - name: ACME-MPLS-INC
    path_group: MPLS
    trusted: true
  - name: GLOBAL-INTERNET-LIMITED
    path_group: INTERNET
  - name: REGION1-INTERNET-CORP
    path_group: INTERNET
  - name: REGION2-INTERNET-CORP
    path_group: INTERNET

wan_path_groups: # (8)!
  - name: MPLS
    id: 101
  - name: INTERNET
    id: 102

# WAN virtual topologies
wan_virtual_topologies: # (9)!
  vrfs:
    - name: BLUE
      policy: BLUE-POLICY
      wan_vni: 100
    - name: RED
      policy: RED-POLICY
      wan_vni: 101
  policies:
    - name: BLUE-POLICY
      application_virtual_topologies:
        - application_profile: VIDEO
          id: 2
          path_groups:
            - names: [INTERNET]
              preference: preferred
            - names: [MPLS] # (10)!
              preference: alternate
        - application_profile: VOICE
          id: 3
          dscp: 46
          lowest_hop_count: true
          constraints:
            jitter: 30
            latency: 150
            loss_rate: 1
          path_groups:
            - names: [MPLS]
              preference: preferred
            - names: [INTERNET]
              preference: alternate
      default_virtual_topology:
        path_groups:
          - names: [INTERNET, MPLS]
            preference: preferred
    - name: RED-POLICY
      application_virtual_topologies:
        - application_profile: CRITICAL-SECRET-DATA
          id: 2
          path_groups:
            - names: [MPLS]
              preference: preferred
        - application_profile: NORMAL-DATA
          id: 3
          path_groups:
            - names: [INTERNET]
              preference: preferred
            - names: [MPLS]
              preference: alternate
        - application_profile: NOT-SO-IMPORTANT-DATA
          id: 4
          path_groups:
            - names: [INTERNET]
              preference: preferred
      default_virtual_topology:
        # dropping unmatched traffic in VRF RED
        drop_unmatched: true

# Traffic classification
application_classification:
  field_sets:
    l4_ports:
      - name: VOICE-PORTS
        port_values:
          - 666-667
      - name: VIDEO-PORTS
        port_values:
          - 4242-4244
  applications:
    ipv4_applications:
      - name: VOICE-APP
        protocols:
          - tcp
        tcp_dest_port_set_name: VOICE-PORTS
      - name: VIDEO-APP
        protocols:
          - tcp
          - udp
        tcp_dest_port_set_name: VIDEO-PORTS
        udp_dest_port_set_name: VIDEO-PORTS
      - name: CRITICAL-SECRET-DATA-APP
        dscp_ranges: [46]
      - name: NORMAL-DATA-APP
        dscp_ranges: [af23]
      - name: NOT-SO-IMPORTANT-DATA-APP
        dscp_ranges: [0]
  application_profiles:
    - name: VOICE
      applications:
        - name: VOICE-APP
    - name: VIDEO
      applications:
        - name: VIDEO-APP
    - name: CRITICAL-SECRET-DATA
      applications:
        - name: CRITICAL-SECRET-DATA-APP
    - name: NORMAL-DATA
      applications:
        - name: NORMAL-DATA-APP
    - name: NOT-SO-IMPORTANT-DATA
      applications:
        - name: NOT-SO-IMPORTANT-DATA-APP
  1. cv_pathfinder_regions is used to declare the Regions, Sites, and their location in the WAN network
  2. cv_pathfinder_global_sites is used to add location for “Global” Pathfinders.
  3. wan_route_servers defines the Pathfinders each router should connect to. This variable can also be used to have per-region pathfinders, for instance, at a group level.
  4. wan_ipsec_profiles is used to control IP Security configuration; these settings are required.
  5. For lab purposes, Flow tracking is enabled on uplinks and downlinks. Make sure to verify the scalability of CloudVision depending on your network in Production.
  6. bgp_peer_groups uses listen range to establish sessions between WAN routers and the Pathfinders. wan_rr_overlay_peers controls the connection between Pathfinders, which is a full mesh.
  7. When a carrier is not trusted (like the Internet ones), AVD requires an ingress ACL on the WAN interfaces. Each carrier is tied to a path-group.
  8. More on virtual topologies after the snippet.
  9. Application classification is used to configure application_profiles that match traffic categories in the policies and apply them to a virtual topology.
  10. When policies are defined but no path exists (in this case MPLS for site 3) the path groups configuration will not be included in the final rendering.

Virtual topologies

The cornerstone of the CV Pathfinder solution is the Virtual Topologies. The Virtual Topologies are grouped into Policies. A policy is a list of match statements, each matching a specific application profile and applying a profile (the Virtual Topology) to this traffic. A policy may or may not have any default match. If no default match is configured, unmatched traffic is dropped.

A profile is used to apply a load balancing policy and potentially an internet-exit policy (not in this example). The load balancing policy defines which path-groups can be used for the traffic.

As explained above, the wan_virtual_topologies variable must be global. AVD decides how to configure the policies for each device based on the locally present path groups. For example, if some traffic being matched is configured to only be sent over the MPLS path group but there is no local WAN interface connected to MPLS, then the match statement, profile, and load-balance policy are not generated by AVD on the device.

For this example, we define two policies, one for each VRF BLUE and RED. The figure below is a graphical representation of a policy for video-based applications. Within our application virtual topologies, we specify a video application profile. This application profile can utilize two path groups. In this case, the internet provider is the preferred path, with MPLS acting as our alternative.

AVT and Policies

Applications

The figure below shows a graphical view of how we define applications. We have a high-level field set that allows us to define our application’s attributes. These can be either IPs or ports. In this case, we use the port range of 4242-4244.

We can leverage field set definitions within our application definitions. For this example, we have an application called VIDEO-APP with additional attributes like the protocol type in use and the destination ports. Please note that we are leveraging the VIDEO-PORTS definition from our field sets.

We can then define our application profiles. The application profiles can be a list of applications if they share the same policies. In this example, we have a fairly one-to-one mapping of applications and policies, but that is not a requirement. Within our VIDEO application profile, we leverage the VIDEO-APP application definition that was created previously.

Applications

L3 interfaces and L3 interface profiles

In AVD, an L3 interface configured under a node is considered a WAN interface when the wan_carrier setting is set. The wan_carrier allows AVD to know which path-group to use for the interface based on the Carrier to Path-Group mapping in the top level wan_carriers key.

AVD WAN interfaces example
wan_router:
  node_groups:
    - group: SITE1
      cv_pathfinder_region: REGION1
      cv_pathfinder_site: SITE1
      # Making this site a transit site
      cv_pathfinder_transit_mode: region
      wan_ha:
        enabled: true
      nodes:
        - name: site1-wan1
          id: 3
          l3_interfaces:
            - name: Ethernet3  # (1)!
              profile: MPLS-WAN-INTERFACE
              peer_interface: Ethernet5
              peer_ip: 172.18.10.1
              ip_address: 172.18.10.2/24
              wan_carrier: ACME-MPLS-INC
              wan_circuit_id: mpls-site1-wan1
            - name: Ethernet42 # (2)!
              ip_address: 100.64.10.2/24
  1. Ethernet3 is a WAN interface because wan_carrier is defined
  2. Ethernet42 is not a WAN interface because wan_carrier is not defined

L3 interface profiles are defined globally and reused on the site WAN routers to apply common configurations.

group_vars/WAN/l3_interface_profiles.yml
---
l3_interface_profiles:
  - profile: MPLS-WAN-INTERFACE
    peer: mpls-cloud
    # The static_routes uses the peer_ip as next-hop
    static_routes:
      - prefix: 172.18.0.0/16
    # Enabling flow tracking on WAN interfaces
    flow_tracking:
      enabled: true
  - profile: INTERNET-WAN-INTERFACE
    peer: inet-cloud
    ip_address: dhcp
    dhcp_accept_default_route: true
    ipv4_acl_in: ACL-INTERNET-IN
    # Enabling flow tracking on WAN interfaces
    flow_tracking:
      enabled: true

Tenants

The tenants are defined globally. In this example, we create SVIs in the BLUE and RED VRFs for testing purposes.

Notice that the vrf_vni is configured for the BLUE and RED VRFs. This potentially differs from the wan_vni configured under wan_virtual_topologies.vrfs. The former is used for EVPN, while the latter is used for DPS.

group_vars/WAN/tenants.yml
---
# WAN Tenants network services

tenants:
  - name: WAN-EXAMPLE-TENANT
    mac_vrf_vni_base: 10000
    vrfs:
      - name: BLUE
        vrf_vni: 100
        svis:
          - id: 666
            name: BLUE-TEST
            enabled: true
            nodes:
              - node: site1-border1
                ip_address: 10.66.1.1/24
              - node: site1-border2
                ip_address: 10.66.11.1/24
              - node: site2-leaf1
                ip_address: 10.66.2.1/24
              - node: site2-leaf2
                ip_address: 10.66.22.1/24
              - node: site3-wan1
                ip_address: 10.66.3.1/24
      - name: RED
        vrf_vni: 101
        svis:
          - id: 42
            name: RED-TEST
            enabled: true
            nodes:
              - node: site1-border1
                ip_address: 10.42.1.1/24
              - node: site1-border2
                ip_address: 10.42.11.1/24
              - node: site2-leaf1
                ip_address: 10.42.2.1/24
              - node: site2-leaf2
                ip_address: 10.42.22.1/24
              - node: site3-wan1
                ip_address: 10.42.3.1/24

Setting pathfinders specific configuration parameters

The pathfinder configuration can be found in group_vars/PATHFINDERS.yml:

group_vars/PATHFINDERS.yml
---
# WAN Pathfinder nodes variables
wan_rr:
  defaults:
    data_plane_cpu_allocation_max: 1  # (1)!
    loopback_ipv4_pool: 192.168.255.0/24
    vtep_loopback_ipv4_pool: 192.168.42.0/24
    bgp_as: 65000
  nodes:
    - name: pf1
      id: 1
      mgmt_ip: 192.168.17.10/24
      cv_pathfinder_site: PF1-GLOBAL  # (2)!
      # WAN interfaces
      l3_interfaces:
        - name: Ethernet1
          peer: mpls-cloud
          peer_interface: Ethernet1
          peer_ip: 172.18.100.1
          ip_address: 172.18.100.2/24
          wan_carrier: ACME-MPLS-INC
          wan_circuit_id: mpls-pf1
          # The static_routes uses the peer_ip as next-hop
          static_routes:
            - prefix: 172.18.0.0/16
        - name: Ethernet2
          peer: inet-cloud
          peer_interface: Ethernet1
          peer_ip: 100.64.100.1
          ip_address: 100.64.100.2/24
          wan_carrier: GLOBAL-INTERNET-LIMITED
          wan_circuit_id: inet-pf1
          ipv4_acl_in: ACL-PF-INTERNET-IN
          static_routes:
            - prefix: 0.0.0.0/0
    - name: pf2
      id: 2
      mgmt_ip: 192.168.17.11/24
      cv_pathfinder_site: PF2-GLOBAL
      # WAN interfaces
      l3_interfaces:
        - name: Ethernet1
          peer: mpls-cloud
          peer_interface: Ethernet2
          peer_ip: 172.18.200.1
          ip_address: 172.18.200.2/24
          wan_carrier: ACME-MPLS-INC
          wan_circuit_id: mpls-pf2
          # The static_routes uses the peer_ip as next-hop
          static_routes:
            - prefix: 172.18.0.0/16
        - name: Ethernet2
          peer: inet-cloud
          peer_interface: Ethernet2
          peer_ip: 100.64.200.1
          ip_address: 100.64.200.2/24
          wan_carrier: GLOBAL-INTERNET-LIMITED
          wan_circuit_id: inet-pf2
          ipv4_acl_in: ACL-PF-INTERNET-IN
          static_routes:
            - prefix: 0.0.0.0/0

Setting site specific configuration parameters

Note

For convenience in the example, both the WAN routers and the LAN devices (borders, leaf, etc.) are defined in the same group_vars file. It would be possible to separate them.

As general principles:

  • WAN routers should configure the LAN devices as uplink_switches.
  • Each WAN router is assigned to a site. When two routers are part of a node_group, HA will be configured.

Site 1

The following diagrams describe the connectivity of Site 1’s physical, LAN, and HA tunnels.

Figure: Site 1 Physical

Site 1 borders are using BGP AS 65101

Figure: Site 1 LAN

By default, AVD uses all uplink interfaces for the LAN_HA path groups. EOS then establishes an IPsec tunnel between all pairs of local and remote connections. In this scenario, four tunnels are created for the LAN_HA path group.

Figure: Site 1 HA Tunnels

group_vars/SITE1.yml
---
# SITE1 variables
# Use eBGP as underlay routing protocol on SITE1, the default is 'none' for WAN routers.
underlay_routing_protocol: ebgp

l3leaf:
  defaults:
    loopback_ipv4_pool: 192.168.255.0/24
    vtep_loopback_ipv4_pool: 192.168.42.0/24
    filter:
      always_include_vrfs_in_tenants: [all]
    mlag_interfaces: [Ethernet5, Ethernet6]
    mlag_peer_l3_ipv4_pool: 10.255.251.0/24
    mlag_peer_ipv4_pool: 10.255.252.0/24
  node_groups:
    - group: SITE1
      bgp_as: 65101
      nodes:
        - name: site1-border1
          mgmt_ip: 192.168.17.14/24
          id: 5
        - name: site1-border2
          mgmt_ip: 192.168.17.15/24
          id: 6

wan_router:
  defaults:
    loopback_ipv4_pool: 192.168.255.0/24
    vtep_loopback_ipv4_pool: 192.168.42.0/24
    uplink_ipv4_pool: 10.0.1.0/24
    filter:
      always_include_vrfs_in_tenants: [all]
    uplink_interfaces: [Ethernet1, Ethernet2]
    uplink_switches: [site1-border1, site1-border2]
    uplink_type: p2p-vrfs # (1)!
    bgp_as: 65000
  node_groups:
    - group: SITE1
      cv_pathfinder_region: REGION1
      cv_pathfinder_site: SITE1
      # Making this site a transit site
      cv_pathfinder_transit_mode: region # (2)!
      wan_ha:
        enabled: true
        # Use this knob to disable IPSec on WAN HA tunnel(s)
        # ipsec: false
      nodes:
        - name: site1-wan1
          id: 3
          mgmt_ip: 192.168.17.12/24
          uplink_switch_interfaces: [Ethernet3, Ethernet3]
          l3_interfaces:
            - name: Ethernet3
              profile: MPLS-WAN-INTERFACE
              peer_interface: Ethernet5
              peer_ip: 172.18.10.1
              ip_address: 172.18.10.2/24
              wan_carrier: ACME-MPLS-INC
              wan_circuit_id: mpls-site1-wan1
            - name: Ethernet4
              peer_interface: Ethernet5
              peer_ip: 100.64.10.1
              ip_address: 100.64.10.2/24
              profile: INTERNET-WAN-INTERFACE
              wan_carrier: REGION1-INTERNET-CORP
              wan_circuit_id: inet-site1-wan1
        - name: site1-wan2
          id: 4
          mgmt_ip: 192.168.17.13/24
          uplink_switch_interfaces: [Ethernet4, Ethernet4]
          l3_interfaces:
            - name: Ethernet3
              profile: MPLS-WAN-INTERFACE
              peer_interface: Ethernet6
              peer_ip: 172.18.11.1
              ip_address: 172.18.11.2/24
              wan_carrier: ACME-MPLS-INC
              wan_circuit_id: mpls-site1-wan2
            - name: Ethernet4
              peer_interface: Ethernet6
              dhcp_ip: 100.64.11.2
              profile: INTERNET-WAN-INTERFACE
              wan_carrier: REGION1-INTERNET-CORP
              wan_circuit_id: inet-site1-wan2
  1. The uplink type p2p-vrfs is used to connect to the LAN switches.
  2. cv_pathfinder_transit_mode defaults to none and the site is considered an edge site. To use it as transit, the variable must be set to region as it is the case here.

Site 2

The following diagrams describe the Site2 physical, LAN and HA tunnels connectivity.

Figure: Site 2 Physical

Site 2 borders are using BGP AS 65101

Figure: Site 2 LAN

Figure: Site 2 HA Tunnels

group_vars/SITE2.yml
---
# SITE2 variables
# Use eBGP as underlay routing protocol on SITE2, the default is 'none' for WAN routers.
underlay_routing_protocol: ebgp

ipv4_prefix_list_catalog:
  - name: ALLOW-DEFAULT
    sequence_numbers:
      - sequence: 10
        action: permit 0.0.0.0/0

# Internet ACL
ipv4_acls:
  - name: ACL-INTERNET-IN
    entries:
      - sequence: 1
        remark: "Not for PRODUCTION: This ACL is built this way because the lab has an out-of-band interface"
      - sequence: 10
        # IPSec traffic from anywhere
        action: permit
        protocol: udp
        source: any
        destination: interface_ip
        destination_ports_match: eq
        destination_ports: [isakmp, non500-isakmp]
      - sequence: 20
        # Permit BGP
        action: permit
        protocol: tcp
        source: any
        destination: interface_ip
        destination_ports_match: eq
        destination_ports: [bgp]
      - sequence: 30
        # Troubleshooting
        action: permit
        protocol: icmp
        source: any
        destination: interface_ip
      - action: deny
        protocol: ip
        source: any
        destination: any

l3leaf:
  defaults:
    loopback_ipv4_pool: 192.168.255.0/24
    vtep_loopback_ipv4_pool: 192.168.42.0/24
    filter:
      always_include_vrfs_in_tenants: [all]
    mlag_interfaces: [Ethernet5, Ethernet6]
    mlag_peer_l3_ipv4_pool: 10.255.251.0/24
    mlag_peer_ipv4_pool: 10.255.252.0/24
  node_groups:
    - group: SITE2
      bgp_as: 65102
      nodes:
        - name: site2-leaf1
          id: 9
          mgmt_ip: 192.168.17.18/24
        - name: site2-leaf2
          id: 10
          mgmt_ip: 192.168.17.19/24

wan_router:
  defaults:
    loopback_ipv4_pool: 192.168.255.0/24
    vtep_loopback_ipv4_pool: 192.168.42.0/24
    filter:
      always_include_vrfs_in_tenants: [all]
    uplink_type: p2p-vrfs
    uplink_interfaces: [Ethernet1]
    uplink_switch_interfaces: [Ethernet3]
    uplink_ipv4_pool: 10.0.2.0/24
    bgp_as: 65000
  node_groups:
    - group: SITE2
      cv_pathfinder_region: REGION2
      cv_pathfinder_site: SITE2
      # Making this site a transit site
      cv_pathfinder_transit_mode: region
      wan_ha:
        enabled: true
        # Direct HA
        ha_interfaces: [Ethernet5]
        ha_ipv4_pool: 10.42.0.0/24
        use_port_channel_for_direct_ha: false
        # Use this knob to disable IPSec on WAN HA tunnel(s)
        # ipsec: false
      nodes:
        - name: site2-wan1
          id: 7
          mgmt_ip: 192.168.17.16/24
          uplink_switches: [site2-leaf1]
          l3_interfaces:
            - name: Ethernet3
              profile: MPLS-WAN-INTERFACE
              peer_interface: Ethernet7
              peer_ip: 172.18.20.1
              ip_address: 172.18.20.2/24
              wan_carrier: ACME-MPLS-INC
              wan_circuit_id: mpls-site2-wan1
        - name: site2-wan2
          id: 8
          mgmt_ip: 192.168.17.17/24
          uplink_switches: [site2-leaf2]
          l3_interfaces:
            - name: Ethernet4
              peer_interface: Ethernet7
              peer_ip: 100.64.21.1
              ip_address: 100.64.21.2/24
              profile: INTERNET-WAN-INTERFACE
              wan_carrier: REGION2-INTERNET-CORP
              wan_circuit_id: inet-site2-wan2
              # Configure a BGP peering with the Service Provider.
              bgp:
                peer_as: 65666
                ipv4_prefix_list_in: ALLOW-DEFAULT

Site 3

The following diagrams describe the Site3 physical and LAN connectivity.

Figure: Site 3 Physical

Figure: Site 3 LAN

group_vars/SITE3.yml
---
# SITE3 variables

l2leaf:
  nodes:
    - name: site3-leaf1
      mgmt_ip: 192.168.17.21/24
      id: 12

wan_router:
  defaults:
    loopback_ipv4_pool: 192.168.255.0/24
    vtep_loopback_ipv4_pool: 192.168.42.0/24
    uplink_ipv4_pool: 10.0.3.0/24
    uplink_interfaces: [Ethernet1]
    uplink_switch_interfaces: [Ethernet1]
    uplink_type: lan
    bgp_as: 65000
  nodes:
    - name: site3-wan1
      cv_pathfinder_region: REGION2
      cv_pathfinder_site: SITE3
      id: 11
      mgmt_ip: 192.168.17.20/24
      uplink_switches: [site3-leaf1]
      l3_interfaces:
        - name: Ethernet4
          peer_interface: Ethernet8
          dhcp_ip: 100.64.30.2
          profile: INTERNET-WAN-INTERFACE
          wan_carrier: REGION2-INTERNET-CORP
          wan_circuit_id: inet-site3-wan1

The playbooks

The playbook is also the same, as the actions to execute in the fabric are the same. It is important to validate the hosts variable in the playbook to include all Ansible groups.

Testing AVD output without a lab

Example of using this playbook without devices (local tasks):

---
# build.yml
- name: Build Configs
  hosts: WAN
  gather_facts: false
  tasks:

    - name: Generate AVD Structured Configurations and Fabric Documentation
      ansible.builtin.import_role:
        name: arista.avd.eos_designs

    - name: Generate Device Configurations and Documentation
      ansible.builtin.import_role:
        name: arista.avd.eos_cli_config_gen

Please look through the folders and files described above to learn more about the output generated by AVD.

The metadata section in the structured_config

The CV Pathfinder visualization in CloudVision relies on the inputs of a hidden metadata studio to work. When using AVD to generate the CV Pathfinder configurations, AVD also generates a metadata section with all the information required to be pushed to CVaaS to populate the studio.

The arista.avd.cv_deploy role will then pick up the metadata section and populate the metadata studio as part of the same workspace, thus keeping the configurations and metadata in sync.

Playbook Run

To build and deploy the configurations to CVaaS, run first the build.yml playbook and then the deploy.yml playbook.

Note

In this example, the deploy.yml playbook is configured to auto approve and execute the Change Control created by the Workspace using cv_run_change_control: true.

---
# deploy.yml
- name: Deploy Configurations via CVaaS
  hosts: WAN
  gather_facts: false
  tasks:
    - name: Deploy Configurations to Devices
      vars:
        # Automatically execute the Change Control
        cv_run_change_control: true
      ansible.builtin.import_role:
        # By default this will use ansible_host and ansible_password
        # from the 'cloudvision' host in the inventory
        name: arista.avd.cv_deploy
### Build the configurations
ansible-playbook playbooks/build.yml

### Deploy Configurations to CVaaS
ansible-playbook playbooks/deploy.yml
user@ubuntu:~/cv-pathfinder$ ansible-playbook build.yml

PLAY [Build Configs] ***************************************************************************

TASK [arista.avd.eos_designs : Verify Requirements] ********************************************
AVD version 5.0.0
ok: [pf1 -> localhost]

TASK [arista.avd.eos_designs : Create required output directories if not present] **************
ok: [pf1 -> localhost] => (item=//home/user/Documents/git_projects/ansible-avd-examples/cv-pathfinder/intended/structured_configs)
ok: [pf1 -> localhost] => (item=/home/user/Documents/git_projects/ansible-avd-examples/cv-pathfinder/documentation/fabric)
(...)

Troubleshooting

For any question reach out to AVD maintainer over Github discussions. Please share the relevant logs as well as the following information:

  • AVD version
  • Python version
  • EOS version in your lab (if relevant)
  • Whether you are using CVaaS or CV on-premises (and, in this case, the version)