Skip to content

AVD example for a MPLS-VPN based WAN Network

Introduction

This example is the logical second step in introducing AVD to new users, following the Introduction to Ansible and AVD section. New users with access to virtual routers (using Arista vEOS-lab or cEOS) can learn how to generate configuration and documentation for a complete fabric environment. Users with access to physical routers will have to adapt a few settings. This is all documented inline in the comments included in the YAML files. If a lab with virtual or physical routers is not accessible, this example can also be used to only generate the output from AVD if desired.

The example includes and describes all the AVD files and their content used to build a MPLS-VPN WAN network covering two sites using the following:

  • Four (virtual) p routers.
  • Three (virtual) pe routers serving aggregation devices and CPEs.
  • Two (virtual) route reflectors act as route servers for the WAN.

This example does not include Integration with CloudVision to keep everything as simple as possible. In this case, the Ansible host will communicate directly with the routers using eAPI.

Installation

Requirements to use this example:

  • Follow the installation guide for AVD found here.
  • Run the following playbook to copy the examples to your current working directory, for example ansible-avd-examples:

ansible-playbook arista.avd.install_examples

This will show the following:

 ~/ansible-avd-examples# ansible-playbook arista.avd.install_examples

PLAY [Install Examples]***************************************************************************************************************************************************************************************************************************************************************

TASK [Copy all examples to ~/ansible-avd-examples]*****************************************************************************************************************************************************
changed: [localhost]

PLAY RECAP
****************************************************************************************************************************************************************************************************************************************************************************
localhost                  : ok=1    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

After the playbook has run successfully, the directory structure will look as shown below, the contents of which will be covered in later sections:

ansible-avd-examples/ (or wherever the playbook was run)
  |── isis-ldp-ipvpn
    ├── ansible.cfg
    ├── build.yml
    ├── deploy.yml
    ├── documentation
    ├── group_vars
    ├── images
    ├── intended
    ├── inventory.yml
    ├── README.md
    └── switch-basic-configurations

Info

If the content of any file is modified and the playbook is rerun, the file will not be overwritten. However, if any file in the example is deleted and the playbook is rerun, Ansible will re-create the file.

Overall design overview

Physical topology

The drawing below shows the physical topology used in this example. The interface assignment shown here is referenced across the entire example, so keep that in mind if this example must be adapted to a different topology. Finally, the Ansible host is connected to the dedicated out-of-band management port (Management1 when using vEOS-lab):

Figure: Arista MPLS-VPN physical topology

IP ranges used

Out-of-band management IP allocation for WAN1 172.16.1.0/24
Default gateway 172.16.1.1
p1 172.16.1.11
p2 172.16.1.12
p3 172.16.1.13
p4 172.16.1.14
pe1 172.16.1.101
pe2 172.16.1.102
pe3 172.16.1.103
rr1 172.16.1.151
rr2 172.16.1.152
Point-to-point links between network nodes (Underlay)
WAN1 10.255.3.0/24
Loopback0 interfaces for router ID (p) 10.255.0.0/27
Loopback0 interfaces for overlay peering (pe) 10.255.1.0/27
Loopback0 interfaces for overlay peering (rr) 10.255.2.0/27
L3 Interfaces 10.0-1.1.0/24
For example pe1 Ethernet3.10 has the IP address: 10.0.1.1
For example pe3 Ethernet4 has the IP address: 10.1.1.9

ISIS-LDP design

BGP design

Basic EOS config

Basic connectivity between the Ansible host and the routers must be established before Ansible can be used to push configurations. You must configure the following on all routers:

  • A hostname configured purely for ease of understanding.
  • An IP enabled interface - in this example, the dedicated out-of-band management interface is used.
  • A username and password with the proper access privileges.

Below is the basic configuration file for p1:

p1-basic-configuration.txt
! ansible-avd-examples/single-dc-l3ls/switch-basic-configurations/p1-basic-configuration.txt
! Basic EOS config
!
! Hostname of the device
hostname p1
!
! Configures username and password for the ansible user
username ansible privilege 15 role network-admin secret sha512 $6$7u4j1rkb3VELgcZE$EJt2Qff8kd/TapRoci0XaIZsL4tFzgq1YZBLD9c6f/knXzvcYY0NcMKndZeCv0T268knGKhOEwZAxqKjlMm920
!
! Defines the VRF for MGMT
vrf instance MGMT
!
! Defines the settings for the Management1 interface through which Ansible reaches the device
interface Management1
   description oob_management
   no shutdown
   vrf MGMT
   ! IP address - must be set uniquely per device
   ip address 172.16.1.11/24
!
! Static default route for VRF MGMT
ip route vrf MGMT 0.0.0.0/0 172.16.1.1
!
! Enables API access in VRF MGMT
management api http-commands
   protocol https
   no shutdown
   !
   vrf MGMT
      no shutdown
!
end
!
! Save configuration to flash
copy running-config startup-config

Note

The folder isis-ldp-ipvpn/switch-basic-configurations/ contains a file per device for the initial configurations.

Ansible inventory, group vars, and naming scheme

The following drawing shows a graphic overview of the Ansible inventory, group variables, and naming scheme used in this example:

Figure: Ansible inventory and vars

Note

The CPE’s and aggregation nodes are not configured by AVD, but the ports used to connect to them are.

Group names use uppercase and underscore syntax:

  • FABRIC
  • WAN1
  • WAN1_P_ROUTERS
  • WAN1_PE_ROUTERS
  • WAN1_RR_ROUTERS

All hostnames use lowercase, for example:

  • p4
  • pe1
  • rr2

The drawing also shows the relationships between groups and their children:

  • For example, p1, p2, p3, and p4 are all children of the group called WAN1_P_ROUTERS.

Additionally, groups themselves can be children of another group, for example:

  • WAN1_P_ROUTERS is a child of the group WAN1.
  • WAN1_PE_ROUTERS is a child of both WAN1 and NETWORK_SERVICES.

This naming convention makes it possible to extend anything easily, but as always, this can be changed based on your preferences. Just ensure that the names of all groups and hosts are unique.

Content of the inventory.yml file

This section describes the entire ansible-avd-examples/isis-ldp-ipvpn/inventory.yml file used to represent the above topology.

The hostnames specified in the inventory must exist either in DNS or in the hosts file on your Ansible host to allow successful name lookup and be able to reach the routers directly. A successful ping from the Ansible host to each inventory host verifies name resolution(e.g., ping p1).

Alternatively, if there is no DNS available, or if devices need to be reached using a fully qualified domain name (FQDN), define ansible_host to be an IP address or FQDN for each device - see below for an example:

inventory.yml
---
all:
  children:
    FABRIC:
      children:
        WAN1:
          children:
            WAN1_P_ROUTERS:
              hosts:
                p1:
                  ansible_host: 172.16.1.11
                p2:
                  ansible_host: 172.16.1.12
                p3:
                  ansible_host: 172.16.1.13
                p4:
                  ansible_host: 172.16.1.14
            WAN1_PE_ROUTERS:
              hosts:
                pe1:
                  ansible_host: 172.16.1.101
                pe2:
                  ansible_host: 172.16.1.102
                pe3:
                  ansible_host: 172.16.1.103
            WAN1_RR_ROUTERS:
              hosts:
                rr1:
                  ansible_host: 172.16.1.151
                rr2:
                  ansible_host: 172.16.1.152

        NETWORK_SERVICES:
          children:
            WAN1_PE_ROUTERS:

The above is included in this example, purely to make it as simple as possible. However, in the future, please do not carry over this practice to a production environment, where an inventory file for an identical topology should look as follows when using DNS:

inventory.yml
---
all:
  children:
    FABRIC:
      children:
        WAN1:
          children:
            WAN1_P_ROUTERS:
              hosts:
                p1:
                p2:
                p3:
                p4:
            WAN1_PE_ROUTERS:
              hosts:
                pe1:
                pe2:
                pe3:
            WAN1_RR_ROUTERS:
              hosts:
                rr1:
                rr2:

        NETWORK_SERVICES:
          children:
            WAN1_PE_ROUTERS:
  1. NETWORK_SERVICES

    - Creates a group named NETWORK_SERVICES. Ansible variable resolution resolves this group name to the identically named group_vars file (ansible-avd-examples/isis-ldp-ipvpn/group_vars/NETWORK_SERVICES.yml).

    - The file’s contents are specifications of tenant VRFs and their associated routed interfaces, BGP peers, and OSPF interfaces, then applied to the group’s children. In this case, the group WAN1_PE_ROUTERS.

Defining device types

Since this example covers building a MPLS WAN network, AVD must know about the device types, for example, p, pe, rr routers, etc. The devices are already grouped in the inventory, so the device types are specified in the group variable files with the following names and content:

---
type: p
---
type: pe
---
type: rr

For example, all routers that are children of the WAN1_P_ROUTERS group defined in the inventory will be of type p.

Setting fabric-wide configuration parameters

The ansible-avd-examples/isis-ldp-ipvpn/group_vars/FABRIC.yml file defines generic settings that apply to all children of the FABRIC group as specified in the inventory described earlier.

The first section defines how the Ansible host connects to the devices:

FABRIC.yml
ansible_connection: ansible.netcommon.httpapi 
ansible_network_os: arista.eos.eos 
ansible_user: ansible 
ansible_password: ansible
ansible_become: true
ansible_become_method: enable 
ansible_httpapi_use_ssl: true 
ansible_httpapi_validate_certs: false 

The following section specifies variables that generate configuration to be applied to all devices in the fabric:

FABRIC.yml
fabric_name: FABRIC 

underlay_routing_protocol: isis-ldp
overlay_routing_protocol: ibgp

local_users: 
  - name: ansible
    privilege: 15
    role: network-admin
    sha512_password: $6$QJUtFkyu9yoecsq.$ysGzlb2YXaIMvezqGEna7RE8CMALJHnv7Q1i.27VygyKUtSeX.n2xRTyOtCR8eOAl.4imBLyhXFc4o97P5n071
  - name: admin
    privilege: 15
    role: network-admin
    no_password: true

bgp_peer_groups: 
  mpls_overlay_peers:
    password: Q4fqtbqcZ7oQuKfuWtNGRQ==

p2p_uplinks_mtu: 1500 

Setting device-specific configuration parameters

The ansible-avd-examples/isis-ldp-ipvpn/group_vars/WAN1.yml file defines settings that apply to all children of the WAN1 group as specified in the inventory described earlier. However, this time the settings defined are no longer fabric-wide but are limited to WAN1. This example is of limited benefit with only a single data center. Still, it allows us to scale the configuration to a scenario with multiple data centers in the future.

WAN1.yml
---
mgmt_gateway: 172.16.1.1 

p:
  defaults:
    platform: vEOS-lab 
    loopback_ipv4_pool: 10.255.0.0/27 

  nodes: 
    - name: p1
      id: 1 
      mgmt_ip: 172.16.1.11/24 

    - name: p2
      id: 2
      mgmt_ip: 172.16.1.12/24

The following section covers the pe routers. Significantly more settings need to be set compared to the p routers:

WAN1.yml
# PE router group
pe:
  defaults:
    platform: vEOS-lab 
    loopback_ipv4_pool: 10.255.1.0/27 
    virtual_router_mac_address: 00:1c:73:00:dc:00 
    mpls_route_reflectors: [ rr1, rr2 ] 
    isis_system_id_prefix: '0000.0001' 
    spanning_tree_mode: none 

  node_groups: 
    - group: WAN1-PE1-2
      nodes:
        - name: pe1
          id: 1
          mgmt_ip: 172.16.1.101/24
        - name: pe2
          id: 2
          mgmt_ip: 172.16.1.102/24

    - group: WAN1-PE3
      nodes:
        - name: pe3
          id: 3
          mgmt_ip: 172.16.1.103/24

Finally, more of the same, but this time for the rr routers:

WAN1.yml
rr:
  defaults:
    platform: vEOS-lab
    loopback_ipv4_pool: 10.255.2.0/27
    mpls_route_reflectors: [ rr1, rr2 ] 
    isis_system_id_prefix: '0000.0002'
    spanning_tree_mode: none

  node_groups:
    - group: WAN1_RR1-2
      nodes:
        - name: rr1
          id: 1
          mgmt_ip: 172.16.1.151/24
        - name: rr2
          id: 2
          mgmt_ip: 172.16.1.152/24

Defining underlay connectivity between network nodes

A free-standing list of core_interfaces dictionaries and their associated profiles and ip pools defines the underlay connectivity between nodes.

WAN1.yml
core_interfaces:
  p2p_links_ip_pools:
    - name: core_pool 
      ipv4_pool: 10.255.3.0/24

  p2p_links_profiles:
    - name: core_profile 
      mtu: 1500
      isis_metric: 50
      ip_pool: core_pool
      isis_circuit_type: level-2
      isis_authentication_mode: md5
      isis_authentication_key: $1c$sTNAlR6rKSw=

  p2p_links: 
    - nodes: [ pe1, p1 ] 
      interfaces: [ Ethernet1, Ethernet1 ]
      profile: core_profile 
      id: 1 

    - nodes: [ pe1, p2 ]
      interfaces: [ Ethernet2, Ethernet2 ]
      profile: core_profile
      id: 2

Specifying network services (VRFs and routed interfaces) and endpoint connectivity in the VPN-IPv4 fabric

NETWORK_SERVICES.yml
---
tenants:
  # Definition of tenants. Additional level of abstraction to VRFs
  - name: CUSTOMER1
    vrfs:
      # VRF definitions inside the tenant.
      - name: C1_VRF1
        # VRF ID definition.
        vrf_id: 10
        # Select address families for the VRF.
        address_families:
          - vpn-ipv4
        # Enable OSPF on selected PEs in the VRF.
        ospf:
          enabled: true
          nodes:
            - pe1
            - pe2
            - pe3
        l3_interfaces:
          # L3 interfaces
          - interfaces: [ Ethernet3.10, Ethernet4.10, Ethernet2 ]
            nodes: [ pe1, pe2, pe3 ]
            description: C1_L3_SERVICE
            enabled: true
            ip_addresses: [ 10.0.1.1/29, 10.0.1.2/29, 10.0.1.9/30 ]
            # Enable OSPF on the interfaces.
            ospf:
              enabled: true

  - name: CUSTOMER2
    vrfs:
      - name: C2_VRF1
        vrf_id: 20
        address_families:
          - vpn-ipv4
        l3_interfaces:
          # L3 interfaces
          - interfaces: [ Ethernet3.20, Ethernet4.20, Ethernet4 ]
            nodes: [ pe1, pe2, pe3 ]
            description: C2_L3_SERVICE
            enabled: true
            ip_addresses: [ 10.1.1.1/29, 10.1.1.2/29, 10.1.1.9/30 ]
        # Define BGP peers inside the VRF.
        bgp_peers:
          - ip_address: 10.1.1.3
            remote_as: 65123
            description: C2_ROUTER1
            send_community: standard
            maximum_routes: 100
            nodes: [ pe1, pe2 ]
          - ip_address: 10.1.1.10
            remote_as: 65124
            description: C2_ROUTER2
            send_community: standard
            maximum_routes: 100
            nodes: [ pe3 ]

All tenant VRFs and routed interfaces for endpoint connectivity in the network are defined here.

Two tenants called CUSTOMER1 and CUSTOMER2 are specified. Each of these tenants has a single VRF defined, and under those VRFs, we define the routed interfaces, tenant (PE-CE) routing protocols and address families in use:

NETWORK_SERVICES.yml
      - name: C1_VRF1
        vrf_id: 10
        address_families:
          - vpn-ipv4
        ospf:
          enabled: true
          nodes:
            - pe1
            - pe2
            - pe3
        l3_interfaces:
          - interfaces: [ Ethernet3.10, Ethernet4.10, Ethernet2 ]
            nodes: [ pe1, pe2, pe3 ]
            description: C1_L3_SERVICE
            enabled: true
            ip_addresses: [ 10.0.1.1/29, 10.0.1.2/29, 10.0.1.9/30 ]
            ospf:
              enabled: true

This defines C1_VRF1, with a VRF ID of 10, enables OSPF routing for PE-CE connections inside the VRF on selected pe routers and defines routed interfaces that are used to connect to the CE devices/aggregation nodes. Each interface has an IP address assigned, a description, and has OSPF routing enabled.

The lists of interfaces, nodes, and ip_addresses used in the above definition of the l3 interface are read by the ansible logic as follows: interface Ethernet3.10 belongs to the node pe1 and has the IP address of 10.0.1.1/29. In other words, the list indices are used to form the basic parameters for one interface.

The playbook

In this example, the deploy playbook looks like the following:

deploy.yml
---
# Please note, comments below are intended for site documentation only
- name: Run AVD 
  hosts: FABRIC
  gather_facts: false
  tasks:
    - name: Generate AVD Structured Configurations and Fabric Documentation 
      ansible.builtin.import_role:
        name: arista.avd.eos_designs

    - name: Generate Device Configurations and Documentation 
      ansible.builtin.import_role:
        name: arista.avd.eos_cli_config_gen

    - name: Deploy Configurations to Devices 
      ansible.builtin.import_role:
        name: arista.avd.eos_config_deploy_eapi

Testing AVD output without a lab

Example of using the build playbook without devices (local tasks):

build.yml
---
# Please note, comments below are intended for site documentation only
- name: Run AVD
  hosts: FABRIC
  gather_facts: false
  tasks:
    - name: Generate AVD Structured Configurations and Fabric Documentation
      ansible.builtin.import_role:
        name: arista.avd.eos_designs

    - name: Generate Device Configurations and Documentation
      ansible.builtin.import_role:
        name: arista.avd.eos_cli_config_gen

The build playbook will generate all of the output (variables, configurations, documentation) but will not attempt to communicate with any devices.

Please look through the folders and files described above to learn more about the output generated by AVD.

Executing the playbook

The execution of the playbook should produce the following output:

user@ubuntu:~/isis-ldp-ipvpn$ ansible-playbook deploy.yml

PLAY [Run AVD] *****************************************************************************************************************************************************************************

TASK [arista.avd.eos_designs : Collection arista.avd version 3.5.0 loaded from /home/user/.ansible/collections/ansible_collections] ******************************************************
ok: [p1]

TASK [arista.avd.eos_designs : Create required output directories if not present] **********************************************************************************************************
ok: [p1 -> localhost] => (item=/home/user/Documents/git_projects/ansible-avd-examples/isis-ldp-ipvpn/intended/structured_configs)
ok: [p1 -> localhost] => (item=/home/user/Documents/git_projects/ansible-avd-examples/isis-ldp-ipvpn/documentation/fabric)
(...)

If similar output is not shown, make sure:

  1. The documented requirements are met.
  2. The latest arista.avd collection is installed.

Troubleshooting

VPN-IPv4 Overlay not working

If after doing the following steps:

  1. Manually copy/paste the switch-basic-configuration to the devices.
  2. Run the playbook and push the generated configuration to the fabric.
  3. Login to a pe or rr device, for example, pe1 and run the command show bgp vpn-ipv4 summary to view VPN routes.

The following error message is shown:

pe1#show bgp vpn-ipv4 summary
% Not supported
pe1#

This is caused by AVD pushing the configuration line service routing protocols model multi-agent, which enables the multi-agent routing process supporting VPN-IPv4 and EVPN. This change requires a reboot of the device.

VPN-IPv4 Overlay in Arista Cloud Test (ACT)

Suppose you are running this lab in the Arista Cloud Test service, and the overlay services are not working (no connectivity from CPE to CPE) after performing the abovementioned steps. In that case, you may need to change the default forwarding engine of the vEOS nodes.

Add the following line to the starting configurations for each node:

platform tfa personality arfa

Currently, this command must be manually entered into the device configurations before trying to push the command with AVD. After you have entered it manually on each node, add the following YAML to group_vars/WAN1.yml and run the deployment playbook:

eos_cli: |
  platform tfa personality arfa

Retest the services. They should now work, provided the CPEs and aggregation node are correctly configured.