ISL#
Fabrics → IISLs
An inter-switch link, or ISL, is a logical representation of one or more physical links interconnecting two network nodes. ISLs are usually created as derived resources by a Fabric, as a way to model the physical connectivity of network switches that are part of that Fabric.
Inter-switch links vs edge links
Inter-switch links always connect to network switches, and are never connected to client equipment such as servers. Use Links to model a physical connection from an edge interface to a compute.
In addition to modeling physical connectivity between switches, the ISL resource is responsible for the configuration of underlay protocols like BGP and OSPF for the exchange of underlay routes.
BFD#
BFD parameters can be configured on an ISL, which are passed down to the derived DefaultInterface that the ISL creates. The configured BFD session will monitor the neighboring interface, improving the fault detection time significantly if there is layer-2-only equipment in between the two switches.
Not seeing BFD sessions being established?
BFD requires a protocol to subscribe before a BFD session is created. This could be either a static route, a BGP peer, or OSPF neighbor. For example, a DefaultBGPPeer with BFD enabled will only establish a session with its peer if the underlying DefaultInterface (which is created as a derived resource by the ISL resource) has BFD enabled as well, and vice versa.
Dependencies#
DefaultRouter#
ISL resources provide underlay connectivity in the default VRF. Therefore, it is required that DefaultRouter resources are created on each switch that the ISL interconnects.
Interface#
Both endpoints of an ISL are Interface resources, which represent a (set of) physical port(s). For each endpoint, the ISL will use the DefaultRouter resource and the Interface resource to create a derived DefaultInterface resource.
Once the ISL is created, the derived DefaultInterface are referenced in the state of the ISL.
LAGs vs individual links
In a datacenter architecture, it is common practice to avoid LAGs (Link Aggregation Groups) and instead create individual links - each with their own BGP or OSPF session - even if they interconnect the same switches. In some scenarios, this enables more powerful load-balancing behavior compared to bundling physical links.
SubnetAllocationPool#
IP addresses are required if the switches are not using IPv6 unnumbered addresses to communicate. These inter-switch IP addresses are taken from a SubnetAllocationPool. This allocation pool specifies a bigger subnet (e.g. 172.16.0.0/12) that can be subdivided into smaller subnets. For the ISL resource, the SubnetAllocationPool must contain a segment that hands out subnets of size /31.
Referenced resources#
IngressPolicy#
Quality of Services (QoS) mechanisms can be configured on the ISL, ensuring that traffic is properly classified and prioritized. If an IngressPolicy is used, these policies must be created before they can be attached to an ISL.
Read the QoS documentation
The QoS application documentation contains important information about QoS and what is supported by EDA. Specifically, QoS on an inter-switch link is typically referred to as Network QoS, which has different capabilities compared to Access QoS on most hardware platforms.
EgressPolicy#
Quality of Services (QoS) mechanisms can be configured on the ISL, ensuring that traffic is properly prioritized. If an EgressPolicy is used, these policies must be created before they can be attached to an ISL.
DefaultBGPGroup#
If BGP is enabled on an ISL, the resource will automatically create derived DefaultBGPPeer resources: one for each endpoint. These peers will exchange (MP-)BGP routes to advertise reachability information throughout the network. In EDA, DefaultBGPPeers always belong to a DefaultBGPGroup, which configures common parameters that are re-used across multiple BGP sessions.
Certain BGP session parameters, such as importPolicies and exportPolicies can be overridden in the ISL resource: if they are not specified, the policies of the DefaultBGPGroup are used instead.
Policy#
Routing policies determine which reachability information is advertised to BGP and OSPF neighbors. In typical datacenter fabrics, the /31 point-to-point subnets are not advertised to peers: the underlay BGP / OSPF sessions are only used for the exchange of system IP addresses. To accomplish this, routing policies are required.
DefaultOSPFInstance#
An OSPF instance is the top-most container in an OSPF hierarchy, and specifies which address families are being exchanged.
OSPF hierarchy concepts
If OSPF is enabled, a DefaultOSPFInterface will be created for each derived DefaultInterface resource, which is then assigned to a DefaultOSPFArea that in turn belongs to a DefaultOSPFInstance.
Both resources must be created when configuring an ISL that uses OSPF to exchange reachability information.
DefaultOSPFArea#
An OSPF area is a logical grouping of routers that share the same area ID, configured within an OSPF instance.
Examples#
apiVersion: fabrics.eda.nokia.com/v1
kind: ISL
metadata:
name: isl-leaf1-spine2
namespace: eda
spec:
bfd:
desiredMinTransmitIntMs: 1000
detectionMultiplier: 3
enabled: true
requiredMinEchoReceiveIntMs: 1000
requiredMinReceiveIntMs: 1000
bgp:
afiSAFIs:
- ipv4unicast
- l2vpnevpn
bgpGroup: bgpgroup-ebgp-fabric
enabled: true
localInterfaceAS: 100
remoteInterfaceAS: 101
localDefaultRouter: router-leaf1
localInterface: leaf1-ethernet-1-2
poolIPv4: ipv4-pool
remoteDefaultRouter: router-spine2
remoteInterface: spine2-ethernet-1-1
cat << 'EOF' | kubectl apply -f -
apiVersion: fabrics.eda.nokia.com/v1
kind: ISL
metadata:
name: isl-leaf1-spine2
namespace: eda
spec:
bfd:
desiredMinTransmitIntMs: 1000
detectionMultiplier: 3
enabled: true
requiredMinEchoReceiveIntMs: 1000
requiredMinReceiveIntMs: 1000
bgp:
afiSAFIs:
- ipv4unicast
- l2vpnevpn
bgpGroup: bgpgroup-ebgp-fabric
enabled: true
localInterfaceAS: 100
remoteInterfaceAS: 101
localDefaultRouter: router-leaf1
localInterface: leaf1-ethernet-1-2
poolIPv4: ipv4-pool
remoteDefaultRouter: router-spine2
remoteInterface: spine2-ethernet-1-1
EOF
Custom Resource Definition#
To browse the Custom Resource Definition go to crd.eda.dev.
ISL
SPEC
The ISL enables the configuration and management of direct links between Nodes. This resource allows for specifying IPv4 and IPv6 allocation pools, enabling BFD for fast failure detection, and configuring VLAN IDs for the ISL. It also supports BGP peering between the endpoints, with options for setting autonomous systems, AFI/SAFI configurations, and import/export routing policies.
-
Enable or disable BFD on the ISL. [default=false]
-
The minimum interval in milliseconds between transmission of BFD control packets.
default: 1000range: 10 to 100000 -
The number of packets that must be missed to declare this session as down.
default: 3range: 3 to 20 -
Enables Biforward Detection.
default: false -
The minimum interval between echo packets the local node can receive.
default: 1000range: 0 to 100000 -
The minimum interval in milliseconds between received BFD control packets that this system should support.
default: 1000range: 10 to 100000 -
Sets custom IP TTL or Hop Limit for multi-hop BFD sessions packets. Not applicable to single-hop BFD sessions.
range: 2 to 255
-
-
-
Which AFI and SAFI to advertise on the BGP peering session. Options: ipv4unicast, ipv6unicast, l2vpnevpn
-
Reference to a DefaultBgpGroup.
-
Enable or disable BGP peering between the two endpoints of the ISL. [default=false]
default: false -
Reference to a RoutingPolicy to use when evaluating route exports from the DefaultRouter.
-
Reference to a RoutingPolicy to use when evaluating route imports into the DefaultRouter.
-
Keychain to be used for authentication
-
The Autonomous System to configure on the Local Interface.
format: int32range: 0 to 4294967295 -
The Autonomous System to configure on the Remote Interface.
format: int32range: 0 to 4294967295
-
-
Sets the IP MTU for the local and remote Interfaces
range: 1280 to 9486 -
Reference to the DefaultRouter associated with the local Interface in which the ISL will be provisioned.
-
Reference to an Interface.
-
Enable or disable OSPF on the ISL.
-
Enable or disable OSPF between the two endpoints of the ISL. kubebuilder:validation:Boolean
-
OSPFv3 Parameters.
-
Reference to a IPV4 DefaultOSPFArea on the local interface.
-
Reference to a IPV4 DefaultOSPFInstance on the local interface
-
Reference to a IPV6 DefaultOSPFArea on the local interface.
-
Reference to a IPV6 DefaultOSPFInstance on the local interface.
-
Reference to a IPV4 DefaultOSPFArea on the remote interface.
-
Reference to a IPV4 DefaultOSPFInstance on the remote interface
-
Reference to a IPV6 DefaultOSPFArea on the remote interface.
-
Reference to a IPV6 DefaultOSPFInstance on the remote interface.
-
-
-
Reference to an IPv4 allocation pool to use for ISL subnet allocations.
-
Reference to an IPv6 allocation pool to use for ISL subnet allocations.
-
Reference to the DefaultRouter associated with the remote Interface in which the ISL will be provisioned.
-
Reference to an Interface.
-
Enables the use of unnumbered interfaces on the ISL. For IPv6, no IP address are configured on the sub-interface and only the link local address will be used. If any allocation pool is specified for IPv6 that will take precedence and IPs will be assigned to the interfaces. When using eBGP for an underlay protocol, the DefaultInterfaces which are a part of the ISL will be added to the BGP dynamic neighbor list.
enum: "IPv6" -
Single VLAN tag value between 1-4094.
range: 1 to 4094
STATUS
ISLStatus defines the observed state of ISL
-
Indicates the health score of the ISL
-
Indicates the reason for the health score
-
The time when the state of the resource last changed
-
Operational state of the ISL
enum: "Up", "Down", "Degraded", "Unknown"