Chapter 13 VNET Hybrid Connectivity with ExpressRoute

This Chapter covers following Topic Lessons

• Virtual Network Hybrid Connectivity using Virtual Network Gateway
• VNET Hybrid Connectivity over ExpressRoute connection
• Connecting Azure Virtual Network to On-Premises
• ExpressRoute Routing Domains
• ExpressRoute Connectivity Options (3 Options)
• ExpressRoute Connection Tiers
• ExpressRoute Gateway SKU
• ExpressRoute Bandwidth options
• ExpressRoute Health
• Comparing ExpressRoute and VPN
• Connecting Virtual Networks (VNET) to ExpressRoute circuit
• ExpressRoute Gateway Pricing
• ExpressRoute Connection Pricing
• ExpressRoute Direct

This Chapter covers following Lab Exercises

• Create Virtual Network Gateway of Type ExpressRoute
• Create ExpressRoute Circuit and Connect to VNETCloud

Chapter Topology

In this Chapter we will add ExpressRoute to the topology. We will create Virtual Network Gateway of type ExpressRoute in GatewaySubnet of Virtual Network VNETCloud. We will also Create ExpressRoute Circuit. Provisioning of Circuit will not be done as we don’t have access to ExpressRoute Service Provider.

Virtual Network Hybrid Connectivity using Virtual Network Gateway

You can connect Virtual Network to on-premises Datacenter through virtual network gateway located in GatewaySubnet using either Internet VPN (P2S or S2S VPN) or ExpressRoute Private WAN connectivity.

For Internet VPN you deploy virtual network gateway of type VPN. For Private WAN connectivity you deploy virtual network gateway of type ExpressRoute.

Figure below shows Virtual Network Connected to on-premises Datacenter.

Every Azure VPN gateway consists of two instances in an active-standby or active-active configuration.

VNET Hybrid Connectivity over ExpressRoute connection

ExpressRoute is an Azure Managed service, which creates dedicated private connections between Microsoft Datacenters and on-premises infrastructure.

ExpressRoute connections don’t go over the public internet. They offer more reliability, faster speeds, lower latencies and higher security than typical internet connections.

Azure ExpressRoute connects Virtual Network (VNET), Azure PaaS Services (Azure SQL, Azure Storage etc) and Microsoft Online Services (Dynamics 365 & Office 365) to your on-premises infrastructure.

ExpressRoute, connections to Azure are established at an Exchange provider facility. Each ExpressRoute circuit consists of two connections to two Microsoft Enterprise edge routers (MSEEs) from the connectivity provider.

Figure below shows ExpressRoute Circuit Dual Connection (Primary & Secondary) between Microsoft Edge Routers and Partner Service Provider. From Service Provider to Customer Network it can be dual or single connection.

Connecting Azure Virtual Network to On-Premises

For Connecting Virtual Network (VNET) to on-premises infrastructure, ExpressRoute Gateway is created in GatewaySubnet. A GatewaySubnet is created in Azure Virtual Network (VNET).

ExpressRoute Private WAN connection connects ExpressRoute Gateway to On-Premises infrastructure.

Figure Below shows Virtual Network with ExpressRoute Gateway installed in Gateway Subnet. Virtual Network is connected to ExpressRoute Circuit at Azure Side. On-premises Infrastructure is connected to ExpressRoute Circuit at Service Provide end.

Note 1: Azure ExpressRoute Gateway consists of two instances.

Note 2: There is dual Connectivity from Microsoft to Service Provider edge.

Note 3: Connectivity from Customer Network to Service Provider can be single or dual.

ExpressRoute Routing Domains

An ExpressRoute circuit has multiple routing domains associated with it: Azure private and Microsoft. See Figure on page 487.

Private peering domain

On premises infrastructure connects with Azure virtual network (VNET) through the private peering domain. The private peering domain is an extension of your on premises network into Microsoft Azure Virtual Network. Private peering lets you connect to virtual machines directly on their private IP addresses.

Microsoft Peering

Connectivity to Microsoft online services (Office 365 services & Dynamics 365) and Azure PaaS Services will be through the Microsoft peering. Microsoft Peering enables bi-directional connectivity between your WAN and Microsoft cloud services through the Microsoft peering routing domain.

Note : Peering type is configured through ExpressRoute Circuit Dashboard.

ExpressRoute Connectivity Options (3 Options)

Layer 3 Connectivity: With ExpressRoute you can establish Layer 3 connectivity between your on-premises network and the Microsoft Cloud through a connectivity provider. For layer 3 Connectivity Microsoft uses BGP to exchange routes between your on-premises network, your instances in Azure, and Microsoft public addresses.

Layer 2 Connectivity: With ExpressRoute you can establish Layer 2 connectivity between your on-premises network and the Microsoft Cloud using Point to Point Ethernet links.

Integrating your IPVPN WAN: IPVPN providers (typically MPLS VPN) offer any-to-any connectivity between your branch offices and Datacenters. The Microsoft cloud can be interconnected to your WAN to make it look just like any other branch office as shown below.

Connectivity Redundancy: Each ExpressRoute circuit consists of two connections to two Microsoft Enterprise edge routers (MSEEs) from the connectivity provider. Connectivity from Customer Network to Service Provider can be single or dual.

ExpressRoute Connection Tiers

ExpressRoute Connection Circuit comes in 2 Tiers: Standard & Premium Add on.

ExpressRoute Standard Connection

The ExpressRoute Standard Connection provides the following capabilities:

1. Up to 10 VNET Links per ExpressRoute circuit.
2. An ExpressRoute circuit created in any region will have access to resources across any other region in the same Geographic region. For Example a VNET created in US East can be accessed through ExpressRoute circuit provisoned in any region in United States only. This VNET in US east cannot be accessed by ExpressRoute Circuit provisioned in Europe.
3. Supports Private and Public Peering
4. Supports upto 4000 routes for Azure Public & Private Peering.

ExpressRoute Premium Connection

The ExpressRoute premium is an add-on over the ExpressRoute circuit. The ExpressRoute premium add-on provides the following capabilities:

1. Increased route limits for Azure public and Azure private peering from 4,000 routes to 10,000 routes.
2. An ExpressRoute circuit created in any region (excluding Azure China, Azure Germany and Azure Government cloud) will have access to resources across any other region in the world. For example, a virtual network created in West Europe can be accessed through an ExpressRoute circuit provisioned in Silicon Valley.
3. Increased number of VNet links per ExpressRoute circuit from 10 to a larger limit of 100 (depending on the bandwidth of the circuit).
4. Supports Private, Public Peering & Microsoft Peering.

ExpressRoute Gateway SKU

Standard
High Performance
Ultra High Performance

Comparison of aggregate throughput by gateway SKU.

ExpressRoute Bandwidth options

ExpressRoute connection is available in multiple bandwidth options.

50 Mbps
100 Mbps
200 Mbps
500 Mbps
1 Gbps
2 Gbps
5 Gbps
10 Gbps

Dynamic scaling of bandwidth

You can increase the ExpressRoute circuit bandwidth (on a best effort basis) without having to tear down your connections.

ExpressRoute Health

ExpressRoute circuits may be monitored for availability, connectivity to VirtualNetworks and bandwidth utilization using Network Performance Monitor (NPM).

NPM monitors the health of Azure private peering and Microsoft peering.

ExpressRoute Service Providers

Microsoft has large Service Provider partner network which provide ExpressRoute Circuit across various locations in the world. Some of the Service Provider partners include AT&, Airtel, British Telecom, China Telecom, Comcast, Colt, Equinix, MTN, NTT Communications, Sify, Singtel, Tata Communications, Telenor, Vodafone & Verizon etc.

ExpressRoute System Integrators

Microsoft ExpressRoute System Integrator Partners provide ExpressRoute circuit integration services. These partners help in connecting on-premises Data center with Azure using ExpressRoute circuit. Some of the System Integrator Partners include Avande, Equinix, Bright Skies GmbH, Orange Networks & Presidio etc.

ExpressRoute Limits

Number of Virtual Networks per ExpressRoute Circuit

Comparing ExpressRoute and VPN

Connecting Virtual Networks (VNET) to ExpressRoute circuit

There are 6 steps to connecting Virtual Networks to ExpressRoute circuit. This assumes that VNET is already created. 7 th Step is configured onpremises to connect on-prem Router to Service Provider ExpressRoute Circuit line.

• Create GatewaySubnet in Virtual Network.
• Create Virtual Network Gateway of type ExpressRoute in GatewaySubnet.
• Create an ExpressRoute Circuit.
• From ExpressRoute Circuit Dashboard copy the key and send to your Service Provider for provisioning the circuit.
• After Circuit is Provisioned, Configure Routing (Private Peering) in ExpressRoute Dashboard.
• From ExpressRoute Dashboard Link VNET to ExpressRoute Circuit.
• Connect and configure on-premises Device to Service Provider ExpressRoute Circuit line. Use the same shared key which was specified during Peering Configuration on Azure Side.

Exercise 124: Create Virtual Network Gateway of Type ExpressRoute

In this exercise we will create VPN Gateway of Type ExpressRoute in GatewaySubnet in VNETCloud. VNETCloud was created in Chapter 1 Exercise 3. GatewaySubnet was created in VNETCloud in Chapter 12, Exercise 122.

• Click Create a resource>Networking>Virtual Network gateway> Create virtual network gateway blade opens>Enter a name, Select Location EAST US 2, Select Gateway type as ExpressRoute , Select SKU Standard, Select Virtual Network VNETCloud and Select Create new Public IP and enter a name> Click Review +create (Not Shown)>After validation is passed click create.

  Figure below shows Dashboard of Virtual Network Gateway (Type ExpressRoute) ERCloud.

Exercise 125: Create ExpressRoute Circuit and Connect to VNETCloud

This is a demonstration Exercise. We will create ExpressRoute Circuit and show how to connect it to Virtual Network VNETCloud. Provisioning of Circuit will not be shown as we don’t have access to Service Provider.

• Click Create a resource>Networking>ExpressRoute>Create ExpressRoute Circuit Blade opens>Enter a name>Select Provider AT&T, Select Peering location Washington DC, Bandwidth 50 Mbps, SKU Standard, Select Billing Model Metered> Select Resource Group RGCloud and Location East US 2> Click create.

  Figure below shows Dashboard of ExpressRoute Circuit ERCCloud. Note the Service Key in Right pane. It shows Provider Status as not enabled.

• In Right pane Note down the Service Key and send it to your Service Provider for Provisioning of the Circuit. Go to Next step after ER Circuit is provisioned.

• Configure Routing (Private Peering) in ExpressRoute Dashboard. In ExpressRoute Circuit Dashboard Click Peerings in left pane > Peering pane opens as shown below.

  In right pane Click Azure Private>Private Peering blade opens as shown below. All options are greyed out as Circuit is not provisioned by the Service Provider

  Here Primary/Secondary Subnet is /30 subnet of a Public IP owned by you . From this subnet you will assign the first useable IP address to your router. Microsoft uses the second useable IP for its router. Primary/Secondary Subnet refer to Primary/Secondary links. Specify VLAN & Public/private ASN for peering.

• Connect ExpressRoute Circuit to Virtual Network VNETCloud. In ExpressRoute Circuit Dashboard Click Connections in left pane.

  In Right pane you need to click +Add to open the Connection pane. It is currently greyed out as ExpressRoute circuit is not provisioned.

  Click + Add to open Connection pane. Here I am showing you connection pane from Azure Docs. Here select your Virtual Network and ExpressRoute Circuit.

• Configure on-premises router using the same shared key which was specified in Private Peering Configuration in step 3. Connect it to ER Circuit.

• Monitor ExpressRoute using Network Performance Monitor (NPM ). In ExpressRoute Circuit Dashboard click Health using NPM>

  Note 1: Network Performance Monitor (NPM) will be discussed in Chapter 18.

  Note 2: Delete VPN Gateway and ExpressRoute Circuit as we no longer require it.

ExpressRoute Gateway Pricing

ExpressRoute Connection Pricing

ExpressRoute has 2 Pricing option - Metered Data Plan and Unlimited Data Plan.

Metered Data Plan

Metered Data Plan has 2 components - Fixed monthly port fee (High Availability dual ports) based on Bandwidth and outbound data charge. Figure below show ExpressRoute pricing for port speed of 50 Mbps and 100 Mbps only. Note - speed can go upto 10 Gbps.

Note 1 - Outbound data transfer is charged at a rate of $0.025 per GB for Zone 1, $0.05 per GB for Zone 2 and $0.14 per GB for Zone 3.

Unlimited Data Plan

With Unlimited Data Plan users are charged a single fixed monthly port fee (High Availability dual ports) based on Bandwidth. All inbound and outbound data transfer is free of charge. Figure below show ExpressRoute pricing for port speed of 50 Mbps and 100 Mbps only. Note - speed can go upto 10 Gbps.

ExpressRoute Direct (In Preview)

With ExpressRoute Direct, customers connect directly to Microsoft’s network through a pair of 100Gbps ports to create 5Gbps, 10Gbps, 40Gbps and 100Gbps ExpressRoute Local, Standard and ExpressRoute Premium circuits.

ExpressRoute Direct contains both a monthly Port fee and, for ExpressRoute Premium circuits, a Premium Circuit fee. Outbound data transfer is applicable to Standard and Premium circuits and not applicable to Local circuits.