Some time ago I was tasked with interfacing to a couple of
other multi-site organisations across a large governmental network similar in
operation to the Internet. This was an interim measure prior to integrating
aspects of the three networks into a single entity, and prior to having any
dedicated WAN links in place.
I had to provide connectivity between a variable number of users and servers across all three networks, and with many overlapping IP ranges in place. The idea was to have a flexible enough configuration that I could easily add and change routes at the far end to keep pace with any integration work.
The last requirement was the support of one or more AD trusts between the organisations, with DNS forwarding.
To make it as light a touch a possible for the far-end IT departments, I went for a single interface Cisco router that could be connected directly to a Firewall DMZ on the far end firewall.
The topology essentially looked like this (with extraneous devices stripped away) and Internet substituted for the (private) government network:
I had to provide connectivity between a variable number of users and servers across all three networks, and with many overlapping IP ranges in place. The idea was to have a flexible enough configuration that I could easily add and change routes at the far end to keep pace with any integration work.
The last requirement was the support of one or more AD trusts between the organisations, with DNS forwarding.
To make it as light a touch a possible for the far-end IT departments, I went for a single interface Cisco router that could be connected directly to a Firewall DMZ on the far end firewall.
The topology essentially looked like this (with extraneous devices stripped away) and Internet substituted for the (private) government network:
At a more useful level, showing physical interfaces and IP addresses:
NAT
Ideally, in order to allow variable numbers of users to cross the NAT boundary in either direction, one would be able to use PAT in both directions. However, this is only available to the “inside” interface.As a large number of users were likely to be crossing from the remote site, and only a few from the hub site, I had to make the remote physical interface act as “inside” and the tunnel act as “outside”.
This allowed me to use PAT for remote users and dynamic NAT for hub users.
Servers were easily handled by static NAT in both directions
Non-NAT-Compliant Applications
Nowadays, most applications, including to my surprise, Microsoft domain trusts, work quite well across (Cisco IOS) NAT boundaries. I found only one application which didn’t: an old version of HP Openview ServiceDesk, which embeds the source IP of the HPOV server inside the java client for use in a subsequent return connection.In this particular instance, the server was based at the hub site, and no IP conflict existed at the remote site. I was able to create an identity NAT for the server in the direction of the Hub which worked fine once supporting routes were in place.
MTU issues
Because the remote firewall has not participated in creating the tunnel endpoint, it can’t respond correctly to hub-destined traffic with DF flag set, so we have to ensure that the remote firewall allows ICMP unreachables to be sent from our router to devices on its internal network.Design Notes
Some basic notes might be required to clarify where all the addresses are coming from:IPSEC and GRE tunnel end-points
The physical interfaces representing the Hub and Remote endpoints have internal 192.168 addresses and are mapped to NAT addresses on their upstream firewalls. I have used 10.1.1.10 and 10.2.2.10 respectively.
Inter-Org NAT Allocation
Subnet 192.168.198.0/24 is used to Dynamically NAT all Hub users accessing Remote servers
Subnet 192.168.199.0/24 is used to present Hub servers to Remote users.
IP Address 192.168.200.1 is used to PAT all Remote users accessing Hub servers
Subnet 192.168.200.0/24 is used to present Remote servers to Hub users.
Configuration Fragments
The configurations below have been taken from working devices, with some minimal IP address obfuscation:Hub Distribution Router:
! IKE Phase 1
crypto isakmp policy 1
encr aes
authentication pre-share
group 5
! Pre-shared key for remote site
crypto isakmp key RemoteSiteKey address 10.2.2.10
! IKE Phase 2
crypto ipsec transform-set AES256_SHA_tra esp-aes 256 esp-sha-hmac
mode transport
! Crypto ACL for GRE to remote site
ip access-list extended HUB-INTERNET-REMOTE-CRYACL
remark Tunneled traffic over the Internet to Remote site
permit gre host 192.168.12.9 host 10.2.2.10
! Crypto MAP entry for remote site
crypto map INTERNET-CM 15 ipsec-isakmp
set peer 10.2.2.10
set transform-set AES256_SHA_tra
match address HUB-INTERNET-REMOTE-CRYACL
! Physical interface for termination of all WAN and Internet tunnels
interface GigabitEthernet0/1
description Connects to Local Firewall inside
ip address 192.168.12.9 255.255.255.248
crypto map INTERNET-CM
! Tunnel to remote site
interface Tunnel14200
description Tunnel over Internet to Remote Site
! low bandwidth used (EIGRP) for backup tunnels over Internet
bandwidth 1000
ip address 192.168.14.201 255.255.255.252
! Maximum starting MTU (1500 - 8(NAT-T) - 53(AES256) - 24(GRE))
ip mtu 1415
! high delay used (EIGRP) for backup tunnels over Internet
delay 2000
tunnel source GigabitEthernet0/1
tunnel destination 10.2.2.10
! Tell GRE to copy DF from inner to outer IP header
tunnel path-mtu-discovery
ip prefix-list EIGRP-SITETUNNELS-OUT-PL description Route adverts to remote sites
ip prefix-list EIGRP-SITETUNNELS-OUT-PL seq 5 permit 0.0.0.0/0
ip prefix-list EIGRP-SITETUNNELS-OUT-PL seq 10 permit 192.168.0.0/16 le 32
router eigrp 192
passive-interface GigabitEthernet0/1
network 192.168.12.0
network 192.168.14.0
distribute-list prefix EIGRP-SITETUNNELS-OUT-PL out Tunnel14200
no auto-summary
no eigrp log-neighbor-changes
Hub Firewall:
PIX Version 7.2(2)
name 10.1.1.10 dist-rt02_INTERNET
name 192.168.12.9 dist-rt02_G01
object-group service NAT-T udp
description NAT Traversal
port-object eq 4500
object-group service IPsec_udp udp
description UDP protocols used by IPsec
group-object NAT-T
port-object eq isakmp
object-group network Cisco_Devices
description Cisco devices' Internet interfaces
network-object host remote-rt01_INTERNET
interface Ethernet0
speed 100
duplex full
nameif outside
security-level 0
ip address 10.1.1.4 255.255.255.0 standby 10.1.1.5
interface Ethernet1
speed 100
duplex full
nameif inside
security-level 100
ip address 192.168.12.12 255.255.255.248 standby 192.168.12.13
route outside 0.0.0.0 0.0.0.0 10.1.1.1 1
route inside 192.168.0.0 255.255.0.0 192.168.12.9 1
! Mapping the routable Tunnel endpoint
static (inside,outside) dist-rt02_INTERNET dist-rt02_G01 netmask 255.255.255.255
access-list inside-access-in remark Allow ISAKMP & NAT-T to sites using VPN-over-Internet
access-list inside-access-in extended permit udp host dist-rt02_G01 object-group Cisco_Devices object-group IPsec_udp log disable
access-group inside-access-in in interface inside
Remote Firewall:
PIX Version 6.3(4)
interface ethernet0 100full
interface ethernet1 100full
interface ethernet4 100full
nameif ethernet0 outside security0
nameif ethernet1 inside security100
nameif ethernet4 HUBDMZ security49
ip address outside 10.2.2.4 255.255.255.0
ip address inside 192.168.0.5 255.255.254.0
ip address HUBDMZ 192.168.22.9 255.255.255.248
failover ip address outside 10.2.2.5
failover ip address inside 192.168.0.6
failover ip address HUBDMZ 192.168.22.10
object-group network HUB
description HUBDMZ network
network-object 192.168.22.8 255.255.255.248
description HUB users on this subnet
network-object 192.168.198.0 255.255.255.0
description HUB servers on this subnet
network-object 192.168.199.0 255.255.255.0
! Minimal ACLs to permit traffic flow – not representative!
access-list inside_access_in permit ip any object-group HUB
access-group inside_access_in in interface inside
access-list hubdmz_access_in permit icmp any any
access-list hubdmz_access_in permit ip host 192.168.22.13 host 10.1.1.10
access-list hubdmz_access_in permit ip object-group HUB 192.168.0.0 255.255.0.0
access-group hubdmz_access_in in interface HUBDMZ
access-list outside_access_in permit udp host 10.1.1.10 host 10.2.2.13 eq 4500
access-list outside_access_in permit udp host 10.1.1.10 host 10.2.2.13 eq isakmp
access-group outside_access_in in interface outside
! Bypass NAT for incoming HUB traffic (low security to high security)
access-list NO_NAT_HUBDMZ permit ip object-group HUB 192.168.0.0 255.255.0.0
nat (HUBDMZ) 0 access-list NO_NAT_HUBDMZ
! Mapping the routable Tunnel endpoint
static (HUBDMZ,outside) 10.2.2.10 192.168.22.13 netmask 255.255.255.255 0 0
! Hub users and servers respectively
route HUBDMZ 192.168.198.0 255.255.255.0 192.168.22.13 1
route HUBDMZ 192.168.199.0 255.255.255.0 192.168.22.13 1
Remote VPN Router:
! example hub hosts with pre(real) and post nat addresses (hub perspective)
ip host hubhost01 192.168.4.10 192.168.199.5
ip host hubhost02 192.168.4.20 192.168.199.6
! example remote hosts with "pre" and "post"(real) nat (hub perspective)
ip host remotehost01 192.168.200.7 192.168.4.50
ip host remotehost02 192.168.200.8 192.168.4.51
! need inspection to activate ALGs
ip inspect name INSPECT_LIST dns
ip inspect name INSPECT_LIST ftp
ip inspect name INSPECT_LIST https
ip inspect name INSPECT_LIST icmp
ip inspect name INSPECT_LIST imap
ip inspect name INSPECT_LIST pop3
ip inspect name INSPECT_LIST esmtp
ip inspect name INSPECT_LIST sqlnet
ip inspect name INSPECT_LIST streamworks
ip inspect name INSPECT_LIST tftp
ip inspect name INSPECT_LIST tcp
ip inspect name INSPECT_LIST udp
ip inspect name INSPECT_LIST vdolive
ip inspect name INSPECT_LIST kerberos
ip inspect name INSPECT_LIST ldap
ip inspect name INSPECT_LIST microsoft-ds
! IKE Phase 1
crypto isakmp policy 1
encr aes
authentication pre-share
group 5
! Pre-shared key for this site
crypto isakmp key RemoteSiteKey address 10.1.1.10
! IKE Phase 2
crypto ipsec transform-set AES256_SHA_tra esp-aes 256 esp-sha-hmac
mode transport
! Crypto ACL for GRE to hub site
ip access-list extended REMOTE-INTERNET-HUB-CRYACL
remark Traffic tunnelled over Internet to HUB
permit gre host 192.168.22.13 host 10.1.1.10
! Crypto MAP entry for hub site
crypto map INTERNET-CM 2 ipsec-isakmp
set peer 10.1.1.10
set transform-set AES256_SHA_tra
match address REMOTE-INTERNET-HUB-CRYACL
! Single physical interface for LAN and VPN traffic
! in/out ACL not included in config
interface FastEthernet0/0
description Exit to Internet and Remote LAN via Remote DMZ
ip address 192.168.22.13 255.255.255.248
no ip redirects
ip inspect INSPECT_LIST in
! Treat the remote network as inside so we can use PAT
ip nat inside
! enabled automatically with NAT config
ip virtual-reassembly
duplex full
speed 100
no cdp enable
crypto map INTERNET-CM
interface Loopback0
description Remote PAT address for overlapping client subnets
ip address 192.168.200.1 255.255.255.0
interface Tunnel14200
description Tunnel over Internet to Hub network
! low bandwidth used (EIGRP) for backup tunnels over Internet
bandwidth 1000
ip address 192.168.14.202 255.255.255.252
! Maximum starting MTU (1500-8(NAT-T)-53(AES256)-24(GRE))
ip mtu 1415
! Required to allow PAT in the opposite direction
ip nat outside
! enabled automatically with NAT config
ip virtual-reassembly
! high delay used (EIGRP) for backup tunnels over Internet
delay 2000
tunnel source FastEthernet0/0
tunnel destination 10.1.1.10
tunnel path-mtu-discovery
router eigrp 192
passive-interface Loopback0
network 192.168.14.0
network 192.168.200.0
distribute-list prefix EIGRP-TUNNEL-OUT-PL out Tunnel14200
no auto-summary
eigrp stub connected
! Floating default route back to the hub over the tunnel
ip route 0.0.0.0 0.0.0.0 192.168.14.201 200
! Example remote site networks - 192.168.4.0 chosen to demonstrate overlaps
ip route 192.168.0.0 255.255.254.0 192.168.22.9
ip route 192.168.4.0 255.255.254.0 192.168.22.9
ip route 192.168.35.0 255.255.254.0 192.168.22.9
! Explicit route for our tunnel destination to avoid recursion
ip route 10.1.1.0 255.255.255.0 192.168.22.9
! We need the flexibility of PAT to be applied to the remote network
ip nat inside source list REMOTE-USERS interface Loopback0 overload
! Which leaves us on the "outside" using dynamic NAT
ip nat pool HUB-POOL 192.168.198.1 192.168.198.254 prefix-length 24
ip nat outside source list HUB-USERS pool HUB-POOL
! Example remote servers - DNS ALG will use these to translate our queries
ip nat inside source static 192.168.4.50 192.168.200.7
ip nat inside source static 192.168.4.51 192.168.200.8
! Example hub servers - DNS ALG will use these to translate their queries
ip nat outside source static 192.168.4.10 192.168.199.5
ip nat outside source static 192.168.4.20 192.168.199.6
! Define which remote subnets hide behind PAT
ip access-list standard REMOTE-USERS
remark Remote main site
permit 192.168.0.0 0.0.1.255
remark Remote secondary site example
permit 192.168.35.0 0.0.1.255
! Define which hub subnets hide behind Dynamic NAT
ip access-list standard HUB-USERS
remark Hub IT department
permit 192.168.32.0 0.0.0.255
remark Hub main site
permit 192.168.125.0 0.0.0.255
remark Hub secondary site example
permit 192.168.35.0 0.0.0.255
ip prefix-list EIGRP-TUNNEL-OUT-PL description Routes to be advertised from site
ip prefix-list EIGRP-TUNNEL-OUT-PL seq 5 permit 192.168.200.0/24
Postscript
The creation and ongoing support of Microsoft domain trusts across this two-way NAT boundary was reasonably straight forward. There were a couple of issues, neither of which were caused by or really impinged upon the configuration itself, but might be worth mentioning:1. Problems creating a domain Trust across two-way NAT
I found it useful to ensure that all DNS servers in both domains could see and forward to each other. In one of the organisations I needed to connect to, this was tiresome because they had at least 6 DCs of which 4 were DNS servers. This requires static NAT entries to be configured for each server.
I also found that physical DCs were more reliable than VMs, in part due to VMware tools not being installed thoughtfully - the Shared Folders option should not be installed as it causes network (RPC) problems. However, you can't chose in advance which DCs will participate on each side, so it becomes useful to be able to mask off the suspect ones by removing their NAT entries.
2. Kerberos-related fragmentation
Depending upon the server and workstation versions, Kerberos may still default to UDP, which may cause performance problems due to fragmentation. This is particularly noticable where W2K3 and XP are in use, and where there are many nested groups and SID histories to bloat the packets. This manifests itself as a delay in accessing resources across the trust. Debugging ip virtual-reassembly may show maximum fragments or fragmentation buffer being exceeded and some additional tweaking may be required to prevent timeouts and retransmissions within Kerberos.
Sources
I found the following document very useful in getting this to production:NAT Order of Operation
