Campus Floor Connectivity Design

Design Context

This project was an expansion of an operating campus network. The FortiGate 600-series HA firewalls were already in place; the work focused on extending connectivity, upgrading distribution, and integrating new access switches with existing infrastructure. In this public version, I use Orion and Atlas as fictional building names and replace provider names and public IP addresses in the original plan with examples.

Orion Floor 2 housed the WAN handoff and a five-switch C9300 StackWise stack on the trusted side of the firewalls. At Atlas, an existing fiber rack on Floor 4 became the connection point for the new C9500 distribution pair, the Floor 7 access groups, and a separate Floor 4 extension. The C9500s took over the distribution role previously served by C9200-series switches, which were retained downstream for access.

I put particular care into how the new switching would fit the existing fiber plant. A contractor installed the fiber links, and their routing and endpoint mapping had to be checked on site. That coordination mattered as much as the logical topology: a redundant line on a drawing is only useful when the actual patching and failure domains support it.

Hardware Inventory and Roles

The inventory below follows the original drawing and my implementation context. It separates existing equipment from the expansion work and distinguishes device counts from representative diagram symbols.

Detailed Material Reference

The material reference below captures the equipment and media involved in the expansion. Exact optic and interface selections belong in the implementation port schedule; generic device artwork is not a module specification.

Switch Port and Uplink Plan

The port plan follows the traffic boundary: ISP transport first, firewall policy next, and trusted campus switching after that. Fiber racks are passive patching points, not forwarding devices.

Specific Uplink and Optic Notes

The 10G annotation in the source applies to the new campus uplink design; it should not be read as upgrading every connected device to 10G. The C9200L artwork is specifically labeled with 1G uplinks. I distinguish those links here so the port plan does not overstate their capacity.

Design Requirements

Connectivity Options

The design decision was driven by the need to expand an existing network without replacing every layer. Reusing the firewall edge, the Floor 4 rack, and downstream access equipment kept the scope focused on distribution and connectivity.

Proposed Architecture

The service path starts with the existing ISP handoffs on Orion Floor 2: ISP services → SG300 WAN aggregation → FortiGate 600-series HA → five-switch C9300 stack → OM3 inter-building connection → Atlas Floor 4 C9500 distribution → campus access. This places the C9300 stack behind the firewalls, not ahead of them. SD-WAN is a logical function on the FortiGate, not a separate physical device.

At Atlas Floor 4, the new C9500 pair took over distribution and connected through the existing fiber rack to the Floor 7 access groups and the Floor 4 extension. Floor 7 contains 12 C9300 switches across D1, D2, and D3. The original drawing uses representative switch icons; the inventory records the actual total so those symbols are not mistaken for a device count.

The C9500 interconnection uses a different architecture from the C9300 rear-panel StackWise stack. StackWise Virtual presents the two C9500 chassis as one logical switch with an active and standby control plane; both chassis can forward traffic. If the active chassis fails, the standby takes over the control-plane role. This is more than copying a configuration: availability also depends on the SVL, dual-active detection, surviving uplinks, and remaining capacity. Cisco's StackWise Virtual guide explains these roles.

Internet traffic follows the firewall path. No WAN VLAN or alternative route should bridge around the FortiGate into trusted campus switching. Traffic routed locally between campus VLANs does not automatically cross the firewall; segments requiring stateful inspection need a forwarding path through it. The security boundary therefore depends on gateway placement and routing as well as the physical topology.

Transceiver and Media Plan

OM3 multimode fiber was used for the interconnections shown in the design. The fiber work was contracted out, and we had to verify how the links were routed and terminated. The existing Atlas Floor 4 rack was the central patching point for Floor 7 and the local extension.

OM3 does not imply unlimited reach. Optic selection must match the measured path and both platforms. In particular, 10GBASE-LR single-mode optics are not a drop-in replacement for multimode optics on an OM3 run. Labeling and contractor handoff records should make the physical path traceable without relying on the colors in the drawing.

Resiliency Decisions

The distribution upgrade provided two C9500 chassis and redundant connection points, with paired fiber connections for the Floor 7 groups. Equipment redundancy, link redundancy, and physical route diversity need to be assessed separately. The contractor's cable routing and termination work determines whether a single physical incident can affect both connections.

• Distribution failure: StackWise Virtual supports control-plane switchover, while surviving links carry traffic through the remaining chassis. Size those links and the surviving switch for the required failure-state load; switchover alone does not guarantee sufficient bandwidth.
• Uplink redundancy: A port-channel spanning both C9500 chassis is a Multi-Chassis EtherChannel (MEC). Paired physical links are not automatically a MEC: their aggregation and peer configuration must match. Keep the SVL and dual-active detection function distinct from downstream port-channels.
• Fiber failure domains: Two strands or cables on the same route can fail together. A shared conduit, riser, patching point, or technical-room dependency remains a physical single point of failure for the connections using it. Contractor route verification is what establishes the extent of physical diversity.
• Floor 2 power constraint: The five C9300s used StackWise for data, but not StackPower. I was told this was a budget decision. That does not disable data stacking, but it removes shared stack-power capability; PSU and feed redundancy must be evaluated per member.
• Existing WAN dependencies: The FortiGate HA pair remains behind the existing SG300 handoff. Firewall HA does not remove the failure domain of that upstream switch or any shared provider path.
• Operations: Test link loss, distribution-chassis failure, and firewall failover separately. Check reachability, session impact, convergence, interface utilization, and the recovery procedure. Labels and accurate endpoint records shorten fault isolation.

Security and Layer 2 Controls

The expansion retained the existing FortiGate security edge. WAN services terminate on the upstream aggregation layer, and the trusted campus stack sits downstream of the firewalls. Maintaining that separation during the cutover is essential: access growth must not introduce an unintended bypass around the existing policy boundary.

The access design also considered 802.1X, VLAN segmentation, ACLs, storm control, PortFast, and BPDU Guard. These controls have different jobs; a VLAN boundary alone is not stateful inspection.

• 802.1X for authenticated endpoint access.
• Separate VLANs for users, voice, management, cameras, and access-control systems.
• ACLs at the relevant routing boundary for controlled inter-segment access.
• Storm-control thresholds suited to the expected traffic profile.
• PortFast and BPDU Guard on appropriate endpoint-facing ports.
• Document gateway and firewall-policy ownership so troubleshooting follows the actual forwarding path.

FortiGate Configuration Examples: SD-WAN, SLA, and Policy

These anonymized examples adapt patterns from a historical FortiGate 300D backup to FortiOS 7.4.x syntax. They are illustrative migration examples, not a captured 7.4.x configuration or confirmation that this release runs on the original 300D. The backup contains several VDOMs, so the examples retain their separate contexts. They do not establish that the proposed campus redesign was implemented.

Interface and object names, rule IDs, probe targets, and addresses have been replaced with generic labels and documentation values. Credentials, encrypted secrets, certificates, serial numbers, and customer identifiers are excluded. Referenced interfaces and address objects are not defined here; these are adapted excerpts, not a complete deployment configuration.

SD-WAN Members and Explicit Traffic Steering

In one VDOM, two enabled WAN members carried the SD-WAN configuration; a third disabled member is omitted here. The excerpt preserves the measured-volume balancing mode, member cost, volume ratio, and one source/destination-specific rule. That rule selects member 1; it does not demonstrate an SLA-based alternate path. The omitted second rule selects member 2 for a different destination.

config system sdwan
    set status enable
    set load-balance-mode measured-volume-based
    config zone
        edit "virtual-wan-link"
        next
    end
    config members
        edit 1
            set interface "WAN_A"
            set gateway 192.0.2.1
            set cost 5
        next
        edit 2
            set interface "WAN_B"
            set gateway 198.51.100.1
            set volume-ratio 99
        next
    end
    config health-check
        edit "WAN_REACHABILITY"
            set server "203.0.113.10"
            set members 1 2
        next
    end
    config service
        edit 10
            set name "CLIENT_A_VIA_WAN_A"
            set mode manual
            set priority-members 1
            set dst "CLIENT_A_REMOTE"
            set src "CLIENT_A_USERS" "CLIENT_A_SERVERS"
        next
    end
end

The example addresses are not live gateways or probe targets. The volume ratio is a configured balancing weight, not a measured traffic split or a bandwidth requirement. Explicit steering and link monitoring must be reviewed together with routing and failure behavior.

Performance SLA in a Separate VDOM

A different VDOM has one configured WAN member and the HTTP probe shown below. Its configured SLA thresholds are 250 ms latency, 50 ms jitter, and 5% packet loss, with a 1,000 ms probe interval and 10 successful responses for recovery. These are historical settings, not measured performance or universal targets. A single WAN member cannot provide provider failover, and an SLA definition alone does not prove that a steering rule uses it.

config system sdwan
    config health-check
        edit "HTTP_SERVICE_SLA"
            set server "probe.example.com"
            set protocol http
            set interval 1000
            set recoverytime 10
            config sla
                edit 1
                    set latency-threshold 250
                    set jitter-threshold 50
                    set packetloss-threshold 5
                next
            end
        next
    end
end

Firewall Policy: Authorization Is Separate from Path Selection

This adapted policy originates from the same VDOM as the two enabled WAN members. It permits specified source and destination objects toward the SD-WAN zone, applies source NAT, and logs accepted traffic. SD-WAN selects the egress path; the firewall policy authorizes the session. Neither replaces the other.

config firewall policy
    edit 100
        set name "SERVICE_OUTBOUND"
        set srcintf "INTERNAL_A" "INTERNAL_B"
        set dstintf "virtual-wan-link"
        set srcaddr "AUTHORIZED_USERS"
        set dstaddr "REMOTE_SERVICE"
        set action accept
        set schedule "always"
        set service "ALL"
        set inspection-mode proxy
        set logtraffic all
        set nat enable
    next
end

The original rule uses ALL services. That is broader than a service-specific allow list, even with restricted address objects. A refinement would validate application dependencies and restrict allowed services through the change process. This excerpt does not establish web filtering or application-control enforcement; those require the relevant policy settings and profile definitions.

Viewing Configuration in the Correct VDOM

The source backup contains multiple VDOMs. To inspect one, select an existing VDOM before displaying its SD-WAN, address objects, routing, and firewall policies. VDOM_A below is an anonymized label: replace it with a verified existing name, since edit with a new name can create a VDOM. These commands inspect configuration; they do not enable VDOM mode or define a new segmentation design.

config vdom
    edit "VDOM_A"
        show
        show system settings
        show system sdwan
        show firewall address
        show firewall policy
        show router static
        show full-configuration system sdwan
    next
end

Within the selected VDOM, show displays configuration with default settings generally omitted. show full-configuration includes defaults; the scoped command above limits that output to SD-WAN. Repeat the review in the VDOM containing the SLA example rather than combining settings from different domains. These are command examples, not captured live output. Sanitize configuration output before sharing it, including encrypted credential fields.

Operational Verification

Run these read-only commands in the relevant VDOM on the target FortiOS 7.4.x release. Compare configured members and rules with live health and route selection; the backup alone contains no runtime evidence of successful failover.

show system sdwan
show firewall policy
get router info routing-table all
diagnose sys sdwan health-check
diagnose sys sdwan service4

In this adaptation, config system sdwan replaces the older configuration namespace, the manual rule uses priority-members, and virtual-wan-link remains the SD-WAN zone referenced by the firewall policy. See Fortinet's 7.4 SD-WAN configuration guide and diagnostic command reference.

Release selection: 7.4.x identifies the example syntax, not a recommended patch level. Before deployment, select a supported maintenance release for the exact hardware using the release notes, Fortinet PSIRT advisories, and the upgrade path tool. No release is guaranteed to be free of vulnerabilities; validate the adapted configuration and failover behavior on the selected build.

Growth Planning

This expansion combined new distribution capacity with reused access equipment and fiber infrastructure. The practical growth plan was to add access where needed while preserving the existing security edge and extending from the Floor 4 rack. The trade-off is that retained access interfaces keep their original limits; a faster distribution layer does not make every downstream link faster.

For the next expansion, I would review free ports, PoE budgets, uplink utilization, spare fiber pairs, and capacity after a chassis or link failure. I would also include the budget for power resilience explicitly. The Floor 2 StackPower decision is a useful reminder that data-plane redundancy and power resilience are separate design choices.

Design Improvement I Would Apply Today

During the project, I recommended a further phase to strengthen the WAN edge and improve service recovery. The site operated 24/7, and the Change Advisory Board (CAB) and executive leadership did not authorize the proposed production changes. Operational continuity, internal policy, and budget constraints prevented this phase from proceeding. The following describes the design I proposed and how I would refine it today; it was not part of the completed campus expansion.

WAN Aggregation and Firewall Connectivity

The first objective was to replace the existing SG300 WAN aggregation switch, a shared point of failure ahead of the FortiGate HA pair, with a dedicated redundant switching block. ISP handoffs would remain upstream of the firewalls: ISP circuits → isolated WAN aggregation → FortiGate HA → trusted Cisco campus switching. This would preserve a clear security boundary while reducing dependence on a single WAN handoff switch.

My initial cabling proposal called for eight links: four on the WAN side and four on the LAN side. These were separate functional groups, not one eight-member EtherChannel or unidirectional inbound and outbound links. The final allocation across HA units and switch members would need to be documented in the port schedule and validated against interface availability, throughput, and the capacity required after a failure.

Each untagged ISP handoff would use an access port in that provider's VLAN. Trunks toward the firewall would carry only the required tagged WAN VLANs, with matching firewall subinterfaces. A provider delivering a tagged service would need a matching handoff configuration. Separate LAN-side aggregates would connect the firewalls to trusted campus switching; ISP VLANs would not be extended into the core as a path around the firewall.

Each switch-side LACP bundle would terminate on one FortiGate unit. Ports leading to both HA units would not be combined into one EtherChannel. Across a C9300 StackWise stack, this is a cross-stack EtherChannel; across the C9500 StackWise Virtual pair, it is a Multi-Chassis EtherChannel (MEC). LACP negotiates the bundle, while the forwarding hash distributes traffic. A single flow does not normally use the sum of all member speeds. Physical route and power diversity would still need to be addressed separately.

VDOM Separation and Application Policy

I also planned separate FortiGate virtual domains: an Internet VDOM and a VDOM for critical services. Each would have its own interfaces, routing, and security policies, subject to the platform's VDOM support and licensing. Any communication between them would use explicitly controlled inter-VDOM connectivity. This provides logical separation while retaining shared hardware and its associated failure domain.

The policy design included web filtering, application control, and service-specific access rules for selected customer applications. Where supported by the chosen inspection feature, I considered pattern-based matching, including regular expressions. Those rules would need application testing and an appropriate inspection strategy for encrypted traffic; they would complement explicit source, destination, and service controls.

AWS Recovery Option for Critical Services

For critical migrations and selected customer services, I considered an AWS recovery environment connected to the on-premises FortiGate edge through redundant IPsec tunnels. The cloud firewall role would require a virtual appliance such as FortiGate-VM for AWS, rather than a physical FortiGate 600 in AWS. This remained a recovery concept; the cloud service model, appliance sizing, and availability design were not finalized.

AWS Site-to-Site VPN provides two tunnels per connection. Both would need routing and failover configuration. In that model, the tunnels terminate on the AWS VPN gateway service; routing would then steer protected traffic through the cloud firewall before it reaches the EC2 workloads, with a corresponding return path. Tunnels terminated directly on FortiGate-VM would be a different design and would not automatically receive the managed service's two-tunnel arrangement. Two tunnels also do not remove a shared on-premises ISP, device, or power failure.

EC2 workloads would sit in protected subnets behind the inspection path. Security groups would restrict each workload to its required peers and service ports, while subnet network ACLs would provide additional controls. Security groups are stateful; network ACLs are stateless and require explicit return-traffic rules. Firewall policy and VPC routing would need to preserve inspection in both directions, rather than assume that placing a firewall in the VPC makes every flow pass through it. AWS documents the distinction between these controls.

Connectivity alone would not make this a disaster recovery solution. The next design steps would define recovery time and recovery point objectives, data replication and backups, application dependencies, recovery capacity, and tested failover and failback procedures. A production migration would also need an approved change window, rollback criteria, and named operational owners. None of this recovery phase reached implementation or validation because the required policy, budget, and management approvals were not granted.

LACP Port-Channels from the Cisco Core Stack to the FortiGate Firewalls

This member-interface example shows the proposed eight-link allocation across three Cisco StackWise members: four WAN-side links and four LAN-side links, organized into four two-member LACP bundles. It illustrates a shared switching arrangement, distinct from the dedicated WAN switching option discussed above. WAN transport VLANs must remain isolated from trusted LAN VLANs, with no routing or bridging that bypasses the firewalls. The three-member numbering is not the interface layout of the two-chassis C9500 StackWise Virtual pair in the campus diagram.

• Port-channel101 (WAN) and Port-channel100 (LAN): members 1 and 2, assigned to the first firewall in this example.
• Port-channel103 (WAN) and Port-channel102 (LAN): members 2 and 3, assigned to the second firewall in this example.

Each bundle must terminate on one firewall unit. The repeated FW-601-E descriptions below are the original example labels; the implementation port schedule must distinguish the two units. Member 2 participates in all four bundles, so its loss leaves each bundle with one surviving link. Remaining capacity and minimum-link settings must support that failure condition.

This excerpt configures physical members only. Before deployment, define Port-channel100 through Port-channel103, restrict each trunk to its intended WAN or LAN VLANs, and match the FortiGate aggregates and subinterfaces. Verify actual interface numbering and supported HA/LACP behavior for the selected platform and release. See Fortinet's HA active/passive cluster setup guide.

interface TenGigabitEthernet1/0/1
 description FortigateFW-601-E-to-Wan
 switchport mode trunk
 channel-protocol lacp
 channel-group 101 mode active
!
interface TenGigabitEthernet1/0/2
 description Lan-To-FortigateFW-601-E
 switchport mode trunk
 channel-protocol lacp
 channel-group 100 mode active

interface TenGigabitEthernet2/0/1
 description FortigateFW-601-E-to-Wan
 switchport mode trunk
 channel-protocol lacp
 channel-group 101 mode active
!
interface TenGigabitEthernet2/0/2
 description Lan-To-FortigateFW-601-E
 switchport mode trunk
 channel-protocol lacp
 channel-group 100 mode active
!
interface TenGigabitEthernet2/0/3
 description Lan-To-FortigateFW-601-E
 switchport mode trunk
 channel-protocol lacp
 channel-group 102 mode active
!
interface TenGigabitEthernet2/0/4
 description FortigateFW-601-E-to-Wan
 switchport mode trunk
 channel-protocol lacp
 channel-group 103 mode active

interface TenGigabitEthernet3/0/1
 description FortigateFW-601-E-to-Wan
 switchport mode trunk
 channel-protocol lacp
 channel-group 103 mode active
!
interface TenGigabitEthernet3/0/2
 description Lan-To-FortigateFW-601-E
 switchport mode trunk
 channel-protocol lacp
 channel-group 102 mode active