Storage Area Network & Fabric Engineering — Fibre Channel, iSCSI and NVMe-oF
Dual-fabric SANs with zoning a human can read, multipathing proven by actually pulling a path, and NVMe-oF over lossless Ethernet that does not drop a frame under load.
What this usually looks like
Storage performance problems get blamed on the array. The fabric is the last place anyone looks — and a surprising share of the time, the fabric is the answer.
- Both fabrics exist, but half the hosts are silently single-pathed because somebody forgot the second HBA zone.
- Multipath is installed and has never been failed over, so nobody knows whether the second path actually carries IO.
- NVMe-oF or RoCE was deployed on the same Ethernet as everything else, with no PFC and no ECN, and it works "mostly".
- Zoning was inherited from an integrator three engineers ago, so no one will touch it — and no one can prove what depends on it.
Three ways to buy this
Audit if you want to know how bad it is. Build if you have the team and want the capability. Managed if you don't want to own the pager. Most engagements start with the audit and decide afterwards.
Audit
A fixed-price health check — the cheapest way in
- Fabric topology, ISL sizing and oversubscription review across both fabrics
- Zoning audit: single-initiator/single-target compliance, and every deviation explained
- Host-side path audit — real path counts, queue depths, ALUA state, and path-policy correctness
- Latency and throughput baseline per path, not just per array
Build
One-off delivery: design, code, hardening, handover
- Dual-fabric SAN (FC, iSCSI or NVMe-oF) with zoning generated from a source of truth rather than hand-typed
- Fibre Channel fabric: VSANs, ISLs, F_Port trunking, and FC-NVMe where the HBAs and array support it
- NVMe-oF over RDMA (RoCE v2) or TCP, with a properly configured lossless Ethernet domain — PFC, ECN and DCQCN, not just jumbo frames
- Host-side multipathing via dm-multipath (or native NVMe multipath) with tested path-failover policy and sane queue and timeout values
- MC-LAG / MLAG dual-homing for iSCSI and NVMe/TCP storage networks
- SAN monitoring path by path, with optics health and CRC error counters alarmed
Managed
Build plus 7×24 on-call, SLA and quarterly drills
- Quarterly path-failover drill: pull a cable or disable a switch port, and report the measured outcome
- Firmware and zoning review for both fabrics as arrays and hosts change
- Optic and transceiver health tracking before degradation becomes an outage
- Path-level latency regression detection against the stored baseline
What we actually deploy
Specific tooling, not categories. If a version matters to your decision, ask — we will tell you which one we would pick and why, including when the answer is the older one.
Fibre Channel
- Cisco MDS (NX-OS 9.x)
- Brocade / Broadcom FOS 9.x
- 32G / 64G FC
- VSANs
- ISL / IFL
- F_Port trunking
- NPIV
- zone merge management
- FC-NVMe
iSCSI
- open-iscsi (iscsid)
- CHAP authentication
- iSCSI offload HBAs
- jumbo frames (MTU 9000)
- target portals
- MC-LAG dual-homing
- LACP with correct hashing
NVMe over Fabrics
- NVMe/RDMA (RoCE v2)
- NVMe/TCP
- NVMe/FC
- kernel nvme-tcp / nvme-rdma
- SPDK targets
- ConnectX-6 / ConnectX-7
- namespace sharing and ANA
Host multipathing
- dm-multipath
- ALUA (implicit / explicit)
- path_selector (queue-length / service-time)
- path_grouping_policy
- queue_if_no_path and no_path_retry tuning
- native NVMe multipath
- udev rules and SCSI timeout tuning
Lossless Ethernet & QoS
- PFC (priority flow control)
- ECN + DCQCN
- DSCP marking and trust boundary
- ETS / strict priority queues
- buffer and headroom tuning
- per-priority pause verification
HBAs & adapters
- Marvell QLogic
- Broadcom Emulex LPe
- NVIDIA / Mellanox ConnectX
- driver and firmware matrix validation
- systool / lspci cross-check
Array integration
- Pure FlashArray
- NetApp ONTAP
- Dell PowerStore / PowerVault ME
- Hitachi VSP
- IBM FlashSystem
- Huawei OceanStor
- software-defined block: Ceph RBD, LINSTOR, Mayastor
Validation & diagnostics
- fio (libaio / io_uring)
- vdbench
- ioping
- nvme-cli
- multipath -ll
- fcping / fctraceroute
- sanbulk-style IO validation
- ib_write_bw / ib_send_bw
- optic DDM/DOM readings
What we will be measured on
These are the numbers we put in the delivery report. If we cannot measure it, we do not promise it.
- A path-failover drill performed on purpose: IO continues with zero errors, and the measured switchover time is written down.
- One entire fabric taken out of service — every host keeps reading and writing through the surviving fabric, demonstrated.
- Zoning 100% single-initiator/single-target, or every exception documented with a reason you accept in writing.
- A lossless Ethernet domain for RDMA that shows zero pause-frame-induced drops under a sustained load test — not "it seems fine".
- A per-path latency baseline (p50 / p99), so a future regression is a measurement rather than an argument.
What we would build, and where the weak point is
Two fabrics on one power feed, or two fabrics sharing the same rack and the same upstream switch, is a single fabric with extra cabling. The host-side multipath layer is also a quiet single point of failure: misconfigured, it will happily keep sending IO down a path it should have abandoned.
marks a single point of failure we would raise with you rather than hide. Every architecture has one; the question is whether you know where it is.
What we don't do
Every vendor says yes to everything. Here is where our answer is no, or where someone else needs to be in the room. If this list looks wrong for your situation, tell us — we would rather argue about scope before the contract than after.
- Requires partner
Array hardware warranty, firmware release engineering and RMA
We integrate with your array vendor and gather the evidence for a case. We do not resell arrays or replace vendor support.
- Out of scope
Data recovery from failed media
Clean-room forensic work. We design so you never need it, and we will tell you which replica to fail over to instead.
- Requires partner
Legacy tape and archive fabric operations
We integrate a tape tier into the design if you have one; we do not operate tape libraries.
- Requires partner
Windows and ESXi host-side path policy as a primary deliverable
Linux host multipathing is ours end to end. For hypervisor-hosted paths we implement and hand over, working alongside your virtualisation team.
What it looked like last time
A dual-fabric SAN where the array was blamed for slow writes: the fabric review found an under-sized ISL and an FC port negotiating at half speed after a failing optic, while a third of the hosts were single-pathed without anyone noticing. Host multipath corrected, path failover drilled, p99 write latency roughly halved — with no storage purchased.
Anonymised under NDA. No client names, no logos, no invented metrics. Full case library on the work page.
Talk to the storage network engineer about your san & storage networking
Send us the architecture, or just describe what broke most recently. You'll get an engineer's answer, not a discovery questionnaire.
Talk to a storage network engineer: botonwa@gmail.com · +86 130 5181 3099