INFO:
Method of Procedure (MOP)
Upgrade Dell PowerSwitch S4148F-ON (VLT pairs) from OS10 10.5.6.x to OS10 10.6.1.3P1
Item | Details |
Purpose | Fix the current Critical and High security findings on the CALEA ToR switches by moving to the Dell OS10 release that contains the fixes. |
Devices | Dell PowerSwitch S4148F-ON ToR switches, deployed in VLT pairs. |
Current version | OS10 10.5.6.x on all switches (confirm on each switch in Step A2). |
Upgrade path | 10.5.6.x → 10.6.1.3P1 in ONE step. Dell supports this direct upgrade for VLT pairs, so no in-between version is needed. |
Target version | OS10 10.6.1.3P1. Do NOT use plain 10.6.1.3: Dell pulled it because of a functional issue. |
Expected impact | Each switch reboots once. Because the pair is VLT, traffic keeps flowing through the other switch. Short blips are possible, so book a maintenance window. |
Time needed | Prep (no outage): about 45 min per pair. Maintenance window: about 60 to 90 min per pair. |
Back-out | Switch back to the old version that is still saved on the switch (Section D). About 15 min per switch. |
Prepared by / Date | ______________________ / ____________ |
Plain-English summary: Every OS10 switch has two "slots" for the operating system: the one it is running now (active) and a spare (standby). I load the new version into the spare slot ahead of time while the switch keeps running normally. During the window I tell each switch to start from the spare slot and reboot it, one switch at a time. The old version stays in the other slot, so going back is quick.
A. Preparation (before the window, no outage)
A1. Download the software
Sign in at dell.com with the account that holds the switch entitlement, then go to My Account → Software Licenses and Subscriptions (this is Dell Digital Locker).
Open the entitlement (order number) for the switches, then go to the Available Downloads tab. Set the page size to 75 per page so the list is easier to scroll.
Download the OS10 10.6.1.3P1 switch installer and its Release Notes. The file name looks like PKGS_OS10-Enterprise-10.6.1.3P1…bookworm-installer-x86_64.bin. Also note its SHA-256 checksum from the download page.
Wrong file: OS10_Virtualization_10.6.1.3P1….zip ("GNS3 Bundle") is the virtual lab switch. It will NOT install on a real S4148F-ON.
Download the ONIE Firmware Updater release notes for the S4100-ON series (see A3).
Put the files on an SCP/SFTP server that the switch management interfaces can reach. Write down the file names exactly.
If the download is not listed: the entitlement or support contract may have expired, or the files may sit under a different order number. Open a Dell support case and ask for access to 10.6.1.3P1.
A2. Confirm every switch is on 10.5.6.x
On each switch, run the command below and write the exact version in the tracking table.
show version
Switch name | VLT role (Primary / Secondary) | Current version (10.5.6.__) | Upgraded to 10.6.1.3P1 (✓ / time) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Stop if: any switch shows a version that is NOT 10.5.6.x. The one-step path in this MOP only applies to 10.5.6.x. Any other version needs extra upgrades first.
Good to know: 10.5.6.x runs on Debian 10 ("buster") and 10.6.x runs on Debian 12 ("bookworm"). It is still a normal OS10 upgrade, but the underlying Linux changes, which is one more reason to test on the lab switch first.
A3. Check the ONIE firmware (one-time check)
For the S4100-ON family (which includes the S4148F-ON), Dell's upgrade order is: ONIE update → ONIE firmware update → SSD update → OS10 upgrade. Read the S4100-ON ONIE Firmware Updater release notes. If they say the switches need a newer firmware updater, do that first in its own step, test it on the lab switch, and note that it adds one extra reboot per switch.
A4. Prove it on the lab switch first
Follow Section B and Section C on the test switch, using the same image file.
Confirm that the switch boots, the config loads, the interfaces come up, OME still sees the switch, and SNMP/syslog still work.
Practice the back-out (Section D) once, so it is familiar and its timing is known.
Note: from 10.6.1.3 onward, show license status shows "Virtual License Applied". This is expected and not an error.
B. Pre-stage the new version (can be done days ahead, no outage)
Do this on BOTH switches in the pair. It loads the new version into the spare slot only. Nothing changes on the running switch until I reboot it in Section C. This is what makes the window short.
Save and back up the config to the SCP server (use a different file name for each switch):
write memory
copy running-configuration scp://<user>:<password>@<server>/<path>/<switchname>-pre-upgrade.txt
Download the image onto the switch, then watch the progress until the transfer shows success:
image download sftp://<user>:<password>@<server>/<path>/<10.6.1.3P1-file>.bin
show image status
If SCP/SFTP will not connect: a common fix is allowing the older ssh-rsa host key on the file server (HostkeyAlgorithms +ssh-rsa). You can also use http:// or a USB stick (usb://).
Make sure the file is not damaged: compare its checksum with the SHA-256 value from Dell's download page. They must match exactly.
system "sha256sum /var/os10-image/<10.6.1.3P1-file>.bin"
Install the image into the spare (standby) slot. This usually takes about 30 minutes. Keep running the status command until it shows "Completed: Success":
image install image://<10.6.1.3P1-file>.bin
show image status
Confirm that the standby slot now holds 10.6.1.3P1 and that the active slot still holds 10.5.6.x:
show boot detail
Do not run "boot system standby" or "reload" yet. Those happen inside the maintenance window.
C. Maintenance window: upgrade the pair one switch at a time
C1. Pre-checks (both switches)
Find out which switch is Primary and which is Secondary, and confirm that the VLT link (VLTi, usually Port-channel 1000) is Up:
show vlt all
Confirm that the backup (heartbeat) link shows Peer Heartbeat Status = Up:
show vlt all backup-link
Save a "before" snapshot so I can compare afterwards. Paste the output into a text file:
show running-configuration | no-more
show interface status | no-more
show port-channel summary
show lldp neighbors
Check that there are no active alarms in OME, and tell the NOC and the CALEA team that work is starting.
C2. Upgrade the SECONDARY switch first
Tell the switch to start from the new version next time, and confirm that Next-Boot points to standby (10.6.1.3P1):
boot system standby
show boot
Reboot the switch. The other switch carries all the traffic while this one restarts (about 5 to 10 minutes):
reload
Log back in and confirm the new version and that VLT is back Up with its peer:
show version
show vlt all
show vlt all backup-link
Compare the interfaces and port-channels against the "before" snapshot. Wait about 5 minutes and confirm that everything is stable.
Rule: while the two switches run different versions, make NO configuration changes on either switch.
C3. Upgrade the PRIMARY switch
Repeat C2 steps 1 to 4 on the Primary switch. During its reboot, the already-upgraded switch carries the traffic.
C4. Final checks (both switches)
Both switches show 10.6.1.3P1, VLT is Up, the heartbeat is Up, and all expected ports and port-channels are Up.
Servers and uplinks can reach their gateways, and the CALEA / lawful-intercept team confirms that their sessions and mirrors work.
OME shows both switches as healthy, with the new version in inventory.
Save the config and take an "after" backup:
write memory
copy running-configuration scp://<user>:<password>@<server>/<path>/<switchname>-post-upgrade.txt
Ask the CSO / security team to re-scan the switches to confirm that the findings are closed.
D. Back-out plan (if something goes wrong)
The old 10.5.6.x version is still saved in the other slot, so going back is the same steps in reverse. Back out only the switch that has the problem.
Point the switch back to the old slot, confirm the change, and reboot:
boot system standby
show boot
reload
Confirm that 10.5.6.x is running and VLT is Up. If the config does not look right, compare it with the "pre-upgrade" backup file and open a Dell support case before making changes.
Stop the upgrade on the second switch and record what happened for Dell support.
Back-out triggers: the switch does not boot, VLT does not come back Up within 15 minutes, important ports or port-channels stay down, or the CALEA team reports that intercept is broken.
E. After the upgrade: turn off risky services
Telnet, the REST API and SNMP v1/v2c are currently enabled. Do this only after BOTH switches are upgraded, and try it on the lab switch first.
configure terminal
no ip telnet server enable
end
write memory
Telnet: turn it off. Use SSH only. Dell's April 2026 security update (10.6.0.8) fixed a well-known Telnet flaw (CVE-2026-24061, scored 9.8 out of 10) that attackers use to get full control. Turning Telnet off removes that risk no matter which version is running.
REST API and SNMP v1/v2c: before turning these off, check how OME and any monitoring tools talk to the switches. If OME uses SNMP v2c, move it to SNMP v3 first, then remove the old community strings. If nothing uses the REST API, disable it ("no rest api restconf").
Limit who can log in: only allow SSH, SNMP and the API from the management/jump-host network, using a control-plane access list.
F. Note on Dell OME
OME is useful here for inventory, health and confirming versions before and after. Dell offers OS10 as an OME-friendly update package (DUP) only for some models (E3200-ON, N3200-ON and S3248T-ON), and the S4148F-ON is not on that list. So this MOP does the upgrade from the switch command line, and I use OME to check the results.
Before production: run every command exactly as written on the lab switch and check it against the 10.6.1.3P1 Release Notes. Command output can look slightly different between releases.
Comments
Post a Comment