MahaffeyNAS - WAN UDP/MTU Blackhole Runbook

MahaffeyNAS - WAN UDP/MTU Blackhole Runbook

Background

The law firm's WAN connection silently drops UDP packets above a certain size threshold. This has been observed twice, both times presenting as Tailscale/remote GUI access to the NAS (mahaffeynas, 100.75.148.52) becoming unreliable or completely unresponsive after a few minutes of use, while local LAN access remains largely unaffected.

This is consistent with a DS-Lite or MAP-T style IPv4-over-IPv6 transition mechanism on the ISP side, which adds encapsulation overhead and reduces the usable MTU below the standard 1500 bytes. When client traffic (especially larger GUI payloads over Tailscale, which adds its own WireGuard encapsulation overhead) exceeds the real usable MTU, packets get silently dropped rather than triggering a clean fragmentation response. This produces hangs and timeouts instead of clear errors.

Confirmed blackhole boundary as of two incidents:

• First incident: usable up to roughly 1268-1288 bytes on the wire
• Second incident: usable at 1188 bytes on the wire (tested via Tailscale tunnel), failed at 1200 bytes

The exact boundary may shift slightly depending on path/tunnel overhead at the time of testing. Treat any figure as approximate, not fixed.

Part 1: Remote Workaround (use when you cannot get onsite)

This is a temporary fix. It does not solve the underlying problem. On macOS it is session-only and will need to be reapplied any time the affected Tailscale interface resets (sleep/wake, reconnect, network change). On Windows it can be set to persist across reboots, but should still be revisited if the boundary shifts (see notes at the end of this section).

Steps are provided separately for macOS and Windows below. Use whichever matches the machine you are troubleshooting from.

macOS

Step 1: Confirm the blackhole is present

From the affected Mac, run fragmentation-disabled pings at decreasing sizes against the NAS's Tailscale IP (100.75.148.52):

ping -D -s 1300 100.75.148.52
ping -D -s 1200 100.75.148.52
ping -D -s 1180 100.75.148.52
ping -D -s 1150 100.75.148.52

• If a size fails immediately with "Message too long," that size exceeds the local Tailscale interface MTU and never left the machine. Not evidence of the blackhole itself.
• If a size sends successfully but times out with no reply, that is the blackhole in action.
• Narrow down until you find the largest size that consistently returns replies with 0% packet loss.

Step 2: Identify the active Tailscale interface

ifconfig | grep -B4 "100\."

Look for the utun interface that lists your Tailscale IP (100.x.x.x) in its output. Note the interface name (e.g. utun7).

Step 3: Lower the MTU on that interface

sudo ifconfig utun4 mtu 1180

Replace utunX with the interface identified in Step 2. Use a value comfortably below the failure point identified in Step 1 (1180 was the last confirmed-safe value as of August 2026, but always confirm against current test results first, since the boundary has shifted between incidents).

Step 4: Retest

Reload the DSM web GUI (https://mahaffeynas:5001 or the Tailscale IP directly) and confirm pages load fully and consistently, not just partially.

macOS Notes

• This fix applies only to the machine it was run on. It does not help office staff on the local LAN, who are typically unaffected by this issue.
• This will reset on sleep/wake, VPN reconnect, or network change, and will need to be reapplied.

Windows

Use Windows PowerShell directly (not WSL) for these steps. WSL runs in its own network namespace and does not have direct control over the Windows-level Tailscale virtual adapter.

Step 1: Confirm the blackhole is present

Windows ping uses different flags than macOS/Linux: -f (don't fragment) instead of -D, and -l (payload size) instead of -s. The numeric values translate directly, no adjustment needed.

ping -f -l 1300 100.75.148.52
ping -f -l 1200 100.75.148.52
ping -f -l 1180 100.75.148.52
ping -f -l 1150 100.75.148.52

• If a size exceeds the local interface MTU, Windows responds with "Packet needs to be fragmented but DF set." Not evidence of the blackhole itself.
• If a size sends successfully but returns "Request timed out," that is the blackhole in action.
• Narrow down until you find the largest size that consistently returns replies with 0% loss.

Step 2: Identify the active Tailscale interface

netsh interface ipv4 show interfaces

Look for the entry named "Tailscale" in the Name column. Example observed output:

Idx     Met         MTU          State                Name
---  ----------  ----------  ------------  ---------------------------
 52           5        1280  connected     Tailscale

Note that Tailscale's default interface MTU on Windows is 1280, not 1500 - this is expected and separate from the WAN blackhole issue.

Step 3: Lower the MTU on that interface

netsh interface ipv4 set subinterface "Tailscale" mtu=1180 store=persistent

Use the exact interface name found in Step 2 (it may not always be exactly "Tailscale"). Use a value comfortably below the failure point identified in Step 1.

Step 4: Verify it applied

netsh interface ipv4 show subinterfaces

Confirm the MTU column now shows the new value for the Tailscale interface.

Step 5: Retest

Reload the DSM web GUI and confirm pages load fully and consistently, not just partially.

Windows Notes

• store=persistent means this setting survives reboots, unlike the macOS fix. This is more durable, but also means it will not self-correct if the blackhole boundary shifts again later. Revisit if symptoms return even with this applied.
• This fix applies only to the machine it was run on. It does not help office staff on the local LAN, who are typically unaffected by this issue.

General Notes (both platforms)

• This is a stopgap only. Do not treat repeated use of either workaround as a substitute for the permanent fix in Part 2.
• The confirmed-safe boundary has shifted between incidents (roughly 1268-1288 bytes in the first incident, 1180-1188 bytes in the second). Always re-run the Step 1 test rather than assuming a previously-documented value still holds.

Part 2: Onsite Permanent Fix Checklist

Goal: identify and correct the actual WAN-side MTU/encapsulation issue at the firm's router so this stops recurring.

Step 1: Identify the WAN connection type

Log into the firm's router/gateway admin panel directly (not through DSM, not through Tailscale).

• Locate the WAN/Internet connection status page.
• Look specifically for any of the following terms: DS-Lite, MAP-T, MAP-E, 464XLAT, CGNAT, Dual-Stack Lite.
• Note the ISP name and the exact connection type shown.

Step 2: Check negotiated MTU values

• Check what MTU the router's WAN interface reports as negotiated/active.
• Compare this against the router's own LAN-side or DHCP-advertised MTU to client devices.
• A mismatch between the two (router downstream advertising 1500 while the actual WAN path only supports something smaller) is the likely root cause.

Step 3: Test from a device with a direct local connection (no VPN/Tailscale in the path)

This isolates the router/WAN path from any tunnel overhead, giving a cleaner number than testing through Tailscale.

ping -D -s 1472 8.8.8.8

(1472 + 28 bytes of ICMP/IP header = 1500 total, the standard MTU ceiling.) Step this down in increments (1400, 1300, 1200, etc.) until replies succeed consistently, to find the true WAN-side ceiling independent of any tunnel overhead.

Step 4: Apply a permanent fix

Depending on what Step 1 and Step 2 reveal, one of the following will likely apply:

• If the router allows manually setting a lower WAN MTU to match the actual DS-Lite/MAP-T tunnel overhead, set it explicitly rather than relying on auto-negotiation.
• If the router supports MSS clamping, enable it so TCP connections automatically negotiate a safe segment size, which helps with TCP-based traffic (web GUI, SMB, etc.) even if UDP/Tailscale still needs individual handling.
• If the ISP-provided router/ONT is limited in its configuration options, consider whether a router swap (e.g., to something like pfsense, which is already in use in the homelab environment and known to handle this kind of configuration well) is a reasonable recommendation for the client.

Step 5: Confirm the fix

• Repeat the ping boundary test from Step 3, confirming no drop at 1472 bytes (or as close to full 1500 MTU as the WAN type allows).
• Repeat the same test over Tailscale from a remote client to confirm both the raw WAN path and the tunnel overhead are now within safe limits.
• Have someone access DSM's web GUI remotely for an extended session (10+ minutes, multiple page loads) to confirm no recurrence of the hang/timeout symptoms.

Step 6: Document the outcome

Record the following for future reference:

• Confirmed WAN connection type (DS-Lite, MAP-T, etc.)
• The MTU value that was configured
• Whether MSS clamping was enabled
• Date fix was applied and by whom
• Confirmation test results

Quick Reference Summary