Advice to eduroam® Identity Providers and Service Providers following the release of Wi-Fi CERTIFIED #WPA3™ Security

Executive Summary

The biggest change in Wi-Fi CERTIFIED WPA3™ Security is about pre-shared key deployments, which are not in scope for eduroam IdPs. There are small but useful benefits for WPA3-Enterprise, and there is a new optional operation mode WPA3-Enterprise with 192-Bit Security with significant interoperability issues on the deployed base of eduroam WPA-Enterprise hotspots.

The only action, which typically does not require monetary investments at all, is to turn on support for Protected Management Frames (PMFs) and to turn off WPA1 in the existing eduroam SP network deployment.

WPA3-Enterprise with 192-Bit Security MUST NOT be configured.

Wi-Fi CERTIFIED WPA3™ is a trademark of Wi-Fi Alliance®.
eduroam® is a registered trademark and servicemark of GEANT Association.

Changes in WPA3-Personal with pre-shared keys

What was formerly a WPA2-PSK network (one passphrase protects the entire SSID) gets modernised. The passphrase remains as-is, but the algorithm to derive the Wi-Fi master session key from the user-supplied WPA passphrase gets changed and is now based on the
“Dragonfly” algorithm. It is called WPA3-SAE (Simultaneous Authentication of Equals). eduroam Identity Providers may know the Dragonfly algorithm for its appearance in the EAP type “EAP-pwd”.

Changes in WPA3-Enterprise
Changes for all WPA3-Enterprise deployment modes, including eduroam’s WPA-EAP

The changes are very small.

  • WPA3-Enterprise includes a roll-up of several late additions to WPA2-Enterprise certification. This ensures that WPA3-Enterprise certified devices are guaranteed to
    o be immune against the KRACK vulnerability
    o support Protected Management Frames (PMF)
    o validate a server certificate to a root CA, if such a root CA is configured. Note:

    • There is still not a requirement to actually have a CA certificate configured.
  • Support for Protected Management Frames (PMFs) is a requirement

There is no explicit “mixed mode”, nor is one required: a WPA3-Enterprise network is identical to a WPA2-Enterprise network which has configured support for Protected Management Frames (PMF). So long as PMFs are only configured as supported, rather than required, older WPA2 devices can continue to connect to the network as if it were a normal WPA2 network.

Only the backwards compatibility to WPA(1) is discontinued, which should not have any significant relevance in the eduroam environment any more. This makes it easy and useful for eduroam Service Providers to activate WPA3-Enterprise by marking Protected Management Frames as “supported, but not required” in their network equipment and by turning off support for WPA1.

Since most equipment supports PMFs since many years, typically, no new investment in Wi-Fi CERTIFIED WPA3™ hardware is required for this.

Considering that a BYOD environment such as eduroam typically has a very diverse set of user devices, including old devices which potentially do not support PMFs, it is considered premature to make PMFs required at this point in time. eduroam operations will revisit this position in due time.

The new operation mode WPA3-Enterprise with 192-Bit security

There is an optional new operation mode in Wi-Fi CERTIFIED WPA3™: WPA3-Enterprise with 192-Bit security. This operation mode is based on EAP just like the normal WPA3-Enterprise mode, but has further constraints regarding the permitted cipher suites; both during the TLS negotiation inside of tunnelling EAP methods [2] and for group ciphers on the wireless medium. The list of cipher suites and key lengths mirrors a regulatory requirement by a committee of various intelligence agencies in the United States of America[1] designed for their local government agencies and military operation (“Commercial National Security
Algorithm Suite”,CNSA). Despite being part of the Wi-Fi CERTIFIED WPA3™ certification, which suggests that it has an immediate effect only between client devices and access points, its security requirements would also induce a stringent required feature set on the RADIUS/EAP server equipment of eduroam Identity Providers[2]. Early studies in the eduroam IdP landscape indicate that the cipher suite and key length requirements are not met by a majority of eduroam IdPs at this point in time. Furthermore, EAP methods not based on TLS – notably EAP-pwd – are not permitted at all on WPA3-Enterprise with 192-Bit Security networks.

Since WPA3-Enterprise with 192-Bit Security is configured on the access point, but needs compatible EAP servers at the Identity Provider side, with no signalling between the Access Point and the eduroam Identity Provider server, there is a significant risk for interoperability issues inside eduroam.

WPA3-Enterprise with 192-Bit Security, if enabled, must be the only key management on a given SSID. In order to continue to serve client devices which are uncapable of 192-Bit Security, there must be two distinct SSIDs: ‘eduroam’ with the classic WPA3-Enterprise, and a new SSID for WPA3-Enterprise with 192-Bit Security

To work around those issues, there are three possibilities:

Option A: step-change upgrade sequence: all IdPs -> then SPs
A global upgrade in eduroam infrastructure would need to be done in two phases:

1) first, all eduroam Identity Providers globally upgrade their RADIUS/EAP servers to meet the IdP-side requirements of WPA3-Enterprise with 192-Bit Security

2) strictly only after that, eduroam Service Providers start to upgrade their Access Point configuration to WPA3-Enterprise with 192-Bit Security at their own pace

Option B: per-IdP regime of client configurations

All eduroam Identity Providers individually need to make sure that all their own users’ client configurations do not allow the respective device to connect to any WPA3-Enterprise with 192-Bit Security eduroam network until the eduroam Identity Provider has upgraded their authentication server to support the TLS cipher suites required by WPA3-Enterprise with 192-Bit Security (at which point in time its users can be signalled to change their configuration at their own pace).

eduroam Identity Providers not enforcing such a configuration restraint would subject their users to the aforementioned interoperability problems. Given the diverse implementation quality of EAP supplicants, the vastly varying richness of expression of configuration formats, and the fact that a significant fraction of users will not follow configuration advice (including connecting without pre-configuration to non-standard SSIDs), realising this modus operandi appears unrealistic. This option also still needs a new SSID, with the same legacy support reasoning as in Option A, above.

Option C: Do not transition to WPA3-Enterprise with 192-Bit Security
eduroam Identity Providers that are interested in the level of security that WPA3-Enterprise with 192-Bit Security brings can upgrade their RADIUS/EAP server to support only the three cipher suites in question at any time, while remaining compatible with the existing eduroam SP setup in WPA2-Enterprise and WPA3-Enterprise.
This will achieve almost the same result as WPA3-Enterrprise with 192-Bit Security by steering and enforcing cipher suite selection from the IdP-side, and without the interoperability problems of the actual change of operation mode. The only minor difference is then the group cipher on the medium.

Advice
Considering that the WPA3-Enterprise with 192-Bit Security operation mode’s primary use case is in one country outside its educational community, combined with the fact that option B above is unrealistic, and the significant deployment complexities of option A above,

eduroam currently pursues option C above, i.e. does not currently have any plans to engage in any transition in any way.

Due to that, until further notice, eduroam Service Providers are advised NOT to configure WPA3-Enterprise with 192-Bit Security. Doing so anyway will lead to hard to debug authentication failures for users of the majority of eduroam Identity Providers.

[1] https://www.cnss.gov – yes, the web site really has a certificate which is not in browser trust stores.
[2] EAP peers and servers need to support TLS1.2 with the cipher suites ECDHE-RSA-AES256-GCM-SHA384,
ECDHE-ECDSA-AES256-GCM-SHA384 and/or DHE_RSA_WITH_AES_256_GCM_SHA384. Further to this, where RSA keys are used, they need to be at least 3072 bit long; where ECDSA keys are used, they need to be based on curve P-384

Source : Download the original advisory notice

Understanding Various AP-IOS Flash Corruption Issues

 Introduction

This document describes flash corruption problems reported on IOS based Cisco Access Points(AP).

Requirements

Cisco recommends that you have basic knowledge of:

  • AireOS WLCs
  • Lightweight APs

 

Components Used

  • Cisco Aironet 1040, 1140, 1250, 1260, 1600, 1700, 2600, 2700, 3500, 3600, 3700, 700, AP801, and AP802 Series indoor access points
  • Cisco Aironet 1520 (1522, 1524), 1530, 1550 (1552), 1570, and Industrial Wireless 3700 Series outdoor and industrial wireless access points

Note: Problem is present AireOS 8.0.x to 8.5.x release trains. There is a much higher prevalence in Wave1 AP models like 1700/2700/3700  and 2600/3600  on this issue vs other AP types due to flash HW type.

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.

Background Information

Wave 1 APs can report no flash access or file system corruption, especially when upgrading the WLC.

The corruption can cause the following status on the AP

  • AP unable to save the configuration
  • AP unable to perform an upgrade
  • AP lose configuration
  • AP stuck in a booting loop
  • AP in ROMMON status –  console to the AP is need to recover the AP

Note: The issue is prelevant during upgrade scenarios where AP’s get hung/stuck, but not neccesarily is limited to WLC upgrade scenarios. AP’s may be working fine, servicing clients, etc, while on this problem state which is not easily detectable.

Symptoms of the problem

Flash Inaccessible

Command: show file system

The size and free space on flash would show as “-” instead of the actual value

Problem Output:

==============================
       Size(b)       Free(b)      Type  Flags  Prefixes				
*            -             -     flash     rw   flash:	
==============================

Normal output:

================================
       Size(b)       Free(b)      Type    Flags  Prefixes				
*     40900608       25282560     flash     rw     flash:
===============================

Command:  show flash

AP cannot display its files

Directory of flash:/
 %Error opening flash:/ (Invalid argument)

Command: show logging

AP can display de following messages.

*Mar  1 00:01:04.159: %LWAPP-3-CLIENTERRORLOG: Save LWAPP Config: error saving config file

.....


*Mar  1 00:01:04.159: %LWAPP-3-CLIENTERRORLOG: Load nvram:/lwapp_ap.cfg config
failed, trying backup...
*Mar  1 00:01:04.159: %LWAPP-3-CLIENTERRORLOG: Load nvram:/lwapp_ap.cfg.bak
config failed...

Command: fsck flash:

AP cannot verify its file system.

AP# fsck flash:
Fsck operation may take a while. Continue? [confirm]

%Error fscking flash: (Device or resource busy)

Note: This can occur in two scenarios: 1. When flash is inaccessible or 2. When the flash is still accessible, yet a file descriptor is leaked. Refer CSCvf28459 . The only workaround is to reload the AP.

 

Caution: When AP is in flash inaccessible state, the outcome of the reload is not always predictable. The AP may come back just fine or it may end up in the ROMMON or reboot loop at which point a console based recovery would be required. Please be prepared with console access to the site when performing this operation.

 

Corrupted Files

AP stuck in the following loop.

Loading "flash:/ap1g1-k9w8-mx.v153_80mr_esc.201603111020/ap1g1-k9w8-mx.v153_80mr_esc.201603111020"
...###############################################################################################
##################################################################################################
##################################################################################################
##################################################################################################
##################################################################################################
####################bad mzip file, unknown zip method?

Note: If You have a WLC with a different code than AP runs, forcing AP to join that WLC can fix the problem

 

Solution

Upgrade WLC as per the following document TAC Recommended AireOS Builds.

Caution: Before Upgrade, complete reading this document :

 

Before Upgrade

In order to identify affected AP on the network and fix them before an upgrade. Follow the next steps:

  • Download wlanpoller tool. (Python script.)
    • This is the Wireless Escalation tool (made by Federico Lovison @flovison)
  • Install the tool. For instructions look follow “README.md” file downloaded with the tool
  • Run script. For instructions look follow “README.md” file downloaded with the tool.

Tool Description

Every time the script is run it verifies whether an AP’s flash is accessible or not.

If it is accessible, it runs the command fsck flash:

  • If all is OK, move on to the next AP
  • else repeat the command up to 4 times

if it is inaccessible or command fsck flash is not successful after four times.

  • the script will flag AP on its final report in order to recover on the third run.

The script needs to be run three times.

  1. Run
    • The script will build MD5 database looking at MD5 checksum value for every file on the AP. The final MD5 value for a specific file is the one that has the more hits across same AP family on WLC.
  2. Run
    • The script starts comparing MD5 checksum values vs its database. If value matches then files is ok, if not then AP is flaged in order to recover on the third run
  3. Run
    • Enable recover flag on the configuration file.
    • The script triggers command test capwap image capwap only on the APs where the flash is accessible but some errors were found either fsck flash command failed  after 4 times and/or MD5 mismatch

Note: This recovery method causes the AP to reload once the image is downloaded and installed. Make sure you run it in a maintenance window.

 Tool Output

File: <timestamp>_ap_fs.csv – Summary of the checks executed on APs and their results

Columns description

  • ap_name: Name of the AP.
  • ap_type: AP model.
  • ap_uptime: Uptime for the AP (days).
  • ap_ios_ver: IOS version.
  • fs_free_bytes: Number of free bytes in flash file system.
  • flash_issue: True if any flash corruption has been observed.
  • fs_zero_size: True when flash hung has been detected file system showing “-“
  • fsck_fail: True if file system check has failed.
  • fsck_busy: True device or resource busy when doing flash fsck.
  • fsck_recovered: True when an error occurred on fsck but it is fixed in next fsck.
  • fsck_attempts: Number of attemps of fsck to recover the AP (max 4)
  • md5_fail: True when md5 at least one file is different from the stored in database.
  • rcv_trigger: True when AP tried to download image from WLC when issue has been detected and recovery has been enabled.

File: <timestamp>_ap_md5.csv   Details of the MD5 checksum values of all files (on all APs)

Columns description

  • ap_name: Name of the AP.
  • ap_type: AP model.
  • ap_uptime: Uptime for the AP (days).
  • filename: IOS image file name.
  • md5_hash: md5 value for filename.
  • is_good: True md5 value matches with value stored in db. False md5 mismatch observed for this file.
  • is_zero_bytes: True when filename has 0 bytes based on md5checksum so file is incorrect.
  • md5_error: Error message retriving md5 value if it was not possible to get md5 for filename.

Note: There could be scenarios where the WLANPOLLER recovery script is unable to recover certain AP’s and those AP remains flagged as failed in the report. In those scenarios, manual AP recovery by telnet/SSH/console into AP CLI is recommended. Please open TAC SR if you needed assistance on this process.

 

Reference Bug IDs

Bug ID Headline Symptoms Status Fixed releases
CSCuz47559  Error saving configuration file happens on Cisco Wave1 APs Flash hung Fixed 8.0MR5, 8.2MR6, 8.3MR3, 8.4+
CSCvd07423 AP firmware corrupt after power cycle bad mzip file, unknown zip method reboot loop Crash loop Fixed 8.3MR2, 8.5+
CSCvf16302 Flash on lightweight IOS APs gets corrupted AP stuck on rommon Fixed 8.5MR1, 8.3MR3(via CSCvg98786 )
CSCvf28459 Write of the Private File nvram:/lwapp_ap.cfg Failed on compare RCA needed (try = 1) Flash accessible but fsck not working Fixed  8.3MR4,8.2MR7,8.5MR1
CSCvg98786 IOS AP dtls flap issue seen in pre commit sanity Collateral Fixed 8.5MR1, 8.3MR3

 

source : Understanding Various AP-IOS Flash Corruption Issues

Updated:May 9, 2018
Document ID:213317