tests/cipher-and-key-mgmt-testing.txt

   1 Cipher suite (CCMP, TKIP, GCMP, ..) and key management testing
   2 ==============================================================
   3
   4 wpa_supplicant and hostapd include number of extensions that allow
   5 special test builds to be used for testing functionality related to
   6 correct implementation of IEEE 802.11. These extensions allow behavior
   7 to be modified and invalid operations to be performed to verify behavior
   8 of other devices in unexpected situations. While most of the testing
   9 extensions are focused on the fully automated testing framework with
  10 mac80211_hwsim (see tests/hwsim subdirectory), many of these can be used
  11 for over-the-air testing of the protocol as well.
  12
  13 Since some of the testing extensions can result in exposing key
  14 information or allowing non-compliant behavior, these changes are
  15 disabled in default wpa_supplicant and hostapd builds for production
  16 purposes. Testing functionality can be enabled by adding
  17 CONFIG_TESTING_OPTIONS=y into build configuration (hostapd/.config and
  18 wpa_supplicant/.config).
  19
  20
  21 Testing setup
  22 -------------
  23
  24 These tests can be run as black-box testing without having to modify the
  25 tested device at all or without knowing details of its
  26 functionality. The test commands in wpa_supplicant/hostapd control
  27 interfaces are used to perform unexpected operations and normal data
  28 traffic is used to verify reaction of the tested device to such
  29 operations.
  30
  31 In theory, the test functionality is available with most drivers
  32 supported by wpa_supplicant/hostapd, but the most reliable results are
  33 likely available through ath9k-based devices. If you are using something
  34 else, it is strongly recommended that you'll run the first tests with
  35 sniffer captures and verify that the test tools are behaving correctly.
  36
  37 wpa_supplicant is used to control a test device in station mode to test
  38 an AP and hostapd is similarly used to control a test device in AP mode
  39 to test a station.
  40
  41 Various data traffic generators could be used to test the behavior, but
  42 this document focuses on using ping to test unicast traffic and arping
  43 to test broadcast traffic. To keep things simple and to reduce
  44 interference from unrelated traffic, the steps here assume static IPv4
  45 addresses are used and IPv6 is disabled.
  46
  47 The tests here use WPA2-Personal for simplicity. WPA2-Enterprise and
  48 other cipher suites can also be tested for more complete coverage.
  49
  50 Example hostapd.conf for the test tool in AP mode:
  51
  52 driver=nl80211
  53 hw_mode=g
  54 channel=1
  55 ieee80211n=1
  56 interface=wlan0
  57 ctrl_interface=/var/run/hostapd
  58 ctrl_interface_group=adm
  59 ssid=test-psk
  60 wpa=2
  61 wpa_key_mgmt=WPA-PSK
  62 wpa_pairwise=CCMP
  63 wpa_passphrase=12345678
  64
  65 Example wpa_supplicant.conf for the test tool in station mode:
  66
  67 ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=adm
  68
  69 network={
  70     ssid="test-psk"
  71     key_mgmt=WPA-PSK
  72     psk="12345678"
  73 }
  74
  75 The examples in this document assume following IPv4 address
  76 configuration:
  77
  78 Test tool (either AP or station mode): 192.168.1.1/24
  79 Device under test: 192.168.1.2/24
  80
  81
  82 Data traffic tests
  83 ------------------
  84
  85 ping is used to test whether unicast frames go through on the data
  86 link. It should be noted that ping may need to use broadcast ARP at the
  87 beginning if the other device is not yet in the ARP table, so working
  88 broadcast and unicast connectivity may be needed to get this started.
  89
  90 Example:
  91
  92 $ ping -n -c 5 192.168.1.2
  93 PING 192.168.1.2 (192.168.1.2) 56(84) bytes of data.
  94 64 bytes from 192.168.1.2: icmp_seq=1 ttl=64 time=43.7 ms
  95 64 bytes from 192.168.1.2: icmp_seq=2 ttl=64 time=67.9 ms
  96 64 bytes from 192.168.1.2: icmp_seq=3 ttl=64 time=900 ms
  97 64 bytes from 192.168.1.2: icmp_seq=4 ttl=64 time=5.81 ms
  98 64 bytes from 192.168.1.2: icmp_seq=5 ttl=64 time=135 ms
  99
 100 --- 192.168.1.2 ping statistics ---
 101 5 packets transmitted, 5 received, 0% packet loss, time 4004ms
 102 rtt min/avg/max/mdev = 5.811/230.605/900.223/337.451 ms
 103
 104 This shows working unicast data connectivity.
 105
 106 $ ping -n -c 5 192.168.1.2
 107 PING 192.168.1.2 (192.168.1.2) 56(84) bytes of data.
 108
 109 --- 192.168.1.2 ping statistics ---
 110 5 packets transmitted, 0 received, 100% packet loss, time 4033ms
 111
 112 This shows not working unicast data connectivity.
 113
 114
 115 arping is used to test broadcast connectivity.
 116
 117 Example:
 118
 119 $ arping -b -I wlan0 192.168.1.2 -c 5
 120 ARPING 192.168.1.2 from 192.168.1.1 wlan0
 121 Unicast reply from 192.168.1.2 [<DUT MAC address>]  119.695ms
 122 Unicast reply from 192.168.1.2 [<DUT MAC address>]  144.496ms
 123 Unicast reply from 192.168.1.2 [<DUT MAC address>]  166.788ms
 124 Unicast reply from 192.168.1.2 [<DUT MAC address>]  2.283ms
 125 Unicast reply from 192.168.1.2 [<DUT MAC address>]  2.234ms
 126 Sent 5 probes (5 broadcast(s))
 127 Received 5 response(s)
 128
 129 This shows working broadcast data connectivity.
 130
 131 $ arping -b -I wlan0 192.168.1.2 -c 5
 132 ARPING 192.168.1.2 from 192.168.1.1 wlan0
 133 Sent 5 probes (5 broadcast(s))
 134 Received 0 response(s)
 135
 136 This shows not working broadcast data connectivity.
 137
 138 If testing results do not look consistent, the testing state can be
 139 cleared by disconnection and reconnecting the station (the test tool or
 140 the DUT) to the network.
 141
 142
 143 Sniffer and wlantest
 144 --------------------
 145
 146 It is useful to get a wireless sniffer capture from the operating
 147 channel of the AP to be able to confirm DUT behavior if any of the data
 148 tests indicate reason to believe something is not working as expected.
 149
 150 wlantest (from the wlantest directory of hostap.git) can be used to
 151 decrypt and analyze a sniffer capture. For example:
 152
 153 wlantest -r wlan0.pcap -n decrypted.pcap -p 12345678
 154
 155 The debug prints and comments in the generated file indicate where
 156 unexpected behavior has been detected, e.g., when the test tool ends up
 157 clearing its packet number to test replay protection. That can help in
 158 checking whether the DUT actually replies to a frame that it was
 159 supposed to drop due replay.
 160
 161
 162 Testing replay protection on a station device
 163 ---------------------------------------------
 164
 165 Start hostapd and use hostapd_cli on the test device to control testing
 166 operations. Connect the DUT to the network.
 167
 168 <3>AP-STA-CONNECTED <DUT MAC address>
 169
 170 This indicates that the connection was completed successfully.
 171
 172 Verify that broadcast and unicast traffic works correctly (if not,
 173 something is wrong in the test setup and that needs to be resolved
 174 before being able to run any tests).
 175
 176 Verify that unicast traffic works and issue the following command in
 177 hostapd_cli:
 178
 179 > raw RESET_PN <DUT MAC address>
 180 OK
 181
 182 Verify that unicast traffic does not work anymore. If it does, the DUT
 183 does not implement replay protection correctly for unicast frames. Note
 184 that unicast traffic can recover once the packet number from the test
 185 device increases beyond the value used prior to that RESET_PN command.
 186
 187
 188 Verify that broadcast traffic works and issue the following command in
 189 hostapd_cli:
 190
 191 > raw RESET_PN ff:ff:ff:ff:ff:ff
 192 OK
 193
 194 Verify that broadcast traffic does not work anymore. If it does, the DUT
 195 does not implement replay protection correctly for broadcast
 196 frames. Note that broadcast traffic can recover once the packet number
 197 from the test device increases beyond the value used prior to that
 198 RESET_PN command.
 199
 200
 201 Testing replay protection on an AP device
 202 -----------------------------------------
 203
 204 Start the AP (DUT) and start wpa_supplicant on the test device to
 205 connect to the network. Use wpa_cli to control the test device.
 206
 207 <3>SME: Trying to authenticate with <DUT MAC address> (SSID='test-psk' freq=5240 MHz)
 208 <3>CTRL-EVENT-REGDOM-CHANGE init=CORE type=WORLD
 209 <3>Trying to associate with <DUT MAC address> (SSID='test-psk' freq=5240 MHz)
 210 <3>Associated with <DUT MAC address>
 211 <3>WPA: Key negotiation completed with <DUT MAC address> [PTK=CCMP GTK=CCMP]
 212 <3>CTRL-EVENT-CONNECTED - Connection to <DUT MAC address> completed [id=0 id_str=]
 213
 214 Verify that unicast traffic works and issue the following command in
 215 wpa_cli:
 216
 217 > raw RESET_PN
 218 OK
 219
 220 Verify that unicast traffic does not work anymore. If it does, the DUT
 221 does not implement replay protection correctly. Note that unicast
 222 traffic can recover once the packet number from the test device
 223 increases beyond the value used prior to that RESET_PN command.
 224
 225 IEEE 802.11 protocol uses unicast frames in station-to-AP direction, so
 226 there is no need to test AP replay protection behavior separately with
 227 the broadcast IPv4 traffic (which would be converted to unicast frames
 228 on the link layer).
 229
 230
 231 Testing GTK reinstallation protection on a station device (group handshake)
 232 ---------------------------------------------------------------------------
 233
 234 Use the procedure describe above for testing replay protection, but with
 235 the following hostapd_cli commands:
 236
 237 Test broadcast connectivity; should work
 238
 239 > raw RESEND_GROUP_M1 <DUT MAC address>
 240 OK
 241 > raw RESET_PN ff:ff:ff:ff:ff:ff
 242 OK
 243
 244 Test broadcast connectivity; should not work; if it does, the device
 245 does not implement protection for delayed retransmission of Group Key
 246 Message 1/2.
 247
 248
 249 Testing GTK reinstallation protection on a station device (4-way handshake)
 250 ---------------------------------------------------------------------------
 251
 252 Use the procedure described above for testing replay protection for
 253 broadcast traffic, but with the following hostapd_cli commands:
 254
 255 Test broadcast connectivity; should work
 256
 257 > raw RESEND_M3 <DUT MAC address>
 258 OK
 259 > raw RESET_PN ff:ff:ff:ff:ff:ff
 260 OK
 261
 262 Test broadcast connectivity; should not work; if it does, the device
 263 does not implement protection for delayed retransmission of 4-way
 264 handshake EAPOL-Key Message 3/4.
 265
 266 Variant 1: Include extra Message 1/4
 267
 268 Otherwise same as above, but replace RESEND_M3 command with:
 269
 270 > raw RESEND_M1 <DUT MAC address>
 271 OK
 272 > raw RESEND_M3 <DUT MAC address>
 273 OK
 274
 275 Variant 2: Include two extra Message 1/4
 276
 277 Otherwise same as above, but replace RESEND_M3 command with:
 278
 279 > raw RESEND_M1 <DUT MAC address> change-anonce
 280 OK
 281 > raw RESEND_M1 <DUT MAC address>
 282 OK
 283 > raw RESEND_M3 <DUT MAC address>
 284 OK
 285
 286
 287 Testing TK reinstallation protection on a station device (4-way handshake)
 288 --------------------------------------------------------------------------
 289
 290 Use the procedure described above for testing replay protection for
 291 unicast traffic, but with the following hostapd_cli commands:
 292
 293 Test unicast connectivity; should work
 294
 295 > raw RESEND_M3 <DUT MAC address>
 296 OK
 297 > raw RESET_PN <DUT MAC address>
 298 OK
 299
 300 Test unicast connectivity; should not work; if it does, the device
 301 does not implement protection for delayed retransmission of 4-way
 302 handshake EAPOL-Key Message 3/4.
 303
 304 Variant 1: Include extra Message 1/4
 305
 306 Otherwise same as above, but replace RESEND_M3 command with:
 307
 308 > raw RESEND_M1 <DUT MAC address>
 309 OK
 310 > raw RESEND_M3 <DUT MAC address>
 311 OK
 312
 313 Variant 2: Include two extra Message 1/4
 314
 315 Otherwise same as above, but replace RESEND_M3 command with:
 316
 317 > raw RESEND_M1 <DUT MAC address> change-anonce
 318 OK
 319 > raw RESEND_M1 <DUT MAC address>
 320 OK
 321 > raw RESEND_M3 <DUT MAC address>
 322 OK
 323
 324
 325 Testing ANonce generation on an AP device
 326 -----------------------------------------
 327
 328 Start the AP (DUT) and start wpa_supplicant on the test device to
 329 connect to the network. Use wpa_cli to control the test device.
 330
 331 <3>SME: Trying to authenticate with <DUT MAC address> (SSID='test-psk' freq=5240 MHz)
 332 <3>CTRL-EVENT-REGDOM-CHANGE init=CORE type=WORLD
 333 <3>Trying to associate with <DUT MAC address> (SSID='test-psk' freq=5240 MHz)
 334 <3>Associated with <DUT MAC address>
 335 <3>WPA: Key negotiation completed with <DUT MAC address> [PTK=CCMP GTK=CCMP]
 336 <3>CTRL-EVENT-CONNECTED - Connection to <DUT MAC address> completed [id=0 id_str=]
 337
 338 Show the ANonce from the first 4-way handshake, request PTK rekeying,
 339 and show the ANonce from the second 4-way handshake:
 340
 341 > GET anonce
 342 df8c61d1f1f7aca9f1739dd888199547f4af2b8b07f8bf15b45ea271da0072b2
 343 > raw KEY_REQUEST 0 1
 344 OK
 345 > GET anonce
 346 d8ddcb716f28abfdf1352a05d51e7a70f58802122e99d13c730c3c0f09594aac
 347
 348 If the ANonce values are same, the AP did not update the ANonce for
 349 rekeying (it should have as shown in the example above).
 350
 351
 352 Testing FT Reassociation Request frame retransmission on an AP device
 353 ---------------------------------------------------------------------
 354
 355 This test case requires a sniffer to be used and manually analyzed.
 356
 357 Enable FT on the DUT AP (likely two AP devices needed), connect test
 358 tool to the AP using FT protocol (e.g., connect to another AP first and
 359 then use the "ROAM <BSSID>" command), and do the following steps:
 360
 361 - verify unicast traffic from the AP to test station (either ping from
 362   the AP or from a device behind the AP); this needs to work
 363 - wpa_cli "raw RESEND_ASSOC"
 364 - verify unicast traffic from the AP to test station (either ping from
 365   the AP or from a device behind the AP); this is likely to fail, but
 366   the real analysis is done based on the sniffer capture
 367
 368 In the sniffer capture, find the last Reassociation Request frame from
 369 the test station (this is more or less identical to the previous one and
 370 the only one that should not have Authentication frame exchange before
 371 it). Look at the last used PN in a unicast Data frame from the AP to the
 372 test station before the last Reassociation Request frame and the PN in
 373 the following unicast Data frame after the last Reassociation Request
 374 frame. If the PN goes down (e.g., is reset to 1), this would be a sign
 375 of a likely security vulnerability. The AP's TK configuration should be
 376 verified (i.e., whether it is configuring the same TK again and then
 377 allowing it to be used with reused PN values).