]>
Commit | Line | Data |
---|---|---|
9eadcfbf JM |
1 | Step-by-step guide for setting up hostapd/wpa_supplicant test framework |
2 | ----------------------------------------------------------------------- | |
3 | ||
4 | This document can be used as a quick guide for getting started with | |
5 | hostapd/wpa_supplicant test framework with mac80211_hwsim. While the | |
6 | example here uses Ubuntu 14.04.1 server to have a list of exact steps, | |
7 | there are no requirements for using that specific distribution in the | |
8 | testing setup. | |
9 | ||
10 | The steps here describe how to run a full Linux installation in a | |
11 | virtual machine with any host system (e.g., Linux, Windows, or OS X as | |
12 | the host and using kvm, VirtualBox, etc. for running the virtual guest | |
13 | system). For more advanced (and significantly faster and with more | |
14 | testing coverage) configuration on a Linux host system, parallel virtual | |
15 | machines can be used as an alternative setup. See tests/hwsim/vm/README | |
16 | for more details on that. | |
17 | ||
18 | ||
19 | Install Ubuntu Server 14.04.1 in the virtual machine | |
20 | ||
21 | - download installation image, e.g., | |
22 | http://releases.ubuntu.com/14.04.1/ubuntu-14.04.1-server-amd64.iso | |
23 | - use virtualization software specific steps to create a new VM and | |
24 | install the the guest system with default settings (i.e., no need to | |
25 | select any extra packages during initial installation) | |
26 | - if the host system has multiple CPU cores, it is likely a good idea to | |
27 | enabled at least two CPUs in the guest; 1024 MB of RAM should be enough | |
28 | for testing purposes | |
29 | - 8 GB of virtual hard driver should be fine for this purpose | |
30 | - boot to the installed operating system | |
31 | ||
32 | ||
33 | Install the prerequisite packages that may not have been installed by default | |
34 | ||
78bb62b2 MB |
35 | sudo apt-get install build-essential git libpcap-dev libsqlite3-dev binutils-dev libnl-3-dev libnl-genl-3-dev libnl-route-3-dev libssl-dev libiberty-dev libdbus-1-dev iw bridge-utils python-pyrad python-crypto tshark |
36 | ||
37 | optional: | |
38 | sudo apt-get install python-netifaces | |
9eadcfbf JM |
39 | |
40 | ||
41 | Install a recent kernel wireless components (mac80211_hwsim, mac80211, | |
42 | cfg80211) | |
43 | ||
44 | For this step, the kernel version may be updated, but the simpler option | |
45 | is to install the latest version of Backports package. For example: | |
46 | ||
47 | wget http://www.kernel.org/pub/linux/kernel/projects/backports/stable/v3.19-rc1/backports-3.19-rc1-1.tar.xz | |
48 | tar xJf backports-3.19-rc1-1.tar.xz | |
49 | cd backports-3.19-rc1-1 | |
50 | ||
51 | cat > defconfigs/mac80211_hwsim <<EOF | |
52 | CPTCFG_CFG80211=m | |
53 | CPTCFG_CFG80211_WEXT=y | |
54 | CPTCFG_MAC80211=m | |
55 | CPTCFG_MAC80211_LEDS=y | |
56 | CPTCFG_MAC80211_MESH=y | |
57 | CPTCFG_WLAN=y | |
58 | CPTCFG_MAC80211_HWSIM=m | |
59 | EOF | |
60 | ||
61 | make defconfig-mac80211_hwsim | |
62 | make | |
63 | sudo make install | |
64 | cd .. | |
65 | ||
66 | ||
67 | Update iw based on custom iw.git build | |
68 | ||
69 | Couple of the test cases expect iw to have support for requesting | |
70 | cfg80211 scan results to be flushed. That functionality is not included | |
71 | in the version that Ubuntu 14.04.1 includes (iw v3.4). Following steps | |
72 | can be used to replace that version with a custom build. This is | |
73 | optional, i.e., most test cases will work with the old iw version, but | |
74 | some test cases are skipped and some are more likely to fail if iw does | |
75 | not get updated. | |
76 | ||
77 | wget https://www.kernel.org/pub/software/network/iw/iw-3.17.tar.gz | |
78 | tar xf iw-3.17.tar.gz | |
79 | cd iw-3.17 | |
80 | make | |
81 | sudo mv /sbin/iw{,-distro} | |
82 | sudo cp iw /sbin/iw | |
83 | cd .. | |
84 | ||
85 | ||
86 | Update wireless-regdb | |
87 | ||
88 | Number of VHT and DFS test cases are skipped if the old wireless-regdb | |
89 | version from Ubuntu 14.04 (2013.02.13) is used. Following steps can | |
90 | optionally be used to update wireless-regdb to a newer snapshot to | |
91 | enable additional test cases: | |
92 | ||
93 | wget http://kernel.org/pub/software/network/wireless-regdb/wireless-regdb-2014.10.07.tar.xz | |
94 | tar xJf wireless-regdb-2014.10.07.tar.xz | |
95 | sudo mv /lib/crda/regulatory.bin{,-distro} | |
96 | sudo cp wireless-regdb-2014.10.07/regulatory.bin /lib/crda/regulatory.bin | |
97 | ||
98 | # following command can be used to verify that the new version is trusted | |
99 | regdbdump /lib/crda/regulatory.bin | |
100 | ||
101 | ||
102 | Download a snapshot of the hostap.git repository and build the programs | |
103 | ||
104 | git clone git://w1.fi/hostap.git | |
105 | cd hostap/tests/hwsim | |
106 | ./build.sh | |
107 | ||
108 | ||
109 | Setup is now ready for testing. You can run a quick test to confirm that | |
110 | things work as expected: | |
111 | ||
112 | # load mac80211_hwsim and start test software | |
113 | ./start.sh | |
114 | ||
115 | # run a single test case ap_open | |
116 | sudo ./run-tests.py ap_open | |
117 | ||
118 | This should print out following style results: | |
119 | ||
120 | DEV: wlan0: 02:00:00:00:00:00 | |
121 | DEV: wlan1: 02:00:00:00:01:00 | |
122 | DEV: wlan2: 02:00:00:00:02:00 | |
123 | APDEV: wlan3 | |
124 | APDEV: wlan4 | |
125 | START ap_open 1/1 | |
126 | Test: AP with open mode (no security) configuration | |
127 | Starting AP wlan3 | |
128 | Connect STA wlan0 to AP | |
129 | PASS ap_open 0.175895 2015-01-17 20:12:07.486006 | |
130 | passed all 1 test case(s) | |
131 | ||
132 | (If that "PASS ap_open" line does not show up, something unexpected has | |
133 | happened and the setup is not in working condition.) | |
134 | ||
135 | # to stop test software and unload mac80211_hwsim | |
136 | ./stop.sh | |
137 | ||
138 | ||
139 | To run all available test cases (about thousand or so), you can run following: | |
140 | ||
141 | ./run-all.sh | |
142 | ||
143 | This will take about half an hour to hour to run (if that sounds long, see | |
144 | vm/README for information on how parallel VMs can be used to speed this | |
145 | up; e.g., a 4-core i7-4770K can run these in under 10 minutes with 7 | |
146 | parallel VMs). | |
147 | ||
148 | The results may look something like this: | |
149 | ||
150 | START grpform_goneg_fail_with_group_iface 1/981 | |
151 | PASS grpform_goneg_fail_with_group_iface 0.371424 2015-01-17 22:17:16.659803 | |
152 | START grpform2 2/981 | |
153 | PASS grpform2 1.476142 2015-01-17 22:17:18.136539 | |
154 | ... | |
155 | START ext_password_psk_not_found 981/981 | |
156 | PASS ext_password_psk_not_found 1.544709 2015-01-17 22:46:56.489764 | |
157 | failed tests: wext_wpa2_psk wext_wep_open_auth wext_open wext_rfkill wext_scan_hidden wext_pmksa_cache wext_wep_shared_key_auth | |
158 | ||
159 | ||
160 | In this example, about 860 test cases passed and about 100 were skipped. | |
161 | ||
162 | Most of the skipped test cases are in following categories: | |
163 | - D-Bus (requires kvm-based test run, see vm/README) | |
164 | - VHT 80 and 160 MHz channels (requires wireless-regdb update) | |
165 | - DFS (requires wireless-regdb updates) | |
166 | ||
167 | The following test failed every time (i.e., other failed cases could be | |
168 | passed on second attempt): | |
169 | ||
170 | wext_pmf wext_wpa2_psk wext_wep_open_auth wext_open wext_rfkill wext_scan_hidden wext_pmksa_cache wext_wep_shared_key_auth | |
171 | ||
172 | WEXT failures are due to the specific cfg80211/mac80211 version from | |
173 | Backports not allowing WEXT support to be enabled. A newer build | |
174 | addresses that and these WEXT test cases pass, e.g., with this snapshot | |
175 | build: | |
176 | http://buildbot.w1.fi/backports-wireless-testing/backports-wireless-testing-2015-01-18-ba3f765.tar.bz2 | |
177 | ||
178 | With that version, ibss_rsn is failing due to a known cfg80211 | |
179 | regression in the specific snapshot build. All other test cases passed | |
180 | at least on retry or were skipped due to missing testing capability. | |
0a503e0d MB |
181 | |
182 | With systemd based distros, e.g., Ubuntu 16.04, systemd-rfkill.service might | |
183 | block the mac80211_hwsim network devices. | |
184 | The tests will fail with | |
185 | Exception: Failed to enable hostapd interface wlan3 | |
186 | In the *.hostapd log, would will read | |
187 | nl80211: Could not yet enable interface 'wlan3' due to rfkill | |
188 | Your syslog will read | |
189 | systemd[1]: Starting Load/Save RF Kill Switch Status... | |
190 | This can be fixed by | |
191 | systemctl mask systemd-rfkill.service |