Xiao Ni [Tue, 28 May 2024 13:51:47 +0000 (21:51 +0800)]
mdadm/tests: bitmap cases enhance
It fails because bitmap dirty number is smaller than 400 sometimes. It's not
good to compare bitmap dirty bits with a number. It depends on the test
machine, it can flush soon before checking the number. So remove related codes.
Signed-off-by: Xiao Ni <xni@redhat.com> Signed-off-by: Mariusz Tkaczyk <mariusz.tkaczyk@linux.intel.com>
Xiao Ni [Wed, 22 May 2024 08:50:55 +0000 (16:50 +0800)]
mdadm/tests: 07changelevelintr
It needs to specify a 2 powered array size when updating array size.
If not, it can't change chunksize.
And sometimes it reports error reshape doesn't happen. In fact the
reshape has finished. It doesn't need to wait before checking
reshape action. Because check function waits itself.
Signed-off-by: Xiao Ni <xni@redhat.com> Signed-off-by: Mariusz Tkaczyk <mariusz.tkaczyk@linux.intel.com>
Xiao Ni [Wed, 22 May 2024 08:50:53 +0000 (16:50 +0800)]
mdadm/tests: 07autoassemble
This test is used to test stacked array auto assemble.
There are two different cases depends on if array is foreign or not.
If the array is foreign, the stacked array (md0 is on md1 and md2)
can't be assembled with name md0. Because udev rule will run when md1
and md2 are assembled and mdadm -I doesn't specify homehost. So it
will treat stacked array (md0) as foreign array and choose md127 as
the device node name (/dev/md127)
Add the case that stacked array is local.
Signed-off-by: Xiao Ni <xni@redhat.com> Signed-off-by: Mariusz Tkaczyk <mariusz.tkaczyk@linux.intel.com>
Xiao Ni [Wed, 22 May 2024 08:50:44 +0000 (16:50 +0800)]
mdadm/tests: 03assem-incr enhance
It fails when hostname lenght > 32. Because the super1 metadata name
doesn't include hostname when hostname length > 32. Then mdadm thinks
the array is a foreign array if no device link is specified when
assembling the array. It chooses a minor number from 127.
Signed-off-by: Xiao Ni <xni@redhat.com> Signed-off-by: Mariusz Tkaczyk <mariusz.tkaczyk@linux.intel.com>
Xiao Ni [Wed, 22 May 2024 08:50:43 +0000 (16:50 +0800)]
mdadm/tests: names_template enhance
For super1, if the length of hostname is >= 32, it doesn't add hostname
in metadata name. Fix this problem by checking the length of hostname.
Because other cases may use need to check this, so do the check in
do_setup.
And this patch adds a check if link /dev/md/name exists.
Signed-off-by: Xiao Ni <xni@redhat.com> Signed-off-by: Mariusz Tkaczyk <mariusz.tkaczyk@linux.intel.com>
Xiao Ni [Wed, 22 May 2024 08:50:41 +0000 (16:50 +0800)]
mdadm/tests: test enhance
There are two changes.
First, if md module is not loaded, it gives error when reading
speed_limit_max. So read the value after loading md module which
is done in do_setup
Second, sometimes the test reports error sync action doesn't
happen. But dmesg shows sync action is done. So limit the sync
speed before test. It doesn't affect the test run time. Because
check wait sets the max speed before waiting sync action. And
recording speed_limit_max/min in do_setup.
Fixes: 4c12714d1ca0 ('test: run tests on system level mdadm') Signed-off-by: Xiao Ni <xni@redhat.com> Signed-off-by: Mariusz Tkaczyk <mariusz.tkaczyk@linux.intel.com>
Xiao Ni [Wed, 22 May 2024 08:50:39 +0000 (16:50 +0800)]
mdadm: Start update_opt from 0
Before f2e8393bd722 ('Manage&Incremental: code refactor, string to enum'), it uses
NULL to represent it doesn't need to update. So init UOPT_UNDEFINED to 0. This
problem is found by test case 05r6tor0.
Signed-off-by: Xiao Ni <xni@redhat.com> Signed-off-by: Mariusz Tkaczyk <mariusz.tkaczyk@linux.intel.com>
Valery Ushakov [Wed, 22 May 2024 14:07:38 +0000 (17:07 +0300)]
Makefile: fix make -s detection
Only check the first word of MAKEFLAGS for 's', that's where all the
single letter options are collected.
MAKEFLAGS contains _all_ make flags, so if any command line argument
contains a letter 's', the silent test will be false positive. Think
e.g. make 'DESTDIR=.../aports/main/mdadm/pkg/mdadm' install
Mariusz Tkaczyk [Fri, 29 Mar 2024 14:21:54 +0000 (15:21 +0100)]
mdadm: deprecate bitmap custom file
This option has been deprecated in kernel by Christoph in commit 0ae1c9d38426 ("md: deprecate bitmap file support"). Do the same in
mdadm.
With this change, user must acknowledge it, it is not
skippable. The implementation of custom bitmap file looks like it's
abandoned. It cannot be done by Incremental so it is not respected by
any udev based system and it seems to not be recorded by metadata.
User must assemble such volume manually.
Tests for bitmap custom file are removed because now they will not
pass because interaction with user is mandatory.
Nigel Croxon [Wed, 22 May 2024 20:53:22 +0000 (16:53 -0400)]
mdadm: super-intel fix bad shift
In the expression "1 << i", left shifting by more than 31 bits has undefined behavior.
The shift amount, "i", is as much as 63. The operand has type "int" (32 bits) and will
be shifted as an "int". The fix is to change to a 64 bit int.
Nigel Croxon [Wed, 22 May 2024 20:05:25 +0000 (16:05 -0400)]
mdadm: super-intel remove dead code
Execution cannot reach this statement: "while (devlist) { dv = de...".
Local variable "err" is assigned only once, to a constant value,
making it effectively constant throughout its scope.
Remove dead code.
Blazej Kucman [Wed, 15 May 2024 11:26:28 +0000 (13:26 +0200)]
mdadm: Fix compilation for 32-bit arch
Casting void pointer to __u64 works for 64-bit arch but fails to compile
on 32-bit arch like i686.
Fail on i686 platform:
drive_encryption.c: In function ‘nvme_security_recv_ioctl’:
drive_encryption.c:236:25: error: cast from pointer to integer of
different size [-Werror=pointer-to-int-cast]
236 | nvme_cmd.addr = (__u64)response_buffer;
| ^
drive_encryption.c: In function ‘nvme_identify_ioctl’:
drive_encryption.c:271:25: error: cast from pointer to integer of
different size [-Werror=pointer-to-int-cast]
271 | nvme_cmd.addr = (__u64)response_buffer;
| ^
cc1: all warnings being treated as errors
make: *** [Makefile:211: drive_encryption.o] Error 1
This change adds cast void pointer to uintptr_t first to ensure that
proper pointer size is used for casting from pointer type. Then is safe to
cast it to __u64 because it is tracked as u_int, regardless it is 32-bit
or 64-bit arch.
Introduce review.yml used by GitHub actions. Add make probe, checkpatch
and hardening-check on every pull request.
Add dependabot.yml file which check for updates of actions used in this
repository. This option enables to automatically fill new PR with action
updated to the latest version.
Kinga Stefaniuk [Tue, 7 May 2024 03:38:56 +0000 (05:38 +0200)]
Wait for mdmon when it is stared via systemd
When mdmon is being started it may need few seconds to start.
For now, we didn't wait for it. Introduce wait_for_mdmon()
function, which waits up to 5 seconds for mdmon to start completely.
Move -pie from LDLIBS to LDFLAGS and make LDFLAGS configurable to allow
the user to drop it by setting their own LDFLAGS (e.g. PIE could be
enabled or disabled by the buildsystem such as buildroot).
Xiao Ni [Thu, 18 Apr 2024 10:23:19 +0000 (18:23 +0800)]
tests/01r5fail enhance
After removing dev0, the recovery starts because it already has a spare
disk. It's good to check recovery. But it's not right to check recovery
after adding dev3. Because the recovery may finish. It depends on the
recovery performance of the testing machine. If the recovery finishes,
it will fail. But dev3 is only added as a spare disk, we can't expect
there is a recovery happens.
So remove the codes about adding dev3.
Signed-off-by: Xiao Ni <xni@redhat.com> Signed-off-by: Mariusz Tkaczyk <mariusz.tkaczyk@linux.intel.com>
VROC UEFI driver does not support RAID 10 with more than 4 drives.
Add user prompts if such layout is being created and for R0->R10
reshapes.
Refactor ask() function:
- simplify the code,
- remove dialog reattempts,
- do no pass '?' sign on function calls,
- highlight default option on output.
This patch completes adding support for R10D4+ to IMSM.
IMSM version 1.3 (called ATTRIBS) brought attributes used to define array
properties which require support in driver. The goal of this change was
to avoid changing version when adding new features.
For some reasons migration has never been completed and currently (after
10 years of implementing) IMSM can use older versions.
It is right time to finally switch it. There is no point in using old
versions, use 1.3.00 as minimal one.
Add imsm_level_ops struct for better handling and unifying raid level
support. Add helper methods and move "orom_has_raid[...]" methods from
header to source file.
RAID 1e is not supported under Linux, remove RAID 1e associated code.
Refactor imsm_analyze_change() and is_raid_level_supported().
Remove hardcoded check for 4 drives and make devNumChange a multiplier
for RAID 10.
As for now, IMSM supports only 4 drive RAID 1+0. This patch is first in
series to add support for literal RAID 10 (with more than 4 drives) to
imsm.
Allow setting RAID 10 as raid level for imsm arrays.
Add update_imsm_raid_level() to handle raid level updates. Set RAID10 as
default level for imsm R0 to R10 migrations. Replace magic numbers with
defined values for RAID level checks/assigns.
Define FALLOC_FL_ZERO_RANGE if needed as FALLOC_FL_ZERO_RANGE is only
defined for aarch64 on uclibc-ng resulting in the following or1k build
failure since commit 577fd10486d8d1472a6b559066f344ac30a3a391:
Create.c: In function 'write_zeroes_fork':
Create.c:155:35: error: 'FALLOC_FL_ZERO_RANGE' undeclared (first use in this function)
155 | if (fallocate(fd, FALLOC_FL_ZERO_RANGE | FALLOC_FL_KEEP_SIZE,
| ^~~~~~~~~~~~~~~~~~~~
Mariusz Tkaczyk [Tue, 26 Mar 2024 12:21:10 +0000 (13:21 +0100)]
mdadm: add CHANGELOG.md
Bring changelog back to life. Remove ANNOUCEs. It will use markdown
format, to have one style. All releases are migrated to new
changelog. It was a exercise I have taken, to familiarize with the
mdadm history.
Blazej Kucman [Fri, 22 Mar 2024 11:51:20 +0000 (12:51 +0100)]
imsm: drive encryption policy implementation
IMSM cares about drive encryption state. It is not allowed to mix disks
with different encryption state within one md device. This policy will
verify that attempt to use disks with different encryption states will
fail. Verification is performed for devices NVMe/SATA Opal and SATA.
There is one exception, Opal SATA drives encryption is not checked when
ENCRYPTION_NO_VERIFY key with "sata_opal" value is set in conf, for this
reason such drives are treated as without encryption support.
Blazej Kucman [Fri, 22 Mar 2024 11:51:19 +0000 (12:51 +0100)]
imsm: print disk encryption information
Print SATA/NVMe disk encryption information in --detail-platform.
Encryption Ability and Status will be printed for each disk.
There is one exception, Opal SATA drives encryption is not checked when
ENCRYPTION_NO_VERIFY key with "sata_opal" value is set in conf, for this
reason such drives are treated as without encryption support.
To test this feature, drives SATA/NVMe with Opal support or SATA drives
with encryption support have to be used.
Example outputs of --detail-platform:
Non Opal, encryption enabled, SATA drive:
Port0 : /dev/sdc (CVPR050600G3120LGN)
Encryption(Ability|Status): Other|Unlocked
NVMe drive without Opal support:
NVMe under VMD : /dev/nvme2n1 (PHLF737302GB1P0GGN)
Encryption(Ability|Status): None|Unencrypted
Unencrypted SATA drive with OPAL support:
- default allow_tpm, we will get an error from mdadm:
Port6 : /dev/sdi (CVTS4246015V180IGN)
mdadm: Detected SATA drive /dev/sdi with Trusted Computing support.
mdadm: Cannot verify encryption state. Requires libata.tpm_enabled=1.
mdadm: Failed to get drive encrytpion information.
- added "libata.allow_tpm=1" to boot parameters(requires reboot),
the status will be read correctly:
Port6 : /dev/sdi (CVTS4246015V180IGN)
Encryption(Ability|Status): SED|Unencrypted
Blazej Kucman [Fri, 22 Mar 2024 11:51:18 +0000 (12:51 +0100)]
Add key ENCRYPTION_NO_VERIFY to conf
Add ENCRYPTION_NO_VERIFY config key and allow to disable checking
encryption status for given type of drives.
The key is introduced because of SATA Opal disks for which TPM commands
must be enabled in libata kernel module, (libata.allow_tpm=1), otherwise
it is impossible to verify encryption status. TPM commands are disabled by
default.
Currently the key only supports the "sata_opal" value, if necessary,
the functionality is ready to support more types of disks. This
functionality will be used in the next patches.
Blazej Kucman [Fri, 22 Mar 2024 11:51:17 +0000 (12:51 +0100)]
Add reading SATA encryption information
Functionality reads information about SATA disk encryption. Technical
documentation used is given in the implementation.
The implementation is able to recognized two encryption standards for SATA
drives, OPAL and ATA security.
If the SATA drive supports OPAL, encryption status and ability are
determined based on Opal Level 0 discovery response, for ATA security,
based on ATA identify response. If SATA supports OPAL, ability is set to
"SED", for ATA security to "Other".
SED(Self-Encrypting Drive) is commonly used to describe drive which using
OPAL or Enterprise standards developed by Trusted Computing Group. Ability
"Other" is used for ATA security because we rely only on information from
ATA identify which describe the overall state of encryption.
It is allowed to mix disks with different encryption ability such as "SED"
and "Other" and it is not security gap.
Motivation for adding this functionality is to block mixing of disks in
IMSM arrays with encryption enabled and disabled. The main goal is to not
allow stealing data by rebuilding array to not encrypted drive which can be
read elsewhere.
For SATA Opal drives, libata allow_tmp parameter enabled is required, which
is necessary for Opal Security commands to work, therefore, if the
parameter is not enabled, SATA Opal disk cannot be used in case the
encryption will be checked by metadata.
Implemented functions will be used in one of the next patches. In one of
the next patches, a flag will be added to enable disabling SATA Opal
encryption checking due to allow_tpm kernel setting dependency.
Blazej Kucman [Fri, 22 Mar 2024 11:51:16 +0000 (12:51 +0100)]
Add reading Opal NVMe encryption information
For NVMe devices with Opal support, encryption information, status and
ability are determined based on Opal Level 0 discovery response. Technical
documentation used is given in the implementation.
Ability in general describes what type of encryption is supported, Status
describes in what state the disk with encryption support is. The current
patch includes only the implementation of reading encryption information,
functions will be used in one of the next patches.
Motivation for adding this functionality is to block mixing of disks in
IMSM arrays with encryption enabled and disabled. The main goal is to not
allow stealing data by rebuilding array to not encrypted drive which can be
read elsewhere.
Value ENA_OTHER from enum encryption_ability will be used in the next
patch.
Mariusz Tkaczyk [Mon, 18 Mar 2024 15:19:30 +0000 (16:19 +0100)]
mdadm: Fix native --detail --export
Mentioned commit (see Fixes) causes that UUID is not swapped as expected
for native superblock. Fix this problem.
For detail, we should avoid superblock calls, we can have information
about supertype from map, use that.
Simplify fname_from_uuid() by removing dependencies to metadata
handler, it is not needed. Decision is taken at compile time, expect
super1 but this function is not used by super1. Add warning about that.
Remove separator, it is always ':'.
Mariusz Tkaczyk [Thu, 29 Feb 2024 11:52:14 +0000 (12:52 +0100)]
Monitor, Incremental: use device policies
spare_criteria is expanded to contain policies which will be generated
by handler's get_spare_criteria() function. It provides a way to
test device for metadata specific policies earlier than during
add_do_super(), when device is already removed from previous
array/container for Monitor.
For Incremental, it ensures that all criteria are tested when trying
spare. It is not tested when device contains valid metadata.
Mariusz Tkaczyk [Thu, 29 Feb 2024 11:52:12 +0000 (12:52 +0100)]
Create: Use device policies
Generate and compare policies, abort if policies do not match.
It is tested for both create modes, with container and disk list
specified directly. It is used if supertype supports it.
For a case when disk list is specified, container may contain more
devices, so additional check on container is done to analyze all disks.
Add support for three scenarios:
- obtaining array wide policies via fd,
- obtaining array wide policies via struct mdinfo,
- getting policies for particular drive from the request.
Add proper functions and make them extern. These functions are used
in next patches.
Mariusz Tkaczyk [Thu, 29 Feb 2024 11:52:10 +0000 (12:52 +0100)]
mdadm.h: Introduce custom device policies
The approach proposed here is to test drive policies outside
validate_geometry() separately per every drive and add determined
policies to list. The implementation reuses dev_policy we have in
mdadm.
This concept addresses following problems:
- test drives if they fit together to criteria required by metadata
handler,
- test all drives assigned to the container even if some of them are not
target of the request, mdmon is free to use any drive in the same
container,
- extensibility, new policies can be added to handler easy,
- fix issues related to imsm controller domain verifying.
Add superswitch function. It is used in next patches.
Mariusz Tkaczyk [Thu, 29 Feb 2024 11:52:09 +0000 (12:52 +0100)]
mdadm: introduce sysfs_get_container_devnm()
There at least two places where it is done directly, so replace them
with function. Print message about creating external array, add "/dev/"
prefix to refer directly to devnode.
Mariusz Tkaczyk [Thu, 29 Feb 2024 11:52:06 +0000 (12:52 +0100)]
mdadm: drop get_required_spare_criteria()
Only IMSM implements get_spare_criteria, so load_super() in
get_required_spare_criteria() is dead code. It is moved inside
metadata handler, because only IMSM implements it.
Give possibility to provide devnode to be opened. With that we can hide
load_container() used only to fill spare criteria inside handler
and simplify implementation in generic code.
Add helper function for testing spare criteria in Incremental and
error messages.
File descriptor in get_spare_criteria_imsm() is always opened on purpose.
New functionality added in next patches will require it. For the same
reason, function is moved to other place.
Mariusz Tkaczyk [Thu, 29 Feb 2024 11:52:05 +0000 (12:52 +0100)]
mdadm: Add functions for spare criteria verification
It is done similar way in few places. As a result, two almost identical
functions (dev_size_from_id() and dev_sector_size_from_id()) are
removed. Now, it uses same file descriptor to send two ioctls.
Two extern functions are added, in next patches
disk_fd_matches_criteria() is used.
Next optimization is inline zeroing struct spare_criteria. With that,
we don't need to reset values in get_spare_criteria_imsm().
Dedicated boolean field for checking if criteria are filled is added.
We don't need to execute the code if it is not set.
Kinga Tanska [Tue, 27 Feb 2024 02:36:14 +0000 (03:36 +0100)]
Detail: remove duplicated code
Remove duplicated code from Detail(), where MD_UUID and MD_DEVNAME
are being set. Superblock is no longer required to print system
properties. Now it tries to obtain map in two ways.
Mariusz Tkaczyk [Fri, 23 Feb 2024 14:51:45 +0000 (15:51 +0100)]
mdadm: remove mkinitramfs stuff
This script uses mdadm.static which is known to not be abandoned
(probably not working) from years. Mdadm is integrated with dracut
and mkinitramfs these days.
Kinga Tanska [Tue, 27 Feb 2024 06:36:39 +0000 (07:36 +0100)]
super-intel: respect IMSM_DEVNAME_AS_SERIAL flag
IMSM_DEVNAME_AS_SERIAL flag was respected only when searching
serial using nvme or scsi device wasn't successful. This
flag shall be applied first, to have user settings with
the highest priority.
Mateusz Kusiak [Wed, 28 Feb 2024 15:37:20 +0000 (16:37 +0100)]
Monitor: Allow no PID in check_one_sharer()
Commit 5fb5479ad100 ("Monitor: open file before check in
check_one_sharer()") introduced a regression that prohibits monitor
from starting if PID file does not exist.
Mateusz Kusiak [Tue, 20 Feb 2024 16:04:44 +0000 (17:04 +0100)]
test: run tests on system level mdadm
The tests run with MDADM_NO_SYSTEMCTL flag by default, however it has
no effect on udev. In case of external metadata, even if flag is set,
udev will trigger systemd to launch mdmon.
This commit changes test execution level, so the tests are run on system
level mdadm, meaning local build must be installed prior to running
tests.
Add warning that the tests are run on system level mdadm and local
build must be installed first.
Do not call mdadm with "quiet" as it makes it not display critical
messages necessary for debug.
Remove forcing speed_limit and add restoring system speed_limit_max
after test execution.
Mateusz Kusiak [Tue, 20 Feb 2024 10:56:12 +0000 (11:56 +0100)]
mdmon: refactor md device name check in main()
Refactor mdmon main function to verify if fd is valid prior to checking
device name. This is due to static code analysis complaining after
change b938519e7719 ("util: remove obsolete code from get_md_name").
Mateusz Kusiak [Thu, 18 Jan 2024 10:30:18 +0000 (11:30 +0100)]
Grow: Move update_tail assign to Grow_reshape()
Due to e919fb0af245 ("FIX: Enable metadata updates for raid0") code
can't enter super-intel.c:3415, resulting in checkpoint not being
saved to metadata for second volume in matrix raid array.
This results in checkpoint being stuck at last value for the
first volume.
Move st->update_tail to Grow_reshape() so it is assigned for each
volume.
Mateusz Kusiak [Thu, 18 Jan 2024 10:30:17 +0000 (11:30 +0100)]
Super-intel: Fix first checkpoint restart
When imsm based array is stopped after reaching first checkpoint and
then assembled, first checkpoint is reported as 0.
This behaviour is valid only for initial checkpoint, if the array was
stopped while performing some action.
Last checkpoint value is not taken from metadata but always starts
with 0 and it's incremented when sync_completed in sysfs changes.
In simplification, read_and_act() is responsible for checkpoint updates
and is executed each time sysfs checkpoint update happens. For first
checkpoint it is executed twice and due to marking checkpoint before
triggering any action on the array, it is impossible to read
sync_completed from sysfs in just two iterations.
The workaround to this is not marking any checkpoint for first
sysfs checkpoint after RAID assembly, to preserve checkpoint value
stored in metadata.
Mateusz Kusiak [Thu, 18 Jan 2024 10:30:15 +0000 (11:30 +0100)]
Remove hardcoded checkpoint interval checking
Mdmon assumes that kernel marks checkpoint every 1/16 of the volume size
and that the checkpoints are equal in size. This is not true, kernel may
mark checkpoints more frequently depending on several factors, including
sync speed. This results in checkpoints reported by mdadm --examine
falling behind the one reported by kernel.
Mariusz Tkaczyk [Mon, 5 Feb 2024 14:50:29 +0000 (15:50 +0100)]
Revert "mdadm: remove container_enough logic"
Mentioned patch changes way of IMSM member arrays assembling, they are
updated by every new drive incremental processes. Previously, member
arrays were created and filled once, by last drive incremental process.
We determined regressions with various impact. Unfortunately, initial
testing didn't show them.
Regressions are connected to drive appearance order and may not be
reproducible on every configuration, there are at least two know
issues for now:
- sysfs attributes are filled using old metadata if there is
outdated drive and it is enumerated first.
- rebuild may be aborted and started from beginning after reboot,
if drive under rebuild is enumerated as the last one.
Mariusz Tkaczyk [Thu, 1 Feb 2024 11:32:41 +0000 (12:32 +0100)]
super1: remove support for name= in config
Only super1 provides "name=" to config. It is recoreded in metadata
so there is no need to duplicate same information.
UUID is our main key.
It is not used by Incremental and Assemble handles empty name well
because other supertypes don't set it in conf.
Expectation that the name in config is same as in metadata is bug prone.
Config should be the place where use can define customized settings.
Remove printing "name=" from mdadm config creation commands. Ignore
the name in config file to keep backward compatibility. Remove
description from man mdadm.conf.
Update 00conftest because "name" is no longer accepted.
As the name is ignored, error for mdadm --detail is not printed.
Reported-by: Stefan Fleischmann <sfle@kth.se> Fixes: e2eb503bd797 ("mdadm: Follow POSIX Portable Character Set") Signed-off-by: Mariusz Tkaczyk <mariusz.tkaczyk@linux.intel.com>
projects / thirdparty / mdadm.git / log
