Xiubo Li [Thu, 14 Apr 2022 01:07:21 +0000 (09:07 +0800)]
ceph: fix possible NULL pointer dereference for req->r_session
The request will be inserted into the ci->i_unsafe_dirops before
assigning the req->r_session, so it's possible that we will hit
NULL pointer dereference bug here.
Cc: stable@vger.kernel.org
URL: https://tracker.ceph.com/issues/55327 Signed-off-by: Xiubo Li <xiubli@redhat.com> Reviewed-by: Jeff Layton <jlayton@kernel.org> Tested-by: Aaron Tomlin <atomlin@redhat.com> Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Xiubo Li [Tue, 22 Mar 2022 03:03:13 +0000 (11:03 +0800)]
ceph: remove incorrect session state check
Once the session is opened the s->s_ttl will be set, and when receiving
a new mdsmap and the MDS map is changed, it will be possibly will close
some sessions and open new ones. And then some sessions will be in
CLOSING state evening without unmounting.
URL: https://tracker.ceph.com/issues/54979 Signed-off-by: Xiubo Li <xiubli@redhat.com> Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Niels Dossche [Tue, 15 Mar 2022 15:29:47 +0000 (16:29 +0100)]
ceph: get snap_rwsem read lock in handle_cap_export for ceph_add_cap
ceph_add_cap says in its function documentation that the caller should
hold the read lock on the session snap_rwsem. Furthermore, not only
ceph_add_cap needs that lock, when it calls to ceph_lookup_snap_realm it
eventually calls ceph_get_snap_realm which states via lockdep that
snap_rwsem needs to be held. handle_cap_export calls ceph_add_cap
without that mdsc->snap_rwsem held. Thus, since ceph_get_snap_realm
and ceph_add_cap both need the lock, the common place to acquire that
lock is inside handle_cap_export.
Signed-off-by: Niels Dossche <dossche.niels@gmail.com> Reviewed-by: Xiubo Li <xiubli@redhat.com> Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Bjørn Mork [Mon, 28 Mar 2022 16:11:08 +0000 (18:11 +0200)]
mtdblock: warn if opened on NAND
Warning on every translated mtd partition results in excessive log noise
if this driver is loaded:
nand: device found, Manufacturer ID: 0xc2, Chip ID: 0xf1
nand: Macronix MX30LF1G18AC
nand: 128 MiB, SLC, erase size: 128 KiB, page size: 2048, OOB size: 64
mt7621-nand 1e003000.nand: ECC strength adjusted to 4 bits
read_bbt: found bbt at block 1023
10 fixed-partitions partitions found on MTD device mt7621-nand
Creating 10 MTD partitions on "mt7621-nand":
0x000000000000-0x000000080000 : "Bootloader"
mtdblock: MTD device 'Bootloader' is NAND, please consider using UBI block devices instead.
0x000000080000-0x000000100000 : "Config"
mtdblock: MTD device 'Config' is NAND, please consider using UBI block devices instead.
0x000000100000-0x000000140000 : "Factory"
mtdblock: MTD device 'Factory' is NAND, please consider using UBI block devices instead.
0x000000140000-0x000002000000 : "Kernel"
mtdblock: MTD device 'Kernel' is NAND, please consider using UBI block devices instead.
0x000000540000-0x000002000000 : "ubi"
mtdblock: MTD device 'ubi' is NAND, please consider using UBI block devices instead.
0x000002140000-0x000004000000 : "Kernel2"
mtdblock: MTD device 'Kernel2' is NAND, please consider using UBI block devices instead.
0x000004000000-0x000004100000 : "wwan"
mtdblock: MTD device 'wwan' is NAND, please consider using UBI block devices instead.
0x000004100000-0x000005100000 : "data"
mtdblock: MTD device 'data' is NAND, please consider using UBI block devices instead.
0x000005100000-0x000005200000 : "rom-d"
mtdblock: MTD device 'rom-d' is NAND, please consider using UBI block devices instead.
0x000005200000-0x000005280000 : "reserve"
mtdblock: MTD device 'reserve' is NAND, please consider using UBI block devices instead.
mtk_soc_eth 1e100000.ethernet eth0: mediatek frame engine at 0xbe100000, irq 21
This is more likely to annoy than to help users of embedded distros where
this driver is enabled by default. Making the blockdevs available does
not imply that they are in use, and warning about bootloader partitions
or other devices which obviously never will be mounted is more confusing
than helpful.
Move the warning to open(), where it will be of more use - actually warning
anyone who mounts a file system on NAND using mtdblock.
Fixes: e07403a8c6be ("mtdblock: Warn if added for a NAND device") Signed-off-by: Bjørn Mork <bjorn@mork.no> Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar> Signed-off-by: Miquel Raynal <miquel.raynal@bootlin.com> Link: https://lore.kernel.org/linux-mtd/20220328161108.87757-1-bjorn@mork.no
Allow phram to be probed from the devicetree. It expects to be in a
reserved-memory node as documented by the bindings. This allows things
like partitioning to be specified via the devicetree.
Allow the MTD index to be specified via a devicetree alias, so that the
number does not just depend on probe order. This is useful to allow
pseudo-devices like phram to be optionally used on systems, without
having this affect the numbering of the real hardware MTD devices.
This custom maps driver was never used with the device tree
enabled IXP4xx systems, which use the physmap add-on from
commit 2aba2f2a704d
("mtd: physmap_of: add a hook for Intel IXP4xx flash probing")
Since kernel v5.18-rc1 IXP4xx is only booting from the device
tree so drop this old driver.
ALSA: fireworks: fix wrong return count shorter than expected by 4 bytes
ALSA fireworks driver has a bug in its initial state to return count
shorter than expected by 4 bytes to userspace applications when handling
response frame for Echo Audio Fireworks transaction. It's due to missing
addition of the size for the type of event in ALSA firewire stack.
Zihao Wang [Sun, 24 Apr 2022 08:41:20 +0000 (16:41 +0800)]
ALSA: hda/realtek: Add quirk for Yoga Duet 7 13ITL6 speakers
Lenovo Yoga Duet 7 13ITL6 has Realtek ALC287 and built-in
speakers do not work out of the box. The fix developed for
Yoga 7i 14ITL5 also enables speaker output for this model.
firewire: core: extend card->lock in fw_core_handle_bus_reset
card->local_node and card->bm_retries are both always accessed under
card->lock.
fw_core_handle_bus_reset has a check whose condition depends on
card->local_node and whose body writes to card->bm_retries.
Both of these accesses are not under card->lock. Move the lock acquiring
of card->lock to before this check such that these accesses do happen
when card->lock is held.
fw_destroy_nodes is called inside the check.
Since fw_destroy_nodes already acquires card->lock inside its function
body, move this out to the callsites of fw_destroy_nodes.
Also add a comment to indicate which locking is necessary when calling
fw_destroy_nodes.
Jakob Koschel [Sat, 9 Apr 2022 04:12:42 +0000 (13:12 +0900)]
firewire: remove check of list iterator against head past the loop body
When list_for_each_entry() completes the iteration over the whole list
without breaking the loop, the iterator value will be a bogus pointer
computed based on the head element.
While it is safe to use the pointer to determine if it was computed
based on the head element, either with list_entry_is_head() or
&pos->member == head, using the iterator variable after the loop should
be avoided.
In preparation to limit the scope of a list iterator to the list
traversal loop, use a dedicated pointer to point to the found element [1].
Chengfeng Ye [Sat, 9 Apr 2022 04:12:41 +0000 (13:12 +0900)]
firewire: fix potential uaf in outbound_phy_packet_callback()
&e->event and e point to the same address, and &e->event could
be freed in queue_event. So there is a potential uaf issue if
we dereference e after calling queue_event(). Fix this by adding
a temporary variable to maintain e->client in advance, this can
avoid the potential uaf issue.
Colin Ian King [Sun, 24 Apr 2022 20:59:45 +0000 (21:59 +0100)]
ALSA: pcm: Check for null pointer of pointer substream before dereferencing it
Pointer substream is being dereferenced on the assignment of pointer card
before substream is being null checked with the macro PCM_RUNTIME_CHECK.
Although PCM_RUNTIME_CHECK calls BUG_ON, it still is useful to perform the
the pointer check before card is assigned.
Mark Brown [Thu, 21 Apr 2022 11:50:20 +0000 (12:50 +0100)]
selftests: alsa: Start validating control names
Not much of a test but we keep on getting problems with boolean controls
not being called Switches so let's add a few basic checks to help people
spot problems.
FWIW, you probably need to make sure that a barrier instruction inside
an lr/sc loop is a good thing. IIUC, the execution time of a barrier
instruction is determined by the status of store buffers and invalidate
queues (and probably other stuffs), so it may increase the execution
time of the lr/sc loop, and make it unlikely to succeed. But this really
depends on how the arch executes these instructions.
The generic atomic.h used cmpxchg to implement the atomic
operations, it will cause daul loop to reduce the forward
guarantee. The patch implement csky custom atomic operations with
ldex/stex instructions for the best performance.
Important comment by Rutland: 8e86f0b409a4 ("arm64: atomics: fix use of acquire + release for
full barrier semantics")
csky: atomic: Optimize cmpxchg with acquire & release
Optimize cmpxchg with ASM acquire/release fence ASM instructions
instead of previous generic based. Prevent a fence when cmxchg's
first load != old.
Comments by Rutland:
8e86f0b409a4 ("arm64: atomics: fix use of acquire + release for
full barrier semantics")
Comments by Boqun:
FWIW, you probably need to make sure that a barrier instruction inside
an lr/sc loop is a good thing. IIUC, the execution time of a barrier
instruction is determined by the status of store buffers and invalidate
queues (and probably other stuffs), so it may increase the execution
time of the lr/sc loop, and make it unlikely to succeed. But this really
depends on how the arch executes these instructions.
if sun50i_cpufreq_get_efuse failed, then opp_tables leak.
Fixes: f328584f7bff ("cpufreq: Add sun50i nvmem based CPU scaling driver") Signed-off-by: Xiaobing Luo <luoxiaobing0926@gmail.com> Reviewed-by: Samuel Holland <samuel@sholland.org> Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
drm/i915: Check EDID for HDR static metadata when choosing blc
We have now seen panel (XMG Core 15 e21 laptop) advertizing support
for Intel proprietary eDP backlight control via DPCD registers, but
actually working only with legacy pwm control.
This patch adds panel EDID check for possible HDR static metadata and
Intel proprietary eDP backlight control is used only if that exists.
Missing HDR static metadata is ignored if user specifically asks for
Intel proprietary eDP backlight control via enable_dpcd_backlight
parameter.
v2 :
- Ignore missing HDR static metadata if Intel proprietary eDP
backlight control is forced via i915.enable_dpcd_backlight
- Printout info message if panel is missing HDR static metadata and
support for Intel proprietary eDP backlight control is detected
Fixes: 4a8d79901d5b ("drm/i915/dp: Enable Intel's HDR backlight interface (only SDR for now)") Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/5284 Cc: Lyude Paul <lyude@redhat.com> Cc: Mika Kahola <mika.kahola@intel.com> Cc: Jani Nikula <jani.nikula@intel.com> Cc: Filippo Falezza <filippo.falezza@outlook.it> Cc: stable@vger.kernel.org Signed-off-by: Jouni Högander <jouni.hogander@intel.com> Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220413082826.120634-1-jouni.hogander@intel.com Reviewed-by: Lyude Paul <lyude@redhat.com>
(cherry picked from commit b4b157577cb1de13bee8bebc3576f1de6799a921) Signed-off-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Hans de Goede [Mon, 18 Apr 2022 15:09:36 +0000 (17:09 +0200)]
drm/i915: Fix DISP_POS_Y and DISP_HEIGHT defines
Commit 428cb15d5b00 ("drm/i915: Clean up pre-skl primary plane registers")
introduced DISP_POS_Y and DISP_HEIGHT defines but accidentally set these
their masks to REG_GENMASK(31, 0) instead of REG_GENMASK(31, 16).
This breaks the primary display pane on at least pineview machines, fix
the mask to fix the primary display pane only showing black.
Tested on an Acer One AO532h with an Intel N450 SoC.
Fixes: 428cb15d5b00 ("drm/i915: Clean up pre-skl primary plane registers") Cc: José Roberto de Souza <jose.souza@intel.com> Cc: Ville Syrjälä <ville.syrjala@linux.intel.com> Signed-off-by: Hans de Goede <hdegoede@redhat.com> Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com> Link: https://patchwork.freedesktop.org/patch/msgid/20220418150936.5499-1-hdegoede@redhat.com Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
(cherry picked from commit 681f8a5c6e372dbfd2a313ace417e7749543de1d) Signed-off-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Ondrej Jirman [Mon, 25 Apr 2022 01:02:17 +0000 (18:02 -0700)]
Input: sun4i-lradc-keys - add wakeup support
Allow the driver to wake the system on key press if the "wakeup-source"
property is provided in the device tree. Using the LRADC as a wakeup
source requires keeping the AVCC domain active during sleep. Since this
has a nontrivial impact on power consumption (sometimes doubling it),
disable the LRADC wakeup source by default.
Signed-off-by: Ondrej Jirman <x@xff.cz> Acked-by: Jernej Skrabec <jernej.skrabec@gmail.com> Signed-off-by: Samuel Holland <samuel@sholland.org> Link: https://lore.kernel.org/r/20220424161328.61103-1-samuel@sholland.org Signed-off-by: Dmitry Torokhov <dmitry.torokhov@gmail.com>
The status of the keys connected to the KPDPWR_N and RESIN_N pins
is identified by reading corresponding bits in the interrupt real
time status register. If the status has changed by the time that
the interrupt is handled then a press event will be missed.
Maintain a last known status variable to find unbalanced release
events and simulate press events for each accordingly.
David Collins [Mon, 25 Apr 2022 01:20:05 +0000 (18:20 -0700)]
Input: pm8941-pwrkey - add software key press debouncing support
On certain PMICs, an unexpected assertion of KPDPWR_DEB (the
positive logic hardware debounced power key signal) may be seen
during the falling edge of KPDPWR_N (i.e. a power key press) when
it occurs close to the rising edge of SLEEP_CLK. This then
triggers a spurious KPDPWR interrupt.
Handle this issue by adding software debouncing support to ignore
key events that occur within the hardware debounce delay after the
most recent key release event.
Input: pm8941-pwrkey - add support for PON GEN3 base addresses
Currently, PON address is read from the "reg" property. For PON GEN3,
which starts with PMK8350, the "reg" property will have both the PON
HLOS and PON PBS addesses defined. Add support so that all PON
generations can be configured.
Input: ep93xx_keypad - use devm_platform_ioremap_resource() helper
Use the devm_platform_ioremap_resource() helper instead of calling
platform_get_resource() and devm_ioremap_resource() separately. This
makes the code simpler without functional changes.
Charles Mirabile [Wed, 20 Apr 2022 21:00:20 +0000 (14:00 -0700)]
Input: add Raspberry Pi Sense HAT joystick driver
This patch adds the driver for the Sense HAT joystick. It outputs BTN_DPAD
key events when moved in any of the four directions and the BTN_SELECT
event when depressed.
Co-developed-by: Daniel Bauman <dbauman@redhat.com> Signed-off-by: Daniel Bauman <dbauman@redhat.com> Co-developed-by: Mwesigwa Guma <mguma@redhat.com> Signed-off-by: Mwesigwa Guma <mguma@redhat.com> Co-developed-by: Joel Savitz <jsavitz@redhat.com> Signed-off-by: Joel Savitz <jsavitz@redhat.com> Signed-off-by: Charles Mirabile <cmirabil@redhat.com> Link: https://lore.kernel.org/r/20220419205158.28088-3-cmirabil@redhat.com Signed-off-by: Dmitry Torokhov <dmitry.torokhov@gmail.com>
Alexander Shiyan [Wed, 20 Apr 2022 20:58:45 +0000 (13:58 -0700)]
Input: clps711x-keypad - use syscon_regmap_lookup_by_phandle
Since version 5.13, the standard syscon bindings have been added
to all clps711x DT nodes, so we can now use the more general
syscon_regmap_lookup_by_phandle function to get the syscon pointer.
io_uring: cleanup error-handling around io_req_complete
Move common error-handling to io_req_complete, so that various callers
avoid repeating that. Few callers (io_tee, io_splice) require slightly
different handling. These are changed to use __io_req_complete instead.
This works like __sys_socket(), except instead of allocating and
returning a socket fd, it just returns the file associated with the
socket. No fd is installed into the process file table.
This is similar to do_accept(), and allows io_uring to use this without
instantiating a file descriptor in the process file table.
Stefan Roesch [Mon, 25 Apr 2022 00:10:46 +0000 (18:10 -0600)]
fs: split off setxattr_copy and do_setxattr function from setxattr
This splits of the setup part of the function setxattr in its own
dedicated function called setxattr_copy. In addition it also exposes a new
function called do_setxattr for making the setxattr call.
This makes it possible to call these two functions from io_uring in the
processing of an xattr request.
Right now io_uring will not actively inform userspace if a CQE is
dropped. This is extremely rare, requiring a CQ ring overflow, as well as
a GFP_ATOMIC kmalloc failure. However the consequences could cause for
example applications to go into an undefined state, possibly waiting for a
CQE that never arrives.
Return an error code (EBADR) in these cases. Since this is expected to be
incredibly rare, try and avoid as much as possible affecting the hot code
paths, and so it only is returned lazily and when there is no other
available CQEs.
Once the error is returned, reset the error condition assuming the user is
either ok with it or will clean up appropriately.
io_uring: rework io_uring_enter to simplify return value
io_uring_enter returns the count submitted preferrably over an error
code. In some code paths this check is not required, so reorganise the
code so that the check is only done as needed.
This is also a prep for returning error codes only in waiting scenarios.
We currently check REQ_F_POLLED before arming async poll for a
notification to retry. If it's set, then we don't allow poll and will
punt to io-wq instead. This is done to prevent a situation where a buggy
driver will repeatedly return that there's space/data available yet we
get -EAGAIN.
However, if we already transferred data, then it should be safe to rely
on poll again. Gate the check on whether or not REQ_F_PARTIAL_IO is
also set.
io_uring: support MSG_WAITALL for IORING_OP_SEND(MSG)
Like commit 7ba89d2af17a for recv/recvmsg, support MSG_WAITALL for the
send side. If this flag is set and we do a short send, retry for a
stream of seqpacket socket.
Rather than match on a specific key, be it user_data or file, allow
canceling any request that we can lookup. Works like
IORING_ASYNC_CANCEL_ALL in that it cancels multiple requests, but it
doesn't key off user_data or the file.
Can't be set with IORING_ASYNC_CANCEL_FD, as that's a key selector.
Only one may be used at the time.
io_uring: allow IORING_OP_ASYNC_CANCEL with 'fd' key
Currently sqe->addr must contain the user_data of the request being
canceled. Introduce the IORING_ASYNC_CANCEL_FD flag, which tells the
kernel that we're keying off the file fd instead for cancelation. This
allows canceling any request that a) uses a file, and b) was assigned the
file based on the value being passed in.
The current cancelation will lookup and cancel the first request it
finds based on the key passed in. Add a flag that allows to cancel any
request that matches they key. It completes with the number of requests
found and canceled, or res < 0 if an error occured.
io_uring: pass in struct io_cancel_data consistently
In preparation for being able to not only key cancel off the user_data,
pass in the io_cancel_data struct for the various functions that deal
with request cancelation.
Pavel Begunkov [Wed, 20 Apr 2022 12:40:54 +0000 (13:40 +0100)]
io_uring: move timeout locking in io_timeout_cancel()
Move ->timeout_lock grabbing inside of io_timeout_cancel(), so
we can do io_req_task_queue_fail() outside of the lock. It's much nicer
than relying on triple nested locking.
io_uring: store SCM state in io_fixed_file->file_ptr
A previous commit removed SCM accounting for non-unix sockets, as those
are the only ones that can cause a fixed file reference. While that is
true, it also means we're now dereferencing the file as part of the
workqueue driven __io_sqe_files_unregister() after the process has
exited. This isn't safe for SCM files, as unix gc may have already
reaped them when the process exited. KASAN complains about this:
We have an extra bit we can use in file_ptr on 64-bit, use that to store
whether this file is SCM'ed or not, avoiding the need to look at the
file contents itself. This does mean that 32-bit will be stuck with SCM
for all registered files, just like 64-bit did before the referenced
commit.
Fixes: 1f59bc0f18cf ("io_uring: don't scm-account for non af_unix sockets") Signed-off-by: Jens Axboe <axboe@kernel.dk>
Pavel Begunkov [Mon, 18 Apr 2022 19:51:14 +0000 (20:51 +0100)]
io_uring: add a helper for putting rsrc nodes
Add a simple helper to encapsulating dropping rsrc nodes references,
it's cleaner and will help if we'd change rsrc refcounting or play with
percpu_ref_put() [no]inlining.
Pavel Begunkov [Mon, 18 Apr 2022 19:51:13 +0000 (20:51 +0100)]
io_uring: store rsrc node in req instead of refs
req->fixed_rsrc_refs keeps a pointer to rsrc node pcpu references, but
it's more natural just to store rsrc node directly. There were some
reasons for that in the past but not anymore.
Pavel Begunkov [Mon, 18 Apr 2022 19:51:12 +0000 (20:51 +0100)]
io_uring: refactor io_assign_file error path
All io_assign_file() callers do error handling themselves,
req_set_fail() in the io_assign_file()'s fail path needlessly bloats the
kernel and is not the best abstraction to have. Simplify the error path.
Pavel Begunkov [Fri, 15 Apr 2022 21:08:28 +0000 (22:08 +0100)]
io_uring: refactor io_queue_sqe()
io_queue_sqe() is a part of the submission path and we try hard to keep
it inlined, so shed some extra bytes from it by moving the error
checking part into io_queue_sqe_arm_apoll() and renaming it accordingly.
note: io_queue_sqe_arm_apoll() is not inlined, thus the patch doesn't
change the number of function calls for the apoll path.
Pavel Begunkov [Fri, 15 Apr 2022 21:08:25 +0000 (22:08 +0100)]
io_uring: helper for prep+queuing linked timeouts
We try to aggresively inline the submission path, so it's a good idea to
not pollute it with colder code. One of them is linked timeout
preparation + queue, which can be extracted into a function.
Pavel Begunkov [Fri, 15 Apr 2022 21:08:23 +0000 (22:08 +0100)]
io_uring: kill io_put_req_deferred()
We have several spots where a call to io_fill_cqe_req() is immediately
followed by io_put_req_deferred(). Replace them with
__io_req_complete_post() and get rid of io_put_req_deferred() and
io_fill_cqe_req().
> size ./fs/io_uring.o
text data bss dec hex filename
86942 13734 8 100684 1894c ./fs/io_uring.o
> size ./fs/io_uring.o
text data bss dec hex filename
86438 13654 8 100100 18704 ./fs/io_uring.o
Pavel Begunkov [Fri, 15 Apr 2022 21:08:20 +0000 (22:08 +0100)]
io_uring: clean poll tw PF_EXITING handling
When we meet PF_EXITING in io_poll_check_events(), don't overcomplicate
the code with io_poll_mark_cancelled() but just return -ECANCELED and
the callers will deal with the rest.
Pavel Begunkov [Tue, 12 Apr 2022 14:09:51 +0000 (15:09 +0100)]
io_uring: optimise io_get_cqe()
io_get_cqe() is expensive because of a bunch of loads, masking, etc.
However, most of the time we should have enough of entries in the CQ,
so we can cache two pointers representing a range of contiguous CQE
memory we can use. When the range is exhausted we'll go through a slower
path to set up a new range. When there are no CQEs avaliable, pointers
will naturally point to the same address.
Pavel Begunkov [Tue, 12 Apr 2022 14:09:50 +0000 (15:09 +0100)]
io_uring: optimise submission left counting
Considering all inlining io_submit_sqe() is huge and usually ends up
calling some other functions.
We decrement @left in io_submit_sqes() just before calling
io_submit_sqe() and use it later after the call. Considering how huge
io_submit_sqe() is, there is not much hope @left will be treated
gracefully by compilers.
Decrement it after the call, not only it's easier on register spilling
and probably saves stack write/read, but also at least for x64 uses
CPU flags set by the dec instead of doing (read/write and tests).
Pavel Begunkov [Tue, 12 Apr 2022 14:09:49 +0000 (15:09 +0100)]
io_uring: optimise submission loop invariant
Instead of keeping @submitted in io_submit_sqes(), which for each
iteration requires comparison with the initial number of SQEs, store the
number of SQEs left to submit. We'll need nr only for when we're done
with SQE handling.
note: if we can't allocate a req for the first SQE we always has been
returning -EAGAIN to the userspace, save this behaviour by looking into
the cache in a slow path.
Pavel Begunkov [Tue, 12 Apr 2022 14:09:47 +0000 (15:09 +0100)]
io_uring: helper for empty req cache checks
Add io_req_cache_empty(), which checks if there are requests in the
inline req cache or not. It'll be needed in the future, but also nicely
cleans up a few spots poking into ->free_list directly.
Pavel Begunkov [Tue, 12 Apr 2022 14:09:45 +0000 (15:09 +0100)]
io_uring: shrink final link flush
All good users should not set IOSQE_IO_*LINK flags for the last request
of a link. io_uring flushes collected links at the end of submission,
but it's not the optimal way and so we don't care too much about it.
Replace io_queue_sqe() call with io_queue_sqe_fallback() as the former
one is inlined and will generate a bunch of extra code. This will also
help compilers with the submission path inlining.
> size ./fs/io_uring.o
text data bss dec hex filename
87265 13734 8 101007 18a8f ./fs/io_uring.o
> size ./fs/io_uring.o
text data bss dec hex filename
87073 13734 8 100815 189cf ./fs/io_uring.o
Pavel Begunkov [Tue, 12 Apr 2022 14:09:44 +0000 (15:09 +0100)]
io_uring: memcpy CQE from req
We can do CQE filling a bit more efficiently when req->cqe is fully
filled by memcpy()'ing it to the userspace instead of doing it field by
field. It's easier on register spilling, removes a couple of extra
loads/stores and write combines two u32 memory writes.
Pavel Begunkov [Tue, 12 Apr 2022 14:09:43 +0000 (15:09 +0100)]
io_uring: explicitly keep a CQE in io_kiocb
We already have req->{result,user_data,cflags}, which mimic struct
io_uring_cqe and are intended to store CQE data. Combine them into a
struct io_uring_cqe field.
Pavel Begunkov [Thu, 7 Apr 2022 12:40:03 +0000 (13:40 +0100)]
io_uring: don't pass around fixed index for scm
There is an old API nuisance where io_uring's SCM accounting functions
traverse fixed file tables and so requires them to be set in advance,
which leads to some implicit rules of how io_sqe_file_register() should
be used.
__io_sqe_files_scm() now works with only one file at a time, pass a file
directly and get rid of all fixed table dereferencing inside. Clean
io_sqe_file_register() callers.
projects / people / ms / linux.git / log
