tty: n_tty: use uint for space returned by tty_write_room()

tty_write_room() returns an "unsigned int". So in case some insane
driver (like my tty test driver) returns (legitimate) UINT_MAX from its
tty_operations::write_room(), n_tty is confused on several places.

For example, in process_output_block(), the result of tty_write_room()
is stored into (signed) "int". So this UINT_MAX suddenly becomes -1. And
that is extended to ssize_t and returned from process_output_block().
This causes a write() to such a node to receive -EPERM (which is -1).

Fix that by using proper "unsigned int" and proper "== 0" test. And
return 0 constant directly in that "if", so that it is immediately clear
what is returned ("space" equals to 0 at that point).

Similarly for process_output() and __process_echoes().

Note this does not fix any in-tree driver as of now.

If you want "Fixes: something", it would be commit 03b3b1a2405c ("tty:
make tty_operations::write_room return uint"). I intentionally do not
mark this patch by a real tag below.

Signed-off-by: Jiri Slaby (SUSE) <jirislaby@kernel.org>
Link: https://lore.kernel.org/r/20250317070046.24386-6-jirislaby@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

diff --git a/drivers/tty/n_tty.c b/drivers/tty/n_tty.c
index 2c5995019dd1a0965d32d776ab7e86d3fbb52a6c..765d24268d75ca90430327168e4a753117ad6985 100644 (file)
--- a/drivers/tty/n_tty.c
+++ b/drivers/tty/n_tty.c
@@ -488,7 +488,8 @@ static int do_output_char(u8 c, struct tty_struct *tty, int space)
 static int process_output(u8 c, struct tty_struct *tty)
 {
        struct n_tty_data *ldata = tty->disc_data;
-       int     space, retval;
+       unsigned int space;
+       int retval;
 
        mutex_lock(&ldata->output_lock);
 
@@ -524,16 +525,16 @@ static ssize_t process_output_block(struct tty_struct *tty,
                                    const u8 *buf, unsigned int nr)
 {
        struct n_tty_data *ldata = tty->disc_data;
-       int     space;
-       int     i;
+       unsigned int space;
+       int i;
        const u8 *cp;
 
        mutex_lock(&ldata->output_lock);
 
        space = tty_write_room(tty);
-       if (space <= 0) {
+       if (space == 0) {
                mutex_unlock(&ldata->output_lock);
-               return space;
+               return 0;
        }
        if (nr > space)
                nr = space;
@@ -698,7 +699,7 @@ static int n_tty_process_echo_ops(struct tty_struct *tty, size_t *tail,
 static size_t __process_echoes(struct tty_struct *tty)
 {
        struct n_tty_data *ldata = tty->disc_data;
-       int     space, old_space;
+       unsigned int space, old_space;
        size_t tail;
        u8 c;