git.ipfire.org Git - thirdparty/kernel/stable-queue.git/blob

   1 From ff6781fd1bb404d8a551c02c35c70cec1da17ff1 Mon Sep 17 00:00:00 2001
   2 From: Nicholas Piggin <npiggin@gmail.com>
   3 Date: Wed, 21 Mar 2018 12:22:28 +1000
   4 Subject: powerpc/64s: Fix lost pending interrupt due to race causing lost update to irq_happened
   5
   6 From: Nicholas Piggin <npiggin@gmail.com>
   7
   8 commit ff6781fd1bb404d8a551c02c35c70cec1da17ff1 upstream.
   9
  10 force_external_irq_replay() can be called in the do_IRQ path with
  11 interrupts hard enabled and soft disabled if may_hard_irq_enable() set
  12 MSR[EE]=1. It updates local_paca->irq_happened with a load, modify,
  13 store sequence. If a maskable interrupt hits during this sequence, it
  14 will go to the masked handler to be marked pending in irq_happened.
  15 This update will be lost when the interrupt returns and the store
  16 instruction executes. This can result in unpredictable latencies,
  17 timeouts, lockups, etc.
  18
  19 Fix this by ensuring hard interrupts are disabled before modifying
  20 irq_happened.
  21
  22 This could cause any maskable asynchronous interrupt to get lost, but
  23 it was noticed on P9 SMP system doing RDMA NVMe target over 100GbE,
  24 so very high external interrupt rate and high IPI rate. The hang was
  25 bisected down to enabling doorbell interrupts for IPIs. These provided
  26 an interrupt type that could run at high rates in the do_IRQ path,
  27 stressing the race.
  28
  29 Fixes: 1d607bb3bd60 ("powerpc/irq: Add mechanism to force a replay of interrupts")
  30 Cc: stable@vger.kernel.org # v4.8+
  31 Reported-by: Carol L. Soto <clsoto@us.ibm.com>
  32 Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
  33 Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
  34 Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
  35
  36 ---
  37  arch/powerpc/kernel/irq.c |    8 ++++++++
  38  1 file changed, 8 insertions(+)
  39
  40 --- a/arch/powerpc/kernel/irq.c
  41 +++ b/arch/powerpc/kernel/irq.c
  42 @@ -430,6 +430,14 @@ void force_external_irq_replay(void)
  43          */
  44         WARN_ON(!arch_irqs_disabled());
  45
  46 +       /*
  47 +        * Interrupts must always be hard disabled before irq_happened is
  48 +        * modified (to prevent lost update in case of interrupt between
  49 +        * load and store).
  50 +        */
  51 +       __hard_irq_disable();
  52 +       local_paca->irq_happened |= PACA_IRQ_HARD_DIS;
  53 +
  54         /* Indicate in the PACA that we have an interrupt to replay */
  55         local_paca->irq_happened |= PACA_IRQ_EE;
  56  }