From: Lehua Ding Date: Wed, 23 Aug 2023 03:25:20 +0000 (+0800) Subject: RISC-V: Add conditional unary neg/abs/not autovec patterns X-Git-Tag: basepoints/gcc-15~6731 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=92f2ec417c57e980b92b8966226fc2bfbf042af8;p=thirdparty%2Fgcc.git RISC-V: Add conditional unary neg/abs/not autovec patterns Hi, This patch add conditional unary neg/abs/not autovec patterns to RISC-V backend. For this C code: void test_3 (float *__restrict a, float *__restrict b, int *__restrict pred, int n) { for (int i = 0; i < n; i += 1) { a[i] = pred[i] ? __builtin_fabsf (b[i]) : a[i]; } } Before this patch: ... vsetvli a7,zero,e32,m1,ta,ma vfabs.v v2,v2 vmerge.vvm v1,v1,v2,v0 ... After this patch: ... vsetvli a7,zero,e32,m1,ta,mu vfabs.v v1,v2,v0.t ... For int neg/not and FP neg patterns, Defining the corresponding cond_xxx paterns is enough. For the FP abs pattern, We need to change the definition of `abs2` and `@vcond_mask_` pattern from define_expand to define_insn_and_split in order to fuse them into a new pattern `*cond_abs` at the combine pass. A fusion process similar to the one below: (insn 30 29 31 4 (set (reg:RVVM1SF 152 [ vect_iftmp.15 ]) (abs:RVVM1SF (reg:RVVM1SF 137 [ vect__6.14 ]))) "float.c":15:56 discrim 1 12799 {absrvvm1sf2} (expr_list:REG_DEAD (reg:RVVM1SF 137 [ vect__6.14 ]) (nil))) (insn 31 30 32 4 (set (reg:RVVM1SF 140 [ vect_iftmp.19 ]) (if_then_else:RVVM1SF (reg:RVVMF32BI 136 [ mask__27.11 ]) (reg:RVVM1SF 152 [ vect_iftmp.15 ]) (reg:RVVM1SF 139 [ vect_iftmp.18 ]))) 12707 {vcond_mask_rvvm1sfrvvmf32bi} (expr_list:REG_DEAD (reg:RVVM1SF 152 [ vect_iftmp.15 ]) (expr_list:REG_DEAD (reg:RVVM1SF 139 [ vect_iftmp.18 ]) (expr_list:REG_DEAD (reg:RVVMF32BI 136 [ mask__27.11 ]) (nil))))) ==> (insn 31 30 32 4 (set (reg:RVVM1SF 140 [ vect_iftmp.19 ]) (if_then_else:RVVM1SF (reg:RVVMF32BI 136 [ mask__27.11 ]) (abs:RVVM1SF (reg:RVVM1SF 137 [ vect__6.14 ])) (reg:RVVM1SF 139 [ vect_iftmp.18 ]))) 13444 {*cond_absrvvm1sf} (expr_list:REG_DEAD (reg:RVVM1SF 137 [ vect__6.14 ]) (expr_list:REG_DEAD (reg:RVVMF32BI 136 [ mask__27.11 ]) (expr_list:REG_DEAD (reg:RVVM1SF 139 [ vect_iftmp.18 ]) (nil))))) Best, Lehua gcc/ChangeLog: * config/riscv/autovec-opt.md (*cond_abs): New combine pattern. (*copysign_neg): Ditto. * config/riscv/autovec.md (@vcond_mask_): Adjust. (2): Ditto. (cond_): New. (cond_len_): Ditto. * config/riscv/riscv-protos.h (enum insn_type): New. (expand_cond_len_unop): New helper func. * config/riscv/riscv-v.cc (shuffle_merge_patterns): Adjust. (expand_cond_len_unop): New helper func. gcc/testsuite/ChangeLog: * gcc.target/riscv/rvv/autovec/cond/cond_unary-1.c: New test. * gcc.target/riscv/rvv/autovec/cond/cond_unary-2.c: New test. * gcc.target/riscv/rvv/autovec/cond/cond_unary-3.c: New test. * gcc.target/riscv/rvv/autovec/cond/cond_unary-4.c: New test. * gcc.target/riscv/rvv/autovec/cond/cond_unary-5.c: New test. * gcc.target/riscv/rvv/autovec/cond/cond_unary-6.c: New test. * gcc.target/riscv/rvv/autovec/cond/cond_unary-7.c: New test. * gcc.target/riscv/rvv/autovec/cond/cond_unary-8.c: New test. * gcc.target/riscv/rvv/autovec/cond/cond_unary_run-1.c: New test. * gcc.target/riscv/rvv/autovec/cond/cond_unary_run-2.c: New test. * gcc.target/riscv/rvv/autovec/cond/cond_unary_run-3.c: New test. * gcc.target/riscv/rvv/autovec/cond/cond_unary_run-4.c: New test. * gcc.target/riscv/rvv/autovec/cond/cond_unary_run-5.c: New test. * gcc.target/riscv/rvv/autovec/cond/cond_unary_run-6.c: New test. * gcc.target/riscv/rvv/autovec/cond/cond_unary_run-7.c: New test. * gcc.target/riscv/rvv/autovec/cond/cond_unary_run-8.c: New test. --- diff --git a/gcc/config/riscv/autovec-opt.md b/gcc/config/riscv/autovec-opt.md index 99b609a99d97..8247eb87ddb5 100644 --- a/gcc/config/riscv/autovec-opt.md +++ b/gcc/config/riscv/autovec-opt.md @@ -684,3 +684,42 @@ } [(set_attr "type" "vfwmuladd") (set_attr "mode" "")]) + +;; Combine and vcond_mask generated by midend into cond_len_ +;; Currently supported operations: +;; abs(FP) +(define_insn_and_split "*cond_abs" + [(set (match_operand:VF 0 "register_operand") + (if_then_else:VF + (match_operand: 3 "register_operand") + (abs:VF (match_operand:VF 1 "nonmemory_operand")) + (match_operand:VF 2 "register_operand")))] + "TARGET_VECTOR && can_create_pseudo_p ()" + "#" + "&& 1" + [(const_int 0)] +{ + emit_insn (gen_cond_len_abs (operands[0], operands[3], operands[1], + operands[2], + gen_int_mode (GET_MODE_NUNITS (mode), Pmode), + const0_rtx)); + DONE; +}) + +;; Combine vlmax neg and UNSPEC_VCOPYSIGN +(define_insn_and_split "*copysign_neg" + [(set (match_operand:VF 0 "register_operand") + (neg:VF + (unspec:VF [ + (match_operand:VF 1 "register_operand") + (match_operand:VF 2 "register_operand") + ] UNSPEC_VCOPYSIGN)))] + "TARGET_VECTOR && can_create_pseudo_p ()" + "#" + "&& 1" + [(const_int 0)] +{ + riscv_vector::emit_vlmax_insn (code_for_pred_ncopysign (mode), + riscv_vector::RVV_BINOP, operands); + DONE; +}) diff --git a/gcc/config/riscv/autovec.md b/gcc/config/riscv/autovec.md index acca4c22b907..e1addc07036e 100644 --- a/gcc/config/riscv/autovec.md +++ b/gcc/config/riscv/autovec.md @@ -552,12 +552,16 @@ ;; - vfmerge.vf ;; ------------------------------------------------------------------------- -(define_expand "@vcond_mask_" - [(match_operand:V 0 "register_operand") - (match_operand: 3 "register_operand") - (match_operand:V 1 "nonmemory_operand") - (match_operand:V 2 "register_operand")] - "TARGET_VECTOR" +(define_insn_and_split "@vcond_mask_" + [(set (match_operand:V 0 "register_operand") + (if_then_else:V + (match_operand: 3 "register_operand") + (match_operand:V 1 "nonmemory_operand") + (match_operand:V 2 "register_operand")))] + "TARGET_VECTOR && can_create_pseudo_p ()" + "#" + "&& 1" + [(const_int 0)] { /* The order of vcond_mask is opposite to pred_merge. */ std::swap (operands[1], operands[2]); @@ -979,11 +983,14 @@ ;; Includes: ;; - vfneg.v/vfabs.v ;; ------------------------------------------------------------------------------- -(define_expand "2" +(define_insn_and_split "2" [(set (match_operand:VF 0 "register_operand") (any_float_unop_nofrm:VF (match_operand:VF 1 "register_operand")))] - "TARGET_VECTOR" + "TARGET_VECTOR && can_create_pseudo_p ()" + "#" + "&& 1" + [(const_int 0)] { insn_code icode = code_for_pred (, mode); riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_UNOP, operands); @@ -1499,6 +1506,80 @@ DONE; }) +;; ------------------------------------------------------------------------- +;; ---- [INT] Conditional unary operations +;; ------------------------------------------------------------------------- +;; Includes: +;; - vneg/vnot +;; ------------------------------------------------------------------------- + +(define_expand "cond_" + [(match_operand:VI 0 "register_operand") + (match_operand: 1 "vector_mask_operand") + (any_int_unop:VI + (match_operand:VI 2 "register_operand")) + (match_operand:VI 3 "register_operand")] + "TARGET_VECTOR" +{ + /* Normalize into cond_len_* operations. */ + emit_insn (gen_cond_len_ (operands[0], operands[1], operands[2], + operands[3], + gen_int_mode (GET_MODE_NUNITS (mode), Pmode), + const0_rtx)); + DONE; +}) + +(define_expand "cond_len_" + [(match_operand:VI 0 "register_operand") + (match_operand: 1 "vector_mask_operand") + (any_int_unop:VI + (match_operand:VI 2 "register_operand")) + (match_operand:VI 3 "register_operand") + (match_operand 4 "autovec_length_operand") + (match_operand 5 "const_0_operand")] + "TARGET_VECTOR" +{ + riscv_vector::expand_cond_len_unop (, operands); + DONE; +}) + +;; ------------------------------------------------------------------------- +;; ---- [FP] Conditional unary operations +;; ------------------------------------------------------------------------- +;; Includes: +;; - vfneg/vfabs +;; ------------------------------------------------------------------------- + +(define_expand "cond_" + [(match_operand:VF 0 "register_operand") + (match_operand: 1 "vector_mask_operand") + (any_float_unop_nofrm:VF + (match_operand:VF 2 "register_operand")) + (match_operand:VF 3 "register_operand")] + "TARGET_VECTOR" +{ + /* Normalize into cond_len_* operations. */ + emit_insn (gen_cond_len_ (operands[0], operands[1], operands[2], + operands[3], + gen_int_mode (GET_MODE_NUNITS (mode), Pmode), + const0_rtx)); + DONE; +}) + +(define_expand "cond_len_" + [(match_operand:VF 0 "register_operand") + (match_operand: 1 "vector_mask_operand") + (any_float_unop_nofrm:VF + (match_operand:VF 2 "register_operand")) + (match_operand:VF 3 "register_operand") + (match_operand 4 "autovec_length_operand") + (match_operand 5 "const_0_operand")] + "TARGET_VECTOR" +{ + riscv_vector::expand_cond_len_unop (, operands); + DONE; +}) + ;; ------------------------------------------------------------------------- ;; ---- [INT] Conditional binary operations ;; ------------------------------------------------------------------------- diff --git a/gcc/config/riscv/riscv-protos.h b/gcc/config/riscv/riscv-protos.h index 472c00dc4395..2c4405c98606 100644 --- a/gcc/config/riscv/riscv-protos.h +++ b/gcc/config/riscv/riscv-protos.h @@ -184,6 +184,10 @@ enum insn_type { RVV_MISC_OP = 1, RVV_UNOP = 2, + RVV_UNOP_M = RVV_UNOP + 2, + RVV_UNOP_MU = RVV_UNOP + 2, + RVV_UNOP_TU = RVV_UNOP + 2, + RVV_UNOP_TUMU = RVV_UNOP + 2, RVV_BINOP = 3, RVV_BINOP_MU = RVV_BINOP + 2, RVV_BINOP_TU = RVV_BINOP + 2, @@ -191,8 +195,6 @@ enum insn_type RVV_MERGE_OP = 4, RVV_CMP_OP = 4, RVV_CMP_MU_OP = RVV_CMP_OP + 2, /* +2 means mask and maskoff operand. */ - RVV_UNOP_MU = RVV_UNOP + 2, /* Likewise. */ - RVV_UNOP_M = RVV_UNOP + 2, /* Likewise. */ RVV_TERNOP = 5, RVV_TERNOP_MU = RVV_TERNOP + 1, RVV_TERNOP_TU = RVV_TERNOP + 1, @@ -294,6 +296,7 @@ bool neg_simm5_p (rtx); bool has_vi_variant_p (rtx_code, rtx); void expand_vec_cmp (rtx, rtx_code, rtx, rtx); bool expand_vec_cmp_float (rtx, rtx_code, rtx, rtx, bool); +void expand_cond_len_unop (rtx_code, rtx *); void expand_cond_len_binop (rtx_code, rtx *); void expand_reduction (rtx_code, rtx *, rtx, reduction_type = reduction_type::UNORDERED); diff --git a/gcc/config/riscv/riscv-v.cc b/gcc/config/riscv/riscv-v.cc index b01028c62013..14eda581d004 100644 --- a/gcc/config/riscv/riscv-v.cc +++ b/gcc/config/riscv/riscv-v.cc @@ -2992,7 +2992,9 @@ shuffle_merge_patterns (struct expand_vec_perm_d *d) emit_vlmax_cmp_insn (icode, ops); /* TARGET = MASK ? OP0 : OP1. */ - emit_insn (gen_vcond_mask (vmode, vmode, d->target, d->op0, d->op1, mask)); + /* swap op0 and op1 since the order is opposite to pred_merge. */ + rtx ops2[] = {d->target, d->op1, d->op0, mask}; + emit_vlmax_merge_insn (code_for_pred_merge (vmode), riscv_vector::RVV_MERGE_OP, ops2); return true; } @@ -3384,10 +3386,58 @@ needs_fp_rounding (rtx_code code, machine_mode mode) { if (!FLOAT_MODE_P (mode)) return false; - return code != SMIN && code != SMAX; + return code != SMIN && code != SMAX && code != NEG && code != ABS; } -/* Expand COND_LEN_*. */ +/* Expand unary ops COND_LEN_*. */ +void +expand_cond_len_unop (rtx_code code, rtx *ops) +{ + rtx dest = ops[0]; + rtx mask = ops[1]; + rtx src = ops[2]; + rtx merge = ops[3]; + rtx len = ops[4]; + machine_mode mode = GET_MODE (dest); + machine_mode mask_mode = GET_MODE (mask); + + poly_int64 value; + bool is_dummy_mask = rtx_equal_p (mask, CONSTM1_RTX (mask_mode)); + bool is_vlmax_len + = poly_int_rtx_p (len, &value) && known_eq (value, GET_MODE_NUNITS (mode)); + rtx cond_ops[] = {dest, mask, merge, src}; + insn_code icode = code_for_pred (code, mode); + + if (is_dummy_mask) + { + /* Use TU, MASK ANY policy. */ + if (needs_fp_rounding (code, mode)) + emit_nonvlmax_fp_tu_insn (icode, RVV_UNOP_TU, cond_ops, len); + else + emit_nonvlmax_tu_insn (icode, RVV_UNOP_TU, cond_ops, len); + } + else + { + if (is_vlmax_len) + { + /* Use TAIL ANY, MU policy. */ + if (needs_fp_rounding (code, mode)) + emit_vlmax_masked_fp_mu_insn (icode, RVV_UNOP_MU, cond_ops); + else + emit_vlmax_masked_mu_insn (icode, RVV_UNOP_MU, cond_ops); + } + else + { + /* Use TU, MU policy. */ + if (needs_fp_rounding (code, mode)) + emit_nonvlmax_fp_tumu_insn (icode, RVV_UNOP_TUMU, cond_ops, len); + else + emit_nonvlmax_tumu_insn (icode, RVV_UNOP_TUMU, cond_ops, len); + } + } +} + +/* Expand binary ops COND_LEN_*. */ void expand_cond_len_binop (rtx_code code, rtx *ops) { diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary-1.c new file mode 100644 index 000000000000..8076243f7d4a --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary-1.c @@ -0,0 +1,43 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-march=rv64gcv_zvfh -mabi=lp64d --param riscv-autovec-preference=scalable -fno-vect-cost-model" } */ + +#include + +#define abs(A) ((A) < 0 ? -(A) : (A)) +#define neg(A) (-(A)) +#define not(A) (~(A)) + +#define DEF_LOOP(TYPE, OP) \ + void __attribute__ ((noipa)) \ + test_##TYPE##_##OP (TYPE *__restrict r, TYPE *__restrict a, \ + TYPE *__restrict pred, int n) \ + { \ + for (int i = 0; i < n; ++i) \ + r[i] = pred[i] ? OP (a[i]) : a[i]; \ + } + +#define TEST_INT_TYPE(T, TYPE) \ + T (TYPE, abs) \ + T (TYPE, neg) \ + T (TYPE, not ) + +#define TEST_FLOAT_TYPE(T, TYPE, SUFFIX) \ + T (TYPE, __builtin_fabs##SUFFIX) \ + T (TYPE, neg) + +#define TEST_ALL(T) \ + TEST_INT_TYPE (T, int8_t) \ + TEST_INT_TYPE (T, int16_t) \ + TEST_INT_TYPE (T, int32_t) \ + TEST_INT_TYPE (T, int64_t) \ + TEST_FLOAT_TYPE (T, _Float16, f16) \ + TEST_FLOAT_TYPE (T, float, f) \ + TEST_FLOAT_TYPE (T, double, ) + +TEST_ALL (DEF_LOOP) + +/* NOTE: int abs operator is converted to vmslt + vneg.v */ +/* { dg-final { scan-assembler-times {\tvneg\.v\tv[0-9]+,v[0-9]+,v0\.t} 8 } } */ +/* { dg-final { scan-assembler-times {\tvnot\.v\tv[0-9]+,v[0-9]+,v0\.t} 4 } } */ +/* { dg-final { scan-assembler-times {\tvfabs\.v\tv[0-9]+,v[0-9]+,v0\.t} 3 } } */ +/* { dg-final { scan-assembler-times {\tvfneg\.v\tv[0-9]+,v[0-9]+,v0\.t} 3 } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary-2.c new file mode 100644 index 000000000000..8e44301ae805 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary-2.c @@ -0,0 +1,46 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-march=rv64gcv_zvfh -mabi=lp64d --param riscv-autovec-preference=scalable -fno-vect-cost-model" } */ + +#include + +#define abs(A) ((A) < 0 ? -(A) : (A)) +#define neg(A) (-(A)) +#define not(A) (~(A)) + +#define DEF_LOOP(TYPE, OP) \ + void __attribute__ ((noipa)) \ + test_##TYPE##_##OP (TYPE *__restrict r, TYPE *__restrict a, \ + TYPE *__restrict b, TYPE *__restrict pred, int n) \ + { \ + for (int i = 0; i < n; ++i) \ + { \ + TYPE bi = b[i]; \ + r[i] = pred[i] ? OP (a[i]) : bi; \ + } \ + } + +#define TEST_INT_TYPE(T, TYPE) \ + T (TYPE, abs) \ + T (TYPE, neg) \ + T (TYPE, not ) + +#define TEST_FLOAT_TYPE(T, TYPE, SUFFIX) \ + T (TYPE, __builtin_fabs##SUFFIX) \ + T (TYPE, neg) + +#define TEST_ALL(T) \ + TEST_INT_TYPE (T, int8_t) \ + TEST_INT_TYPE (T, int16_t) \ + TEST_INT_TYPE (T, int32_t) \ + TEST_INT_TYPE (T, int64_t) \ + TEST_FLOAT_TYPE (T, _Float16, f16) \ + TEST_FLOAT_TYPE (T, float, f) \ + TEST_FLOAT_TYPE (T, double, ) + +TEST_ALL (DEF_LOOP) + +/* NOTE: int abs operator is converted to vmslt + vneg.v */ +/* { dg-final { scan-assembler-times {\tvneg\.v\tv[0-9]+,v[0-9]+,v0\.t} 8 } } */ +/* { dg-final { scan-assembler-times {\tvnot\.v\tv[0-9]+,v[0-9]+,v0\.t} 4 } } */ +/* { dg-final { scan-assembler-times {\tvfabs\.v\tv[0-9]+,v[0-9]+,v0\.t} 3 } } */ +/* { dg-final { scan-assembler-times {\tvfneg\.v\tv[0-9]+,v[0-9]+,v0\.t} 3 } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary-3.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary-3.c new file mode 100644 index 000000000000..6da5b6e42e3c --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary-3.c @@ -0,0 +1,43 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-march=rv64gcv_zvfh -mabi=lp64d --param riscv-autovec-preference=scalable -fno-vect-cost-model" } */ + +#include + +#define abs(A) ((A) < 0 ? -(A) : (A)) +#define neg(A) (-(A)) +#define not(A) (~(A)) + +#define DEF_LOOP(TYPE, OP) \ + void __attribute__ ((noipa)) \ + test_##TYPE##_##OP (TYPE *__restrict r, TYPE *__restrict a, \ + TYPE *__restrict pred, int n) \ + { \ + for (int i = 0; i < n; ++i) \ + r[i] = pred[i] ? OP (a[i]) : 5; \ + } + +#define TEST_INT_TYPE(T, TYPE) \ + T (TYPE, abs) \ + T (TYPE, neg) \ + T (TYPE, not ) + +#define TEST_FLOAT_TYPE(T, TYPE, SUFFIX) \ + T (TYPE, __builtin_fabs##SUFFIX) \ + T (TYPE, neg) + +#define TEST_ALL(T) \ + TEST_INT_TYPE (T, int8_t) \ + TEST_INT_TYPE (T, int16_t) \ + TEST_INT_TYPE (T, int32_t) \ + TEST_INT_TYPE (T, int64_t) \ + TEST_FLOAT_TYPE (T, _Float16, f16) \ + TEST_FLOAT_TYPE (T, float, f) \ + TEST_FLOAT_TYPE (T, double, ) + +TEST_ALL (DEF_LOOP) + +/* NOTE: int abs operator is converted to vmslt + vneg.v */ +/* { dg-final { scan-assembler-times {\tvneg\.v\tv[0-9]+,v[0-9]+,v0\.t} 8 } } */ +/* { dg-final { scan-assembler-times {\tvnot\.v\tv[0-9]+,v[0-9]+,v0\.t} 4 } } */ +/* { dg-final { scan-assembler-times {\tvfabs\.v\tv[0-9]+,v[0-9]+,v0\.t} 3 } } */ +/* { dg-final { scan-assembler-times {\tvfneg\.v\tv[0-9]+,v[0-9]+,v0\.t} 3 } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary-4.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary-4.c new file mode 100644 index 000000000000..5428c289d22f --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary-4.c @@ -0,0 +1,43 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-march=rv64gcv_zvfh -mabi=lp64d --param riscv-autovec-preference=scalable -fno-vect-cost-model" } */ + +#include + +#define abs(A) ((A) < 0 ? -(A) : (A)) +#define neg(A) (-(A)) +#define not(A) (~(A)) + +#define DEF_LOOP(TYPE, OP) \ + void __attribute__ ((noipa)) \ + test_##TYPE##_##OP (TYPE *__restrict r, TYPE *__restrict a, \ + TYPE *__restrict pred, int n) \ + { \ + for (int i = 0; i < n; ++i) \ + r[i] = pred[i] ? OP (a[i]) : 0; \ + } + +#define TEST_INT_TYPE(T, TYPE) \ + T (TYPE, abs) \ + T (TYPE, neg) \ + T (TYPE, not ) + +#define TEST_FLOAT_TYPE(T, TYPE, SUFFIX) \ + T (TYPE, __builtin_fabs##SUFFIX) \ + T (TYPE, neg) + +#define TEST_ALL(T) \ + TEST_INT_TYPE (T, int8_t) \ + TEST_INT_TYPE (T, int16_t) \ + TEST_INT_TYPE (T, int32_t) \ + TEST_INT_TYPE (T, int64_t) \ + TEST_FLOAT_TYPE (T, _Float16, f16) \ + TEST_FLOAT_TYPE (T, float, f) \ + TEST_FLOAT_TYPE (T, double, ) + +TEST_ALL (DEF_LOOP) + +/* NOTE: int abs operator is converted to vmslt + vneg.v */ +/* { dg-final { scan-assembler-times {\tvneg\.v\tv[0-9]+,v[0-9]+,v0\.t} 8 } } */ +/* { dg-final { scan-assembler-times {\tvnot\.v\tv[0-9]+,v[0-9]+,v0\.t} 4 } } */ +/* { dg-final { scan-assembler-times {\tvfabs\.v\tv[0-9]+,v[0-9]+,v0\.t} 3 } } */ +/* { dg-final { scan-assembler-times {\tvfneg\.v\tv[0-9]+,v[0-9]+,v0\.t} 3 } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary-5.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary-5.c new file mode 100644 index 000000000000..8e567378d0d9 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary-5.c @@ -0,0 +1,36 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-march=rv64gcv_zvfh -mabi=lp64d --param riscv-autovec-preference=scalable -fno-vect-cost-model" } */ + +#include + +#define abs(A) ((A) < 0 ? -(A) : (A)) +#define neg(A) (-(A)) +#define not(A) (~(A)) + +#define DEF_LOOP(TYPE1, TYPE2, COUNT, OP) \ + void __attribute__ ((noipa)) \ + test_##TYPE1##_##TYPE2##_##OP (TYPE2 *__restrict r, TYPE2 *__restrict a, \ + TYPE1 *__restrict pred) \ + { \ + for (int i = 0; i < COUNT; ++i) \ + r[i] = pred[i] ? OP (a[i]) : a[i]; \ + } + +#define TEST_TYPES(T, TYPE1, TYPE2, COUNT) \ + T (TYPE1, TYPE2, COUNT, abs) \ + T (TYPE1, TYPE2, COUNT, neg) \ + T (TYPE1, TYPE2, COUNT, not ) + +#define TEST_ALL(T) \ + TEST_TYPES (T, int16_t, int8_t, 7) \ + TEST_TYPES (T, int32_t, int8_t, 3) \ + TEST_TYPES (T, int32_t, int16_t, 3) \ + TEST_TYPES (T, int64_t, int8_t, 42) \ + TEST_TYPES (T, int64_t, int16_t, 42) \ + TEST_TYPES (T, int64_t, int32_t, 42) + +TEST_ALL (DEF_LOOP) + +/* NOTE: int abs operator is converted to vmslt + vneg.v */ +/* { dg-final { scan-assembler-times {\tvneg\.v\tv[0-9]+,v[0-9]+,v0\.t} 12 } } */ +/* { dg-final { scan-assembler-times {\tvnot\.v\tv[0-9]+,v[0-9]+,v0\.t} 6 } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary-6.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary-6.c new file mode 100644 index 000000000000..65a36d0e52a4 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary-6.c @@ -0,0 +1,39 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-march=rv64gcv_zvfh -mabi=lp64d --param riscv-autovec-preference=scalable -fno-vect-cost-model" } */ + +#include + +#define abs(A) ((A) < 0 ? -(A) : (A)) +#define neg(A) (-(A)) +#define not(A) (~(A)) + +#define DEF_LOOP(TYPE1, TYPE2, COUNT, OP) \ + void __attribute__ ((noipa)) \ + test_##TYPE1##_##TYPE2##_##OP (TYPE2 *__restrict r, TYPE2 *__restrict a, \ + TYPE2 *__restrict b, TYPE1 *__restrict pred) \ + { \ + for (int i = 0; i < COUNT; ++i) \ + { \ + TYPE2 bi = b[i]; \ + r[i] = pred[i] ? OP (a[i]) : bi; \ + } \ + } + +#define TEST_TYPES(T, TYPE1, TYPE2, COUNT) \ + T (TYPE1, TYPE2, COUNT, abs) \ + T (TYPE1, TYPE2, COUNT, neg) \ + T (TYPE1, TYPE2, COUNT, not ) + +#define TEST_ALL(T) \ + TEST_TYPES (T, int16_t, int8_t, 7) \ + TEST_TYPES (T, int32_t, int8_t, 3) \ + TEST_TYPES (T, int32_t, int16_t, 3) \ + TEST_TYPES (T, int64_t, int8_t, 42) \ + TEST_TYPES (T, int64_t, int16_t, 42) \ + TEST_TYPES (T, int64_t, int32_t, 42) + +TEST_ALL (DEF_LOOP) + +/* NOTE: int abs operator is converted to vmslt + vneg.v */ +/* { dg-final { scan-assembler-times {\tvneg\.v\tv[0-9]+,v[0-9]+,v0\.t} 12 } } */ +/* { dg-final { scan-assembler-times {\tvnot\.v\tv[0-9]+,v[0-9]+,v0\.t} 6 } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary-7.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary-7.c new file mode 100644 index 000000000000..356fe9fc25a0 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary-7.c @@ -0,0 +1,36 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-march=rv64gcv_zvfh -mabi=lp64d --param riscv-autovec-preference=scalable -fno-vect-cost-model" } */ + +#include + +#define abs(A) ((A) < 0 ? -(A) : (A)) +#define neg(A) (-(A)) +#define not(A) (~(A)) + +#define DEF_LOOP(TYPE1, TYPE2, COUNT, OP) \ + void __attribute__ ((noipa)) \ + test_##TYPE1##_##TYPE2##_##OP (TYPE2 *__restrict r, TYPE2 *__restrict a, \ + TYPE1 *__restrict pred) \ + { \ + for (int i = 0; i < COUNT; ++i) \ + r[i] = pred[i] ? OP (a[i]) : 5; \ + } + +#define TEST_TYPES(T, TYPE1, TYPE2, COUNT) \ + T (TYPE1, TYPE2, COUNT, abs) \ + T (TYPE1, TYPE2, COUNT, neg) \ + T (TYPE1, TYPE2, COUNT, not ) + +#define TEST_ALL(T) \ + TEST_TYPES (T, int16_t, int8_t, 7) \ + TEST_TYPES (T, int32_t, int8_t, 3) \ + TEST_TYPES (T, int32_t, int16_t, 3) \ + TEST_TYPES (T, int64_t, int8_t, 42) \ + TEST_TYPES (T, int64_t, int16_t, 42) \ + TEST_TYPES (T, int64_t, int32_t, 42) + +TEST_ALL (DEF_LOOP) + +/* NOTE: int abs operator is converted to vmslt + vneg.v */ +/* { dg-final { scan-assembler-times {\tvneg\.v\tv[0-9]+,v[0-9]+,v0\.t} 12 } } */ +/* { dg-final { scan-assembler-times {\tvnot\.v\tv[0-9]+,v[0-9]+,v0\.t} 6 } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary-8.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary-8.c new file mode 100644 index 000000000000..5208a858882d --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary-8.c @@ -0,0 +1,36 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-march=rv64gcv_zvfh -mabi=lp64d --param riscv-autovec-preference=scalable -fno-vect-cost-model" } */ + +#include + +#define abs(A) ((A) < 0 ? -(A) : (A)) +#define neg(A) (-(A)) +#define not(A) (~(A)) + +#define DEF_LOOP(TYPE1, TYPE2, COUNT, OP) \ + void __attribute__ ((noipa)) \ + test_##TYPE1##_##TYPE2##_##OP (TYPE2 *__restrict r, TYPE2 *__restrict a, \ + TYPE1 *__restrict pred) \ + { \ + for (int i = 0; i < COUNT; ++i) \ + r[i] = pred[i] ? OP (a[i]) : 0; \ + } + +#define TEST_TYPES(T, TYPE1, TYPE2, COUNT) \ + T (TYPE1, TYPE2, COUNT, abs) \ + T (TYPE1, TYPE2, COUNT, neg) \ + T (TYPE1, TYPE2, COUNT, not ) + +#define TEST_ALL(T) \ + TEST_TYPES (T, int16_t, int8_t, 7) \ + TEST_TYPES (T, int32_t, int8_t, 3) \ + TEST_TYPES (T, int32_t, int16_t, 3) \ + TEST_TYPES (T, int64_t, int8_t, 42) \ + TEST_TYPES (T, int64_t, int16_t, 42) \ + TEST_TYPES (T, int64_t, int32_t, 42) + +TEST_ALL (DEF_LOOP) + +/* NOTE: int abs operator is converted to vmslt + vneg.v */ +/* { dg-final { scan-assembler-times {\tvneg\.v\tv[0-9]+,v[0-9]+,v0\.t} 12 } } */ +/* { dg-final { scan-assembler-times {\tvnot\.v\tv[0-9]+,v[0-9]+,v0\.t} 6 } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary_run-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary_run-1.c new file mode 100644 index 000000000000..3de65aa7af18 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary_run-1.c @@ -0,0 +1,27 @@ +/* { dg-do run { target { riscv_vector } } } */ +/* { dg-additional-options "--param riscv-autovec-preference=scalable -fno-vect-cost-model" } */ + +#include "cond_unary-1.c" + +#define N 99 + +#define TEST_LOOP(TYPE, OP) \ + { \ + TYPE r[N], a[N], pred[N]; \ + for (int i = 0; i < N; ++i) \ + { \ + a[i] = (i & 1 ? i : 3 * i) * (i % 3 == 0 ? 1 : -1); \ + pred[i] = (i % 7 < 4); \ + asm volatile ("" ::: "memory"); \ + } \ + test_##TYPE##_##OP (r, a, pred, N); \ + for (int i = 0; i < N; ++i) \ + if (r[i] != (pred[i] ? OP (a[i]) : a[i])) \ + __builtin_abort (); \ + } + +int main () +{ + TEST_ALL (TEST_LOOP) + return 0; +} diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary_run-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary_run-2.c new file mode 100644 index 000000000000..799f1f7f9d26 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary_run-2.c @@ -0,0 +1,28 @@ +/* { dg-do run { target { riscv_vector } } } */ +/* { dg-additional-options "--param riscv-autovec-preference=scalable -fno-vect-cost-model" } */ + +#include "cond_unary-2.c" + +#define N 99 + +#define TEST_LOOP(TYPE, OP) \ + { \ + TYPE r[N], a[N], b[N], pred[N]; \ + for (int i = 0; i < N; ++i) \ + { \ + a[i] = (i & 1 ? i : 3 * i) * (i % 3 == 0 ? 1 : -1); \ + b[i] = (i % 9) * (i % 7 + 1); \ + pred[i] = (i % 7 < 4); \ + asm volatile ("" ::: "memory"); \ + } \ + test_##TYPE##_##OP (r, a, b, pred, N); \ + for (int i = 0; i < N; ++i) \ + if (r[i] != (pred[i] ? OP (a[i]) : b[i])) \ + __builtin_abort (); \ + } + +int main () +{ + TEST_ALL (TEST_LOOP) + return 0; +} diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary_run-3.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary_run-3.c new file mode 100644 index 000000000000..6ad7e6178768 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary_run-3.c @@ -0,0 +1,27 @@ +/* { dg-do run { target { riscv_vector } } } */ +/* { dg-additional-options "--param riscv-autovec-preference=scalable -fno-vect-cost-model" } */ + +#include "cond_unary-3.c" + +#define N 99 + +#define TEST_LOOP(TYPE, OP) \ + { \ + TYPE r[N], a[N], pred[N]; \ + for (int i = 0; i < N; ++i) \ + { \ + a[i] = (i & 1 ? i : 3 * i) * (i % 3 == 0 ? 1 : -1); \ + pred[i] = (i % 7 < 4); \ + asm volatile ("" ::: "memory"); \ + } \ + test_##TYPE##_##OP (r, a, pred, N); \ + for (int i = 0; i < N; ++i) \ + if (r[i] != (pred[i] ? OP (a[i]) : 5)) \ + __builtin_abort (); \ + } + +int main () +{ + TEST_ALL (TEST_LOOP) + return 0; +} diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary_run-4.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary_run-4.c new file mode 100644 index 000000000000..cd7934b4f1aa --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary_run-4.c @@ -0,0 +1,27 @@ +/* { dg-do run { target { riscv_vector } } } */ +/* { dg-additional-options "--param riscv-autovec-preference=scalable -fno-vect-cost-model" } */ + +#include "cond_unary-4.c" + +#define N 99 + +#define TEST_LOOP(TYPE, OP) \ + { \ + TYPE r[N], a[N], pred[N]; \ + for (int i = 0; i < N; ++i) \ + { \ + a[i] = (i & 1 ? i : 3 * i) * (i % 3 == 0 ? 1 : -1); \ + pred[i] = (i % 7 < 4); \ + asm volatile ("" ::: "memory"); \ + } \ + test_##TYPE##_##OP (r, a, pred, N); \ + for (int i = 0; i < N; ++i) \ + if (r[i] != (pred[i] ? OP (a[i]) : 0)) \ + __builtin_abort (); \ + } + +int main () +{ + TEST_ALL (TEST_LOOP) + return 0; +} diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary_run-5.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary_run-5.c new file mode 100644 index 000000000000..38e9fb86b7e2 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary_run-5.c @@ -0,0 +1,26 @@ +/* { dg-do run { target { riscv_vector } } } */ +/* { dg-additional-options "--param riscv-autovec-preference=scalable -fno-vect-cost-model" } */ + +#include "cond_unary-5.c" + +#define TEST_LOOP(TYPE1, TYPE2, N, OP) \ + { \ + TYPE1 pred[N]; \ + TYPE2 r[N], a[N]; \ + for (int i = 0; i < N; ++i) \ + { \ + a[i] = (i & 1 ? i : 3 * i) * (i % 3 == 0 ? 1 : -1); \ + pred[i] = (i % 4 < 2); \ + asm volatile ("" ::: "memory"); \ + } \ + test_##TYPE1##_##TYPE2##_##OP (r, a, pred); \ + for (int i = 0; i < N; ++i) \ + if (r[i] != (pred[i] ? OP (a[i]) : a[i])) \ + __builtin_abort (); \ + } + +int main () +{ + TEST_ALL (TEST_LOOP) + return 0; +} diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary_run-6.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary_run-6.c new file mode 100644 index 000000000000..71f846437092 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary_run-6.c @@ -0,0 +1,27 @@ +/* { dg-do run { target { riscv_vector } } } */ +/* { dg-additional-options "--param riscv-autovec-preference=scalable -fno-vect-cost-model" } */ + +#include "cond_unary-6.c" + +#define TEST_LOOP(TYPE1, TYPE2, N, OP) \ + { \ + TYPE1 pred[N]; \ + TYPE2 r[N], a[N], b[N]; \ + for (int i = 0; i < N; ++i) \ + { \ + a[i] = (i & 1 ? i : 3 * i) * (i % 3 == 0 ? 1 : -1); \ + b[i] = (i % 5) * (i % 6 + 3); \ + pred[i] = (i % 4 < 2); \ + asm volatile ("" ::: "memory"); \ + } \ + test_##TYPE1##_##TYPE2##_##OP (r, a, b, pred); \ + for (int i = 0; i < N; ++i) \ + if (r[i] != (pred[i] ? OP (a[i]) : b[i])) \ + __builtin_abort (); \ + } + +int main () +{ + TEST_ALL (TEST_LOOP) + return 0; +} diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary_run-7.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary_run-7.c new file mode 100644 index 000000000000..7a52b0cea739 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary_run-7.c @@ -0,0 +1,26 @@ +/* { dg-do run { target { riscv_vector } } } */ +/* { dg-additional-options "--param riscv-autovec-preference=scalable -fno-vect-cost-model" } */ + +#include "cond_unary-7.c" + +#define TEST_LOOP(TYPE1, TYPE2, N, OP) \ + { \ + TYPE1 pred[N]; \ + TYPE2 r[N], a[N]; \ + for (int i = 0; i < N; ++i) \ + { \ + a[i] = (i & 1 ? i : 3 * i) * (i % 3 == 0 ? 1 : -1); \ + pred[i] = (i % 4 < 2); \ + asm volatile ("" ::: "memory"); \ + } \ + test_##TYPE1##_##TYPE2##_##OP (r, a, pred); \ + for (int i = 0; i < N; ++i) \ + if (r[i] != (pred[i] ? OP (a[i]) : 5)) \ + __builtin_abort (); \ + } + +int main () +{ + TEST_ALL (TEST_LOOP) + return 0; +} diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary_run-8.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary_run-8.c new file mode 100644 index 000000000000..ee5c2f384968 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_unary_run-8.c @@ -0,0 +1,28 @@ +/* { dg-do run { target { riscv_vector } } } */ +/* { dg-additional-options "--param riscv-autovec-preference=scalable -fno-vect-cost-model" } */ + +#include "cond_unary-8.c" + +#define N 99 + +#define TEST_LOOP(TYPE1, TYPE2, N, OP) \ + { \ + TYPE1 pred[N]; \ + TYPE2 r[N], a[N]; \ + for (int i = 0; i < N; ++i) \ + { \ + a[i] = (i & 1 ? i : 3 * i) * (i % 3 == 0 ? 1 : -1); \ + pred[i] = (i % 4 < 2); \ + asm volatile ("" ::: "memory"); \ + } \ + test_##TYPE1##_##TYPE2##_##OP (r, a, pred); \ + for (int i = 0; i < N; ++i) \ + if (r[i] != (pred[i] ? OP (a[i]) : 0)) \ + __builtin_abort (); \ + } + +int main () +{ + TEST_ALL (TEST_LOOP) + return 0; +}