MEDIUM: regex: Use pcre_study always when PCRE is used, regardless of JIT

pcre_study() has been around long before JIT has been added. It also seems to
affect the performance in some cases (positive).

Below I've attached some test restults. The test is based on
http://sljit.sourceforge.net/regex_perf.html (see bottom). It has been modified
to just test pcre_study vs. no pcre_study. Note: This test does not try to
match specific header it's instead run over a larger text with more and less
complex patterns to make the differences more clear.

% ./runtest
'mark.txt' loaded. (Length: 19665221 bytes)
-----------------
Regex: 'Twain'
[pcre-nostudy] time:    14 ms (2388 matches)
[pcre-study] time:    21 ms (2388 matches)
-----------------
Regex: '^Twain'
[pcre-nostudy] time:   109 ms (100 matches)
[pcre-study] time:   109 ms (100 matches)
-----------------
Regex: 'Twain$'
[pcre-nostudy] time:    14 ms (127 matches)
[pcre-study] time:    16 ms (127 matches)
-----------------
Regex: 'Huck[a-zA-Z]+|Finn[a-zA-Z]+'
[pcre-nostudy] time:   695 ms (83 matches)
[pcre-study] time:    26 ms (83 matches)
-----------------
Regex: 'a[^x]{20}b'
[pcre-nostudy] time:    90 ms (12495 matches)
[pcre-study] time:    91 ms (12495 matches)
-----------------
Regex: 'Tom|Sawyer|Huckleberry|Finn'
[pcre-nostudy] time:  1236 ms (3015 matches)
[pcre-study] time:    34 ms (3015 matches)
-----------------
Regex: '.{0,3}(Tom|Sawyer|Huckleberry|Finn)'
[pcre-nostudy] time:  5696 ms (3015 matches)
[pcre-study] time:  5655 ms (3015 matches)
-----------------
Regex: '[a-zA-Z]+ing'
[pcre-nostudy] time:  1290 ms (95863 matches)
[pcre-study] time:  1167 ms (95863 matches)
-----------------
Regex: '^[a-zA-Z]{0,4}ing[^a-zA-Z]'
[pcre-nostudy] time:   136 ms (4507 matches)
[pcre-study] time:   134 ms (4507 matches)
-----------------
Regex: '[a-zA-Z]+ing$'
[pcre-nostudy] time:  1334 ms (5360 matches)
[pcre-study] time:  1214 ms (5360 matches)
-----------------
Regex: '^[a-zA-Z ]{5,}$'
[pcre-nostudy] time:   198 ms (26236 matches)
[pcre-study] time:   197 ms (26236 matches)
-----------------
Regex: '^.{16,20}$'
[pcre-nostudy] time:   173 ms (4902 matches)
[pcre-study] time:   175 ms (4902 matches)
-----------------
Regex: '([a-f](.[d-m].){0,2}[h-n]){2}'
[pcre-nostudy] time:  1242 ms (68621 matches)
[pcre-study] time:   690 ms (68621 matches)
-----------------
Regex: '([A-Za-z]awyer|[A-Za-z]inn)[^a-zA-Z]'
[pcre-nostudy] time:  1215 ms (675 matches)
[pcre-study] time:   952 ms (675 matches)
-----------------
Regex: '"[^"]{0,30}[?!\.]"'
[pcre-nostudy] time:    27 ms (5972 matches)
[pcre-study] time:    28 ms (5972 matches)
-----------------
Regex: 'Tom.{10,25}river|river.{10,25}Tom'
[pcre-nostudy] time:   705 ms (2 matches)
[pcre-study] time:    68 ms (2 matches)

In some cases it's more or less the same but when it's faster than by a huge margin.
It always depends on the pattern, the string(s) to match against etc.

Signed-off-by: Christian Ruppert <c.ruppert@babiel.com>

diff --git a/include/common/regex.h b/include/common/regex.h
index 762eb56f2d3f0bfce61e81a9e6a9a889b8f11963..58b9dc14aaf1bdff455c20a607ce0081b1215bf3 100644 (file)
--- a/include/common/regex.h
+++ b/include/common/regex.h
@@ -29,6 +29,12 @@
 #ifdef USE_PCRE
 #include <pcre.h>
 #include <pcreposix.h>
+
+/* For pre-8.20 PCRE compatibility */
+#ifndef PCRE_STUDY_JIT_COMPILE
+#define PCRE_STUDY_JIT_COMPILE 0
+#endif
+
 #else /* no PCRE */
 #include <regex.h>
 #endif
@@ -129,9 +135,14 @@ int regex_exec_match2(const struct my_regex *preg, char *subject, int length,
 static inline void regex_free(struct my_regex *preg) {
 #if defined(USE_PCRE) || defined(USE_PCRE_JIT)
        pcre_free(preg->reg);
-#ifdef USE_PCRE_JIT
+/* PCRE < 8.20 requires pcre_free() while >= 8.20 requires pcre_study_free(),
+ * which is easily detected using PCRE_CONFIG_JIT.
+ */
+#ifdef PCRE_CONFIG_JIT
        pcre_free_study(preg->extra);
-#endif
+#else /* PCRE_CONFIG_JIT */
+       pcre_free(preg->extra);
+#endif /* PCRE_CONFIG_JIT */
 #else
        regfree(&preg->regex);
 #endif

diff --git a/src/regex.c b/src/regex.c
index c0b23cb12515f3a584209666938ae902e44e23b5..760a1eda41d90efdc224979e66dd60325b023105 100644 (file)
--- a/src/regex.c
+++ b/src/regex.c
@@ -288,16 +288,12 @@ int regex_comp(const char *str, struct my_regex *regex, int cs, int cap, char **
                return 0;
        }
 
-#ifdef USE_PCRE_JIT
        regex->extra = pcre_study(regex->reg, PCRE_STUDY_JIT_COMPILE, &error);
        if (!regex->extra && error != NULL) {
                pcre_free(regex->reg);
                memprintf(err, "failed to compile regex '%s' (error=%s)", str, error);
                return 0;
        }
-#else
-       regex->extra = NULL;
-#endif
 #else
        int flags = REG_EXTENDED;