From: William Lallemand Date: Fri, 21 Jul 2023 08:54:41 +0000 (+0200) Subject: MEDIUM: sample: implement us and ms variant of utime and ltime X-Git-Tag: v2.9-dev3~100 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=5e63e1636a6c5e2ca454f177fde34e27a6e11c33;p=thirdparty%2Fhaproxy.git MEDIUM: sample: implement us and ms variant of utime and ltime Implement 4 new fetches: - ms_ltime - ms_utime - us_ltime - us_utime Which are the same as ltime and utime but with milliseconds and microseconds input. The converters also suports the %N conversion specifier like in date(1). Unfortunately since %N is not supported by strftime, the format string is parsed twice, once manually to replace %N, and once by strftime. --- diff --git a/doc/configuration.txt b/doc/configuration.txt index e125f59c99..0cf36624e7 100644 --- a/doc/configuration.txt +++ b/doc/configuration.txt @@ -18566,6 +18566,56 @@ mqtt_is_valid acl data_in_buffer req.len ge 4 tcp-request content reject unless { req.payload(0,0),mqtt_is_valid } +ms_ltime([,]) + This works like "ltime" but takes an input in milliseconds. It also supports + the %N conversion specifier inspired by date(1). + Converts an integer supposed to contain a date since epoch to a string + representing this date in local time using a format defined by the + string using strftime(3). The purpose is to allow any date format to be used + in logs. An optional in milliseconds may be applied to the input date + (positive or negative). See the strftime() man page for the format supported + by your operating system. + + The %N conversion specifier allows you to output the nanoseconds part of the + date, precision is limited since the input is milliseconds. + (000000000..999000000). %N can take a width argument between % and N. It is + useful to display milliseconds (%3N) or microseconds (%6N). The default and + maximum width is 9 (%N = %9N). + + See also the utime converter for UTC as well as "ltime" and "us_ltime" + converters. + + Example : + + # Emit 3 colons, the local time, the timezone and another with ip:port + # e.g. 2023/07/24/11:53:02.196 +0200 127.0.0.1:41530 + log-format %[accept_date(ms),ms_ltime("%Y/%m/%d/%H:%M:%S.%3N %z")]\ %ci:%cp + +ms_utime([,]) + This works like "utime" but takes an input in milliseconds. It also supports + the %N conversion specifier inspired by date(1). + Converts an integer supposed to contain a date since epoch to a string + representing this date in UTC time using a format defined by the + string using strftime(3). The purpose is to allow any date format to be used + in logs. An optional in milliseconds may be applied to the input date + (positive or negative). See the strftime() man page for the format supported + by your operating system. + + The %N conversion specifier allows you to output the nanoseconds part of the + date, precision is limited since the input is milliseconds. + (000000000..999000000). %N can take a width argument between % and N. It is + useful to display milliseconds (%3N) or microseconds (%6N). The default and + maximum width is 9 (%N = %9N). + + See also the ltime converter for local as well as "utime" and "us_utime" + converters. + + Example : + + # Emit 3 colons, the UTC time, the timezone and another with ip:port + # e.g. 2023/07/24/09:53:02.196 +0000 127.0.0.1:41530 + log-format %[accept_date(ms),ms_utime("%Y/%m/%d/%H:%M:%S.%3N %z")]\ %ci:%cp + mul() Multiplies the input value of type signed integer by , and returns the product as an signed integer. In case of overflow, the largest possible @@ -19172,13 +19222,64 @@ unset-var() This prefix is followed by a name. The separator is a '.'. The name may only contain characters 'a-z', 'A-Z', '0-9', '.' and '_'. +us_ltime([,]) + This works like "ltime" but takes an input in microseconds. It also supports + the %N conversion specifier inspired by date(1). + Converts an integer supposed to contain a date since epoch to a string + representing this date in local time using a format defined by the + string using strftime(3). The purpose is to allow any date format to be used + in logs. An optional in microseconds may be applied to the input + date (positive or negative). See the strftime() man page for the format + supported by your operating system. + + The %N conversion specifier allows you to output the nanoseconds part of the + date, precision is limited since the input is microseconds. + (000000000..999999000). %N can take a width argument between % and N. It is + useful to display milliseconds (%3N) or microseconds (%6N). The default and + maximum width is 9 (%N = %9N). + + See also the "utime" converter for UTC as well as "ltime" and "ms_ltime" + converters. + + Example : + + # Emit 3 colons, the local time, the timezone and another with ip:port + # e.g. 2023/07/24/09:53:02.196234 +0000 127.0.0.1:41530 + log-format %[accept_date(us),us_ltime("%Y/%m/%d/%H:%M:%S.%6N %z")]\ %ci:%cp + +us_utime([,]) + This works like "utime" but takes an input in microseconds. It also supports + the %N conversion specifier inspired by date(1). + Converts an integer supposed to contain a date since epoch to a string + representing this date in UTC time using a format defined by the + string using strftime(3). The purpose is to allow any date format to be used + in logs. An optional in microseconds may be applied to the input + date (positive or negative). See the strftime() man page for the format + supported by your operating system. + + The %N conversion specifier allows you to output the nanoseconds part of the + date, precision is limited since the input is microseconds. + (000000000..999999000). %N can take a width argument between % and N. It is + useful to display milliseconds (%3N) or microseconds (%6N). The default and + maximum width is 9 (%N = %9N). + + See also the "ltime" converter for local as well as "utime" and "ms_utime" + converters. + + Example : + + # Emit 3 colons, the UTC time, the timezone and another with ip:port + # e.g. 2023/07/24/09:53:02.196234 +0000 127.0.0.1:41530 + log-format %[accept_date(us),us_utime("%Y/%m/%d/%H:%M:%S.%6N %z")]\ %ci:%cp + utime([,]) Converts an integer supposed to contain a date since epoch to a string representing this date in UTC time using a format defined by the string using strftime(3). The purpose is to allow any date format to be used in logs. An optional in seconds may be applied to the input date (positive or negative). See the strftime() man page for the format supported - by your operating system. See also the ltime converter. + by your operating system. See also the "ltime" converter as well as "ms_utime" + and "us_utime". Example : diff --git a/src/sample.c b/src/sample.c index 7bd7b9b0ce..9450042fdd 100644 --- a/src/sample.c +++ b/src/sample.c @@ -2162,6 +2162,202 @@ static int sample_conv_ipmask(const struct arg *args, struct sample *smp, void * return 1; } +/* + * This function implement a conversion specifier seeker for %N so it could be + * replaced before doing strftime. + * + * is the input format string which is used as a haystack + * + * The function fills multiple variables: + * is the len of the conversion specifier string which was found (ex: strlen(%N):2, strlen(%3N):3 strlen(%123N): 5) + * is the width argument, default width is 9 (ex: %3N: 3, %4N: 4: %N: 9, %5N: 5) + * + * Returns a ptr to the first character of the conversion specifier or NULL if not found + */ +static const char *lookup_convspec_N(const char *format, int *skip, int *width) +{ + const char *p, *needle; + const char *digits; + int state; + + p = format; + + /* this looks for % in loop. The iteration stops when a %N conversion + * specifier was found or there is no '%' anymore */ +lookagain: + while (p && *p) { + state = 0; + digits = NULL; + + p = needle = strchr(p, '%'); + /* Once we find a % we try to move forward in the string + * + * state 0: found % + * state 1: digits (precision) + * state 2: N + */ + while (p && *p) { + switch (state) { + case 0: + state = 1; + break; + + case 1: + if (isdigit((unsigned char)*p) && !digits) /* set the start of the digits */ + digits = p; + + if (isdigit((unsigned char)*p)) + break; + else + state = 2; + /* if this is not a number anymore, we + * don't want to increment p but try the + * next state directly */ + __fallthrough; + case 2: + if (*p == 'N') + goto found; + else + /* this was not a %N, start again */ + goto lookagain; + break; + } + p++; + } + } + + *skip = 0; + *width = 0; + return NULL; + +found: + *skip = p - needle + 1; + if (digits) + *width = atoi(digits); + else + *width = 9; + return needle; +} + + /* + * strftime(3) does not implement nanoseconds, but we still want them in our + * date format. + * + * This function implements %N like in date(1) which gives you the nanoseconds part of the timetamp + * An optional field width can be specified, a maximum width of 9 is supported (ex: %3N %6N %9N) + * + * is the format string + * in seconds since epoch + * only the nanoseconds part of the timestamp + * chose the localtime instead of UTC time + * + * Return the results of strftime in the trash buffer + */ +static struct buffer *conv_time_common(const char *format, time_t curr_date, uint64_t ns, int local) +{ + struct buffer *tmp_format = NULL; + struct buffer *res = NULL; + struct tm tm; + const char *p; + char ns_str[10] = {}; + int set = 0; + + if (local) + get_localtime(curr_date, &tm); + else + get_gmtime(curr_date, &tm); + + + /* we need to iterate in order to replace all the %N in the string */ + + p = format; + while (*p) { + const char *needle; + int skip = 0; + int cpy = 0; + int width = 0; + + /* look for the next %N onversion specifier */ + if (!(needle = lookup_convspec_N(p, &skip, &width))) + break; + + if (width > 9) /* we don't handle more that 9 */ + width = 9; + cpy = needle - p; + + if (!tmp_format) { + tmp_format = alloc_trash_chunk(); + tmp_format->data = 0; + } + + if (set != 9) /* if the snprintf wasn't done yet */ + set = snprintf(ns_str, sizeof(ns_str), "%.9llu", (unsigned long long)ns); + + if (chunk_istcat(tmp_format, ist2(p, cpy)) == 0) /* copy before the %N */ + goto error; + if (chunk_istcat(tmp_format, ist2(ns_str, width)) == 0) /* copy the %N result with the right precison */ + goto error; + + p += skip + cpy; /* skip the %N */ + } + + + if (tmp_format) { /* %N was found */ + if (chunk_strcat(tmp_format, p) == 0) /* copy the end of the string if needed or just the \0 */ + goto error; + res = get_trash_chunk(); + res->data = strftime(res->area, res->size, tmp_format->area , &tm); + } else { + res = get_trash_chunk(); + res->data = strftime(res->area, res->size, format, &tm); + } + +error: + free_trash_chunk(tmp_format); + return res; +} + + + +/* + * same as sample_conv_ltime but input is us and %N is supported + */ +static int sample_conv_us_ltime(const struct arg *args, struct sample *smp, void *private) +{ + struct buffer *temp; + time_t curr_date = smp->data.u.sint / 1000000; /* convert us to s */ + uint64_t ns = (smp->data.u.sint % 1000000) * 1000; /* us part to ns */ + + /* add offset */ + if (args[1].type == ARGT_SINT) + curr_date += args[1].data.sint; + + temp = conv_time_common(args[0].data.str.area, curr_date, ns, 1); + smp->data.u.str = *temp; + smp->data.type = SMP_T_STR; + return 1; +} + +/* + * same as sample_conv_ltime but input is ms and %N is supported + */ +static int sample_conv_ms_ltime(const struct arg *args, struct sample *smp, void *private) +{ + struct buffer *temp; + time_t curr_date = smp->data.u.sint / 1000; /* convert ms to s */ + uint64_t ns = (smp->data.u.sint % 1000) * 1000000; /* ms part to ns */ + + /* add offset */ + if (args[1].type == ARGT_SINT) + curr_date += args[1].data.sint; + + temp = conv_time_common(args[0].data.str.area, curr_date, ns, 1); + smp->data.u.str = *temp; + smp->data.type = SMP_T_STR; + return 1; +} + + /* takes an UINT value on input supposed to represent the time since EPOCH, * adds an optional offset found in args[1] and emits a string representing * the local time in the format specified in args[1] using strftime(). @@ -2197,6 +2393,44 @@ static int sample_conv_sdbm(const struct arg *arg_p, struct sample *smp, void *p return 1; } +/* + * same as sample_conv_utime but input is us and %N is supported + */ +static int sample_conv_us_utime(const struct arg *args, struct sample *smp, void *private) +{ + struct buffer *temp; + time_t curr_date = smp->data.u.sint / 1000000; /* convert us to s */ + uint64_t ns = (smp->data.u.sint % 1000000) * 1000; /* us part to ns */ + + /* add offset */ + if (args[1].type == ARGT_SINT) + curr_date += args[1].data.sint; + + temp = conv_time_common(args[0].data.str.area, curr_date, ns, 0); + smp->data.u.str = *temp; + smp->data.type = SMP_T_STR; + return 1; +} + +/* + * same as sample_conv_utime but input is ms and %N is supported + */ +static int sample_conv_ms_utime(const struct arg *args, struct sample *smp, void *private) +{ + struct buffer *temp; + time_t curr_date = smp->data.u.sint / 1000; /* convert ms to s */ + uint64_t ns = (smp->data.u.sint % 1000) * 1000000; /* ms part to ns */ + + /* add offset */ + if (args[1].type == ARGT_SINT) + curr_date += args[1].data.sint; + + temp = conv_time_common(args[0].data.str.area, curr_date, ns, 0); + smp->data.u.str = *temp; + smp->data.type = SMP_T_STR; + return 1; +} + /* takes an UINT value on input supposed to represent the time since EPOCH, * adds an optional offset found in args[1] and emits a string representing * the UTC date in the format specified in args[1] using strftime(). @@ -4564,7 +4798,11 @@ static struct sample_conv_kw_list sample_conv_kws = {ILH, { { "hex2i", sample_conv_hex2int, 0, NULL, SMP_T_STR, SMP_T_SINT }, { "ipmask", sample_conv_ipmask, ARG2(1,MSK4,MSK6), NULL, SMP_T_ADDR, SMP_T_ADDR }, { "ltime", sample_conv_ltime, ARG2(1,STR,SINT), NULL, SMP_T_SINT, SMP_T_STR }, + { "ms_ltime", sample_conv_ms_ltime, ARG2(1,STR,SINT), NULL, SMP_T_SINT, SMP_T_STR }, + { "us_ltime", sample_conv_us_ltime, ARG2(1,STR,SINT), NULL, SMP_T_SINT, SMP_T_STR }, { "utime", sample_conv_utime, ARG2(1,STR,SINT), NULL, SMP_T_SINT, SMP_T_STR }, + { "ms_utime", sample_conv_ms_utime, ARG2(1,STR,SINT), NULL, SMP_T_SINT, SMP_T_STR }, + { "us_utime", sample_conv_us_utime, ARG2(1,STR,SINT), NULL, SMP_T_SINT, SMP_T_STR }, { "crc32", sample_conv_crc32, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT }, { "crc32c", sample_conv_crc32c, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT }, { "djb2", sample_conv_djb2, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },