Bruno Haible [Sun, 12 Mar 2023 23:10:47 +0000 (00:10 +0100)]
xgettext: In language JavaScript, avoid a crash at an XML comment, CDATA, or PI.
* gettext-tools/src/x-javascript.c (phase5_scan_xml_markup): Return int instead
of bool.
(phase5_get): Ignore XML markup parsed by phase5_scan_xml_markup.
* gettext-tools/tests/xgettext-javascript-9: New file.
* gettext-tools/tests/xgettext-javascript-10: New file.
* gettext-tools/tests/xgettext-javascript-11: New file.
* gettext-tools/tests/Makefile.am (TESTS): Add them.
Bruno Haible [Sun, 12 Mar 2023 14:25:33 +0000 (15:25 +0100)]
xgettext: In language JavaScript, avoid a crash for some non-ASCII input.
* gettext-tools/src/xg-encoding.c (non_ascii_error_message,
non_utf8_error_message): Handle all possible lexical_context_ty values.
* gettext-tools/tests/xgettext-javascript-8: New file.
* gettext-tools/tests/Makefile.am (TESTS): Add it.
Bruno Haible [Sun, 12 Mar 2023 12:11:58 +0000 (13:11 +0100)]
Fix python-brace-format: Fix placement of FMTDIR_ERROR marker.
* gettext-tools/src/format-python-brace.c (parse_directive): Simplify. Fix
placement of FMTDIR_ERROR marker and improve error message in one case.
* gettext-tools/tests/format-python-brace-1: Add more test cases.
* gettext-tools/src/format-python-brace.c (parse_directive): After '.', expect
at least one digit.
* gettext-tools/tests/format-python-brace-1: Add a test case.
Bruno Haible [Sun, 12 Mar 2023 10:45:11 +0000 (11:45 +0100)]
xgettext: Avoid a crash due to a python-brace-format string with a NUL byte.
* gettext-tools/src/format-python-brace.c: Improve comments.
(parse_directive): Don't read past the string end if c1 is NUL.
* gettext-tools/tests/xgettext-python-6: New file.
* gettext-tools/tests/Makefile.am (TESTS): Add it.
Bruno Haible [Sun, 12 Mar 2023 00:39:57 +0000 (01:39 +0100)]
xgettext: Fix abort when outputting a msgid that has an invalid UTF-8 character.
* gettext-tools/src/xg-encoding.c: Include unistr.h.
(non_utf8_error_message): New function.
(from_current_source_encoding): When xgettext_current_source_encoding is
"UTF-8", check that the string is well-formed UTF-8.
* gettext-tools/tests/xgettext-c-8: New file.
* gettext-tools/tests/xgettext-python-5: New file.
* gettext-tools/tests/xgettext-elisp-3: New file.
* gettext-tools/tests/xgettext-librep-3: New file.
* gettext-tools/tests/xgettext-awk-3: New file.
* gettext-tools/tests/xgettext-lua-3: New file.
* gettext-tools/tests/xgettext-vala-4: New file.
* gettext-tools/tests/xgettext-php-5: New file.
* gettext-tools/tests/Makefile.am (TESTS): Add them.
Bruno Haible [Sat, 11 Mar 2023 00:21:40 +0000 (01:21 +0100)]
xgettext: In language Vala, report out-of-range hexadecimal escapes.
This avoids a later crash
"Assertion `UNICODE_VALUE (c) >= 0 && UNICODE_VALUE (c) < 0x110000' failed."
* gettext-tools/src/x-vala.c (phase7_getc): When the value of a hexadecimal
escape sequence overflows, warn and use the last value before it overflowed.
* gettext-tools/tests/xgettext-vala-7: New file.
* gettext-tools/tests/Makefile.am (TESTS): Add it.
Bruno Haible [Sat, 11 Mar 2023 00:02:47 +0000 (01:02 +0100)]
xgettext: In language C, report out-of-range hexadecimal escapes.
This avoids a later crash
"Assertion `UNICODE_VALUE (c) >= 0 && UNICODE_VALUE (c) < 0x110000' failed."
* gettext-tools/src/x-c.c (phase7_getc): When the value of a hexadecimal escape
sequence overflows, warn and use the last value before it overflowed.
* gettext-tools/tests/xgettext-c-7: New file.
* gettext-tools/tests/Makefile.am (TESTS): Add it.
Bruno Haible [Fri, 10 Mar 2023 22:32:24 +0000 (23:32 +0100)]
xgettext: In language PO, handle non-ASCII non-UTF-8 input files correctly.
* gettext-tools/src/x-po.c: Include msgl-iconv.h, msgl-ascii.h, po-charset.h.
(extract): If a header charset is present, convert the messages to UTF-8;
otherwise verify that they are all ASCII.
* gettext-tools/tests/xgettext-po-3: New file.
* gettext-tools/tests/xgettext-po-4: New file.
* gettext-tools/tests/testdata/xg-po-3.po: New file.
* gettext-tools/tests/testdata/xg-po-4.po: New file.
* gettext-tools/tests/Makefile.am (TESTS, EXTRA_DIST): Add them.
* NEWS: Mention the change.
Bruno Haible [Thu, 9 Mar 2023 23:07:52 +0000 (00:07 +0100)]
po-mode: Fix incompatibility with the forthcoming Emacs 29.
Patch by Takesi Ayanokoji <ayanokoji.takesi@gmail.com>.
* gettext-tools/emacs/po-mode.el (po-font-lock-keywords): Use
font-lock-constant-face, introduced in Emacs 20.3, instead of the
long-deprecated font-lock-reference-face.
* NEWS: Mention it.
Bruno Haible [Thu, 9 Mar 2023 21:30:17 +0000 (22:30 +0100)]
xgettext: In language YCP, avoid stack overflow.
* gettext-tools/src/x-ycp.c: Include error-progname.h.
(MAX_NESTING_DEPTH): New macro.
(nesting_depth): New variable.
(extract_parenthesized): Increase and check nesting_depth before calling
extract_parenthesized recursively.
(extract_ycp): Initialize nesting_depth.
* gettext-tools/tests/xgettext-ycp-stackovfl-1: New file.
* gettext-tools/tests/xgettext-ycp-stackovfl-2: New file.
* gettext-tools/tests/xgettext-ycp-stackovfl-3: New file.
* gettext-tools/tests/xgettext-ycp-stackovfl-4: New file.
* gettext-tools/tests/Makefile.am (TESTS): Add them.
Bruno Haible [Thu, 9 Mar 2023 20:23:47 +0000 (21:23 +0100)]
xgettext: In language PHP, avoid stack overflow.
* gettext-tools/src/x-php.c: Include error-progname.h.
(MAX_NESTING_DEPTH): New macro.
(paren_nesting_depth, bracket_nesting_depth): New variables.
(extract_balanced): Increase and check paren_nesting_depth or
bracket_nesting_depth before calling extract_balanced recursively.
(extract_python): Initialize paren_nesting_depth and bracket_nesting_depth.
* gettext-tools/tests/xgettext-php-stackovfl-1: New file.
* gettext-tools/tests/xgettext-php-stackovfl-2: New file.
* gettext-tools/tests/xgettext-php-stackovfl-3: New file.
* gettext-tools/tests/xgettext-php-stackovfl-4: New file.
* gettext-tools/tests/Makefile.am (TESTS): Add them.
Bruno Haible [Thu, 9 Mar 2023 19:28:19 +0000 (20:28 +0100)]
xgettext: In language Perl, avoid stack overflow.
* gettext-tools/src/x-perl.c (MAX_NESTING_DEPTH): New macro.
(nesting_depth): New variable.
(interpolate_keywords): Increase and check nesting_depth. Revert it before
returning.
(x_perl_lex): Likewise.
(extract_balanced): Increase and check nesting_depth before calling
extract_balanced recursively.
(extract_perl): Initialize nesting_depth.
* gettext-tools/tests/xgettext-perl-stackovfl-1: New file.
* gettext-tools/tests/xgettext-perl-stackovfl-2: New file.
* gettext-tools/tests/xgettext-perl-stackovfl-3: New file.
* gettext-tools/tests/xgettext-perl-stackovfl-4: New file.
* gettext-tools/tests/Makefile.am (TESTS): Add them.
Bruno Haible [Thu, 9 Mar 2023 10:34:41 +0000 (11:34 +0100)]
xgettext: In language Tcl, avoid stack overflow.
* gettext-tools/src/x-tcl.c: Include error-progname.h.
(MAX_NESTING_DEPTH): New macro.
(bracket_nesting_depth, brace_nesting_depth): New variables.
(accumulate_word, read_word): Increase and check bracket_nesting_depth or
brace_nesting_depth before calling read_command_list recursively.
(extract_tcl): Initialize bracket_nesting_depth and brace_nesting_depth.
* gettext-tools/tests/xgettext-tcl-stackovfl-1: New file.
* gettext-tools/tests/xgettext-tcl-stackovfl-2: New file.
* gettext-tools/tests/xgettext-tcl-stackovfl-3: New file.
* gettext-tools/tests/xgettext-tcl-stackovfl-4: New file.
* gettext-tools/tests/Makefile.am (TESTS): Add them.
Bruno Haible [Wed, 8 Mar 2023 20:37:57 +0000 (21:37 +0100)]
xgettext: In language Lua, avoid stack overflow.
* gettext-tools/src/x-lua.c: Include error-progname.h.
(MAX_NESTING_DEPTH): New macro.
(paren_nesting_depth, bracket_nesting_depth): New variables.
(extract_balanced): Increase and check paren_nesting_depth or
bracket_nesting_depth before calling extract_balanced recursively.
(extract_lua): Initialize paren_nesting_depth and bracket_nesting_depth.
* gettext-tools/tests/xgettext-lua-stackovfl-1: New file.
* gettext-tools/tests/xgettext-lua-stackovfl-2: New file.
* gettext-tools/tests/xgettext-lua-stackovfl-3: New file.
* gettext-tools/tests/xgettext-lua-stackovfl-4: New file.
* gettext-tools/tests/Makefile.am (TESTS): Add them.
Bruno Haible [Tue, 7 Mar 2023 14:59:08 +0000 (15:59 +0100)]
xgettext: In language JavaScript, avoid stack overflow.
* gettext-tools/src/x-javascript.c (MAX_NESTING_DEPTH): New macro.
(paren_nesting_depth, bracket_nesting_depth, brace_nesting_depth,
xml_element_nesting_depth): New variables.
(extract_balanced): Increase and check paren_nesting_depth or
bracket_nesting_depth or brace_nesting_depth or xml_element_nesting_depth before
calling extract_balanced recursively.
(extract_javascript): Initialize paren_nesting_depth, bracket_nesting_depth,
brace_nesting_depth, xml_element_nesting_depth.
* gettext-tools/tests/xgettext-javascript-stackovfl-1: New file.
* gettext-tools/tests/xgettext-javascript-stackovfl-2: New file.
* gettext-tools/tests/xgettext-javascript-stackovfl-3: New file.
* gettext-tools/tests/xgettext-javascript-stackovfl-4: New file.
* gettext-tools/tests/xgettext-javascript-stackovfl-5: New file.
* gettext-tools/tests/xgettext-javascript-stackovfl-6: New file.
* gettext-tools/tests/xgettext-javascript-stackovfl-7: New file.
* gettext-tools/tests/xgettext-javascript-stackovfl-8: New file.
* gettext-tools/tests/Makefile.am (TESTS): Add them.
Bruno Haible [Mon, 6 Mar 2023 21:21:03 +0000 (22:21 +0100)]
xgettext: In language C#, avoid stack overflow.
* gettext-tools/src/x-csharp.c (MAX_NESTING_DEPTH): New macro.
(paren_nesting_depth, brace_nesting_depth): New variables.
(extract_parenthesized): Increase and check paren_nesting_depth or
brace_nesting_depth before calling extract_parenthesized recursively.
(extract_csharp): Initialize paren_nesting_depth and brace_nesting_depth.
* gettext-tools/tests/xgettext-csharp-stackovfl-1: New file.
* gettext-tools/tests/xgettext-csharp-stackovfl-2: New file.
* gettext-tools/tests/xgettext-csharp-stackovfl-3: New file.
* gettext-tools/tests/xgettext-csharp-stackovfl-4: New file.
* gettext-tools/tests/Makefile.am (TESTS): Add them.
Bruno Haible [Mon, 6 Mar 2023 20:49:02 +0000 (21:49 +0100)]
xgettext: In language Java, avoid stack overflow.
* gettext-tools/src/x-java.c (MAX_NESTING_DEPTH): New macro.
(paren_nesting_depth, brace_nesting_depth): New variables.
(extract_parenthesized): Increase and check paren_nesting_depth or
brace_nesting_depth before calling extract_parenthesized recursively.
(extract_java): Initialize paren_nesting_depth and brace_nesting_depth.
* gettext-tools/tests/xgettext-java-stackovfl-1: New file.
* gettext-tools/tests/xgettext-java-stackovfl-2: New file.
* gettext-tools/tests/xgettext-java-stackovfl-3: New file.
* gettext-tools/tests/xgettext-java-stackovfl-4: New file.
* gettext-tools/tests/Makefile.am (TESTS): Add them.
Bruno Haible [Mon, 6 Mar 2023 19:48:35 +0000 (20:48 +0100)]
xgettext: In language Python, avoid stack overflow.
* gettext-tools/src/x-python.c (MAX_NESTING_DEPTH): New macro.
(paren_nesting_depth, bracket_nesting_depth): New variables.
(extract_balanced): Increase and check paren_nesting_depth or
bracket_nesting_depth before calling extract_balanced recursively.
(extract_python): Initialize paren_nesting_depth and bracket_nesting_depth.
* gettext-tools/tests/xgettext-python-stackovfl-1: New file.
* gettext-tools/tests/xgettext-python-stackovfl-2: New file.
* gettext-tools/tests/xgettext-python-stackovfl-3: New file.
* gettext-tools/tests/xgettext-python-stackovfl-4: New file.
* gettext-tools/tests/Makefile.am (TESTS): Add them.
Bruno Haible [Mon, 6 Mar 2023 16:08:23 +0000 (17:08 +0100)]
intl: Fix many warnings like "warning: "glthread_lock_init" redefined".
* gettext-runtime/intl/configure.ac: Move the renamings to the bottom of the
config.h file. Make the glthread*_lock* renamings conditional so that they
don't occur when these symbols are macros.
Daiki Ueno [Sun, 5 Mar 2023 01:25:16 +0000 (10:25 +0900)]
Fix potential NULL pointer dereference in ITS code
Reported by Shashank in: <https://savannah.gnu.org/bugs/?63529> and
<https://savannah.gnu.org/bugs/?63530>.
* gettext-tools/src/locating-rule.c (get_attribute): Check return
value of xmlGetProp.
(document_locating_rule_match): Check return value of
xmlDocGetRootElement.
(locating_rule_list_add_from_file): Likewise.
Bruno Haible [Sun, 26 Feb 2023 10:41:18 +0000 (11:41 +0100)]
Distribute the source code of csharpexec-test.exe.
Reported by Bastien Roucariès <rouca@debian.org>
via Santiago Vila <sanvila@debian.org>
in <https://lists.gnu.org/archive/html/bug-gettext/2023-02/msg00016.html>.
* gettext-tools/m4/csharpexec-test.cs: New file, extracted from pre-2018
versions of gnulib/m4/csharpcomp.m4.
* gettext-tools/m4/Makefile.am (EXTRA_DIST): Add it.
Bruno Haible [Mon, 13 Feb 2023 12:29:05 +0000 (13:29 +0100)]
build: Fix libgettexpo linking.
* gettext-tools/libgettextpo/Makefile.am: Pass $(LTLIBUNISTRING) in
libgettextpo_la_LDFLAGS, not libgettextpo_la_LIBADD. This is needed per the
documentation
https://www.gnu.org/software/automake/manual/html_node/Libtool-Flags.html, and
also for consistency with how we handle the dependencies to libintl and
libiconv.
Bruno Haible [Mon, 13 Feb 2023 00:31:53 +0000 (01:31 +0100)]
build: Don't hint to --with-installed-csharp-dll when C# support is disabled.
When --disable-csharp was specified, gettext-tools/src/Makefile.am does not
produce msgunfmt.net.exe. Therefore it is irrelevant whether the condition
USE_INSTALLED_CSHARP_DLL evaluates to true or false. Thus the message
"WARNING: When building the gettext-tools package without building the entire
gettext package, you need to pass the --with-installed-csharp-dll option to
configure."
is pointless.
Reported by Ryan Schmidt in <https://savannah.gnu.org/bugs/?58668>.
* gettext-tools/configure.ac: Don't emit hint to use --with-installed-csharp-dll
when BUILDCSHARP is not 'yes'.
Bruno Haible [Sat, 11 Feb 2023 16:17:05 +0000 (17:17 +0100)]
AM_GNU_GETTEXT: Define localedir_c and localedir_c_make.
* gettext-runtime/m4/gettext.m4 (AM_GNU_GETTEXT): Define and substitute
localedir_c and localedir_c_make.
* gettext-runtime/src/Makefile.am (AM_CPPFLAGS): Use $(localedir_c_make)
instead of $(localedir). This is useful in native Windows environments.
* gettext-runtime/intl/Makefile.am (AM_CPPFLAGS): Likewise.
* gettext-tools/src/Makefile.am (AM_CPPFLAGS): Likewise.
* gettext-tools/tests/Makefile.am (AM_CPPFLAGS): Likewise.
* gettext-tools/doc/gettext.texi (AM_GNU_GETTEXT): Small updates. Mention
localedir_c and localedir_c_make.
* NEWS: Mention the change.
Bruno Haible [Thu, 9 Feb 2023 15:58:20 +0000 (16:58 +0100)]
Fix compilation errors with ISO C 23 compilers.
* gettext-runtime/intl/dcigettext.c (getwd, getcwd): Assume a declaration in the
system header files.
* gettext-runtime/intl/gettextP.h (SWAP): Drop K&C compatibility.
Bruno Haible [Mon, 2 Jan 2023 07:33:10 +0000 (08:33 +0100)]
libtextstyle: Fix compilation error on Android.
* libtextstyle/gnulib-local/m4/libglib.m4 (gl_LIBGLIB): Test whether struct
lconv has a 'decimal_point' field.
* libtextstyle/gnulib-local/lib/glib/gstrfuncs.c (g_ascii_strtod,
g_ascii_formatd): If struct lconv does not have a 'decimal_point' field, use "."
instead.
Bruno Haible [Fri, 2 Dec 2022 01:52:15 +0000 (02:52 +0100)]
msgfmt: Fix -k/--keyword option handling.
Reported by Daniel <daniel.rf710@gmail.com> in
<https://savannah.gnu.org/bugs/index.php?63455>.
* gettext-tools/src/msgfmt.c (long_options): Declare that --keyword takes an
optional argument.
(main): Recognize -k as an alternative to --keyword.
Ensure that desktop_keywords is initialized before use.
Don't set desktop_default_keywords to false when an option -kKWSPEC or
--keyword=KWSPEC is seen.
Don't destroy the desktop_keywords table too early.
Don't destroy the desktop_keywords table when the program is terminating anyway.
Bruno Haible [Sun, 25 Sep 2022 15:32:07 +0000 (17:32 +0200)]
Fix a link error on MSVC/clang in --enable-shared mode.
* gettext-runtime/intl/printf.c: Include libgnuintl.h. This ensures that the
LIBINTL_DLL_EXPORTED marker from the libintl_*printf function declarations is
seen when the functions get defined.
* gettext-runtime/intl/Makefile.am: Update comment.
Bruno Haible [Mon, 18 Jul 2022 16:56:46 +0000 (18:56 +0200)]
Move gnulib-tool invocation back from autopull.sh to autogen.sh.
* autogen.sh: Revert most of last change. Improve comments.
* autopull.sh: Only call 'gitsub.sh pull', fetch archive.dir.tar, and recurse.
* libtextstyle/autogen.sh: Revert most of last change. Improve comments.
* libtextstyle/autopull.sh: Only call 'gitsub.sh pull', fetch texinfo.tex.
* libtextstyle/m4/init-package-version.m4: Revert last change.
Bruno Haible [Sun, 17 Jul 2022 14:50:33 +0000 (16:50 +0200)]
Split autogen.sh into autopull.sh and autogen.sh.
* autopull.sh: New file, based on autogen.sh.
* autogen.sh: Remove code that was moved to autopull.sh. Remove --skip-gnulib
option.
* HACKING: Mention autopull.sh.
* libtextstyle/autopull.sh: New file, based on libtextstyle/autogen.sh.
* libtextstyle/autogen.sh: Remove code that was moved to
libtextstyle/autopull.sh. Remove --skip-gnulib option.
* libtextstyle/HACKING: Mention autopull.sh.
* libtextstyle/m4/init-package-version.m4: Update comment.
Bruno Haible [Sun, 17 Jul 2022 10:30:11 +0000 (12:30 +0200)]
Fix "make distcheck" failure (regression from 2022-07-03).
* gettext-runtime/intl/export.h: Add a copyright notice.
* gettext-runtime/intl/Makefile.am (libgnuintl.h): Eliminate this copyright
notice when including export.h.
Bruno Haible [Sun, 3 Jul 2022 14:21:52 +0000 (16:21 +0200)]
build: Fix link error for tstgettext.exe on mingw.
* gettext-runtime/intl/export.h: Treat mingw like MSVC. In static builds. don't
use __declspec(dllexport). In shared builds, use also __declspec(dllimport).
intl: On native Windows, actually define the *wprintf functions.
In dcaf8c4d7 (Cygwin portability., 2003-09-17), the *wprintf() family of
functions in `gettext-runtime/intl/printf.c` were no longer guarded by
the constant `HAVE_WPRINTF`, but instead by `HAVE_FWPRINTF`.
This apparently worked even if the corresponding part in
`gettext-runtime/intl/libgnuintl.h.in` uses `HAVE_WPRINTF` to guard the
declarations of those functions.
However, in d84f20745 (Make sure that libintl.h declares the *wprintf
overrides on Windows., 2019-04-08), gettext introduced a change where it
would look for `wprintf()` instead of `fwprintf()`. As a consequence it
would no longer define the `HAVE_FWPRINTF` constant at all.
GCC apparently interprets `#if HAVE_FWPRINTF` as `#if 0` if the constant
has not even been defined.
This leads to the funny situation that previously, the *wprintf()
functions would be defined, but not be declared. Whereas now the
functions are not defined, but declared.
Also funny: Cygwin did not even export the `fwprintf()` function until
2009, as per Cygwin's 45e20e47ba (cygwin.din: Export wprintf,
fwprintf, swprintf, vwprintf, vfwprintf, vswprintf. [...] , 2009-03-06),
while the `wprintf()` function was exported already in 2003. So the
Cygwin portability patch from 2003 seems to have turned off all of the
*wprintf() functions in gettext.
Let's revert that "portability" patch, as it now only does harm and has
no benefit anymore.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
[Bruno's summary]
Thank you for the analysis and explanation, Johannes!
The commits dcaf8c4d7 and f5ca7696d (2003-09-17) introduced a test for
fwprintf rather than wprintf. This was designed as a workaround to a
Cygwin problem that existed from 2003 to 2009.
The commit d84f20745 (2019-04-08) switched back from testing fwprintf
to testing wprintf, but did so incompletely: I forgot to modify printf.c.
So, since then, the *wprintf functions are declared but not defined.
Bruno Haible [Tue, 21 Jun 2022 00:50:34 +0000 (02:50 +0200)]
gettext-runtime: Provide helper script to build against uninstalled binaries.
Prompted by <https://gcc.gnu.org/pipermail/gcc/2022-June/238929.html>.
* gettext-runtime/uninstalled-config.sh.in: New file.
* gettext-runtime/configure.ac: New variable ENABLE_SHARED. Generate
uninstalled-config.sh at configure time.
Bruno Haible [Mon, 20 Jun 2022 06:56:18 +0000 (08:56 +0200)]
Don't check for java nor javac when --disable-java is specified.
This avoid annoying dialogs on macOS when Java has not been installed so far.
Reported by Iain Sandoe <idsandoe@googlemail.com> in
<https://gcc.gnu.org/pipermail/gcc/2022-June/238929.html>.
* gettext-runtime/configure.ac: When JAVA_CHOICE is 'no', bypass the gt_JAVACOMP
and JAR tests.
* gettext-tools/configure.ac: When JAVA_CHOICE is 'no', bypass the gt_JAVAEXEC,
gt_JAVACOMP, and JAR tests.
Bruno Haible [Sun, 22 May 2022 23:15:51 +0000 (01:15 +0200)]
Add GNU Project notice.
* JOIN-GNU: New file. Inspired by a suggestion from José E. Marchesi
on the gnu-prog-discuss mailing list.
* README: Refer to it.
* Makefile.am (EXTRA_DIST): Add it.