From: Bruno Haible Date: Wed, 12 Oct 2005 10:01:16 +0000 (+0000) Subject: Add support for contexts in xgettext. X-Git-Tag: v0.15~368 X-Git-Url: http://git.ipfire.org/gitweb.cgi?a=commitdiff_plain;h=011e7649509f3f0ae7671612e68fd069afd7e0c3;p=thirdparty%2Fgettext.git Add support for contexts in xgettext. --- diff --git a/NEWS b/NEWS index c5602db8d..23da5d3fe 100644 --- a/NEWS +++ b/NEWS @@ -1,4 +1,17 @@ +* GUI program support: + - PO files can now contain messages constrained to a certain context. + Most often such a context is a menu, dialog or panel identification. + The syntax in the PO file is + msgctxt "context" + msgid "original" + msgstr "translation" + - The xgettext program can be told through the --keyword flag which + function/macro argument has the role of a context. + - The (non-public) include file gettext.h defines macros pgettext, dpgettext + etc. that take a context argument. + For more information, see the node "Contexts" in the manual. + * Programming languages support: - Python: diff --git a/gettext-tools/doc/ChangeLog b/gettext-tools/doc/ChangeLog index 2828e4e15..0255fd20e 100644 --- a/gettext-tools/doc/ChangeLog +++ b/gettext-tools/doc/ChangeLog @@ -1,3 +1,8 @@ +2005-10-03 Bruno Haible + + Add support for contexts in xgettext. + * xgettext.texi (--keyword): Document how to specify context arguments. + 2005-10-07 Bruno Haible * gettext.texi (PO Files, Preparing Strings, Contexts): Small fixes. diff --git a/gettext-tools/doc/xgettext.texi b/gettext-tools/doc/xgettext.texi index 61ad09d50..1f9805d9f 100644 --- a/gettext-tools/doc/xgettext.texi +++ b/gettext-tools/doc/xgettext.texi @@ -146,6 +146,7 @@ Additional keyword to be looked for (without @var{keywordspec} means not to use default keywords). @cindex adding keywords, @code{xgettext} +@cindex context, argument specification in @code{xgettext} If @var{keywordspec} is a C identifer @var{id}, @code{xgettext} looks for strings in the first argument of each call to the function or macro @var{id}. If @var{keywordspec} is of the form @@ -154,12 +155,17 @@ for strings in the first argument of each call to the function or macro @samp{@var{id}:@var{argnum1},@var{argnum2}}, @code{xgettext} looks for strings in the @var{argnum1}st argument and in the @var{argnum2}nd argument of the call, and treats them as singular/plural variants for a message -with plural handling. +with plural handling. Also, if @var{keywordspec} is of the form +@samp{@var{id}:@var{contextargnum}c,@var{argnum}} or +@samp{@var{id}:@var{argnum},@var{contextargnum}c}, @code{xgettext} treats +strings in the @var{contextargnum}th argument as a context specifier. @* The default keyword specifications, which are always looked for if not explicitly disabled, are @code{gettext}, @code{dgettext:2}, @code{dcgettext:2}, @code{ngettext:1,2}, @code{dngettext:2,3}, -@code{dcngettext:2,3}, and @code{gettext_noop}. +@code{dcngettext:2,3}, @code{gettext_noop}, and @code{pgettext:1c,2}, +@code{dpgettext:2c,3}, @code{dcpgettext:2c,3}, @code{npgettext:1c,2,3}, +@code{dnpgettext:2c,3,4}, @code{dcnpgettext:2c,3,4}. @* This option has an effect with most languages, namely C, C++, ObjectiveC, Shell, Python, Lisp, EmacsLisp, librep, Java, C#, awk, Tcl, Perl, PHP, diff --git a/gettext-tools/src/ChangeLog b/gettext-tools/src/ChangeLog index 657b6d7b8..0bb8da14d 100644 --- a/gettext-tools/src/ChangeLog +++ b/gettext-tools/src/ChangeLog @@ -1,3 +1,88 @@ +2005-10-03 Bruno Haible + + Add support for contexts in xgettext. + * xgettext.h: Include stdbool.h. + (struct callshape): New structure type. + (split_keywordspec): Return a struct callshape, not two integers, + by reference. + (struct callshapes): New structure type. + (insert_keyword_callshape): New declaration. + (remember_a_message): Add msgctxt argument. + (struct partial_call): New structure type. + (struct arglist_parser): New structure type. + (arglist_parser_alloc, arglist_parser_clone, arglist_parser_remember, + arglist_parser_decidedp, arglist_parser_done): New declarations. + * xgettext.c (split_keywordspec): Return a struct callshape, not two + integers, by reference. Handle the notation Nc for a context argument. + (insert_keyword_callshape): New function. + (remember_a_message): Add msgctxt argument. Free both strings when + the message is excluded. + (arglist_parser_alloc, arglist_parser_clone, arglist_parser_remember, + arglist_parser_decidedp, arglist_parser_done): New functions. + * x-awk.c (x_awk_keyword): Use callshape API: split_keywordspec, + insert_keyword_callshape. + (extract_parenthesized): Replace commas_to_skip, plural_commas + arguments with a single argparser argument. Use arglist_parser_* API. + (extract_awk): Update. + * x-c.c (add_keyword): Use callshape API: split_keywordspec, + insert_keyword_callshape. + (init_keywords): Register the new functions [d[c]][n]pgettext. + (init_flag_table_c, init_flag_table_objc): Likewise. + (struct xgettext_token_ty): Replace argnum1, argnum2 with a callshapes + pointer. + (x_c_lex): Update. + (extract_parenthesized): Replace commas_to_skip, plural_commas + arguments with a single argparser argument. Use arglist_parser_* API. + (extract_whole_file): Update. + * x-csharp.c (x_csharp_keyword): Use callshape API: split_keywordspec, + insert_keyword_callshape. + (extract_parenthesized): Replace commas_to_skip, plural_commas + arguments with a single argparser argument. Use arglist_parser_* API. + (extract_csharp): Update. + * x-elisp.c (x_elisp_keyword): Use callshape API: split_keywordspec, + insert_keyword_callshape. + (read_object): Use arglist_parser_* API. + * x-glade.c (start_element_handler, end_element_handler): Update. + * x-java.c (x_java_keyword): Use callshape API: split_keywordspec, + insert_keyword_callshape. + (extract_parenthesized): Replace commas_to_skip, plural_commas + arguments with a single argparser argument. Use arglist_parser_* API. + (extract_java): Update. + * x-librep.c (x_librep_keyword): Use callshape API: split_keywordspec, + insert_keyword_callshape. + (read_object): Use arglist_parser_* API. + * x-lisp.c (x_lisp_keyword): Use callshape API: split_keywordspec, + insert_keyword_callshape. + (read_object): Use arglist_parser_* API. + * x-perl.c (x_perl_keyword): Use callshape API: split_keywordspec, + insert_keyword_callshape. + (extract_balanced): Replace arg_sg, arg_pl arguments with arg, + argparser arguments. Use arglist_parser_* API. + (extract_variable, interpolate_keywords, extract_perl): Update. + * x-php.c (x_php_keyword): Use callshape API: split_keywordspec, + insert_keyword_callshape. + (extract_parenthesized): Replace commas_to_skip, plural_commas + arguments with a single argparser argument. Use arglist_parser_* API. + (extract_php): Update. + * x-python.c (x_python_keyword): Use callshape API: split_keywordspec, + insert_keyword_callshape. + (extract_parenthesized): Replace commas_to_skip, plural_commas + arguments with a single argparser argument. Use arglist_parser_* API. + (extract_python): Update. + * x-rst.c (extract_rst): Update. + * x-scheme.c (x_scheme_keyword): Use callshape API: split_keywordspec, + insert_keyword_callshape. + (read_object): Use arglist_parser_* API. + * x-sh.c (x_sh_keyword): Use callshape API: split_keywordspec, + insert_keyword_callshape. + (read_word): Update. + (read_command): Use arglist_parser_* API. + * x-smalltalk.c (extract_smalltalk): Update. + * x-tcl.c (x_tcl_keyword): Use callshape API: split_keywordspec, + insert_keyword_callshape. + (read_command): Use arglist_parser_* API. + * x-ycp.c (extract_parenthesized): Update. + 2005-10-03 Bruno Haible * xgettext.h (savable_comment_to_xgettext_comment): Remove diff --git a/gettext-tools/src/x-awk.c b/gettext-tools/src/x-awk.c index b2d3335e7..f961932d7 100644 --- a/gettext-tools/src/x-awk.c +++ b/gettext-tools/src/x-awk.c @@ -67,25 +67,19 @@ x_awk_keyword (const char *name) else { const char *end; - int argnum1; - int argnum2; + struct callshape shape; const char *colon; if (keywords.table == NULL) hash_init (&keywords, 100); - split_keywordspec (name, &end, &argnum1, &argnum2); + split_keywordspec (name, &end, &shape); /* The characters between name and end should form a valid C identifier. A colon means an invalid parse in split_keywordspec(). */ colon = strchr (name, ':'); if (colon == NULL || colon >= end) - { - if (argnum1 == 0) - argnum1 = 1; - hash_insert_entry (&keywords, name, end - name, - (void *) (long) (argnum1 + (argnum2 << 10))); - } + insert_keyword_callshape (&keywords, name, end - name, &shape); } } @@ -688,25 +682,19 @@ static flag_context_list_table_ty *flag_context_list_table; /* Extract messages until the next balanced closing parenthesis. Extracted messages are added to MLP. - When a specific argument shall be extracted, COMMAS_TO_SKIP >= 0 and, - if also a plural argument shall be extracted, PLURAL_COMMAS > 0, - otherwise PLURAL_COMMAS = 0. - When no specific argument shall be extracted, COMMAS_TO_SKIP < 0. Return true upon eof, false upon closing parenthesis. */ static bool extract_parenthesized (message_list_ty *mlp, flag_context_ty outer_context, flag_context_list_iterator_ty context_iter, - int commas_to_skip, int plural_commas) + struct arglist_parser *argparser) { - /* Remember the message containing the msgid, for msgid_plural. */ - message_ty *plural_mp = NULL; - + /* Current argument number. */ + int arg = 1; /* 0 when no keyword has been seen. 1 right after a keyword is seen. */ int state; /* Parameters of the keyword just seen. Defined only in state 1. */ - int next_commas_to_skip = -1; - int next_plural_commas = 0; + const struct callshapes *next_shapes = NULL; /* Whether to implicitly assume the next tokens are arguments even without a '('. */ bool next_is_argument = false; @@ -748,11 +736,7 @@ extract_parenthesized (message_list_ty *mlp, &keyword_value) == 0) { - int argnum1 = (int) (long) keyword_value & ((1 << 10) - 1); - int argnum2 = (int) (long) keyword_value >> 10; - - next_commas_to_skip = argnum1 - 1; - next_plural_commas = (argnum2 > argnum1 ? argnum2 - argnum1 : 0); + next_shapes = (const struct callshapes *) keyword_value; state = 1; } else @@ -771,31 +755,23 @@ extract_parenthesized (message_list_ty *mlp, case token_type_lparen: if (extract_parenthesized (mlp, inner_context, next_context_iter, - state ? next_commas_to_skip : -1, - state ? next_plural_commas : 0)) - return true; + arglist_parser_alloc (mlp, + state ? next_shapes : NULL))) + { + arglist_parser_done (argparser); + return true; + } next_is_argument = false; next_context_iter = null_context_list_iterator; state = 0; continue; case token_type_rparen: + arglist_parser_done (argparser); return false; case token_type_comma: - if (commas_to_skip >= 0) - { - if (commas_to_skip > 0) - commas_to_skip--; - else - if (plural_mp != NULL && plural_commas > 0) - { - commas_to_skip = plural_commas - 1; - plural_commas = 0; - } - else - commas_to_skip = -1; - } + arg++; inner_context = inherited_context (outer_context, flag_context_list_iterator_advance ( @@ -812,34 +788,13 @@ extract_parenthesized (message_list_ty *mlp, pos.line_number = token.line_number; if (extract_all) - remember_a_message (mlp, token.string, inner_context, &pos, + remember_a_message (mlp, NULL, token.string, inner_context, &pos, savable_comment); else - { - if (commas_to_skip == 0) - { - if (plural_mp == NULL) - { - /* Seen an msgid. */ - message_ty *mp = - remember_a_message (mlp, token.string, - inner_context, &pos, - savable_comment); - if (plural_commas > 0) - plural_mp = mp; - } - else - { - /* Seen an msgid_plural. */ - remember_a_message_plural (plural_mp, token.string, - inner_context, &pos, - savable_comment); - plural_mp = NULL; - } - } - else - free (token.string); - } + arglist_parser_remember (argparser, arg, token.string, + inner_context, + pos.file_name, pos.line_number, + savable_comment); } next_is_argument = false; next_context_iter = null_context_list_iterator; @@ -852,7 +807,7 @@ extract_parenthesized (message_list_ty *mlp, pos.file_name = logical_file_name; pos.line_number = token.line_number; - remember_a_message (mlp, token.string, inner_context, &pos, + remember_a_message (mlp, NULL, token.string, inner_context, &pos, savable_comment); } next_is_argument = false; @@ -879,6 +834,7 @@ extract_parenthesized (message_list_ty *mlp, continue; case token_type_eof: + arglist_parser_done (argparser); return true; case token_type_other: @@ -919,7 +875,7 @@ extract_awk (FILE *f, /* Eat tokens until eof is seen. When extract_parenthesized returns due to an unbalanced closing parenthesis, just restart it. */ while (!extract_parenthesized (mlp, null_context, null_context_list_iterator, - -1, 0)) + arglist_parser_alloc (mlp, NULL))) ; fp = NULL; diff --git a/gettext-tools/src/x-c.c b/gettext-tools/src/x-c.c index 7401013bb..d079276c5 100644 --- a/gettext-tools/src/x-c.c +++ b/gettext-tools/src/x-c.c @@ -112,25 +112,19 @@ add_keyword (const char *name, hash_table *keywords) else { const char *end; - int argnum1; - int argnum2; + struct callshape shape; const char *colon; if (keywords->table == NULL) hash_init (keywords, 100); - split_keywordspec (name, &end, &argnum1, &argnum2); + split_keywordspec (name, &end, &shape); /* The characters between name and end should form a valid C identifier. A colon means an invalid parse in split_keywordspec(). */ colon = strchr (name, ':'); if (colon == NULL || colon >= end) - { - if (argnum1 == 0) - argnum1 = 1; - hash_insert_entry (keywords, name, end - name, - (void *) (long) (argnum1 + (argnum2 << 10))); - } + insert_keyword_callshape (keywords, name, end - name, &shape); } } @@ -160,6 +154,12 @@ init_keywords () x_c_keyword ("dngettext:2,3"); x_c_keyword ("dcngettext:2,3"); x_c_keyword ("gettext_noop"); + x_c_keyword ("pgettext:1c,2"); + x_c_keyword ("dpgettext:2c,3"); + x_c_keyword ("dcpgettext:2c,3"); + x_c_keyword ("npgettext:1c,2,3"); + x_c_keyword ("dnpgettext:2c,3,4"); + x_c_keyword ("dcnpgettext:2c,3,4"); x_objc_keyword ("gettext"); x_objc_keyword ("dgettext:2"); @@ -168,6 +168,12 @@ init_keywords () x_objc_keyword ("dngettext:2,3"); x_objc_keyword ("dcngettext:2,3"); x_objc_keyword ("gettext_noop"); + x_objc_keyword ("pgettext:1c,2"); + x_objc_keyword ("dpgettext:2c,3"); + x_objc_keyword ("dcpgettext:2c,3"); + x_objc_keyword ("npgettext:1c,2,3"); + x_objc_keyword ("dnpgettext:2c,3,4"); + x_objc_keyword ("dcnpgettext:2c,3,4"); x_objc_keyword ("NSLocalizedString"); /* similar to gettext */ x_objc_keyword ("_"); /* similar to gettext */ x_objc_keyword ("NSLocalizedStaticString"); /* similar to gettext_noop */ @@ -190,6 +196,16 @@ init_flag_table_c () xgettext_record_flag ("dcngettext:2:pass-c-format"); xgettext_record_flag ("dcngettext:3:pass-c-format"); xgettext_record_flag ("gettext_noop:1:pass-c-format"); + xgettext_record_flag ("pgettext:2:pass-c-format"); + xgettext_record_flag ("dpgettext:3:pass-c-format"); + xgettext_record_flag ("dcpgettext:3:pass-c-format"); + xgettext_record_flag ("npgettext:2:pass-c-format"); + xgettext_record_flag ("npgettext:3:pass-c-format"); + xgettext_record_flag ("dnpgettext:3:pass-c-format"); + xgettext_record_flag ("dnpgettext:4:pass-c-format"); + xgettext_record_flag ("dcnpgettext:3:pass-c-format"); + xgettext_record_flag ("dcnpgettext:4:pass-c-format"); + /* */ xgettext_record_flag ("fprintf:2:c-format"); xgettext_record_flag ("vfprintf:2:c-format"); @@ -231,6 +247,15 @@ init_flag_table_objc () xgettext_record_flag ("dcngettext:2:pass-objc-format"); xgettext_record_flag ("dcngettext:3:pass-objc-format"); xgettext_record_flag ("gettext_noop:1:pass-objc-format"); + xgettext_record_flag ("pgettext:2:pass-objc-format"); + xgettext_record_flag ("dpgettext:3:pass-objc-format"); + xgettext_record_flag ("dcpgettext:3:pass-objc-format"); + xgettext_record_flag ("npgettext:2:pass-objc-format"); + xgettext_record_flag ("npgettext:3:pass-objc-format"); + xgettext_record_flag ("dnpgettext:3:pass-objc-format"); + xgettext_record_flag ("dnpgettext:4:pass-objc-format"); + xgettext_record_flag ("dcnpgettext:3:pass-objc-format"); + xgettext_record_flag ("dcnpgettext:4:pass-objc-format"); xgettext_record_flag ("NSLocalizedString:1:pass-c-format"); xgettext_record_flag ("NSLocalizedString:1:pass-objc-format"); xgettext_record_flag ("_:1:pass-c-format"); @@ -255,6 +280,15 @@ init_flag_table_gcc_internal () xgettext_record_flag ("dcngettext:2:pass-gcc-internal-format"); xgettext_record_flag ("dcngettext:3:pass-gcc-internal-format"); xgettext_record_flag ("gettext_noop:1:pass-gcc-internal-format"); + xgettext_record_flag ("pgettext:2:pass-gcc-internal-format"); + xgettext_record_flag ("dpgettext:3:pass-gcc-internal-format"); + xgettext_record_flag ("dcpgettext:3:pass-gcc-internal-format"); + xgettext_record_flag ("npgettext:2:pass-gcc-internal-format"); + xgettext_record_flag ("npgettext:3:pass-gcc-internal-format"); + xgettext_record_flag ("dnpgettext:3:pass-gcc-internal-format"); + xgettext_record_flag ("dnpgettext:4:pass-gcc-internal-format"); + xgettext_record_flag ("dcnpgettext:3:pass-gcc-internal-format"); + xgettext_record_flag ("dcnpgettext:4:pass-gcc-internal-format"); #if 0 /* This should better be done inside GCC. */ /* grepping for ATTRIBUTE_PRINTF in gcc-3.3/gcc/?*.h */ /* c-format.c */ @@ -1525,9 +1559,8 @@ struct xgettext_token_ty { xgettext_token_type_ty type; - /* These fields are used only for xgettext_token_type_keyword. */ - int argnum1; - int argnum2; + /* This field is used only for xgettext_token_type_keyword. */ + const struct callshapes *shapes; /* This field is used only for xgettext_token_type_string_literal, xgettext_token_type_keyword, xgettext_token_type_symbol. */ @@ -1570,8 +1603,7 @@ x_c_lex (xgettext_token_ty *tp) == 0) { tp->type = xgettext_token_type_keyword; - tp->argnum1 = (int) (long) keyword_value & ((1 << 10) - 1); - tp->argnum2 = (int) (long) keyword_value >> 10; + tp->shapes = (const struct callshapes *) keyword_value; tp->pos.file_name = logical_file_name; tp->pos.line_number = token.line_number; } @@ -1652,25 +1684,19 @@ static flag_context_list_table_ty *flag_context_list_table; /* Extract messages until the next balanced closing parenthesis. Extracted messages are added to MLP. - When a specific argument shall be extracted, COMMAS_TO_SKIP >= 0 and, - if also a plural argument shall be extracted, PLURAL_COMMAS > 0, - otherwise PLURAL_COMMAS = 0. - When no specific argument shall be extracted, COMMAS_TO_SKIP < 0. Return true upon eof, false upon closing parenthesis. */ static bool extract_parenthesized (message_list_ty *mlp, flag_context_ty outer_context, flag_context_list_iterator_ty context_iter, - int commas_to_skip, int plural_commas) + struct arglist_parser *argparser) { - /* Remember the message containing the msgid, for msgid_plural. */ - message_ty *plural_mp = NULL; - + /* Current argument number. */ + int arg = 1; /* 0 when no keyword has been seen. 1 right after a keyword is seen. */ int state; /* Parameters of the keyword just seen. Defined only in state 1. */ - int next_commas_to_skip = -1; - int next_plural_commas = 0; + const struct callshapes *next_shapes = NULL; /* Context iterator that will be used if the next token is a '('. */ flag_context_list_iterator_ty next_context_iter = passthrough_context_list_iterator; @@ -1694,9 +1720,7 @@ extract_parenthesized (message_list_ty *mlp, switch (token.type) { case xgettext_token_type_keyword: - next_commas_to_skip = token.argnum1 - 1; - next_plural_commas = (token.argnum2 > token.argnum1 - ? token.argnum2 - token.argnum1 : 0); + next_shapes = token.shapes; state = 1; goto keyword_or_symbol; @@ -1725,31 +1749,23 @@ extract_parenthesized (message_list_ty *mlp, case xgettext_token_type_lparen: if (extract_parenthesized (mlp, inner_context, next_context_iter, - state ? next_commas_to_skip : -1, - state ? next_plural_commas : 0)) - return true; + arglist_parser_alloc (mlp, + state ? next_shapes : NULL))) + { + arglist_parser_done (argparser); + return true; + } next_context_iter = null_context_list_iterator; selectorcall_context_iter = null_context_list_iterator; state = 0; continue; case xgettext_token_type_rparen: + arglist_parser_done (argparser); return false; case xgettext_token_type_comma: - if (commas_to_skip >= 0) - { - if (commas_to_skip > 0) - commas_to_skip--; - else - if (plural_mp != NULL && plural_commas > 0) - { - commas_to_skip = plural_commas - 1; - plural_commas = 0; - } - else - commas_to_skip = -1; - } + arg++; inner_context = inherited_context (outer_context, flag_context_list_iterator_advance ( @@ -1780,34 +1796,13 @@ extract_parenthesized (message_list_ty *mlp, case xgettext_token_type_string_literal: if (extract_all) - remember_a_message (mlp, token.string, inner_context, &token.pos, - token.comment); + remember_a_message (mlp, NULL, token.string, inner_context, + &token.pos, token.comment); else - { - if (commas_to_skip == 0) - { - if (plural_mp == NULL) - { - /* Seen an msgid. */ - message_ty *mp = - remember_a_message (mlp, token.string, - inner_context, &token.pos, - token.comment); - if (plural_commas > 0) - plural_mp = mp; - } - else - { - /* Seen an msgid_plural. */ - remember_a_message_plural (plural_mp, token.string, - inner_context, &token.pos, - NULL); - plural_mp = NULL; - } - } - else - free (token.string); - } + arglist_parser_remember (argparser, arg, token.string, + inner_context, + token.pos.file_name, token.pos.line_number, + token.comment); drop_reference (token.comment); next_context_iter = null_context_list_iterator; selectorcall_context_iter = null_context_list_iterator; @@ -1821,6 +1816,7 @@ extract_parenthesized (message_list_ty *mlp, continue; case xgettext_token_type_eof: + arglist_parser_done (argparser); return true; default: @@ -1854,7 +1850,7 @@ extract_whole_file (FILE *f, /* Eat tokens until eof is seen. When extract_parenthesized returns due to an unbalanced closing parenthesis, just restart it. */ while (!extract_parenthesized (mlp, null_context, null_context_list_iterator, - -1, 0)) + arglist_parser_alloc (mlp, NULL))) ; /* Close scanner. */ diff --git a/gettext-tools/src/x-csharp.c b/gettext-tools/src/x-csharp.c index a75a274cd..7f8b703f3 100644 --- a/gettext-tools/src/x-csharp.c +++ b/gettext-tools/src/x-csharp.c @@ -75,26 +75,20 @@ x_csharp_keyword (const char *name) else { const char *end; - int argnum1; - int argnum2; + struct callshape shape; const char *colon; if (keywords.table == NULL) hash_init (&keywords, 100); - split_keywordspec (name, &end, &argnum1, &argnum2); + split_keywordspec (name, &end, &shape); /* The characters between name and end should form a valid C# identifier sequence with dots. A colon means an invalid parse in split_keywordspec(). */ colon = strchr (name, ':'); if (colon == NULL || colon >= end) - { - if (argnum1 == 0) - argnum1 = 1; - hash_insert_entry (&keywords, name, end - name, - (void *) (long) (argnum1 + (argnum2 << 10))); - } + insert_keyword_callshape (&keywords, name, end - name, &shape); } } @@ -1865,25 +1859,19 @@ static flag_context_list_table_ty *flag_context_list_table; /* Extract messages until the next balanced closing parenthesis or brace, depending on TERMINATOR. Extracted messages are added to MLP. - When a specific argument shall be extracted, COMMAS_TO_SKIP >= 0 and, - if also a plural argument shall be extracted, PLURAL_COMMAS > 0, - otherwise PLURAL_COMMAS = 0. - When no specific argument shall be extracted, COMMAS_TO_SKIP < 0. Return true upon eof, false upon closing parenthesis or brace. */ static bool extract_parenthesized (message_list_ty *mlp, token_type_ty terminator, flag_context_ty outer_context, flag_context_list_iterator_ty context_iter, - int commas_to_skip, int plural_commas) + struct arglist_parser *argparser) { - /* Remember the message containing the msgid, for msgid_plural. */ - message_ty *plural_mp = NULL; - + /* Current argument number. */ + int arg = 1; /* 0 when no keyword has been seen. 1 right after a keyword is seen. */ int state; /* Parameters of the keyword just seen. Defined only in state 1. */ - int next_commas_to_skip = -1; - int next_plural_commas = 0; + const struct callshapes *next_shapes = NULL; /* Context iterator that will be used if the next token is a '('. */ flag_context_list_iterator_ty next_context_iter = passthrough_context_list_iterator; @@ -1953,11 +1941,7 @@ extract_parenthesized (message_list_ty *mlp, token_type_ty terminator, &keyword_value) == 0) { - int argnum1 = (int) (long) keyword_value & ((1 << 10) - 1); - int argnum2 = (int) (long) keyword_value >> 10; - - next_commas_to_skip = argnum1 - 1; - next_plural_commas = (argnum2 > argnum1 ? argnum2 - argnum1 : 0); + next_shapes = (const struct callshapes *) keyword_value; state = 1; break; } @@ -1994,16 +1978,26 @@ extract_parenthesized (message_list_ty *mlp, token_type_ty terminator, case token_type_lparen: if (extract_parenthesized (mlp, token_type_rparen, inner_context, next_context_iter, - state ? next_commas_to_skip : -1, - state ? next_plural_commas : 0)) - return true; + arglist_parser_alloc (mlp, + state ? next_shapes : NULL))) + { + xgettext_current_source_encoding = po_charset_utf8; + arglist_parser_done (argparser); + xgettext_current_source_encoding = xgettext_global_source_encoding; + return true; + } next_context_iter = null_context_list_iterator; state = 0; continue; case token_type_rparen: if (terminator == token_type_rparen) - return false; + { + xgettext_current_source_encoding = po_charset_utf8; + arglist_parser_done (argparser); + xgettext_current_source_encoding = xgettext_global_source_encoding; + return false; + } if (terminator == token_type_rbrace) { error_with_progname = false; @@ -2019,15 +2013,25 @@ extract_parenthesized (message_list_ty *mlp, token_type_ty terminator, case token_type_lbrace: if (extract_parenthesized (mlp, token_type_rbrace, null_context, null_context_list_iterator, - -1, 0)) - return true; + arglist_parser_alloc (mlp, NULL))) + { + xgettext_current_source_encoding = po_charset_utf8; + arglist_parser_done (argparser); + xgettext_current_source_encoding = xgettext_global_source_encoding; + return true; + } next_context_iter = null_context_list_iterator; state = 0; continue; case token_type_rbrace: if (terminator == token_type_rbrace) - return false; + { + xgettext_current_source_encoding = po_charset_utf8; + arglist_parser_done (argparser); + xgettext_current_source_encoding = xgettext_global_source_encoding; + return false; + } if (terminator == token_type_rparen) { error_with_progname = false; @@ -2041,19 +2045,7 @@ extract_parenthesized (message_list_ty *mlp, token_type_ty terminator, continue; case token_type_comma: - if (commas_to_skip >= 0) - { - if (commas_to_skip > 0) - commas_to_skip--; - else - if (plural_mp != NULL && plural_commas > 0) - { - commas_to_skip = plural_commas - 1; - plural_commas = 0; - } - else - commas_to_skip = -1; - } + arg++; inner_context = inherited_context (outer_context, flag_context_list_iterator_advance ( @@ -2068,43 +2060,16 @@ extract_parenthesized (message_list_ty *mlp, token_type_ty terminator, pos.file_name = logical_file_name; pos.line_number = token.line_number; + xgettext_current_source_encoding = po_charset_utf8; if (extract_all) - { - xgettext_current_source_encoding = po_charset_utf8; - remember_a_message (mlp, token.string, inner_context, &pos, - token.comment); - xgettext_current_source_encoding = xgettext_global_source_encoding; - } + remember_a_message (mlp, NULL, token.string, inner_context, + &pos, token.comment); else - { - if (commas_to_skip == 0) - { - if (plural_mp == NULL) - { - /* Seen an msgid. */ - message_ty *mp; - - xgettext_current_source_encoding = po_charset_utf8; - mp = remember_a_message (mlp, token.string, - inner_context, &pos, - token.comment); - xgettext_current_source_encoding = xgettext_global_source_encoding; - if (plural_commas > 0) - plural_mp = mp; - } - else - { - /* Seen an msgid_plural. */ - xgettext_current_source_encoding = po_charset_utf8; - remember_a_message_plural (plural_mp, token.string, - inner_context, &pos, NULL); - xgettext_current_source_encoding = xgettext_global_source_encoding; - plural_mp = NULL; - } - } - else - free (token.string); - } + arglist_parser_remember (argparser, arg, token.string, + inner_context, + pos.file_name, pos.line_number, + token.comment); + xgettext_current_source_encoding = xgettext_global_source_encoding; } drop_reference (token.comment); next_context_iter = null_context_list_iterator; @@ -2112,6 +2077,9 @@ extract_parenthesized (message_list_ty *mlp, token_type_ty terminator, continue; case token_type_eof: + xgettext_current_source_encoding = po_charset_utf8; + arglist_parser_done (argparser); + xgettext_current_source_encoding = xgettext_global_source_encoding; return true; case token_type_dot: @@ -2154,7 +2122,7 @@ extract_csharp (FILE *f, due to an unbalanced closing parenthesis, just restart it. */ while (!extract_parenthesized (mlp, token_type_eof, null_context, null_context_list_iterator, - -1, 0)) + arglist_parser_alloc (mlp, NULL))) ; fp = NULL; diff --git a/gettext-tools/src/x-elisp.c b/gettext-tools/src/x-elisp.c index f7e967f5d..bc605837e 100644 --- a/gettext-tools/src/x-elisp.c +++ b/gettext-tools/src/x-elisp.c @@ -82,25 +82,19 @@ x_elisp_keyword (const char *name) else { const char *end; - int argnum1; - int argnum2; + struct callshape shape; const char *colon; if (keywords.table == NULL) hash_init (&keywords, 100); - split_keywordspec (name, &end, &argnum1, &argnum2); + split_keywordspec (name, &end, &shape); /* The characters between name and end should form a valid Lisp symbol. */ colon = strchr (name, ':'); if (colon == NULL || colon >= end) - { - if (argnum1 == 0) - argnum1 = 1; - hash_insert_entry (&keywords, name, end - name, - (void *) (long) (argnum1 + (argnum2 << 10))); - } + insert_keyword_callshape (&keywords, name, end - name, &shape); } } @@ -646,9 +640,8 @@ read_object (struct object *op, bool first_in_list, bool new_backquote_flag, { int arg = 0; /* Current argument number. */ flag_context_list_iterator_ty context_iter; - int argnum1 = 0; /* First string position. */ - int argnum2 = 0; /* Plural string position. */ - message_ty *plural_mp = NULL; /* Remember the msgid. */ + const struct callshapes *shapes = NULL; + struct arglist_parser *argparser = NULL; for (;; arg++) { @@ -672,6 +665,8 @@ read_object (struct object *op, bool first_in_list, bool new_backquote_flag, op->type = t_other; /* Don't bother converting "()" to "NIL". */ last_non_comment_line = line_number; + if (argparser != NULL) + arglist_parser_done (argparser); return; } @@ -694,10 +689,9 @@ read_object (struct object *op, bool first_in_list, bool new_backquote_flag, symbol_name, strlen (symbol_name), &keyword_value) == 0) - { - argnum1 = (int) (long) keyword_value & ((1 << 10) - 1); - argnum2 = (int) (long) keyword_value >> 10; - } + shapes = (const struct callshapes *) keyword_value; + + argparser = arglist_parser_alloc (mlp, shapes); context_iter = flag_context_list_iterator ( @@ -712,42 +706,21 @@ read_object (struct object *op, bool first_in_list, bool new_backquote_flag, } else { - /* These are the argument positions. - Extract a string if we have reached the right - argument position. */ - if (arg == argnum1) - { - if (inner.type == t_string) - { - lex_pos_ty pos; - message_ty *mp; - - pos.file_name = logical_file_name; - pos.line_number = inner.line_number_at_start; - mp = remember_a_message (mlp, string_of_object (&inner), - inner_context, &pos, - savable_comment); - if (argnum2 > 0) - plural_mp = mp; - } - } - else if (arg == argnum2) - { - if (inner.type == t_string && plural_mp != NULL) - { - lex_pos_ty pos; - - pos.file_name = logical_file_name; - pos.line_number = inner.line_number_at_start; - remember_a_message_plural (plural_mp, string_of_object (&inner), - inner_context, &pos, - savable_comment); - } - } + /* These are the argument positions. */ + if (inner.type == t_string) + arglist_parser_remember (argparser, arg, + string_of_object (&inner), + inner_context, + logical_file_name, + inner.line_number_at_start, + savable_comment); } free_object (&inner); } + + if (argparser != NULL) + arglist_parser_done (argparser); } op->type = t_other; last_non_comment_line = line_number; @@ -922,7 +895,7 @@ read_object (struct object *op, bool first_in_list, bool new_backquote_flag, pos.file_name = logical_file_name; pos.line_number = op->line_number_at_start; - remember_a_message (mlp, string_of_object (op), + remember_a_message (mlp, NULL, string_of_object (op), null_context, &pos, savable_comment); } last_non_comment_line = line_number; diff --git a/gettext-tools/src/x-glade.c b/gettext-tools/src/x-glade.c index 8281af33a..cf6bc615b 100644 --- a/gettext-tools/src/x-glade.c +++ b/gettext-tools/src/x-glade.c @@ -268,7 +268,7 @@ start_element_handler (void *userData, const char *name, pos.file_name = logical_file_name; pos.line_number = XML_GetCurrentLineNumber (parser); - remember_a_message (mlp, xstrdup (attp[1]), + remember_a_message (mlp, NULL, xstrdup (attp[1]), null_context, &pos, savable_comment); } break; @@ -305,7 +305,7 @@ end_element_handler (void *userData, const char *name) pos.file_name = logical_file_name; pos.line_number = p->lineno; - remember_a_message (mlp, p->buffer, null_context, &pos, + remember_a_message (mlp, NULL, p->buffer, null_context, &pos, savable_comment); p->buffer = NULL; } diff --git a/gettext-tools/src/x-java.c b/gettext-tools/src/x-java.c index c44a55a07..6845db51c 100644 --- a/gettext-tools/src/x-java.c +++ b/gettext-tools/src/x-java.c @@ -73,26 +73,20 @@ x_java_keyword (const char *name) else { const char *end; - int argnum1; - int argnum2; + struct callshape shape; const char *colon; if (keywords.table == NULL) hash_init (&keywords, 100); - split_keywordspec (name, &end, &argnum1, &argnum2); + split_keywordspec (name, &end, &shape); /* The characters between name and end should form a valid Java identifier sequence with dots. A colon means an invalid parse in split_keywordspec(). */ colon = strchr (name, ':'); if (colon == NULL || colon >= end) - { - if (argnum1 == 0) - argnum1 = 1; - hash_insert_entry (&keywords, name, end - name, - (void *) (long) (argnum1 + (argnum2 << 10))); - } + insert_keyword_callshape (&keywords, name, end - name, &shape); } } @@ -1177,25 +1171,19 @@ static flag_context_list_table_ty *flag_context_list_table; /* Extract messages until the next balanced closing parenthesis or brace, depending on TERMINATOR. Extracted messages are added to MLP. - When a specific argument shall be extracted, COMMAS_TO_SKIP >= 0 and, - if also a plural argument shall be extracted, PLURAL_COMMAS > 0, - otherwise PLURAL_COMMAS = 0. - When no specific argument shall be extracted, COMMAS_TO_SKIP < 0. Return true upon eof, false upon closing parenthesis or brace. */ static bool extract_parenthesized (message_list_ty *mlp, token_type_ty terminator, flag_context_ty outer_context, flag_context_list_iterator_ty context_iter, - int commas_to_skip, int plural_commas) + struct arglist_parser *argparser) { - /* Remember the message containing the msgid, for msgid_plural. */ - message_ty *plural_mp = NULL; - + /* Current argument number. */ + int arg = 1; /* 0 when no keyword has been seen. 1 right after a keyword is seen. */ int state; /* Parameters of the keyword just seen. Defined only in state 1. */ - int next_commas_to_skip = -1; - int next_plural_commas = 0; + const struct callshapes *next_shapes = NULL; /* Context iterator that will be used if the next token is a '('. */ flag_context_list_iterator_ty next_context_iter = passthrough_context_list_iterator; @@ -1265,11 +1253,7 @@ extract_parenthesized (message_list_ty *mlp, token_type_ty terminator, &keyword_value) == 0) { - int argnum1 = (int) (long) keyword_value & ((1 << 10) - 1); - int argnum2 = (int) (long) keyword_value >> 10; - - next_commas_to_skip = argnum1 - 1; - next_plural_commas = (argnum2 > argnum1 ? argnum2 - argnum1 : 0); + next_shapes = (const struct callshapes *) keyword_value; state = 1; break; } @@ -1306,16 +1290,26 @@ extract_parenthesized (message_list_ty *mlp, token_type_ty terminator, case token_type_lparen: if (extract_parenthesized (mlp, token_type_rparen, inner_context, next_context_iter, - state ? next_commas_to_skip : -1, - state ? next_plural_commas : 0)) - return true; + arglist_parser_alloc (mlp, + state ? next_shapes : NULL))) + { + xgettext_current_source_encoding = po_charset_utf8; + arglist_parser_done (argparser); + xgettext_current_source_encoding = xgettext_global_source_encoding; + return true; + } next_context_iter = null_context_list_iterator; state = 0; continue; case token_type_rparen: if (terminator == token_type_rparen) - return false; + { + xgettext_current_source_encoding = po_charset_utf8; + arglist_parser_done (argparser); + xgettext_current_source_encoding = xgettext_global_source_encoding; + return false; + } if (terminator == token_type_rbrace) { error_with_progname = false; @@ -1331,15 +1325,25 @@ extract_parenthesized (message_list_ty *mlp, token_type_ty terminator, case token_type_lbrace: if (extract_parenthesized (mlp, token_type_rbrace, null_context, null_context_list_iterator, - -1, 0)) - return true; + arglist_parser_alloc (mlp, NULL))) + { + xgettext_current_source_encoding = po_charset_utf8; + arglist_parser_done (argparser); + xgettext_current_source_encoding = xgettext_global_source_encoding; + return true; + } next_context_iter = null_context_list_iterator; state = 0; continue; case token_type_rbrace: if (terminator == token_type_rbrace) - return false; + { + xgettext_current_source_encoding = po_charset_utf8; + arglist_parser_done (argparser); + xgettext_current_source_encoding = xgettext_global_source_encoding; + return false; + } if (terminator == token_type_rparen) { error_with_progname = false; @@ -1353,19 +1357,7 @@ extract_parenthesized (message_list_ty *mlp, token_type_ty terminator, continue; case token_type_comma: - if (commas_to_skip >= 0) - { - if (commas_to_skip > 0) - commas_to_skip--; - else - if (plural_mp != NULL && plural_commas > 0) - { - commas_to_skip = plural_commas - 1; - plural_commas = 0; - } - else - commas_to_skip = -1; - } + arg++; inner_context = inherited_context (outer_context, flag_context_list_iterator_advance ( @@ -1380,43 +1372,16 @@ extract_parenthesized (message_list_ty *mlp, token_type_ty terminator, pos.file_name = logical_file_name; pos.line_number = token.line_number; + xgettext_current_source_encoding = po_charset_utf8; if (extract_all) - { - xgettext_current_source_encoding = po_charset_utf8; - remember_a_message (mlp, token.string, inner_context, &pos, - token.comment); - xgettext_current_source_encoding = xgettext_global_source_encoding; - } + remember_a_message (mlp, NULL, token.string, inner_context, + &pos, token.comment); else - { - if (commas_to_skip == 0) - { - if (plural_mp == NULL) - { - /* Seen an msgid. */ - message_ty *mp; - - xgettext_current_source_encoding = po_charset_utf8; - mp = remember_a_message (mlp, token.string, - inner_context, &pos, - token.comment); - xgettext_current_source_encoding = xgettext_global_source_encoding; - if (plural_commas > 0) - plural_mp = mp; - } - else - { - /* Seen an msgid_plural. */ - xgettext_current_source_encoding = po_charset_utf8; - remember_a_message_plural (plural_mp, token.string, - inner_context, &pos, NULL); - xgettext_current_source_encoding = xgettext_global_source_encoding; - plural_mp = NULL; - } - } - else - free (token.string); - } + arglist_parser_remember (argparser, arg, token.string, + inner_context, + pos.file_name, pos.line_number, + token.comment); + xgettext_current_source_encoding = xgettext_global_source_encoding; } drop_reference (token.comment); next_context_iter = null_context_list_iterator; @@ -1424,6 +1389,9 @@ extract_parenthesized (message_list_ty *mlp, token_type_ty terminator, continue; case token_type_eof: + xgettext_current_source_encoding = po_charset_utf8; + arglist_parser_done (argparser); + xgettext_current_source_encoding = xgettext_global_source_encoding; return true; case token_type_dot: @@ -1467,7 +1435,7 @@ extract_java (FILE *f, due to an unbalanced closing parenthesis, just restart it. */ while (!extract_parenthesized (mlp, token_type_eof, null_context, null_context_list_iterator, - -1, 0)) + arglist_parser_alloc (mlp, NULL))) ; fp = NULL; diff --git a/gettext-tools/src/x-librep.c b/gettext-tools/src/x-librep.c index 7bcbbac13..56b78b61b 100644 --- a/gettext-tools/src/x-librep.c +++ b/gettext-tools/src/x-librep.c @@ -84,25 +84,19 @@ x_librep_keyword (const char *name) else { const char *end; - int argnum1; - int argnum2; + struct callshape shape; const char *colon; if (keywords.table == NULL) hash_init (&keywords, 100); - split_keywordspec (name, &end, &argnum1, &argnum2); + split_keywordspec (name, &end, &shape); /* The characters between name and end should form a valid Lisp symbol. */ colon = strchr (name, ':'); if (colon == NULL || colon >= end) - { - if (argnum1 == 0) - argnum1 = 1; - hash_insert_entry (&keywords, name, end - name, - (void *) (long) (argnum1 + (argnum2 << 10))); - } + insert_keyword_callshape (&keywords, name, end - name, &shape); } } @@ -621,9 +615,8 @@ read_object (struct object *op, flag_context_ty outer_context) { int arg = 0; /* Current argument number. */ flag_context_list_iterator_ty context_iter; - int argnum1 = 0; /* First string position. */ - int argnum2 = 0; /* Plural string position. */ - message_ty *plural_mp = NULL; /* Remember the msgid. */ + const struct callshapes *shapes = NULL; + struct arglist_parser *argparser = NULL; for (;; arg++) { @@ -646,6 +639,8 @@ read_object (struct object *op, flag_context_ty outer_context) op->type = t_other; /* Don't bother converting "()" to "NIL". */ last_non_comment_line = line_number; + if (argparser != NULL) + arglist_parser_done (argparser); return; } @@ -668,10 +663,9 @@ read_object (struct object *op, flag_context_ty outer_context) symbol_name, strlen (symbol_name), &keyword_value) == 0) - { - argnum1 = (int) (long) keyword_value & ((1 << 10) - 1); - argnum2 = (int) (long) keyword_value >> 10; - } + shapes = (const struct callshapes *) keyword_value; + + argparser = arglist_parser_alloc (mlp, shapes); context_iter = flag_context_list_iterator ( @@ -686,42 +680,21 @@ read_object (struct object *op, flag_context_ty outer_context) } else { - /* These are the argument positions. - Extract a string if we have reached the right - argument position. */ - if (arg == argnum1) - { - if (inner.type == t_string) - { - lex_pos_ty pos; - message_ty *mp; - - pos.file_name = logical_file_name; - pos.line_number = inner.line_number_at_start; - mp = remember_a_message (mlp, string_of_object (&inner), - inner_context, &pos, - savable_comment); - if (argnum2 > 0) - plural_mp = mp; - } - } - else if (arg == argnum2) - { - if (inner.type == t_string && plural_mp != NULL) - { - lex_pos_ty pos; - - pos.file_name = logical_file_name; - pos.line_number = inner.line_number_at_start; - remember_a_message_plural (plural_mp, string_of_object (&inner), - inner_context, &pos, - savable_comment); - } - } + /* These are the argument positions. */ + if (inner.type == t_string) + arglist_parser_remember (argparser, arg, + string_of_object (&inner), + inner_context, + logical_file_name, + inner.line_number_at_start, + savable_comment); } free_object (&inner); } + + if (argparser != NULL) + arglist_parser_done (argparser); } op->type = t_other; last_non_comment_line = line_number; @@ -858,7 +831,7 @@ read_object (struct object *op, flag_context_ty outer_context) pos.file_name = logical_file_name; pos.line_number = op->line_number_at_start; - remember_a_message (mlp, string_of_object (op), + remember_a_message (mlp, NULL, string_of_object (op), null_context, &pos, savable_comment); } last_non_comment_line = line_number; diff --git a/gettext-tools/src/x-lisp.c b/gettext-tools/src/x-lisp.c index dbfd2ae13..bcc5cf6e9 100644 --- a/gettext-tools/src/x-lisp.c +++ b/gettext-tools/src/x-lisp.c @@ -125,8 +125,7 @@ x_lisp_keyword (const char *name) else { const char *end; - int argnum1; - int argnum2; + struct callshape shape; const char *colon; size_t len; char *symname; @@ -135,7 +134,7 @@ x_lisp_keyword (const char *name) if (keywords.table == NULL) hash_init (&keywords, 100); - split_keywordspec (name, &end, &argnum1, &argnum2); + split_keywordspec (name, &end, &shape); /* The characters between name and end should form a valid Lisp symbol. Extract the symbol name part. */ @@ -157,10 +156,7 @@ x_lisp_keyword (const char *name) symname[i] = (name[i] >= 'a' && name[i] <= 'z' ? name[i] - 'a' + 'A' : name[i]); - if (argnum1 == 0) - argnum1 = 1; - hash_insert_entry (&keywords, symname, len, - (void *) (long) (argnum1 + (argnum2 << 10))); + insert_keyword_callshape (&keywords, symname, len, &shape); } } @@ -1012,9 +1008,8 @@ read_object (struct object *op, flag_context_ty outer_context) { int arg = 0; /* Current argument number. */ flag_context_list_iterator_ty context_iter; - int argnum1 = 0; /* First string position. */ - int argnum2 = 0; /* Plural string position. */ - message_ty *plural_mp = NULL; /* Remember the msgid. */ + const struct callshapes *shapes = NULL; + struct arglist_parser *argparser = NULL; for (;; arg++) { @@ -1037,6 +1032,8 @@ read_object (struct object *op, flag_context_ty outer_context) op->type = t_other; /* Don't bother converting "()" to "NIL". */ last_non_comment_line = line_number; + if (argparser != NULL) + arglist_parser_done (argparser); return; } @@ -1070,10 +1067,9 @@ read_object (struct object *op, flag_context_ty outer_context) strlen (symbol_name + prefix_len), &keyword_value) == 0) - { - argnum1 = (int) (long) keyword_value & ((1 << 10) - 1); - argnum2 = (int) (long) keyword_value >> 10; - } + shapes = (const struct callshapes *) keyword_value; + + argparser = arglist_parser_alloc (mlp, shapes); context_iter = flag_context_list_iterator ( @@ -1088,42 +1084,21 @@ read_object (struct object *op, flag_context_ty outer_context) } else { - /* These are the argument positions. - Extract a string if we have reached the right - argument position. */ - if (arg == argnum1) - { - if (inner.type == t_string) - { - lex_pos_ty pos; - message_ty *mp; - - pos.file_name = logical_file_name; - pos.line_number = inner.line_number_at_start; - mp = remember_a_message (mlp, string_of_object (&inner), - inner_context, &pos, - savable_comment); - if (argnum2 > 0) - plural_mp = mp; - } - } - else if (arg == argnum2) - { - if (inner.type == t_string && plural_mp != NULL) - { - lex_pos_ty pos; - - pos.file_name = logical_file_name; - pos.line_number = inner.line_number_at_start; - remember_a_message_plural (plural_mp, string_of_object (&inner), - inner_context, &pos, - savable_comment); - } - } + /* These are the argument positions. */ + if (inner.type == t_string) + arglist_parser_remember (argparser, arg, + string_of_object (&inner), + inner_context, + logical_file_name, + inner.line_number_at_start, + savable_comment); } free_object (&inner); } + + if (argparser != NULL) + arglist_parser_done (argparser); } op->type = t_other; last_non_comment_line = line_number; @@ -1217,7 +1192,7 @@ read_object (struct object *op, flag_context_ty outer_context) pos.file_name = logical_file_name; pos.line_number = op->line_number_at_start; - remember_a_message (mlp, string_of_object (op), + remember_a_message (mlp, NULL, string_of_object (op), null_context, &pos, savable_comment); } last_non_comment_line = line_number; diff --git a/gettext-tools/src/x-perl.c b/gettext-tools/src/x-perl.c index 6e0350c91..0e6880b61 100644 --- a/gettext-tools/src/x-perl.c +++ b/gettext-tools/src/x-perl.c @@ -74,25 +74,19 @@ x_perl_keyword (const char *name) else { const char *end; - int argnum1; - int argnum2; + struct callshape shape; const char *colon; if (keywords.table == NULL) hash_init (&keywords, 100); - split_keywordspec (name, &end, &argnum1, &argnum2); + split_keywordspec (name, &end, &shape); /* The characters between name and end should form a valid C identifier. A colon means an invalid parse in split_keywordspec(). */ colon = strchr (name, ':'); if (colon == NULL || colon >= end) - { - if (argnum1 == 0) - argnum1 = 1; - hash_insert_entry (&keywords, name, end - name, - (void *) (long) (argnum1 + (argnum2 << 10))); - } + insert_keyword_callshape (&keywords, name, end - name, &shape); } } @@ -767,7 +761,7 @@ static bool extract_balanced (message_list_ty *mlp, int state, token_type_ty delim, flag_context_ty outer_context, flag_context_list_iterator_ty context_iter, - int arg_sg, int arg_pl); + int arg, struct arglist_parser *argparser); /* Extract an unsigned hexadecimal number from STRING, considering at @@ -1374,7 +1368,8 @@ extract_variable (message_list_ty *mlp, token_ty *tp, int first) #endif if (extract_balanced (mlp, 0, token_type_rbrace, - null_context, null_context_list_iterator, -1, -1)) + null_context, null_context_list_iterator, + 1, arglist_parser_alloc (mlp, NULL))) return; buffer[bufpos++] = c; } @@ -1498,6 +1493,19 @@ extract_variable (message_list_ty *mlp, token_ty *tp, int first) if (hash_find_entry (&keywords, tp->string, strlen (tp->string), &keyword_value) == 0) { + /* TODO: Shouldn't we use the shapes of the keyword, instead + of hardwiring argnum1 = 1 ? + const struct callshapes *shapes = + (const struct callshapes *) keyword_value; + */ + struct callshapes shapes; + shapes.keyword = tp->string; /* XXX storage duration? */ + shapes.keyword_len = strlen (tp->string); + shapes.nshapes = 1; + shapes.shapes[0].argnum1 = 1; + shapes.shapes[0].argnum2 = 0; + shapes.shapes[0].argnumc = 0; + /* Extract a possible string from the key. Before proceeding we check whether the open curly is followed by a symbol and then by a right curly. */ @@ -1531,8 +1539,8 @@ extract_variable (message_list_ty *mlp, token_ty *tp, int first) pos.file_name = logical_file_name; xgettext_current_source_encoding = po_charset_utf8; - remember_a_message (mlp, xstrdup (t1->string), context, - &pos, savable_comment); + remember_a_message (mlp, NULL, xstrdup (t1->string), + context, &pos, savable_comment); xgettext_current_source_encoding = xgettext_global_source_encoding; free_token (t2); free_token (t1); @@ -1546,7 +1554,8 @@ extract_variable (message_list_ty *mlp, token_ty *tp, int first) { x_perl_unlex (t1); if (extract_balanced (mlp, 1, token_type_rbrace, - null_context, context_iter, 1, -1)) + null_context, context_iter, + 1, arglist_parser_alloc (mlp, &shapes))) return; } } @@ -1575,7 +1584,8 @@ extract_variable (message_list_ty *mlp, token_ty *tp, int first) real_file_name, line_number); #endif extract_balanced (mlp, 0, token_type_rbrace, - null_context, null_context_list_iterator, -1, -1); + null_context, null_context_list_iterator, + 1, arglist_parser_alloc (mlp, NULL)); break; case '[': @@ -1584,7 +1594,8 @@ extract_variable (message_list_ty *mlp, token_ty *tp, int first) real_file_name, line_number); #endif extract_balanced (mlp, 0, token_type_rbracket, - null_context, null_context_list_iterator, -1, -1); + null_context, null_context_list_iterator, + 1, arglist_parser_alloc (mlp, NULL)); break; case '-': @@ -1953,7 +1964,7 @@ interpolate_keywords (message_list_ty *mlp, const char *string, int lineno) token.string = xstrdup (buffer); extract_quotelike_pass3 (&token, EXIT_FAILURE); xgettext_current_source_encoding = po_charset_utf8; - remember_a_message (mlp, token.string, context, &pos, + remember_a_message (mlp, NULL, token.string, context, &pos, savable_comment); xgettext_current_source_encoding = xgettext_global_source_encoding; /* FALLTHROUGH */ @@ -2777,8 +2788,8 @@ collect_message (message_list_ty *mlp, token_ty *tp, int error_level) /* Extract messages until the next balanced closing parenthesis. Extracted messages are added to MLP. - When specific arguments shall be extracted, ARG_SG and ARG_PL are - set to the corresponding argument number or -1 if not applicable. + ARG is the current argument list position, starts with 1. + ARGPARSER is the corresponding argument list parser. Returns true for EOF, false otherwise. @@ -2805,15 +2816,8 @@ static bool extract_balanced (message_list_ty *mlp, int state, token_type_ty delim, flag_context_ty outer_context, flag_context_list_iterator_ty context_iter, - int arg_sg, int arg_pl) + int arg, struct arglist_parser *argparser) { - /* Remember the message containing the msgid, for msgid_plural. */ - message_ty *plural_mp = NULL; - - /* The current argument for a possibly extracted keyword. Counting - starts with 1. */ - int arg_count = 1; - /* Number of left parentheses seen. */ int paren_seen = 0; @@ -2850,6 +2854,9 @@ extract_balanced (message_list_ty *mlp, int state, token_type_ty delim, if (delim == tp->type) { + xgettext_current_source_encoding = po_charset_utf8; + arglist_parser_done (argparser); + xgettext_current_source_encoding = xgettext_global_source_encoding; #if DEBUG_PERL fprintf (stderr, "%s:%d: extract_balanced finished (%d)\n", logical_file_name, tp->line_number, --nesting_level); @@ -2884,11 +2891,16 @@ extract_balanced (message_list_ty *mlp, int state, token_type_ty delim, if (hash_find_entry (&keywords, tp->string, strlen (tp->string), &keyword_value) == 0) { - last_token = token_type_keyword_symbol; + const struct callshapes *shapes = + (const struct callshapes *) keyword_value; - arg_sg = (int) (long) keyword_value & ((1 << 10) - 1); - arg_pl = (int) (long) keyword_value >> 10; - arg_count = 1; + xgettext_current_source_encoding = po_charset_utf8; + arglist_parser_done (argparser); + xgettext_current_source_encoding = xgettext_global_source_encoding; + argparser = arglist_parser_alloc (mlp, shapes); + arg = 1; + + last_token = token_type_keyword_symbol; state = 2; } @@ -2920,13 +2932,21 @@ extract_balanced (message_list_ty *mlp, int state, token_type_ty delim, if (extract_balanced (mlp, state, token_type_rparen, inner_context, next_context_iter, - arg_sg - arg_count + 1, arg_pl - arg_count + 1)) + arg, arglist_parser_clone (argparser))) { + xgettext_current_source_encoding = po_charset_utf8; + arglist_parser_done (argparser); + xgettext_current_source_encoding = xgettext_global_source_encoding; free_token (tp); return true; } if (my_last_token == token_type_keyword_symbol) - arg_sg = arg_pl = -1; + { + xgettext_current_source_encoding = po_charset_utf8; + arglist_parser_done (argparser); + xgettext_current_source_encoding = xgettext_global_source_encoding; + argparser = arglist_parser_alloc (mlp, NULL); + } next_is_argument = false; next_context_iter = null_context_list_iterator; break; @@ -2947,17 +2967,19 @@ extract_balanced (message_list_ty *mlp, int state, token_type_ty delim, fprintf (stderr, "%s:%d: type comma (%d)\n", logical_file_name, tp->line_number, nesting_level); #endif - ++arg_count; - if (arg_count > arg_sg && arg_count > arg_pl) + if (arglist_parser_decidedp (argparser, arg)) { /* We have missed the argument. */ - arg_sg = arg_pl = -1; - arg_count = 0; + xgettext_current_source_encoding = po_charset_utf8; + arglist_parser_done (argparser); + xgettext_current_source_encoding = xgettext_global_source_encoding; + argparser = arglist_parser_alloc (mlp, NULL); + arg = 0; } + arg++; #if DEBUG_PERL - fprintf (stderr, "%s:%d: arg_count: %d, arg_sg: %d, arg_pl: %d\n", - real_file_name, tp->line_number, - arg_count, arg_sg, arg_pl); + fprintf (stderr, "%s:%d: arg: %d\n", + real_file_name, tp->line_number, arg); #endif inner_context = inherited_context (outer_context, @@ -2976,54 +2998,34 @@ extract_balanced (message_list_ty *mlp, int state, token_type_ty delim, if (extract_all) { + char *string = collect_message (mlp, tp, EXIT_SUCCESS); lex_pos_ty pos; - char *string; pos.file_name = logical_file_name; pos.line_number = tp->line_number; - string = collect_message (mlp, tp, EXIT_SUCCESS); xgettext_current_source_encoding = po_charset_utf8; - remember_a_message (mlp, string, inner_context, &pos, - savable_comment); + remember_a_message (mlp, NULL, string, inner_context, &pos, savable_comment); xgettext_current_source_encoding = xgettext_global_source_encoding; } else if (state) { - lex_pos_ty pos; - char *string; + char *string = collect_message (mlp, tp, EXIT_FAILURE); - pos.file_name = logical_file_name; - pos.line_number = tp->line_number; - - if (arg_count == arg_sg) - { - string = collect_message (mlp, tp, EXIT_FAILURE); - xgettext_current_source_encoding = po_charset_utf8; - plural_mp = remember_a_message (mlp, string, inner_context, - &pos, savable_comment); - xgettext_current_source_encoding = xgettext_global_source_encoding; - arg_sg = -1; - } - else if (arg_count == arg_pl) - { - if (plural_mp == NULL) - error (EXIT_FAILURE, 0, _("\ -%s:%d: fatal: plural message seen before singular message\n"), - real_file_name, tp->line_number); - - string = collect_message (mlp, tp, EXIT_FAILURE); - xgettext_current_source_encoding = po_charset_utf8; - remember_a_message_plural (plural_mp, string, inner_context, - &pos, savable_comment); - xgettext_current_source_encoding = xgettext_global_source_encoding; - arg_pl = -1; - } + xgettext_current_source_encoding = po_charset_utf8; + arglist_parser_remember (argparser, arg, + string, inner_context, + logical_file_name, tp->line_number, + savable_comment); + xgettext_current_source_encoding = xgettext_global_source_encoding; } - if (arg_sg == -1 && arg_pl == -1) + if (arglist_parser_decidedp (argparser, arg)) { + xgettext_current_source_encoding = po_charset_utf8; + arglist_parser_done (argparser); + xgettext_current_source_encoding = xgettext_global_source_encoding; + argparser = arglist_parser_alloc (mlp, NULL); state = 0; - plural_mp = NULL; } next_is_argument = false; @@ -3035,6 +3037,9 @@ extract_balanced (message_list_ty *mlp, int state, token_type_ty delim, fprintf (stderr, "%s:%d: type EOF (%d)\n", logical_file_name, tp->line_number, nesting_level); #endif + xgettext_current_source_encoding = po_charset_utf8; + arglist_parser_done (argparser); + xgettext_current_source_encoding = xgettext_global_source_encoding; free_token (tp); return true; @@ -3045,8 +3050,11 @@ extract_balanced (message_list_ty *mlp, int state, token_type_ty delim, #endif if (extract_balanced (mlp, 0, token_type_rbrace, null_context, null_context_list_iterator, - -1, -1)) + 1, arglist_parser_alloc (mlp, NULL))) { + xgettext_current_source_encoding = po_charset_utf8; + arglist_parser_done (argparser); + xgettext_current_source_encoding = xgettext_global_source_encoding; free_token (tp); return true; } @@ -3071,8 +3079,11 @@ extract_balanced (message_list_ty *mlp, int state, token_type_ty delim, #endif if (extract_balanced (mlp, 0, token_type_rbracket, null_context, null_context_list_iterator, - -1, -1)) + 1, arglist_parser_alloc (mlp, NULL))) { + xgettext_current_source_encoding = po_charset_utf8; + arglist_parser_done (argparser); + xgettext_current_source_encoding = xgettext_global_source_encoding; free_token (tp); return true; } @@ -3098,7 +3109,10 @@ extract_balanced (message_list_ty *mlp, int state, token_type_ty delim, state = 0; /* The ultimate sign. */ - arg_sg = arg_pl = -1; + xgettext_current_source_encoding = po_charset_utf8; + arglist_parser_done (argparser); + xgettext_current_source_encoding = xgettext_global_source_encoding; + argparser = arglist_parser_alloc (mlp, NULL); /* FIXME: Instead of resetting outer_context here, it may be better to recurse in the next_is_argument handling above, waiting for @@ -3203,7 +3217,7 @@ extract_perl (FILE *f, const char *real_filename, const char *logical_filename, due to an unbalanced closing brace, just restart it. */ while (!extract_balanced (mlp, 0, token_type_rbrace, null_context, null_context_list_iterator, - -1, -1)) + 1, arglist_parser_alloc (mlp, NULL))) ; fp = NULL; diff --git a/gettext-tools/src/x-php.c b/gettext-tools/src/x-php.c index 0b03fe345..b42d26982 100644 --- a/gettext-tools/src/x-php.c +++ b/gettext-tools/src/x-php.c @@ -67,25 +67,19 @@ x_php_keyword (const char *name) else { const char *end; - int argnum1; - int argnum2; + struct callshape shape; const char *colon; if (keywords.table == NULL) hash_init (&keywords, 100); - split_keywordspec (name, &end, &argnum1, &argnum2); + split_keywordspec (name, &end, &shape); /* The characters between name and end should form a valid C identifier. A colon means an invalid parse in split_keywordspec(). */ colon = strchr (name, ':'); if (colon == NULL || colon >= end) - { - if (argnum1 == 0) - argnum1 = 1; - hash_insert_entry (&keywords, name, end - name, - (void *) (long) (argnum1 + (argnum2 << 10))); - } + insert_keyword_callshape (&keywords, name, end - name, &shape); } } @@ -1216,25 +1210,19 @@ static flag_context_list_table_ty *flag_context_list_table; /* Extract messages until the next balanced closing parenthesis. Extracted messages are added to MLP. - When a specific argument shall be extracted, COMMAS_TO_SKIP >= 0 and, - if also a plural argument shall be extracted, PLURAL_COMMAS > 0, - otherwise PLURAL_COMMAS = 0. - When no specific argument shall be extracted, COMMAS_TO_SKIP < 0. Return true upon eof, false upon closing parenthesis. */ static bool extract_parenthesized (message_list_ty *mlp, flag_context_ty outer_context, flag_context_list_iterator_ty context_iter, - int commas_to_skip, int plural_commas) + struct arglist_parser *argparser) { - /* Remember the message containing the msgid, for msgid_plural. */ - message_ty *plural_mp = NULL; - + /* Current argument number. */ + int arg = 1; /* 0 when no keyword has been seen. 1 right after a keyword is seen. */ int state; /* Parameters of the keyword just seen. Defined only in state 1. */ - int next_commas_to_skip = -1; - int next_plural_commas = 0; + const struct callshapes *next_shapes = NULL; /* Context iterator that will be used if the next token is a '('. */ flag_context_list_iterator_ty next_context_iter = passthrough_context_list_iterator; @@ -1261,11 +1249,7 @@ extract_parenthesized (message_list_ty *mlp, &keyword_value) == 0) { - int argnum1 = (int) (long) keyword_value & ((1 << 10) - 1); - int argnum2 = (int) (long) keyword_value >> 10; - - next_commas_to_skip = argnum1 - 1; - next_plural_commas = (argnum2 > argnum1 ? argnum2 - argnum1 : 0); + next_shapes = (const struct callshapes *) keyword_value; state = 1; } else @@ -1281,30 +1265,22 @@ extract_parenthesized (message_list_ty *mlp, case token_type_lparen: if (extract_parenthesized (mlp, inner_context, next_context_iter, - state ? next_commas_to_skip : -1, - state ? next_plural_commas: 0)) - return true; + arglist_parser_alloc (mlp, + state ? next_shapes : NULL))) + { + arglist_parser_done (argparser); + return true; + } next_context_iter = null_context_list_iterator; state = 0; continue; case token_type_rparen: + arglist_parser_done (argparser); return false; case token_type_comma: - if (commas_to_skip >= 0) - { - if (commas_to_skip > 0) - commas_to_skip--; - else - if (plural_mp != NULL && plural_commas > 0) - { - commas_to_skip = plural_commas - 1; - plural_commas = 0; - } - else - commas_to_skip = -1; - } + arg++; inner_context = inherited_context (outer_context, flag_context_list_iterator_advance ( @@ -1320,34 +1296,13 @@ extract_parenthesized (message_list_ty *mlp, pos.line_number = token.line_number; if (extract_all) - remember_a_message (mlp, token.string, inner_context, &pos, - savable_comment); + remember_a_message (mlp, NULL, token.string, inner_context, + &pos, savable_comment); else - { - if (commas_to_skip == 0) - { - if (plural_mp == NULL) - { - /* Seen an msgid. */ - message_ty *mp = - remember_a_message (mlp, token.string, - inner_context, &pos, - savable_comment); - if (plural_commas > 0) - plural_mp = mp; - } - else - { - /* Seen an msgid_plural. */ - remember_a_message_plural (plural_mp, token.string, - inner_context, &pos, - savable_comment); - plural_mp = NULL; - } - } - else - free (token.string); - } + arglist_parser_remember (argparser, arg, token.string, + inner_context, + pos.file_name, pos.line_number, + savable_comment); } next_context_iter = null_context_list_iterator; state = 0; @@ -1359,6 +1314,7 @@ extract_parenthesized (message_list_ty *mlp, continue; case token_type_eof: + arglist_parser_done (argparser); return true; default: @@ -1394,7 +1350,7 @@ extract_php (FILE *f, /* Eat tokens until eof is seen. When extract_parenthesized returns due to an unbalanced closing parenthesis, just restart it. */ while (!extract_parenthesized (mlp, null_context, null_context_list_iterator, - -1, 0)) + arglist_parser_alloc (mlp, NULL))) ; /* Close scanner. */ diff --git a/gettext-tools/src/x-python.c b/gettext-tools/src/x-python.c index c0f00c2e3..e56ac1aea 100644 --- a/gettext-tools/src/x-python.c +++ b/gettext-tools/src/x-python.c @@ -84,25 +84,19 @@ x_python_keyword (const char *name) else { const char *end; - int argnum1; - int argnum2; + struct callshape shape; const char *colon; if (keywords.table == NULL) hash_init (&keywords, 100); - split_keywordspec (name, &end, &argnum1, &argnum2); + split_keywordspec (name, &end, &shape); /* The characters between name and end should form a valid C identifier. A colon means an invalid parse in split_keywordspec(). */ colon = strchr (name, ':'); if (colon == NULL || colon >= end) - { - if (argnum1 == 0) - argnum1 = 1; - hash_insert_entry (&keywords, name, end - name, - (void *) (long) (argnum1 + (argnum2 << 10))); - } + insert_keyword_callshape (&keywords, name, end - name, &shape); } } @@ -1648,25 +1642,19 @@ static flag_context_list_table_ty *flag_context_list_table; /* Extract messages until the next balanced closing parenthesis. Extracted messages are added to MLP. - When a specific argument shall be extracted, COMMAS_TO_SKIP >= 0 and, - if also a plural argument shall be extracted, PLURAL_COMMAS > 0, - otherwise PLURAL_COMMAS = 0. - When no specific argument shall be extracted, COMMAS_TO_SKIP < 0. Return true upon eof, false upon closing parenthesis. */ static bool extract_parenthesized (message_list_ty *mlp, flag_context_ty outer_context, flag_context_list_iterator_ty context_iter, - int commas_to_skip, int plural_commas) + struct arglist_parser *argparser) { - /* Remember the message containing the msgid, for msgid_plural. */ - message_ty *plural_mp = NULL; - + /* Current argument number. */ + int arg = 1; /* 0 when no keyword has been seen. 1 right after a keyword is seen. */ int state; /* Parameters of the keyword just seen. Defined only in state 1. */ - int next_commas_to_skip = -1; - int next_plural_commas = 0; + const struct callshapes *next_shapes = NULL; /* Context iterator that will be used if the next token is a '('. */ flag_context_list_iterator_ty next_context_iter = passthrough_context_list_iterator; @@ -1693,11 +1681,7 @@ extract_parenthesized (message_list_ty *mlp, &keyword_value) == 0) { - int argnum1 = (int) (long) keyword_value & ((1 << 10) - 1); - int argnum2 = (int) (long) keyword_value >> 10; - - next_commas_to_skip = argnum1 - 1; - next_plural_commas = (argnum2 > argnum1 ? argnum2 - argnum1 : 0); + next_shapes = (const struct callshapes *) keyword_value; state = 1; } else @@ -1713,30 +1697,26 @@ extract_parenthesized (message_list_ty *mlp, case token_type_lparen: if (extract_parenthesized (mlp, inner_context, next_context_iter, - state ? next_commas_to_skip : -1, - state ? next_plural_commas : 0)) - return true; + arglist_parser_alloc (mlp, + state ? next_shapes : NULL))) + { + xgettext_current_source_encoding = po_charset_utf8; + arglist_parser_done (argparser); + xgettext_current_source_encoding = xgettext_current_file_source_encoding; + return true; + } next_context_iter = null_context_list_iterator; state = 0; continue; case token_type_rparen: + xgettext_current_source_encoding = po_charset_utf8; + arglist_parser_done (argparser); + xgettext_current_source_encoding = xgettext_current_file_source_encoding; return false; case token_type_comma: - if (commas_to_skip >= 0) - { - if (commas_to_skip > 0) - commas_to_skip--; - else - if (plural_mp != NULL && plural_commas > 0) - { - commas_to_skip = plural_commas - 1; - plural_commas = 0; - } - else - commas_to_skip = -1; - } + arg++; inner_context = inherited_context (outer_context, flag_context_list_iterator_advance ( @@ -1751,43 +1731,16 @@ extract_parenthesized (message_list_ty *mlp, pos.file_name = logical_file_name; pos.line_number = token.line_number; + xgettext_current_source_encoding = po_charset_utf8; if (extract_all) - { - xgettext_current_source_encoding = po_charset_utf8; - remember_a_message (mlp, token.string, inner_context, &pos, - token.comment); - xgettext_current_source_encoding = xgettext_current_file_source_encoding; - } + remember_a_message (mlp, NULL, token.string, inner_context, + &pos, token.comment); else - { - if (commas_to_skip == 0) - { - if (plural_mp == NULL) - { - /* Seen an msgid. */ - message_ty *mp; - - xgettext_current_source_encoding = po_charset_utf8; - mp = remember_a_message (mlp, token.string, - inner_context, &pos, - token.comment); - xgettext_current_source_encoding = xgettext_current_file_source_encoding; - if (plural_commas > 0) - plural_mp = mp; - } - else - { - /* Seen an msgid_plural. */ - xgettext_current_source_encoding = po_charset_utf8; - remember_a_message_plural (plural_mp, token.string, - inner_context, &pos, NULL); - xgettext_current_source_encoding = xgettext_current_file_source_encoding; - plural_mp = NULL; - } - } - else - free (token.string); - } + arglist_parser_remember (argparser, arg, token.string, + inner_context, + pos.file_name, pos.line_number, + token.comment); + xgettext_current_source_encoding = xgettext_current_file_source_encoding; } drop_reference (token.comment); next_context_iter = null_context_list_iterator; @@ -1795,6 +1748,9 @@ extract_parenthesized (message_list_ty *mlp, continue; case token_type_eof: + xgettext_current_source_encoding = po_charset_utf8; + arglist_parser_done (argparser); + xgettext_current_source_encoding = xgettext_current_file_source_encoding; return true; case token_type_other: @@ -1846,7 +1802,7 @@ extract_python (FILE *f, /* Eat tokens until eof is seen. When extract_parenthesized returns due to an unbalanced closing parenthesis, just restart it. */ while (!extract_parenthesized (mlp, null_context, null_context_list_iterator, - -1, 0)) + arglist_parser_alloc (mlp, NULL))) ; fp = NULL; diff --git a/gettext-tools/src/x-rst.c b/gettext-tools/src/x-rst.c index 345e1d5dc..be3aed1a9 100644 --- a/gettext-tools/src/x-rst.c +++ b/gettext-tools/src/x-rst.c @@ -219,7 +219,7 @@ extract_rst (FILE *f, pos.file_name = location; pos.line_number = (size_t)(-1); - remember_a_message (mlp, msgid, null_context, &pos, NULL); + remember_a_message (mlp, NULL, msgid, null_context, &pos, NULL); /* Here c is the last read character: EOF or '\n'. */ if (c == EOF) diff --git a/gettext-tools/src/x-scheme.c b/gettext-tools/src/x-scheme.c index 3c481f006..d831dfe3f 100644 --- a/gettext-tools/src/x-scheme.c +++ b/gettext-tools/src/x-scheme.c @@ -94,14 +94,13 @@ x_scheme_keyword (const char *name) else { const char *end; - int argnum1; - int argnum2; + struct callshape shape; const char *colon; if (keywords.table == NULL) hash_init (&keywords, 100); - split_keywordspec (name, &end, &argnum1, &argnum2); + split_keywordspec (name, &end, &shape); /* The characters between name and end should form a valid Lisp symbol. Extract the symbol name part. */ @@ -116,10 +115,7 @@ x_scheme_keyword (const char *name) return; } - if (argnum1 == 0) - argnum1 = 1; - hash_insert_entry (&keywords, name, end - name, - (void *) (long) (argnum1 + (argnum2 << 10))); + insert_keyword_callshape (&keywords, name, end - name, &shape); } } @@ -730,9 +726,8 @@ read_object (struct object *op, flag_context_ty outer_context) { int arg = 0; /* Current argument number. */ flag_context_list_iterator_ty context_iter; - int argnum1 = 0; /* First string position. */ - int argnum2 = 0; /* Plural string position. */ - message_ty *plural_mp = NULL; /* Remember the msgid. */ + const struct callshapes *shapes = NULL; + struct arglist_parser *argparser = NULL; for (;; arg++) { @@ -754,6 +749,8 @@ read_object (struct object *op, flag_context_ty outer_context) { op->type = t_other; last_non_comment_line = line_number; + if (argparser != NULL) + arglist_parser_done (argparser); return; } @@ -777,10 +774,9 @@ read_object (struct object *op, flag_context_ty outer_context) symbol_name, strlen (symbol_name), &keyword_value) == 0) - { - argnum1 = (int) (long) keyword_value & ((1 << 10) - 1); - argnum2 = (int) (long) keyword_value >> 10; - } + shapes = (const struct callshapes *) keyword_value; + + argparser = arglist_parser_alloc (mlp, shapes); context_iter = flag_context_list_iterator ( @@ -795,42 +791,20 @@ read_object (struct object *op, flag_context_ty outer_context) } else { - /* These are the argument positions. - Extract a string if we have reached the right - argument position. */ - if (arg == argnum1) - { - if (inner.type == t_string) - { - lex_pos_ty pos; - message_ty *mp; - - pos.file_name = logical_file_name; - pos.line_number = inner.line_number_at_start; - mp = remember_a_message (mlp, string_of_object (&inner), - inner_context, &pos, - savable_comment); - if (argnum2 > 0) - plural_mp = mp; - } - } - else if (arg == argnum2) - { - if (inner.type == t_string && plural_mp != NULL) - { - lex_pos_ty pos; - - pos.file_name = logical_file_name; - pos.line_number = inner.line_number_at_start; - remember_a_message_plural (plural_mp, string_of_object (&inner), - inner_context, &pos, - savable_comment); - } - } + /* These are the argument positions. */ + if (inner.type == t_string) + arglist_parser_remember (argparser, arg, + string_of_object (&inner), + inner_context, + logical_file_name, + inner.line_number_at_start, + savable_comment); } free_object (&inner); } + if (argparser != NULL) + arglist_parser_done (argparser); } op->type = t_other; last_non_comment_line = line_number; @@ -1156,7 +1130,7 @@ read_object (struct object *op, flag_context_ty outer_context) pos.file_name = logical_file_name; pos.line_number = op->line_number_at_start; - remember_a_message (mlp, string_of_object (op), + remember_a_message (mlp, NULL, string_of_object (op), null_context, &pos, savable_comment); } last_non_comment_line = line_number; diff --git a/gettext-tools/src/x-sh.c b/gettext-tools/src/x-sh.c index e9829cb2a..625dd8870 100644 --- a/gettext-tools/src/x-sh.c +++ b/gettext-tools/src/x-sh.c @@ -85,25 +85,19 @@ x_sh_keyword (const char *name) else { const char *end; - int argnum1; - int argnum2; + struct callshape shape; const char *colon; if (keywords.table == NULL) hash_init (&keywords, 100); - split_keywordspec (name, &end, &argnum1, &argnum2); + split_keywordspec (name, &end, &shape); /* The characters between name and end should form a valid C identifier. A colon means an invalid parse in split_keywordspec(). */ colon = strchr (name, ':'); if (colon == NULL || colon >= end) - { - if (argnum1 == 0) - argnum1 = 1; - hash_insert_entry (&keywords, name, end - name, - (void *) (long) (argnum1 + (argnum2 << 10))); - } + insert_keyword_callshape (&keywords, name, end - name, &shape); } } @@ -1024,7 +1018,7 @@ read_word (struct word *wp, int looking_for, flag_context_ty context) grow_token (&string); string.chars[string.charcount++] = (unsigned char) c; } - remember_a_message (mlp, string_of_token (&string), + remember_a_message (mlp, NULL, string_of_token (&string), context, &pos, savable_comment); free_token (&string); @@ -1132,9 +1126,8 @@ read_command (int looking_for, flag_context_ty outer_context) int arg = 0; /* Current argument number. */ bool arg_of_redirect = false; /* True right after a redirection operator. */ flag_context_list_iterator_ty context_iter; - int argnum1 = -1; /* First string position. */ - int argnum2 = -1; /* Plural string position. */ - message_ty *plural_mp = NULL; /* Remember the msgid. */ + const struct callshapes *shapes = NULL; + struct arglist_parser *argparser = NULL; for (;;) { @@ -1155,7 +1148,11 @@ read_command (int looking_for, flag_context_ty outer_context) if (inner.type == t_separator || inner.type == t_backquote || inner.type == t_paren || inner.type == t_eof) - return inner.type; + { + if (argparser != NULL) + arglist_parser_done (argparser); + return inner.type; + } if (extract_all) { @@ -1165,7 +1162,7 @@ read_command (int looking_for, flag_context_ty outer_context) pos.file_name = logical_file_name; pos.line_number = inner.line_number_at_start; - remember_a_message (mlp, string_of_word (&inner), + remember_a_message (mlp, NULL, string_of_word (&inner), inner_context, &pos, savable_comment); } } @@ -1182,7 +1179,7 @@ read_command (int looking_for, flag_context_ty outer_context) } else { - if (argnum1 < 0 && argnum2 < 0) + if (argparser == NULL) { /* This is the function position. */ arg = 0; @@ -1195,10 +1192,9 @@ read_command (int looking_for, flag_context_ty outer_context) function_name, strlen (function_name), &keyword_value) == 0) - { - argnum1 = (int) (long) keyword_value & ((1 << 10) - 1); - argnum2 = (int) (long) keyword_value >> 10; - } + shapes = (const struct callshapes *) keyword_value; + + argparser = arglist_parser_alloc (mlp, shapes); context_iter = flag_context_list_iterator ( @@ -1213,46 +1209,22 @@ read_command (int looking_for, flag_context_ty outer_context) } else { - /* These are the argument positions. - Extract a string if we have reached the right - argument position. */ - if (arg == argnum1) - { - if (inner.type == t_string) - { - lex_pos_ty pos; - message_ty *mp; - - pos.file_name = logical_file_name; - pos.line_number = inner.line_number_at_start; - mp = remember_a_message (mlp, string_of_word (&inner), - inner_context, &pos, - savable_comment); - if (argnum2 > 0) - plural_mp = mp; - } - } - else if (arg == argnum2) - { - if (inner.type == t_string && plural_mp != NULL) - { - lex_pos_ty pos; - - pos.file_name = logical_file_name; - pos.line_number = inner.line_number_at_start; - remember_a_message_plural (plural_mp, string_of_word (&inner), - inner_context, &pos, - savable_comment); - } - } - - if (arg >= argnum1 && arg >= argnum2) + /* These are the argument positions. */ + if (inner.type == t_string) + arglist_parser_remember (argparser, arg, + string_of_word (&inner), + inner_context, + logical_file_name, + inner.line_number_at_start, + savable_comment); + + if (arglist_parser_decidedp (argparser, arg)) { /* Stop looking for arguments of the last function_name. */ /* FIXME: What about context_iter? */ - argnum1 = -1; - argnum2 = -1; - plural_mp = NULL; + arglist_parser_done (argparser); + shapes = NULL; + argparser = NULL; } } diff --git a/gettext-tools/src/x-smalltalk.c b/gettext-tools/src/x-smalltalk.c index 9a8edc1a8..48050199b 100644 --- a/gettext-tools/src/x-smalltalk.c +++ b/gettext-tools/src/x-smalltalk.c @@ -541,8 +541,8 @@ extract_smalltalk (FILE *f, lex_pos_ty pos; pos.file_name = logical_file_name; pos.line_number = token.line_number; - remember_a_message (mlp, token.string, null_context, &pos, - savable_comment); + remember_a_message (mlp, NULL, token.string, null_context, + &pos, savable_comment); state = 0; break; } @@ -551,7 +551,7 @@ extract_smalltalk (FILE *f, lex_pos_ty pos; pos.file_name = logical_file_name; pos.line_number = token.line_number; - plural_mp = remember_a_message (mlp, token.string, + plural_mp = remember_a_message (mlp, NULL, token.string, null_context, &pos, savable_comment); state = 4; diff --git a/gettext-tools/src/x-tcl.c b/gettext-tools/src/x-tcl.c index fb6357e3d..58f90086e 100644 --- a/gettext-tools/src/x-tcl.c +++ b/gettext-tools/src/x-tcl.c @@ -87,23 +87,19 @@ x_tcl_keyword (const char *name) else { const char *end; - int argnum1; - int argnum2; + struct callshape shape; if (keywords.table == NULL) hash_init (&keywords, 100); - split_keywordspec (name, &end, &argnum1, &argnum2); + split_keywordspec (name, &end, &shape); /* The characters between name and end should form a valid Tcl function name. A leading "::" is redundant. */ if (end - name >= 2 && name[0] == ':' && name[1] == ':') name += 2; - if (argnum1 == 0) - argnum1 = 1; - hash_insert_entry (&keywords, name, end - name, - (void *) (long) (argnum1 + (argnum2 << 10))); + insert_keyword_callshape (&keywords, name, end - name, &shape); } } @@ -853,9 +849,8 @@ read_command (int looking_for, flag_context_ty outer_context) { int arg = 0; /* Current argument number. */ flag_context_list_iterator_ty context_iter; - int argnum1 = 0; /* First string position. */ - int argnum2 = 0; /* Plural string position. */ - message_ty *plural_mp = NULL; /* Remember the msgid. */ + const struct callshapes *shapes = NULL; + struct arglist_parser *argparser = NULL; for (;; arg++) { @@ -875,7 +870,11 @@ read_command (int looking_for, flag_context_ty outer_context) /* Recognize end of command. */ if (inner.type == t_separator || inner.type == t_bracket || inner.type == t_brace || inner.type == t_eof) - return inner.type; + { + if (argparser != NULL) + arglist_parser_done (argparser); + return inner.type; + } if (extract_all) { @@ -885,7 +884,7 @@ read_command (int looking_for, flag_context_ty outer_context) pos.file_name = logical_file_name; pos.line_number = inner.line_number_at_start; - remember_a_message (mlp, string_of_word (&inner), + remember_a_message (mlp, NULL, string_of_word (&inner), inner_context, &pos, savable_comment); } } @@ -908,10 +907,9 @@ read_command (int looking_for, flag_context_ty outer_context) stripped_name, strlen (stripped_name), &keyword_value) == 0) - { - argnum1 = (int) (long) keyword_value & ((1 << 10) - 1); - argnum2 = (int) (long) keyword_value >> 10; - } + shapes = (const struct callshapes *) keyword_value; + + argparser = arglist_parser_alloc (mlp, shapes); context_iter = flag_context_list_iterator ( @@ -926,38 +924,14 @@ read_command (int looking_for, flag_context_ty outer_context) } else { - /* These are the argument positions. - Extract a string if we have reached the right - argument position. */ - if (arg == argnum1) - { - if (inner.type == t_string) - { - lex_pos_ty pos; - message_ty *mp; - - pos.file_name = logical_file_name; - pos.line_number = inner.line_number_at_start; - mp = remember_a_message (mlp, string_of_word (&inner), - inner_context, &pos, - savable_comment); - if (argnum2 > 0) - plural_mp = mp; - } - } - else if (arg == argnum2) - { - if (inner.type == t_string && plural_mp != NULL) - { - lex_pos_ty pos; - - pos.file_name = logical_file_name; - pos.line_number = inner.line_number_at_start; - remember_a_message_plural (plural_mp, string_of_word (&inner), - inner_context, &pos, - savable_comment); - } - } + /* These are the argument positions. */ + if (argparser != NULL && inner.type == t_string) + arglist_parser_remember (argparser, arg, + string_of_word (&inner), + inner_context, + logical_file_name, + inner.line_number_at_start, + savable_comment); } free_word (&inner); diff --git a/gettext-tools/src/x-ycp.c b/gettext-tools/src/x-ycp.c index a91534469..d2e8769c0 100644 --- a/gettext-tools/src/x-ycp.c +++ b/gettext-tools/src/x-ycp.c @@ -616,7 +616,7 @@ extract_parenthesized (message_list_ty *mlp, if (plural_mp == NULL) { /* Seen an msgid. */ - plural_mp = remember_a_message (mlp, token.string, + plural_mp = remember_a_message (mlp, NULL, token.string, inner_context, &pos, savable_comment); state = 2; diff --git a/gettext-tools/src/xgettext.c b/gettext-tools/src/xgettext.c index d112a272b..bdfb7c9ab 100644 --- a/gettext-tools/src/xgettext.c +++ b/gettext-tools/src/xgettext.c @@ -955,63 +955,133 @@ read_exclusion_file (char *filename) void split_keywordspec (const char *spec, - const char **endp, int *argnum1p, int *argnum2p) + const char **endp, struct callshape *shapep) { const char *p; + int argnum1 = 0; + int argnum2 = 0; + int argnumc = 0; /* Start parsing from the end. */ p = spec + strlen (spec); - if (p > spec && isdigit ((unsigned char) p[-1])) + while (p > spec) { - const char *last_arg; - - do - p--; - while (p > spec && isdigit ((unsigned char) p[-1])); - - last_arg = p; - - if (p > spec && p[-1] == ',') + if (isdigit ((unsigned char) p[-1]) + || (p[-1] == 'c' && p - 1 > spec && isdigit ((unsigned char) p[-2]))) { - p--; - - if (p > spec && isdigit ((unsigned char) p[-1])) - { - const char *first_arg; + bool contextp = (p[-1] == 'c'); - do - p--; - while (p > spec && isdigit ((unsigned char) p[-1])); + do + p--; + while (p > spec && isdigit ((unsigned char) p[-1])); - first_arg = p; + if (p > spec && (p[-1] == ',' || p[-1] == ':')) + { + char *dummy; + int arg = strtol (p, &dummy, 10); - if (p > spec && p[-1] == ':') + if (contextp) { - /* Parsed "KEYWORD:ARGNUM1,ARGNUM2". */ - char *dummy; + if (argnumc != 0) + /* Only one context argument can be given. */ + break; + argnumc = arg; + } + else + { + if (argnum2 != 0) + /* At most two normal arguments can be given. */ + break; + argnum2 = argnum1; + argnum1 = arg; + } - *endp = p - 1; - *argnum1p = strtol (first_arg, &dummy, 10); - *argnum2p = strtol (last_arg, &dummy, 10); + p--; + if (*p == ':') + { + if (argnum1 == 0 && argnum2 == 0) + /* At least one non-context argument must be given. */ + break; + *endp = p; + shapep->argnum1 = (argnum1 > 0 ? argnum1 : 1); + shapep->argnum2 = argnum2; + shapep->argnumc = argnumc; return; } } + else + break; } - else if (p > spec && p[-1] == ':') - { - /* Parsed "KEYWORD:ARGNUM1. */ - char *dummy; + else + break; + } - *endp = p - 1; - *argnum1p = strtol (last_arg, &dummy, 10); - *argnum2p = 0; - return; + /* Couldn't parse the desired syntax. */ + *endp = spec + strlen (spec); + shapep->argnum1 = 1; + shapep->argnum2 = 0; + shapep->argnumc = 0; +} + + +void +insert_keyword_callshape (hash_table *table, + const char *keyword, size_t keyword_len, + const struct callshape *shape) +{ + void *old_value; + + if (hash_find_entry (table, keyword, keyword_len, &old_value)) + { + /* Create a one-element 'struct callshapes'. */ + struct callshapes *shapes = + (struct callshapes *) xmalloc (sizeof (struct callshapes)); + shapes->nshapes = 1; + shapes->shapes[0] = *shape; + keyword = + (const char *) hash_insert_entry (table, keyword, keyword_len, shapes); + if (keyword == NULL) + abort (); + shapes->keyword = keyword; + shapes->keyword_len = keyword_len; + } + else + { + /* Found a 'struct callshapes'. See whether it already contains the + desired shape. */ + struct callshapes *old_shapes = (struct callshapes *) old_value; + bool found; + size_t i; + + found = false; + for (i = 0; i < old_shapes->nshapes; i++) + if (old_shapes->shapes[i].argnum1 == shape->argnum1 + && old_shapes->shapes[i].argnum2 == shape->argnum2 + && old_shapes->shapes[i].argnumc == shape->argnumc) + { + found = true; + break; + } + + if (!found) + { + /* Replace the existing 'struct callshapes' with a new one. */ + struct callshapes *shapes = + (struct callshapes *) + xmalloc (sizeof (struct callshapes) + + old_shapes->nshapes * sizeof (struct callshape)); + + shapes->keyword = old_shapes->keyword; + shapes->keyword_len = old_shapes->keyword_len; + shapes->nshapes = old_shapes->nshapes + 1; + for (i = 0; i < old_shapes->nshapes; i++) + shapes->shapes[i] = old_shapes->shapes[i]; + shapes->shapes[i] = *shape; + if (hash_set_value (table, keyword, keyword_len, shapes)) + abort (); + free (old_shapes); } } - /* Parsed "KEYWORD". */ - *endp = p + strlen (p); - *argnum1p = 0; - *argnum2p = 0; } @@ -1760,21 +1830,16 @@ set_format_flags_from_context (enum is_format is_format[NFORMATS], message_ty * -remember_a_message (message_list_ty *mlp, char *string, +remember_a_message (message_list_ty *mlp, char *msgctxt, char *msgid, flag_context_ty context, lex_pos_ty *pos, refcounted_string_list_ty *comment) { enum is_format is_format[NFORMATS]; enum is_wrap do_wrap; - char *msgctxt; - char *msgid; message_ty *mp; char *msgstr; size_t i; - msgctxt = NULL; - msgid = string; - /* See whether we shall exclude this message. */ if (exclude != NULL && message_list_search (exclude, msgctxt, msgid) != NULL) { @@ -1783,6 +1848,10 @@ remember_a_message (message_list_ty *mlp, char *string, xgettext_comment_reset (); savable_comment_reset (); + if (msgctxt != NULL) + free (msgctxt); + free (msgid); + return NULL; } @@ -1792,9 +1861,11 @@ remember_a_message (message_list_ty *mlp, char *string, is_format[i] = undecided; do_wrap = undecided; + if (msgctxt != NULL) + CONVERT_STRING (msgctxt); CONVERT_STRING (msgid); - if (msgid[0] == '\0' && !xgettext_omit_header) + if (msgctxt == NULL && msgid[0] == '\0' && !xgettext_omit_header) { char buffer[21]; @@ -1816,6 +1887,8 @@ meta information, not the empty string.\n"))); mp = message_list_search (mlp, msgctxt, msgid); if (mp != NULL) { + if (msgctxt != NULL) + free (msgctxt); free (msgid); for (i = 0; i < NFORMATS; i++) is_format[i] = mp->is_format[i]; @@ -1839,9 +1912,9 @@ meta information, not the empty string.\n"))); msgstr = ""; /* Allocate a new message and append the message to the list. */ - mp = message_alloc (NULL, msgid, NULL, msgstr, strlen (msgstr) + 1, + mp = message_alloc (msgctxt, msgid, NULL, msgstr, strlen (msgstr) + 1, &dummypos); - /* Do not free msgid. */ + /* Do not free msgctxt and msgid. */ message_list_append (mlp, mp); } @@ -2087,6 +2160,339 @@ remember_a_message_plural (message_ty *mp, char *string, } +struct arglist_parser * +arglist_parser_alloc (message_list_ty *mlp, const struct callshapes *shapes) +{ + if (shapes == NULL || shapes->nshapes == 0) + { + struct arglist_parser *ap = + (struct arglist_parser *) + xmalloc (offsetof (struct arglist_parser, alternative[0])); + + ap->mlp = mlp; + ap->keyword = NULL; + ap->keyword_len = 0; + ap->nalternatives = 0; + + return ap; + } + else + { + struct arglist_parser *ap = + (struct arglist_parser *) + xmalloc (sizeof (struct arglist_parser) + + (shapes->nshapes - 1) * sizeof (struct partial_call)); + size_t i; + + ap->mlp = mlp; + ap->keyword = shapes->keyword; + ap->keyword_len = shapes->keyword_len; + ap->nalternatives = shapes->nshapes; + for (i = 0; i < shapes->nshapes; i++) + { + ap->alternative[i].argnumc = shapes->shapes[i].argnumc; + ap->alternative[i].argnum1 = shapes->shapes[i].argnum1; + ap->alternative[i].argnum2 = shapes->shapes[i].argnum2; + ap->alternative[i].msgctxt = NULL; + ap->alternative[i].msgctxt_pos.file_name = NULL; + ap->alternative[i].msgctxt_pos.line_number = (size_t)(-1); + ap->alternative[i].msgid = NULL; + ap->alternative[i].msgid_context = null_context; + ap->alternative[i].msgid_pos.file_name = NULL; + ap->alternative[i].msgid_pos.line_number = (size_t)(-1); + ap->alternative[i].msgid_comment = NULL; + ap->alternative[i].msgid_plural = NULL; + ap->alternative[i].msgid_plural_context = null_context; + ap->alternative[i].msgid_plural_pos.file_name = NULL; + ap->alternative[i].msgid_plural_pos.line_number = (size_t)(-1); + } + + return ap; + } +} + + +struct arglist_parser * +arglist_parser_clone (struct arglist_parser *ap) +{ + struct arglist_parser *copy = + (struct arglist_parser *) + xmalloc (sizeof (struct arglist_parser) - sizeof (struct partial_call) + + ap->nalternatives * sizeof (struct partial_call)); + size_t i; + + copy->mlp = ap->mlp; + copy->keyword = ap->keyword; + copy->keyword_len = ap->keyword_len; + copy->nalternatives = ap->nalternatives; + for (i = 0; i < ap->nalternatives; i++) + { + const struct partial_call *cp = &ap->alternative[i]; + struct partial_call *ccp = ©->alternative[i]; + + ccp->argnumc = cp->argnumc; + ccp->argnum1 = cp->argnum1; + ccp->argnum2 = cp->argnum2; + ccp->msgctxt = (cp->msgctxt != NULL ? xstrdup (cp->msgctxt) : NULL); + ccp->msgctxt_pos = cp->msgctxt_pos; + ccp->msgid = (cp->msgid != NULL ? xstrdup (cp->msgid) : NULL); + ccp->msgid_context = cp->msgid_context; + ccp->msgid_pos = cp->msgctxt_pos; + ccp->msgid_comment = add_reference (cp->msgid_comment); + ccp->msgid_plural = + (cp->msgid_plural != NULL ? xstrdup (cp->msgid_plural) : NULL); + ccp->msgid_plural_context = cp->msgid_plural_context; + ccp->msgid_plural_pos = cp->msgid_plural_pos; + } + + return copy; +} + + +void +arglist_parser_remember (struct arglist_parser *ap, + int argnum, char *string, + flag_context_ty context, + char *file_name, size_t line_number, + refcounted_string_list_ty *comment) +{ + bool stored_string = false; + size_t nalternatives = ap->nalternatives; + size_t i; + + if (!(argnum > 0)) + abort (); + for (i = 0; i < nalternatives; i++) + { + struct partial_call *cp = &ap->alternative[i]; + + if (argnum == cp->argnumc) + { + cp->msgctxt = string; + cp->msgctxt_pos.file_name = file_name; + cp->msgctxt_pos.line_number = line_number; + stored_string = true; + /* Mark msgctxt as done. */ + cp->argnumc = 0; + } + else if (argnum == cp->argnum1) + { + cp->msgid = string; + cp->msgid_context = context; + cp->msgid_pos.file_name = file_name; + cp->msgid_pos.line_number = line_number; + cp->msgid_comment = add_reference (comment); + stored_string = true; + /* Mark msgid as done. */ + cp->argnum1 = 0; + } + else if (argnum == cp->argnum2) + { + cp->msgid_plural = string; + cp->msgid_plural_context = context; + cp->msgid_plural_pos.file_name = file_name; + cp->msgid_plural_pos.line_number = line_number; + stored_string = true; + /* Mark msgid_plural as done. */ + cp->argnum2 = 0; + } + } + /* Note: There is a memory leak here: When string was stored but is later + not used by arglist_parser_done, we don't free it. */ + if (!stored_string) + free (string); +} + + +bool +arglist_parser_decidedp (struct arglist_parser *ap, int argnum) +{ + size_t i; + + /* Test whether all alternatives are decided. + Note: A decided alternative can be complete + cp->argnumc == 0 && cp->argnum1 == 0 && cp->argnum2 == 0 + or it can be failed if no literal strings were found at the specified + argument positions: + cp->argnumc <= argnum && cp->argnum1 <= argnum && cp->argnum2 <= argnum + */ + for (i = 0; i < ap->nalternatives; i++) + { + struct partial_call *cp = &ap->alternative[i]; + + if (!(cp->argnumc <= argnum + && cp->argnum1 <= argnum + && cp->argnum2 <= argnum)) + /* cp is still undecided. */ + return false; + } + return true; +} + + +void +arglist_parser_done (struct arglist_parser *ap) +{ + size_t ncomplete; + size_t i; + + /* Determine the number of complete calls. */ + ncomplete = 0; + for (i = 0; i < ap->nalternatives; i++) + { + struct partial_call *cp = &ap->alternative[i]; + + if (cp->argnumc == 0 && cp->argnum1 == 0 && cp->argnum2 == 0) + ncomplete++; + } + + if (ncomplete > 0) + { + struct partial_call *best_cp = NULL; + bool ambiguous = false; + + /* Find complete calls where msgctxt, msgid, msgid_plural are all + provided. */ + for (i = 0; i < ap->nalternatives; i++) + { + struct partial_call *cp = &ap->alternative[i]; + + if (cp->argnumc == 0 && cp->argnum1 == 0 && cp->argnum2 == 0 + && cp->msgctxt != NULL + && cp->msgid != NULL + && cp->msgid_plural != NULL) + { + if (best_cp != NULL) + { + ambiguous = true; + break; + } + best_cp = cp; + } + } + + if (best_cp == NULL) + { + struct partial_call *best_cp1 = NULL; + struct partial_call *best_cp2 = NULL; + + /* Find complete calls where msgctxt, msgid are provided. */ + for (i = 0; i < ap->nalternatives; i++) + { + struct partial_call *cp = &ap->alternative[i]; + + if (cp->argnumc == 0 && cp->argnum1 == 0 && cp->argnum2 == 0 + && cp->msgctxt != NULL + && cp->msgid != NULL) + { + if (best_cp1 != NULL) + { + ambiguous = true; + break; + } + best_cp1 = cp; + } + } + + /* Find complete calls where msgid, msgid_plural are provided. */ + for (i = 0; i < ap->nalternatives; i++) + { + struct partial_call *cp = &ap->alternative[i]; + + if (cp->argnumc == 0 && cp->argnum1 == 0 && cp->argnum2 == 0 + && cp->msgid != NULL + && cp->msgid_plural != NULL) + { + if (best_cp2 != NULL) + { + ambiguous = true; + break; + } + best_cp2 = cp; + } + } + + if (best_cp1 != NULL) + best_cp = best_cp1; + if (best_cp2 != NULL) + { + if (best_cp != NULL) + ambiguous = true; + else + best_cp = best_cp2; + } + } + + if (best_cp == NULL) + { + /* Find complete calls where msgid is provided. */ + for (i = 0; i < ap->nalternatives; i++) + { + struct partial_call *cp = &ap->alternative[i]; + + if (cp->argnumc == 0 && cp->argnum1 == 0 && cp->argnum2 == 0 + && cp->msgid != NULL) + { + if (best_cp != NULL) + { + ambiguous = true; + break; + } + best_cp = cp; + } + } + } + + if (ambiguous) + { + error_with_progname = false; + error (0, 0, + _("%s:%d: ambiguous argument specification for keyword '%.*s'"), + best_cp->msgid_pos.file_name, best_cp->msgid_pos.line_number, + ap->keyword_len, ap->keyword); + error_with_progname = true; + } + + if (best_cp != NULL) + { + /* best_cp indicates the best found complete call. + Now call remember_a_message. */ + message_ty *mp; + + mp = remember_a_message (ap->mlp, best_cp->msgctxt, best_cp->msgid, + best_cp->msgid_context, + &best_cp->msgid_pos, + best_cp->msgid_comment); + if (best_cp->msgid_plural != NULL) + remember_a_message_plural (mp, best_cp->msgid_plural, + best_cp->msgid_plural_context, + &best_cp->msgid_plural_pos, + NULL); + } + } + else + { + /* No complete call was parsed. */ + /* Note: There is a memory leak here: When there is more than one + alternative, the same string can be stored in multiple alternatives, + and it's not easy to free all strings reliably. */ + if (ap->nalternatives == 1) + { + if (ap->alternative[0].msgctxt != NULL) + free (ap->alternative[0].msgctxt); + if (ap->alternative[0].msgid != NULL) + free (ap->alternative[0].msgid); + if (ap->alternative[0].msgid_plural != NULL) + free (ap->alternative[0].msgid_plural); + } + } + + for (i = 0; i < ap->nalternatives; i++) + drop_reference (ap->alternative[i].msgid_comment); + free (ap); +} + + static message_ty * construct_header () { diff --git a/gettext-tools/src/xgettext.h b/gettext-tools/src/xgettext.h index c2fe3d1fb..619a785ad 100644 --- a/gettext-tools/src/xgettext.h +++ b/gettext-tools/src/xgettext.h @@ -1,5 +1,5 @@ /* xgettext common functions. - Copyright (C) 2001-2003 Free Software Foundation, Inc. + Copyright (C) 2001-2003, 2005 Free Software Foundation, Inc. Written by Peter Miller and Bruno Haible , 2001. @@ -20,6 +20,7 @@ #ifndef _XGETTEXT_H #define _XGETTEXT_H +#include #include #include @@ -47,9 +48,32 @@ extern int xgettext_omit_header; extern bool substring_match; -/* Split keyword spec into keyword, argnum1, argnum2. */ +/* Calling convention for a given keyword. */ +struct callshape +{ + int argnum1; /* argument number to use for msgid */ + int argnum2; /* argument number to use for msgid_plural */ + int argnumc; /* argument number to use for msgctxt */ +}; + +/* Split keyword spec into keyword, argnum1, argnum2, argnumc. */ extern void split_keywordspec (const char *spec, const char **endp, - int *argnum1p, int *argnum2p); + struct callshape *shapep); + +/* Set of alternative calling conventions for a given keyword. */ +struct callshapes +{ + const char *keyword; /* the keyword, not NUL terminated */ + size_t keyword_len; /* the keyword's length */ + size_t nshapes; + struct callshape shapes[1]; /* actually nshapes elements */ +}; + +/* Insert a (keyword, callshape) pair into a hash table mapping keyword to + 'struct callshapes *'. */ +extern void insert_keyword_callshape (hash_table *table, + const char *keyword, size_t keyword_len, + const struct callshape *shape); /* Context representing some flags. */ @@ -192,19 +216,22 @@ extern void savable_comment_reset (void); /* Add a message to the list of extracted messages. - STRING must be malloc()ed string; its ownership is passed to the callee. + msgctxt must be either NULL or a malloc()ed string; its ownership is passed + to the callee. + MSGID must be a malloc()ed string; its ownership is passed to the callee. POS->file_name must be allocated with indefinite extent. COMMENT may be savable_comment, or it may be a saved copy of savable_comment (then add_reference must be used when saving it, and drop_reference while dropping it). Clear savable_comment. */ extern message_ty *remember_a_message (message_list_ty *mlp, - char *string, + char *msgctxt, + char *msgid, flag_context_ty context, lex_pos_ty *pos, refcounted_string_list_ty *comment); /* Add an msgid_plural to a message previously returned by remember_a_message. - STRING must be malloc()ed string; its ownership is passed to the callee. + STRING must be a malloc()ed string; its ownership is passed to the callee. POS->file_name must be allocated with indefinite extent. COMMENT may be savable_comment, or it may be a saved copy of savable_comment (then add_reference must be used when saving it, and drop_reference while @@ -216,6 +243,59 @@ extern void remember_a_message_plural (message_ty *mp, refcounted_string_list_ty *comment); +/* Represents the progressive parsing of an argument list w.r.t. a single + 'struct callshape'. */ +struct partial_call +{ + int argnumc; /* number of context argument, 0 when seen */ + int argnum1; /* number of singular argument, 0 when seen */ + int argnum2; /* number of plural argument, 0 when seen */ + char *msgctxt; /* context - owned string, or NULL */ + lex_pos_ty msgctxt_pos; + char *msgid; /* msgid - owned string, or NULL */ + flag_context_ty msgid_context; + lex_pos_ty msgid_pos; + refcounted_string_list_ty *msgid_comment; + char *msgid_plural; /* msgid_plural - owned string, or NULL */ + flag_context_ty msgid_plural_context; + lex_pos_ty msgid_plural_pos; +}; + +/* Represents the progressive parsing of an argument list w.r.t. an entire + 'struct callshapes'. */ +struct arglist_parser +{ + message_list_ty *mlp; /* list where the message shall be added */ + const char *keyword; /* the keyword, not NUL terminated */ + size_t keyword_len; /* the keyword's length */ + size_t nalternatives; /* number of partial_call alternatives */ + struct partial_call alternative[1]; /* partial_call alternatives */ +}; + +/* Creates a fresh arglist_parser recognizing calls. + You can pass shapes = NULL for a parser not recognizing any calls. */ +extern struct arglist_parser * arglist_parser_alloc (message_list_ty *mlp, + const struct callshapes *shapes); +/* Clones an arglist_parser. */ +extern struct arglist_parser * arglist_parser_clone (struct arglist_parser *ap); +/* Adds a string argument to an arglist_parser. ARGNUM must be > 0. + STRING must be malloc()ed string; its ownership is passed to the callee. + FILE_NAME must be allocated with indefinite extent. + COMMENT may be savable_comment, or it may be a saved copy of savable_comment + (then add_reference must be used when saving it, and drop_reference while + dropping it). Clear savable_comment. */ +extern void arglist_parser_remember (struct arglist_parser *ap, + int argnum, char *string, + flag_context_ty context, + char *file_name, size_t line_number, + refcounted_string_list_ty *comment); +/* Tests whether an arglist_parser has is not waiting for more arguments after + argument ARGNUM. */ +extern bool arglist_parser_decidedp (struct arglist_parser *ap, int argnum); +/* Terminates the processing of an arglist_parser and deletes it. */ +extern void arglist_parser_done (struct arglist_parser *ap); + + #ifdef __cplusplus } #endif diff --git a/gettext-tools/tests/ChangeLog b/gettext-tools/tests/ChangeLog index 3de68d084..647470a1c 100644 --- a/gettext-tools/tests/ChangeLog +++ b/gettext-tools/tests/ChangeLog @@ -1,3 +1,11 @@ +2005-10-03 Bruno Haible + + Add support for contexts in xgettext. + * xgettext-c-10: New file. + * xgettext-sh-1: Pass additional --keyword option because xgettext's + behaviour has changed when too few arguments are given. + * Makefile.am (TESTS): Add xgettext-c-10. + 2005-10-01 Bruno Haible Support for context dependent translations in PO files. diff --git a/gettext-tools/tests/Makefile.am b/gettext-tools/tests/Makefile.am index 189455240..7bc98c5bf 100644 --- a/gettext-tools/tests/Makefile.am +++ b/gettext-tools/tests/Makefile.am @@ -62,7 +62,7 @@ TESTS = gettext-1 gettext-2 gettext-3 gettext-4 gettext-5 gettext-6 gettext-7 \ xgettext-7 xgettext-8 \ xgettext-awk-1 \ xgettext-c-1 xgettext-c-2 xgettext-c-3 xgettext-c-4 xgettext-c-5 \ - xgettext-c-6 xgettext-c-7 xgettext-c-8 xgettext-c-9 \ + xgettext-c-6 xgettext-c-7 xgettext-c-8 xgettext-c-9 xgettext-c-10 \ xgettext-csharp-1 xgettext-csharp-2 xgettext-csharp-3 \ xgettext-csharp-4 xgettext-csharp-5 \ xgettext-elisp-1 \ diff --git a/gettext-tools/tests/xgettext-sh-1 b/gettext-tools/tests/xgettext-sh-1 index f0ef25048..87f919966 100755 --- a/gettext-tools/tests/xgettext-sh-1 +++ b/gettext-tools/tests/xgettext-sh-1 @@ -466,7 +466,9 @@ EOF tmpfiles="$tmpfiles xg-sh-1.po" : ${XGETTEXT=xgettext} -${XGETTEXT} --omit-header --no-location xg-sh-1.sh -d xg-sh-1 +# Here we use ngettext with two signatures: ngettext:1 and ngettext:1,2. +${XGETTEXT} --omit-header --no-location --keyword=ngettext:1 \ + xg-sh-1.sh -d xg-sh-1 test $? = 0 || { rm -fr $tmpfiles; exit 1; } tmpfiles="$tmpfiles xg-sh-1.ok"