libc-package.bbclass: Split locale hard link processing into two parts

The locale-processing in cross-localedef was proven to be unsafe to
run in parallel due to the way it tried to make hard links to files
that could disappear before the link operation was completed.

To avoid corruption of the pseudo database, and create a
deterministically generated link tree, the operation of the locale
generation and the hard link resolution have been split into two
separate parts.  A side effect of this change is that the do_package()
rule for glibc-locale will be slightly smaller because some of the
hard links were missed in the past, particularly if you had a lot of
cpus free to perform the work.

Before the patch:
% du -sk locale-tree
312524  locale-tree

With the patch:
% du -sk locale-tree
290772 locale-tree

A number of comparisons were performed such as diffing the output,
cross checking the link references using tar's listing output, and
comparing against the results with out a parallel build to ensure the
locale-tree is correct in its final form.

[YOCTO #11299]
[YOCTO #12434]

(From OE-Core rev: 93d935927dd0ec0fa6f0a80a71236a7b00a5d1a4)

Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

diff --git a/meta/classes/libc-package.bbclass b/meta/classes/libc-package.bbclass
index 8859dad56656e9302fdadc1c53e9dac7efc15842..a66e540884721244cd2fea34b8dffadfe5b3aa48 100644 (file)
--- a/meta/classes/libc-package.bbclass
+++ b/meta/classes/libc-package.bbclass
@@ -82,6 +82,9 @@ do_collect_bins_from_locale_tree() {
        parent=$(dirname ${localedir})
        mkdir -p ${PKGD}/$parent
        tar -cf - -C $treedir/$parent -p $(basename ${localedir}) | tar -xf - -C ${PKGD}$parent
+
+       # Finalize tree by chaning all duplicate files into hard links
+       cross-localedef-hardlink -c -v ${WORKDIR}/locale-tree
 }
 
 inherit qemu
@@ -265,7 +268,7 @@ python package_do_split_gconvs () {
                 bb.error("locale_arch_options not found for target_arch=" + target_arch)
                 bb.fatal("unknown arch:" + target_arch + " for locale_arch_options")
 
-            localedef_opts += " --force  --no-archive --prefix=%s \
+            localedef_opts += " --force --no-hard-links --no-archive --prefix=%s \
                 --inputfile=%s/%s/i18n/locales/%s --charmap=%s %s/%s" \
                 % (treedir, treedir, datadir, locale, encoding, outputpath, name)
 
@@ -273,7 +276,7 @@ python package_do_split_gconvs () {
                 (path, i18npath, gconvpath, localedef_opts)
         else: # earlier slower qemu way 
             qemu = qemu_target_binary(d) 
-            localedef_opts = "--force --no-archive --prefix=%s \
+            localedef_opts = "--force --no-hard-links --no-archive --prefix=%s \
                 --inputfile=%s/i18n/locales/%s --charmap=%s %s" \
                 % (treedir, datadir, locale, encoding, name)