From: DJ Delorie Date: Wed, 6 Jul 2016 21:38:58 +0000 (-0400) Subject: Add README for testing copr repo of dj/malloc branch X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=e4cff5fd83963c426e3015d3595dfc69a63f1aca;p=thirdparty%2Fglibc.git Add README for testing copr repo of dj/malloc branch Includes some details on tracing and simulating too. --- diff --git a/malloc/README.copr.html b/malloc/README.copr.html new file mode 100644 index 00000000000..00a0e59b04d --- /dev/null +++ b/malloc/README.copr.html @@ -0,0 +1,193 @@ +Using the dj/malloc GLIBC COPR repo +

Using the dj/malloc GLIBC COPR repo

+ +

The purpose of this document is to assist folks in testing out my + custom dj/malloc branch of the upstream GLIBC git repo. This COPR + repo has pre-built RPMs for easy installation in a test + environment.

+ +See https://copr.fedorainfracloud.org/coprs/djdelorie/glibc_dj_malloc/ +for links and other information. + +

Installing the COPR Repo

+ +
+$ cd /etc/yum.repos.d/
+
+ +

RHEL7

+ + +
+$ wget https://copr.fedorainfracloud.org/coprs/djdelorie/glibc_dj_malloc/repo/epel-7/djdelorie-glibc_dj_malloc-epel-7.repo
+
+$ yum update
+
+$ init 6
+
+ +

Fedora

+ +
+$ dnf copr enable djdelorie/glibc_dj_malloc
+
+$ dnf clean all  (optional)
+$ dnf update
+
+$ init 6
+
+ +

Missing Dependencies

+ +If dnf complains about missing dependencies, see if you have +non-x86_64 variants of glibc installed, and remove them: + +
+$ rpm -qa | grep ^glibc | grep -v x86_64
+
+ +

Confirming Installation

+ +
+$ rpm -qa | grep glibc
+glibc-all-langpacks-2.23.90-alphadj9.fc23.x86_64
+glibc-2.23.90-alphadj9.fc23.x86_64
+glibc-common-2.23.90-alphadj9.fc23.x86_64
+
+ +

Capturing to the Trace Buffer

+ +

One key new feature in this malloc is a high-speed trace buffer + that records every malloc, free, etc call with a minimum of added + latency. This is an improvement over the existing trace feature for + applications that are performance-critical. There is a private + (i.e. glibc-internal) API for activating this buffer, which is + enabled via a provided DSO:

+ +
+$ LD_PRELOAD=/lib64/libmtracectl.so ls
+
+ +

Replace lib64 with lib, or whatever suitable path you've installed +into, for 32-bit machines or machines with non-standard layouts, which +I don't support, but you never know...

+ +
+$ ls -l /tmp/mtrace-*
+-rw-r--r--. 1 root root 12422 Jun  2 20:53 mtrace-1188.out
+
+ +

Each generated file is a plain ASCII text file, with some headers + followed by one line per trace record entry. The syntax is not + "official" but intended to be machine-readable, and some scripts are + included in the COPR repo to process the generated files.

+ +
+$ head -1 /tmp/mtrace-1188.out
+158 out of 1000 events captured
+
+ +

If first number is more than the second number, then the trace only +includes the last however-many records. You can specify a +larger buffer via envirionment variables, like this:

+ +
+$ MTRACE_CTL_COUNT=100000 LD_PRELOAD=/lib64/libmtracectl.so ls
+
+ +(again, or /lib/ for 32-bit machines) + +

Sending Us Trace Files

+ +

If we ask you to send us a trace file, please rename and compress +it to make the file easier to transfer and keep track of.

+ + +
+$ cd /tmp
+$ gzip -9 mtrace-1188.out
+$ mv mtrace-1188.out.gz f24-ls-fred.mtrace.gz (or whatever name fits :)
+
+ +

Then mail f24-ls-fred.mtrace.gz to dj@redhat.com (or +whoever is asking for it, of course)

+ +

Workload Simulator

+ +

This build also includes a set of tools to "play back" a recorded +trace, which can be helpful in diagnosing memory-related performance +issues. Such workloads might be locally generated as part of a +benchmark suite, for example.

+ +
+trace2dat outfile [infile ...]
+
+ +If an infile is not provided, input is read from stdin. + +
+$ trace2dat /tmp/ls.wl /tmp/mtrace-22172.out
+
+ +The resulting file is a "workload" - a data file that tells the +simulator how to play back all the malloc/free/etc calls. This file +is not human-readable, but a compact binary datafile intended to be +used only by the simulator. + +
+trace_run workload.wl
+
+ +

Note: trace_run only works on intel processors with the RDTSCP + opcode, which is only available on reasonably modern processors. To + see if your processor supports this opcode, look for + the rdtscp cpu flag: + +

+$ grep rdtscp /proc/cpuinfo
+
+ +If you get lines like "flags : " then you have support +and trace_run will work. If the grep returns nothing, you don't. + +
+$ trace_run /tmp/ls.wl
+488,004 cycles
+106 usec wall time
+0 usec across 1 thread
+0 Kb Max RSS (1,228 -> 1,228)
+
+Avg malloc time:    385 in        154 calls
+Avg calloc time:      0 in          1 calls
+Avg realloc time:     0 in          1 calls
+Avg free time:      194 in         14 calls
+Total call time: 62,033 cycles
+
+ +Note: +see Practical +Micro-Benchmarking with ltrace and sched to get more stable +numbers. + +

Uninstalling

+ +To uninstall the custom build and revert to an official release, you +"simly" disable the COPR repo and downgrade to the latest "released" version: + +
+$ vi /etc/yum.repos.d/_copr_djdelorie-glibc_dj_malloc.repo
+
+ +change this line from 1 to 0: + +
+  enabled=0
+
+ +Then: + +
+$ dnf --allowerasing downgrade glibc
+
+ +(replace "dnf" with "yum" for RHEL 7)