From: Radoslaw Korzeniewski Date: Mon, 21 Mar 2022 16:45:10 +0000 (+0100) Subject: Add Kubernetes bacula-backup tool X-Git-Tag: Release-11.3.2~47 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=4af5582afdc282b762983067a68889c60a776079;p=thirdparty%2Fbacula.git Add Kubernetes bacula-backup tool --- diff --git a/bacula/ReleaseNotes b/bacula/ReleaseNotes index 0b33f0f70..074f7e68c 100644 --- a/bacula/ReleaseNotes +++ b/bacula/ReleaseNotes @@ -1,3 +1,545 @@ + Release Notes for Bacula 11.0 + +This is a major new release with many new features and a number of +changes. Please take care to test this code carefully before putting it into +production. Although the new features have been tested, they have not run in a +production environment. + +Compatibility: +-------------- +As always, both the Director and Storage daemon(s) must be upgraded at +the same time. Any File daemon running on the same machine as a Director +or Storage daemon must be of the same version. + +Older File Daemons should be compatible with the 11.0.0 Director and Storage +daemons. There should be no need to upgrade older File Daemons. + +New Catalog format in version 11.0.0 and greater +----------------------------------------------- + +This release of Bacula uses a new catalog format. We provide a set of scripts +that permit conversion from 9.x and earlier versions to the new 11.0.0 format +(1023). Normally the conversion/upgrade is automatic, though there is a big +change that takes longer than usual. The upgrade process will require about +twice the disk space of the actual database. + +This database upgrade should significantly increase performance when inserting +a large number of Jobs with a lot of Files into the database catalog. + +If you start from scratch, you don't need to run the update_bacula_tables +script because the create_bacula_tables script automatically creates the new +table format. However, if you are using a version of Bacula older than 5.0.0 +(e.g. 3.0.3) then you need to run the update_bacula_tables script that will be +found in the /src/cats directory after you run the ./configure command. + +As mentioned above, before running this script, please backup your catalog +database, be sure to shutdown Bacula and be aware that running the script can +take some time depending on your database size. + +---------------------------------------------------------------- +Release 11.0.6 10 March 2022 + +11.0.6 is an important bug fix and security fix release. +We advise all 11.0.x users to upgrade to this version. + + - Adjust sample-query.sql file for new catalog schema + - Fix #2654 About compilation issue on Alpine Linux + - Fix #2656 About segfault in XATTR code for FreeBSD + - Fix #7776 About FD error not correctly reported in the job log + - Fix #7998 About dir crashing for client intiated backup + - Fix #8126 About strange ouput for the .status client command + - Fix MySQL default connection in the grant_mysql_privileges script + - Fix db_get_accurate_jobids() with concurrent queries on the same jobs + - Fix issue with MySQL 8 in src/cats/grant_mysql_privileges + - Fix detection of PSK + - Fix org#2622 About incorrect behavior of the MaxDiffInterval directive + - Fix org#2623 About .ls/estimate command not printing files correctly + - Fix org#2627 About DIR crashing for copy jobs and resource rename + + - win32: Upgrade OpenSSL to 1.1.1m + + - Got regression testing working correctly on FreeBSD + + - Update depkgs version to use latest libs3 + + - baculum: Add API endpoints for basic user management + - baculum: Add JSON output parameter to show client(s), show job(s), show pool(s) API endpoints + - baculum: Add capability to assign dedicated bconsole config file to API basic users + - baculum: Add capability to close modal windows on clicking gray shadow + - baculum: Add capability to provide translated directive documentation file + - baculum: Add capability to use pre-defined paths in API config wizard - idea proposed by Heitor Faria + - baculum: Add console page to configure consoles + - baculum: Add copy resource function to enable duplicating resources + - baculum: Add director show API endpoint + - baculum: Add documentation for directives + - baculum: Add interface to manage basic users API from Web component side + - baculum: Add jump to previous/next error navigation in messages window + - baculum: Add new columns to job list page - idea proposed by Sergey Zhidkov + - baculum: Add option to enable/disable messages log window - idea proposed by Bill Arlofski + - baculum: Add password generator added to password fields + - baculum: Add time range filters to job history page - idea proposed by Heitor Faria + - baculum: Add to API deleting volume from the catalog endpoint + - baculum: Add to config API endpoint parameter to apply jobdefs in results + - baculum: Add to directive controls option to hide reset button and remove button + - baculum: Add warning to running job status if job needs media + - baculum: Apply PRADO framework patches to support PHP 8 + - baculum: Backup job wizard improvements + - baculum: Change buttons on dasboard page - reported by Sergey Zhidkov + - baculum: Do not require using some job resource values to ease using jobdefs - idea proposed by Heitor Faria + - baculum: Enlarge boxes with resource count in status director - reported by Sergey Zhidkov + - baculum: Fix #2642 add tool to re-assigning volumes from one pool to another + - baculum: Fix #2646 apply new user permissions immediately instead of after logging out and logging in + - baculum: Fix #2647 PHP warning about headers already sent on storage view page + - baculum: Fix #2653 create new resource by copying configuration from other resource + - baculum: Fix auto-scrolling in windows with configuration + - baculum: Fix component autochanger schemas in OpenAPI documentation + - baculum: Fix displaying directive sections in resouce configuration + - baculum: Fix displaying documentation for jobdefs directives + - baculum: Fix displaying issue in restore browser - reported by Sergey Zhidkov + - baculum: Fix error about expected port number when writing component main resource + - baculum: Fix legend in job status pie chart on job view page + - baculum: Fix loading dashboard page if job status is created but not yet running + - baculum: Fix missing texts in translation files - reported by Sergey Zhidkov + - baculum: Fix opening job details in job table on main dashboard page - reported by Sergey Zhidkov + - baculum: Fix problem with listing directories in restore wizard - reported by Tomasz Swiderski + - baculum: Fix remove storage resource if autochanger directive is set + - baculum: Fix required fields in jobdefs forms + - baculum: Fix running job number on some pages + - baculum: Fix table width on schedule list page + - baculum: Fix undefined index error if user did not use Bacula configuration function + - baculum: Improve checking director in status director API endpoint + - baculum: Improve sun icon for displaying job status weather - idea proposed by Heitor Faria + - baculum: Improve wizards view and responsivity + - baculum: Loading pages optimization + - baculum: Make job status pie chart clickable and direct to job history page with filtered results - idea proposed by Bill Arlofski + - baculum: Make job status pie chart smaller - idea proposed by Sergey Zhidkov + - baculum: Make table texts translatable - reported by Sergey Zhidkov + - baculum: Misc visual improvements + - baculum: Move all external dependencies to vendor directory + - baculum: Move resource monitor and error message box to separate modules + - baculum: New advanced schedule settings + - baculum: New copy job wizard + - baculum: New delete volumes bulk action on volume list page + - baculum: New director page with graphical/text status and with configure director resources + - baculum: New migrate job wizard + - baculum: Reduce free space between interface elements - idea proposed by Sergey Zhidkov + - baculum: Reduce size of icons in run job window and on dashboard page + - baculum: Remove old configure page + - baculum: Remove redundant statistics pages + - baculum: Reorganize dasboard page - idea proposed by Sergey Zhidkov + - baculum: Set responsive priority for job list table + - baculum: Unify buttons view + - baculum: Unset default API host setting if default API host is no longer assigned to user + - baculum: Update API documentation + - baculum: Update Polish translations + - baculum: Update Portuguese translations + - baculum: Update Russian translations + + - baculum: Visual improvements in interface + + - rpms: Disable tcp_wrapper for rhel8 in bacula.spec.in + - rpms: Do not build with tcp_wrapper on Fedora 31 + - rpms: Fix #2599 - bacula-postgresql conflicts with bacula-mysql + - rpms: Fix #2615 - Missing bacula-sd-cloud-s3-driver-.so + - rpms: Fix libs3 installation path + - rpms: Fix mysql devel package dependency for rhel/centos 7 + - rpms: Remove tcp_wrappers for cloud-storage rpm + +Bugs fixed/closed since last release: +2599 2615 2622 2623 2627 2642 2646 2647 2653 2654 2656 7776 7998 8126 + +Release 11.0.5 03 June 2021 + +11.0.5 is a minor bug fix release. + + - Fix compilation + - Fix org#2427 About incorrect handling of empty files with Accurate=yes on Windows + - Update MySQL update procedure for 5.6 + +Bugs fixed/closed since last release: +2427 + + +Release 11.0.4 28 May 2021 + +11.0.4 is a minor bug fix release. + + - baculum: Update script version + - Fix org#2618 Disable fix on bvfs_get_jobids() temporarily + - Improve MySQL upgrade procedure + +Bugs fixed/closed since last release: +2618 + +Release 11.0.3 21 May 2021 + +11.0.3 is a minor bug fix release. + + - Check if char **jobid parameter is NULL before modifying it in bvfs_parse_arg_version() + - Enhance the update_mysql_tables script + - Fix compilation of check_bacula.c reported by Dan + - Fix org#2442 About the check of the Control Device during startup + - Fix org#2500 .bvfs_get_jobids jobid=X must return X in the list + - Fix org#2604 About column alignment of 'Terminated Jobs' section + - Fix org#2605 About incorrect messages in restore command + - Fix stored/Makefile.in to install cloud driver object with cloud targets + - Fix various default permissions + + - baculum: Add autochanger management section and improve few other texts + - baculum: Add component action (start/stop/restart) buttons to client and + storage pages + - baculum: Add example working directory path in API install wizard + - baculum: Add new device interface definition to Baculum OpenAPI documentation + - baculum: Fix #2592 logout button on Safari web browser + - baculum: Fix double device error code number + - baculum: Fix opening update slots window reported by Hector Barrera + - baculum: Fix sub-tabs on client and on storage pages + - baculum: Implement autochanger management + - baculum: Implement support for assigning multiple API hosts to one user + - baculum: Restore wizard improvements + - baculum: Update Portuguese translations + - baculum: Update Russian translations + - baculum: Update documentation chapter and screenshots + - baculum: Use catalog access in changer listall endpoint only if it is configured on API host + + - docs: Add information about the git branch used with Bacula + - docs: Fix #7657 Enhance the FSType description + - docs: Fix #7659 About EnhancedWild fileset directive documentation + - docs: Fix org#2578 About missing "restore directory=xxx" keyword documentation + +Bugs fixed/closed since last release: +2442 2500 2578 2592 2604 2605 7657 7659 + + +Release 11.0.2 26 March 2021 + +11.0.2 is a minor bug fix release. + + - Add functions to unittests library + - Add support for store_alist_str() in plugin configuration items + - Enhance bdelete_and_free() macro + - Update baculabackupreport script + + - Fix #7286 DIR segfault when doing a "dir" command in a restore + - Fix #7321 About issue when stopping jobs waiting for resources + - Fix #7396 GRANT command error in granting privileges script for MySQL + - Fix #7449 About incorrect JSON output with 'TLS Allowed CN' directive + - Fix #7451 About deleted files incorrectly kept in Virtual Full + - Fix S3 compilation + - Fix Verify job issue with offset stream and compressed blocks + - Fix bug #2498 - Wrong mode for /etc/logrotate.d/bacula + - Fix check_bacula.c to ignore daemon events + - Fix possible memory corruption in the label process + - Fix reload issue when a Job doesn't have a Pool defined + - Possible fix for SD high memory usage problem + - Remove suspicious debug line on setdebug() + + - baculum: Add Craig Holyoak to AUTHORS + - baculum: Fix #2597 LDAP login with LDAPS option + - baculum: Fix cancel button in new job wizard + - baculum: Fix displaying warning messages in messages window + - baculum: Fix undefined property error in run job API endpoint if level value + is not provided + - baculum: Implement API version 2 + - baculum: Improve updating asset files after upgrade + - baculum: Unify jobs/{jobid}/files endpoint output for detailed and normal + modes + - baculum: Update Polish translations + - baculum: Update Portuguese translations + - baculum: Update Russian translations + - baculum: Use new APIv2 status client request on job history view page + + - rpms: Fix bacula.spec for Fedora 31 + - rpms: Fix bacula.spec for rhel8 / centos 8 + - rpms: Fix missing query.sql + - rpms: Update bacula.spec for rhel8 + + - win32: Fill the version information as CFLAGS + - win32: Fix #7373 binaries are tagged with correct resource + - win32: Fix error message when labeling volume on Windows SD + - win32: Update openssl version to 1.1.1k + + +Bugs fixed/closed since last release: +2498 2597 7286 7321 7373 7396 7449 7451 + + +Release 11.0.1 04 February 2021 + +11.0.1 is a minor bug fix release. + + - Add PGSQL detection for macOS and MacPorts. + - BEE Backport bacula/src/tools/dbcheck.c + - Fix #7079 About a segfault in a copyjob when the fileset is no longer defined + - Fix #7168 About incorrect start time displayed for canceled jobs not yet + running + - Fix #7207 About 'dbcheck -n' not working properly + - Fix #7214 Adapt mtx-changer.conf for GNU cpio mt version + - Fix #7247 About incorrect variable substitution with the query command + - Fix MySQL update procedure with incorrect handling of the FileIndex type + - Fix compilation warnings with Solaris Studio + - Fix copy/migration job selection + - Fix org#2579 About incorrect JSON generated from empty Messages resource + - Fix org#2587 Improve btraceback output + - Fix org#2588 About incorrect Object/ObjectId type in update_postgresql_tables + - Fix update_sqlite3_tables to upgrade from 9.6 to 11.0 + - Initialize StartTime in db_create_job_record() + - Remove deprecated sbrk in macOS and Windows. + - Use PKG_PROG_PKG_CONFIG macro to search for pkg-config. It is cross-compile + safe. + - alist: Fix for memory overflow access. + + - baculum: Add capability to create filedaemon console and schedule on new + resource page + - baculum: Add console messages log API endpoint + - baculum: Add console messages log envelope + - baculum: Add job status weather on job list page + - baculum: Add new icons for job status weather purpose + - baculum: Add to Bvfs lsdirs and lsfiles API endpoints pathid parameter + - baculum: Browser paths in restore browser using pathid + - baculum: Fix #2560 in restore wizard display names encoded in non-UTF encoding + - baculum: Fix finding jobs by filename in restore wizard if filename contains + whitespace characters + - baculum: Fix problem with setting hourly schedule - reported by Elias Pereira + - baculum: Remove excanvas.js dependency + - baculum: Update debian files to latest version + - baculum: Update spec files + + - win32: Backport signing procedures to 11.0 + - win32: Fix #7094 bypass random pwd generation when pwd is provided + - win32: Fix conditional #define's + - win32: Fix #7256 Update Windows version detection with latest versions + (Windows 10). + + +Bugs fixed/closed since last release: +2560 2579 2587 2588 7079 7094 7168 7207 7214 7247 7256 + +---------------------------------------------------------------- + +Release 11.0.0 12 December 2020 + +11.0.0 is a major release. + +New Features: +------------- + - New catalog format + - Automatic TLS PSK encrypted communication + - Support for Client behind NAT + - Continious Data Protection (CDP) Plugin + - Global Director Autoprune flag + - Events/Audit features + - New Baculum features + - Support for GPFS + +Misc: +----- + - New Prune Command Option + - Dynamic Client Address Directive + - Ability to disable Volume Retention + - Ask to mount/create volume when the disk space is low + - Simplification of the Windows FileSet with File=/ + - Use of QT5 for Bat on Windows + - Support for Windows files with non-UTF16 names + - Windows Snapshot management has been improved + - Support for the system.cifs_acl extended attribute backup with Linux CIFS + - Built-in Client Scheduler + - Reload command improvements (Keep IP address, Maximum Concurrent value, ...) + - Support for GLOB pattern in Console ACL directives + - Faster CRC32 algorithm + +Please see the New Features chapter of the manual for documentation on +the new features. + + +---------------------------------------------------------------- + + Release Notes for Bacula 9.6.7 + +Release 9.6.7 + +10Dec20 + +This is a minor bug fix that corrects among other things the MySQL/MariaDB +schemas. This is the last release of the 9.6.x series. + +03Dec20 + - Restore custom permission of symlink on FreeBSD and MacOS + - Fix #2582 bextract is broken for sparse gzip and compressed streams + - Fix org#2567 device capabilities overwritten + - baculum: Update script version + - Fix org#2573 About Syntax error in granting privileges script with MySQL if + --with-db-password parameter is used + - Fix org#2471 About deleted files are listed as being present in an accurate + backup by various sample queries + - Fix org#2571 About errors in es_AR.po file + - Fix org#2568 About compilation issue on gcc10 + - Fix org#2584 About inconsitancies in the make_mysql_tables script + - baculum: Fix English text noticed by Peter McDonald + - baculum: Update Russian translations + - baculum: Update Portuguese translations + - baculum: Implement graphical status storage + - baculum: Add gauge.js library + - baculum: Add json output option to show storages and show single storage + endpoints + - baculum: Add path field to restore wizard to narrow down search file results + to specific path - idea proposed by Wanderlei Huttel + - baculum: Add path parameter to job files API endpoint + - baculum: Fix #2565 status icon overlaps action buttons in API wizard + - baculum: Add Sergey Zhidkov to AUTHORS + - baculum: Add Russian translations + - baculum: Fix access to job resources if no job assigned to user + - baculum: Update Portuguese translations + - baculum: Ajax queue improvement in framework + - baculum: Fix renaming config resources + - baculum: Add graphs to job view page + - baculum: Fix using offset in job file list query with MySQL catalog database + - baculum: Show more detailed job file list - idea proposed by Wanderlei + Huttel + - baculum: Rework job file list API endpoint + - baculum: Add searching jobs by filename in restore wizard - idea proposed by + Wanderlei Huttel + - baculum: Add job files API endpoint + - Add code to retry a MySQL query after a deadlock + - baculum: Update Portuguese translations + - baculum: Add default days and limit values to status schedules API endpoint + - baculum: Fix displaying multitextbox and multicombobox directive controls + - baculum: Fix date filter on status schedule page + - baculum: Fix #2570 fileset excludes settings with PHP 7.4 + - baculum: New features and improvements to multi-user interface and restricted + access + + + +================================================================== + +Release 9.6.6 + +20Sep20 + +This is a minor bug fix release. Unless there is some new major bug found, +this will be the last of the 9.6.x releases. The next release major +release (a really big one) is currently scheduled for December. More +about this in a subsequent status report ... + + + +18Sep20 + - baculum: Fix displaying date and time on volume view page + - baculum: Fix #2564 changing volume status to Read-Only + - baculum: Fix saving multicombobox control values + - baculum: Fix multicombobox and multitextbox controls to work with PHP 7.4 + - baculum: Fix #2562 displaying date and time in tables on Safari + - Update copyright year + - Clarify SD vbackup device error message + - Fix MT2554 :update upload_cache call in dircmd.c + - zero malloced memory when SMARTALLOC not enabled + - baculum: Fix #2558 saving day or day range in schedule resource - reported by + Jose Alberto + - Fix MT2554 :upgrade upload_cache interface. + - Eliminate compiler errors when smartalloc is turned off + - baculum: Fix date formatter to work with PHP 7.4 + - baculum: Fix configure Bacula hosts page to work with PHP 7.4 + - baculum: Update Portuguese translations + - baculum: Change colours in table headers and borders + - baculum: Add remove runscript button + - Fix #6366 About an issue with verify job level=DiskToCatalog + - baculum: Close modal windows on pressing ESC key + - baculum: Fix logout button on authorization failed page + - baculum: Add local user authentication method support + - baculum: Add date and time formatters - idea proposed by Wanderlei Huttel + - baculum: Enable re-running jobs in async way and visual improvements + - baculum: Change cursor over selectable table + - Fix build script copyright detection + - release: add code to detect Bacula Systems copyrights and fail release + - Fix compilation of bsnapshot on Fedora + +Bugs fixed/closed since last release: +2558 6366 + + +=================================================================== + +Release 9.6.4 + +This is a major security and bug fix release. We suggest everyone +to upgrade as soon as possible. + +One significant improvement is for the AWS S3 cloud driver. First the +code base has been brought much closer to the Enterprise version (still +a long ways to go). Second is that the community code now uses the latest +version of libs3 as maintained by Bacula Systems. The libs3 code is +available as a tar file for Bacula version 9.6.4 at: + + www.bacula.org/downloads/libs3-20200523.tar.gz + +Note: Version 9.6.4 must be compiled with the above libs3 version or later. +To build libs3: +- Remove any libs3 package loaded by your OS +- Download above link +- tar xvfz libs3-20200523.tar.gz +- cd libs3-20200523 +- make # should have no errors +- sudo make install + +Then when you do your Bacula ./configure it should automatically +detect and use the libs3. The output from the ./configure will show +whether or not libs3 was found during the configuration. E.g. + S3 support: yes +in the output from ./configure. + +08Jun20 + - Add configure variables to baculabackupreport. Patch from bug #2538 + - Fix orphaned buffers in cloud by adding truncate argument to end_of_job() + - Improve clarity of Cloud part mismatches and make it an INFO message rather + than a WARNING since it corrects the catalog. + - Small trivial change to check_parts + - Backport more Enterprise cloud parts changes + - Backport cloud upload code from Enterprise + - Update s3_driver.c to new libs3 API calling sequence + - Fix tray-monitor installation + - Recompile ./configure + - Add ./configure code to check for and enable/disable S3 support + - win32: Fix org#2547 About possible NULL pointer dereference in get_memory_info + - Ensure cloud driver loaded when listing cloud volumes + - baculum: Request #2546 support for full restore when file records for backup + job are pruned + - baculum: Fix problem with authorization error after upgrade + - baculum: Add UPGRADE file + - baculum: Fix returning value in TStyleDiff - generated notice with PHP + 7.4 + - baculum: Remove execute bit for framework scripts + - baculum: Fix displaying empty column button in table column visibility menu - + reported by Wanderlei Huttel + - baculum: Update Polish translations + - baculum: Update Portuguese translations + - baculum: New user management. LDAP support. Role-based access control. + - Fix new compiler warnings + always use bstrncpy not strncpy to ensure EOS at + end of string + - Return smartalloc buffers zeroed -- future performance improvement + - Improve scanning data/time, fixes bug #2472 + - Make ABORT mention segfault to clarify non-bug #2528 + - Make reading a short block a warning rather than error + - baculum: Fix validators in run job modal window + - Remove unused -t option in dbcheck.c -- fixes bug #2453 + - Fix bug 2523 -- spurious extra linking + - Fix bug #2534 possible double free in error case + - Fix possible sscanf overflows + - Fix overflow from malicious FD reported by Pasi Saarinen + - baculum: Add option to show time in job log - idea proposed by Wanderlei + Huttel + - baculum: Add show log time parameter to job log endpoint + - baculum: Add tip about using table row selection + - Fix bug #2525 seg fault when doing estimate with accurate and MD5 + - baculum: Fix issues with SELinux support reported by Neil MacGregor + - Correct some copyrights + - Add Docker plugin rpm spec files + +Bugs fixed/closed since last release: +2453 2472 2525 2528 2534 2538 2546 2547 + +=================================================================== + Release Notes for Bacula 9.6.3 diff --git a/bacula/scripts/kubernetes-bacula-backup/Dockerfile.in b/bacula/scripts/kubernetes-bacula-backup/Dockerfile.in new file mode 100644 index 000000000..f44040356 --- /dev/null +++ b/bacula/scripts/kubernetes-bacula-backup/Dockerfile.in @@ -0,0 +1,45 @@ +# +# Bacula(R) - The Network Backup Solution +# +# Copyright (C) 2000-2022 Kern Sibbald +# +# The original author of Bacula is Kern Sibbald, with contributions +# from many others, a complete list can be found in the file AUTHORS. +# +# You may use this file and others of this release according to the +# license defined in the LICENSE file, which includes the Affero General +# Public License, v3.0 ("AGPLv3") and some additional permissions and +# terms pursuant to its AGPLv3 Section 7. +# +# This notice must be preserved when any source code is +# conveyed and/or propagated. +# +# Bacula(R) is a registered trademark of Kern Sibbald. +# +# Author: Radosław Korzeniewski, radekk@inteos.pl, Inteos Sp. z o.o. + +FROM python:3.8-slim AS build +ADD requirements.txt / +ADD baculatar.py / +RUN apt-get update && apt-get -y install binutils upx && pip3 install -r requirements.txt +RUN pyinstaller --onefile baculatar.py + +FROM python:3.8-slim +ARG VERSION=@KUBERNETES_IMAGE_VERSION@ +LABEL maintainer="Radosław Korzeniewski " +LABEL org.label-schema.schema-version="1.0" +LABEL org.label-schema.description="This is a Bacula container for incluster Kubernetes Plugin backup." +LABEL org.label-schema.vendor="Bacula" +LABEL org.label-schema.version=$VERSION +ENV PLUGINMODE 'backup' +ENV PLUGINHOST 'localhost' +ENV PLUGINPORT '9104' +ENV PLUGINTOKEN '' +ENV PLUGINJOB '' +ENV PLUGINCONNRETRIES '600' +ENV PLUGINCONNTIMEOUT '600' + +COPY --from=build /dist/baculatar / +COPY tar /tar + +CMD ["/baculatar"] diff --git a/bacula/scripts/kubernetes-bacula-backup/Makefile b/bacula/scripts/kubernetes-bacula-backup/Makefile new file mode 100644 index 000000000..e8b59d722 --- /dev/null +++ b/bacula/scripts/kubernetes-bacula-backup/Makefile @@ -0,0 +1,45 @@ +# +# Simple Makefile for building plugins for Bacula +# +# Copyright (C) 2000-2022 Kern Sibbald +# License: BSD 2-Clause; see file LICENSE-FOSS +# +# The script creates a docker image of the baculatar.py +# program. It uses also a static version of tar. +# +# It requires docker and pyinstaller (installed via pip3) +# The reference platform is bionic +# +D=$(shell date +%d%b%y) +all: bacula-backup-$(D).tar.gz + +bacula-backup-$(D).tar.gz: baculatar.py tar Dockerfile + docker build -t bacula-backup . + docker tag bacula-backup:latest bacula-backup:$(D) + docker save -o bacula-backup-$(D).tar bacula-backup:$(D) + gzip -9 bacula-backup-$(D).tar + +baculatar: baculatar.py + pyinstaller baculatar.py + ./copylibs dist/baculatar/libpython* + rm -rf dist build + pyinstaller -F baculatar.py + mv -f dist/baculatar . + ./copylibs baculatar + +tar: + rm -rf archbuild + git clone https://github.com/ebl/tar-static.git archbuild + (cd archbuild; ./build.sh) + cp archbuild/releases/tar . + rm -rf archbuild + +clean: + rm -rf baculatar.c bacula-backup-*.tar.gz __pycache__ baculatar.spec + rm -rf dist build baculatar libs + +distclean: clean + rm -f tar + +upload: bacula-backup-$(D).tar.gz + rsync -avz bacula-backup-$(D).tar.gz bacula.org:/home/src/depkgs/ diff --git a/bacula/scripts/kubernetes-bacula-backup/baculatar.py b/bacula/scripts/kubernetes-bacula-backup/baculatar.py new file mode 100644 index 000000000..5ddc8467a --- /dev/null +++ b/bacula/scripts/kubernetes-bacula-backup/baculatar.py @@ -0,0 +1,245 @@ +# -*- coding: UTF-8 -*- +# +# Bacula(R) - The Network Backup Solution +# +# Copyright (C) 2000-2020 Kern Sibbald +# +# The original author of Bacula is Kern Sibbald, with contributions +# from many others, a complete list can be found in the file AUTHORS. +# +# You may use this file and others of this release according to the +# license defined in the LICENSE file, which includes the Affero General +# Public License, v3.0 ("AGPLv3") and some additional permissions and +# terms pursuant to its AGPLv3 Section 7. +# +# This notice must be preserved when any source code is +# conveyed and/or propagated. +# +# Bacula(R) is a registered trademark of Kern Sibbald. +# +# Author: Radoslaw Korzeniewski, radekk@inteos.pl, Inteos Sp. z o.o. +# + +from __future__ import unicode_literals +import os +import sys +import ssl +import socket +import time +import logging +import subprocess + +COPYRIGHT = 'Copyright (C) 2000-2022 Kern Sibbald' +VERSION = '1.0-rpk' +TARCMD = '/tar' +TARPIPE = '/tmp/baculatar.fifo' +TARSTDOUT = '/tmp/baculatar.stdout' +TARSTDERR = '/tmp/baculatar.stderr' +CONNRETRIES = 600 +CONNTIMEOUT = 600 +DEFAULTPORT = 9104 +ERR_NO_DIR_FOUND = "No /{dir}/ directory found. Cannot execute backup!" + + +class BaculaConnection(object): + + def __init__(self, *args, **kwargs): + super(BaculaConnection, self).__init__(*args, **kwargs) + self.mode = os.getenv('PLUGINMODE', 'backup') + if self.mode not in ('backup', 'restore'): + self.mode = 'backup' + self.host = os.getenv('PLUGINHOST', 'localhost') + self.port = os.getenv('PLUGINPORT', DEFAULTPORT) + self.token = os.getenv('PLUGINTOKEN', '') + self.jobname = os.getenv('PLUGINJOB', 'undefined') + self.connretries = os.getenv('PLUGINCONNRETRIES', CONNRETRIES) + if isinstance(self.connretries, str): + try: + self.connretries = int(self.connretries) + except ValueError: + logging.error('$PLUGINCONNRETRIES ValueError: {retry} using default.'.format(retry=self.connretries)) + self.connretries = CONNRETRIES + self.conntimeout = os.getenv('PLUGINCONNTIMEOUT', CONNTIMEOUT) + if isinstance(self.conntimeout, str): + try: + self.conntimeout = int(self.conntimeout) + except ValueError: + logging.error('$PLUGINCONNTIMEOUT ValueError: {timeout} using default.'.format(timeout=self.conntimeout)) + self.connretries = CONNTIMEOUT + self.context = ssl.SSLContext(ssl.PROTOCOL_TLS) + self.tarproc = None + self.conn = None + self.tout = None + self.terr = None + if isinstance(self.port, str): + try: + self.port = int(self.port) + except ValueError: + logging.error('$PLUGINPORT ValueError: {port} using default.'.format(port=self.port)) + self.port = DEFAULTPORT + if isinstance(self.connretries, str): + try: + self.connretries = int(self.connretries) + except ValueError: + logging.error('$PLUGINCONNRETRIES ValueError: {retry} using default.'.format(port=self.connretries)) + self.port = CONNRETRIES + + logging.info('BaculaConnection: mode={mode} host={host} port={port} token={token}'.format( + mode=self.mode, host=self.host, port=self.port, token=self.token + )) + + def authenticate(self): + if self.conn is not None: + # first prepare the hello string + self.conn.send(bytes('{strlen:03}:Hello:{job}'.format(job=self.jobname, strlen=len(self.jobname) + 7), 'ascii')) + self.conn.send(bytes('Token: {token:64}\n'.format(token=self.token), 'ascii')) + stat = self.conn.recv(2) + if stat.decode() == 'OK': + logging.info('Authenticated.') + else: + logging.error('Authentication failure.') + self.disconnect() + sys.exit(4) + + def connect(self): + if self.conn is None: + sock = socket.socket(socket.AF_INET) + self.conn = self.context.wrap_socket(sock, server_hostname=self.host) + for attempt in range(self.connretries): + try: + logging.info('Try to connect to: {host}:{port} {att}/{retry}'.format(host=self.host, port=self.port, att=attempt, retry=self.connretries)) + self.conn.connect((self.host, self.port)) + except ssl.SSLError as e: + logging.debug(e) + logging.error('SSLError - cannot continue.') + sys.exit(2) + except OSError as e: + logging.debug(e) + time.sleep(1) + else: + logging.info('Connected.') + break + else: + logging.error('Cannot connect to {host}:{port} max retries exceed.'.format( + host=self.host, port=self.port + )) + sys.exit(1) + self.authenticate() + + def disconnect(self): + if self.conn is not None: + self.conn.close() + logging.info('Disconnected.') + self.conn = None + + def sendfile(self, filename): + with open(filename, 'rb') as fd: + while True: + data = fd.read(65536) + if len(data) > 0: + self.conn.send(data) + else: + break + + def receivefile(self, filename): + with open(filename, 'wb') as fd: + while True: + data = self.conn.recv(65536) + logging.info('recv:D' + str(len(data))) + if len(data) > 0: + fd.write(data) + else: + break + + def prepare_execute(self): + if not os.path.exists(TARPIPE): + os.mkfifo(TARPIPE) + self.tout = open(TARSTDOUT, 'w') + self.terr = open(TARSTDERR, 'w') + + def logging_from_file(self, filename): + with open(filename, 'r') as fd: + while True: + log = fd.readline().rstrip() + if not log: + break + logging.info(log) + + def final_execute(self): + # close current execution descriptors + self.tout.close() + self.terr.close() + self.disconnect() + # wait for tar to terminate + exitcode = -1 + try: + exitcode = self.tarproc.wait(timeout=self.conntimeout) + except subprocess.TimeoutExpired: + logging.error('Timeout waiting {} for tar to proceed. Terminate!'.format(self.conntimeout)) + self.tarproc.terminate() + # send execution logs + self.connect() + logging.info('tar exit status:{}'.format(exitcode)) + self.conn.send("{}\n".format(exitcode).encode()) + self.conn.send(b'---- stderr ----\n') + self.sendfile(TARSTDERR) + self.conn.send(b'---- list ----\n') + self.sendfile(TARSTDOUT) + self.conn.send(b'---- end ----\n') + self.disconnect() + logging.info('-- stderr --') + self.logging_from_file(TARSTDERR) + logging.info('-- list --') + self.logging_from_file(TARSTDOUT) + logging.info('Finish.') + + def err_no_dir_found(self): + logging.error(ERR_NO_DIR_FOUND.format(dir=self.mode)) + self.terr.write('404 ' + ERR_NO_DIR_FOUND.format(dir=self.mode) + '\n') + self.final_execute() + + def execute_backup(self): + self.prepare_execute() + if os.path.isdir('/backup'): + self.tarproc = subprocess.Popen([TARCMD, '-cvvf', TARPIPE, '-C', '/backup', '.'], + stderr=self.terr, + stdout=self.tout) + self.sendfile(TARPIPE) + self.final_execute() + else: + self.err_no_dir_found() + + def execute_restore(self): + self.prepare_execute() + if os.path.isdir('/restore'): + self.tarproc = subprocess.Popen([TARCMD, '-xvvf', TARPIPE, '-C', '/restore'], + stderr=self.terr, + stdout=self.tout) + self.receivefile(TARPIPE) + self.final_execute() + else: + self.err_no_dir_found() + + def execute(self): + method = getattr(self, 'execute_' + self.mode, None) + if method is None: + logging.error('Invalid mode={mode} of execution!'.format(mode=self.mode)) + self.disconnect() + sys.exit(3) + logging.info('BaculaJob: {}'.format(self.jobname)) + self.connect() + method() + + +def main(): + logging.basicConfig(format='%(asctime)s:%(levelname)s:%(message)s', level=logging.DEBUG) + logging.info('Bacula Kubernetes backup helper version {ver}. Copyright {cc}'.format( + ver=VERSION, + cc=COPYRIGHT + )) + bacula = BaculaConnection() + bacula.execute() + + +if __name__ == '__main__': + main() diff --git a/bacula/scripts/kubernetes-bacula-backup/copylibs b/bacula/scripts/kubernetes-bacula-backup/copylibs new file mode 100755 index 000000000..b3bd2e4d2 --- /dev/null +++ b/bacula/scripts/kubernetes-bacula-backup/copylibs @@ -0,0 +1,33 @@ +#!/bin/bash +# Copyright (C) 2000-2020 Kern Sibbald +# License: BSD 2-Clause; see file LICENSE-FOSS +# +# copy the libraries needed for a binary into the +# local directory + +mkdir -p $PWD/libs/usr/lib64 + +function copylib +{ + file=$1 + PWD=`pwd` + F=`dirname $file` + mkdir -p $PWD/libs/$F + cp -v $file $PWD/libs/$F +} + +ldd $1 | awk '/=>/ { print $3 }' | while read file +do + copylib $file +done + +# We need to copy some common libraries used by libc +file=`ldd $1 | awk '/ld-linux/ { print $1 }'` +copylib $file + +file=`ldconfig -p | grep -v lib32 | awk '/libresolv\.so\./ { gsub(/.+=> /, ""); print $1 }'` +copylib $file + +file=`ldconfig -p | grep -v lib32 | awk '/libnss_dns\.so\./ { gsub(/.+=> /, ""); print $1 }'` +copylib $file + diff --git a/bacula/scripts/kubernetes-bacula-backup/requirements.txt b/bacula/scripts/kubernetes-bacula-backup/requirements.txt new file mode 100644 index 000000000..cec9bf477 --- /dev/null +++ b/bacula/scripts/kubernetes-bacula-backup/requirements.txt @@ -0,0 +1,7 @@ +# -*- coding: UTF-8 -*- +# Copyright (C) 2000-2020 Kern Sibbald +# License: BSD 2-Clause; see file LICENSE-FOSS +# +# Author: Radoslaw Korzeniewski, radekk@inteos.pl, Inteos Sp. z o.o. +# +pyinstaller