E.Smith [Sat, 6 Oct 2018 15:18:17 +0000 (16:18 +0100)]
fanart: Comment out version field in module capabilities.
The version field in the scripts should be the same version as
Tvheadend. However it does not seem worthwhile at the moment to
sed a version number in to the scripts since most people will
install via a package manager.
E.Smith [Thu, 4 Oct 2018 17:07:33 +0000 (18:07 +0100)]
fanart: Tidy logging in Python script.
We want to avoid logging stack traces unless the user explicitly
enables debug, otherwise we get tracebacks whenever user does
not specify an apikey for a module they do not wish to use
(but have not disabled).
E.Smith [Wed, 3 Oct 2018 23:17:53 +0000 (00:17 +0100)]
fanart: Update tmdb to support tv lookups.
Previously we only support movie lookups on tmdb since the
module we used did not support tv lookups. Now we no longer
use an external module we can perform tv lookups on tmdb too.
E.Smith [Wed, 3 Oct 2018 22:39:58 +0000 (23:39 +0100)]
fanart: Rewrite to remove dependency on external tmdb module.
The information we need can easily be retrieved via a
query so avoid the dependency on an external module and
implement the tmdb api retrieval logic ourselves. This
also means we are no longer constrained to be python2.7 only.
E.Smith [Tue, 2 Oct 2018 18:00:42 +0000 (19:00 +0100)]
fanart: Add basic tvdb lookup.
The lookup is by title+year (+language) only (episode-specific
fanart is not yet retrieved).
To use with Tvheadend, the extra arguments in the grabber need to
include:
--tvdb-key XX
And an optional two character languages as csv:
--tvdb-languages en,it
E.Smith [Wed, 3 Oct 2018 23:54:45 +0000 (00:54 +0100)]
dvr: Remove fanart entries from the list until we process one.
We might have entries on our list of pending fanart fetches that
no longer need fetching (user changed config, etc.) So keep
removing entries from the list until we find ones that need
a fanart fetch.
E.Smith [Wed, 3 Oct 2018 09:13:24 +0000 (10:13 +0100)]
fanart: Add option to only lookup fanart for identifiable recordings.
OTA broadcasts can have data that will fail a fanart lookup or is
too vague so will match incorrect programmes. So add option so by
default we ignore fanart for programmes without a year (for movie)
or season/episode (for episode).
E.Smith [Tue, 2 Oct 2018 22:28:32 +0000 (23:28 +0100)]
ui: Use background-size to scale fanart to fit dvr dialog box.
Previously the fanart could be too big for the dialog. Now we
scale to fit the dialog.
We use 'cover' which can make the image slightly exceed the
dialog dimensions, compared to 'contain' which would leave
space at the top/bottom if the fanart is an odd dimension.
The 'cover' seems better with the fanart I've seen so far.
E.Smith [Tue, 2 Oct 2018 13:50:03 +0000 (14:50 +0100)]
xmltv: Add option to save epgdb after xmltv import.
The "periodic save database" means that for xmltv you can
import your daily listings, crash, restart, and not have xmltv
data since the periodic epgdb timer has not elapsed.
So, add an option so the user can save the database after the
import has completed, assuming changes occurred.
This save is delayed by a couple of minutes in case the user
is importing from several different xmltv guides, in which case
the save occurs after the last import.
We keep track of uuids for upcoming recordings. Once an hour we
get one entry from the list and pre-fetch artwork for it so it
can be displayed in the UI "upcoming recordings" dialog.
This is only done if the user has explicitly enabled fetch
artwork on the entry's profile.
In Configuration->Recording->Digital Video Recorder Profiles there
is now a "fetch artwork for new recordings" button, and a place
to enter additional arguments (such as --tmdb-key).
python: Support multiple grabber modules for movies and tv artwork.
The "--modules-movie=a,b,c" command line option will load each of
these modules in turn and ask them to provide artwork. TV modules are
selected via "--modules-tv" option.
The default (if no --modules-movies is provided) is to search the
Python path for python files that are named "tv_meta_*py" and then
call the module's "get_capabilities" to determine if the grabber
supports tv, movie, or both.
Each grabber module is called in turn until all artwork is
retrieved. So, if the first module only provides a fanart, then the
next module can supply the poster image.
Modules can be passed command line options from tvhmeta by prefixing
them with the shortened module name.
So, for a grabber module such as "tv_grab_tmdb", we pass through
command line arguments from tvhmeta to it. So if the tvhmeta is called
with an option of "--tmdb-key" then we pass through "key" as an option
to the tv_grab_tmdb (stripping the "--tmdb").
This allows third parties to produce grabber modules that
automatically integrate.
Also switched from deprecated OptionParser to argparse.
install: Add tvhmeta and tv_meta_tmdb.py to install files.
The tvhmeta program allows lookups of metadata for recordings.
The tv_meta_tmdb.py is the underlying file that does lookups
and is both an executable and a library.
python: Add basic tmdb lookup scripts to retrieve artwork.
The scripts attempt a tmdb lookup by using the title+year from
the dvr record associated with a particular uuid.
It then sets artwork and fanart.
The script can either be invoked manually or run automatically
from Tvheadend when a recording occurs (pre-recording rule).
In that case it needs to be passed the arguments:
"/usr/local/bin/tvhmeta --uuid %U --tmdb-key abcdef"
...where the key is from the tmdb website sign up.
The tv_meta_tmdb library is stand-alone and can be used
to test specific lookups to determine why they do not work.
A per-user cache is kept inside the tmdb3 library and this
is stored in /tmp.
The tmdb3 library has to be installed. This can be installed via "pip
install tmdb3" or "synth install www/py-tmdb3" depending on OS.
The scripts support a "--debug" option. I'd expect we are likely to
get several wrong/no results, especially with non-English movies until
we have a larger set of failure reasons to work with.
Once we get the tmdb working, we can try and "modularize" it so
different providers can be installed, and add a grabber for tv
episodes.
We now display fanart (if available) on the background of the dvr
dialog. This fanart image is also displayed every ten seconds where
the existing image is displayed.
We have to put the image inside a fixed width container, otherwise if
you alternate between a long thin image and a wide image then the text
reflows.
E.Smith [Mon, 1 Oct 2018 17:05:26 +0000 (18:05 +0100)]
FreeBSD: Add libunwind trap support for FreeBSD only.
Although the existing backtrace works correctly on Linux, on
FreeBSD it frequently generates a backtrace with completely
wrong function names. (FreeBSD 11.2, current latest version).
For example, making htsp_build_dvrentry crash with SEGV, it
would either not generate a stacktrace or would generate a
backtrace of:
-pthread_sigmask
-pthread_getspecific
-service_remove_unseen
-htsp_get_subscription_status
-htsp_init
-tcp_server_done
-tvhthread_create.
...instead of the correct backtrace of:
-<signal>
-htsp_build_dvrentry
-htsp_method_async
-htsp_read_loop
-htsp_serve...
So on FreeBSD only, we use libunwind to generate the
backtrace and function names. We explicitly make
libunwind and libexecinfo mutually exclusive since
FreeBSD has both.
Line are logged similar to:
CRASH: htsp_build_dvrentry+5d (ip=11f659d sp=7fffd8bc3930)
Note that it does not have line numbers since the addr2line
does not appear to work on FreeBSD (even with the original
backtrace code).
An example of the problem with the old backtrace code using
the frame from htsp_method_async from within the tvheadend
traphandler after the retrieval of the stack frames:
(gdb) print dladdr(0x11f1638, &dli) <--- addr of htsp_method_async from frame 4.
$39 = 1 <--- success
(gdb) print dli
$40 = {dli_fname = 0x7fffffffef97 ".../build.freebsd/tvheadend", dli_fbase = 0x1021000, dli_sname = 0x1044f91 "service_remove_unseen", <--- but wrong name
dli_saddr = 0x11eff80 <service_remove_unseen>} <--- and this is nearest symbol address
(gdb) print htsp_method_async+1640
$41 = (htsmsg_t *(*)(htsp_connection_t *, htsmsg_t *)) 0x11f1638 <htsp_method_async+1640> <---but gdb knows the original address is htsp_method_async
(gdb) print service_remove_unseen
$42 = {void (const char *, int)} 0x11eff80 <service_remove_unseen> <--- and gdb knows sevice_remove_unseen is at the dli_saddr.
By contrast, with libunwind, we get:
(gdb) print buf
$50 = "htsp_method_async", '\000' <repeats 110 times> <--- libunwind detected correct function name
(gdb) where 10 <--- even though our signal has been delivered on its own stack
#0 traphandler_libunwind () at src/trap.c:162
#1 0x000000000120cf06 in traphandler (sig=11, si=0x7fffdbbdb860, UC=0x7fffdbbdb4f0) at src/trap.c:221
#2 0x0000000806673954 in ?? ()
#3 0x0000000000000000 in ?? ()
(gdb) disass 18814904 <--- and gdb knows that ip address is for the same method as libunwind detected
Dump of assembler code for function htsp_method_async:
0x00000000011f1150 <+0>: push %rbp
E.Smith [Mon, 1 Oct 2018 15:57:36 +0000 (16:57 +0100)]
trap: Allow chdir /tmp even if prctl not supported.
Even though prctl is Linux specific, other platforms allow core
dumps to occur in the cwd, so it's useful to allow the "cd /tmp"
for those platforms if the existing --dump option is specified.
E.Smith [Mon, 1 Oct 2018 17:32:26 +0000 (18:32 +0100)]
build: Add hardening options.
Add some hardening options from:
https://wiki.debian.org/Hardening
These protect against basic buffer overruns.
Although debian/rules can have an "export DEB_BUILD_HARDENING=1",
it's useful to have these available across all builds that support
the compiler options.
Previously if multiple services were mapped to same channel then
we would get duplicate entries in the csv, such as a string of
'DVB-T,DVB-T,DVB-S,DVB-S'.
Previously we added the filename to the dvr_entry at the start of the
recording, but did not persist it. This meant that if tvheadend
crashed before the programme completed then we would leave a file on
disk which is not referenced by any recording, hence will never be
deleted.
So we persist after the file is created/stream opened. This entry then
has filename, stream info, and (actual) start time, but no (actual)
stop time.
Currently this fanartImage is set by the user in recording
post-processing. So, a "%U" format specifier gives the user the
uuid of the recording and they can use api/idnode/{load,save}
to add fanart/image artwork from appropriate sources.
This fanart is then displayed in Kodi via pvr.hts.
python: Add tvhmeta program for setting artwork on a dvr recording.
Very basic program for setting artwork/fanart in a dvr entry,
primarily as a proof of concept. Retrieving of artwork from an
external source is not done.
The program demonstrates retrieving existing data from the server for
the recording, updating the artwork, saving it, then re-fetching to
show the change has been applied.
The uuid can be found via the %U format specifier in the recording
post-processing commands.
Fix building with gcc 8
- Patch for nasm taken from Fedora 28
- Added CFLAGS -Wno-stringop-truncation -Wno-stringop-overflow
- fdk-aac requires -fPIC to link properly
Some broadcasts can have different charsets (such as iso-8859-1) but
we assume utf-8 unless user has set it correctly. So when decode fails
we get an exception. So we now attempt to decode with error
replacement so user sees incorrect character.
This gives "u'Denise Th\ufffd\ufffd':" as the string returned instead
when the received name contains an é that is in iso-8859-1 instead of
utf-8.
projects / thirdparty / tvheadend.git / log
