xmltv: Parse credits, category, keyword and more age ratings. (#4441)
The xmltv provides additional information about programmes such
as keywords ("Zookeeper", "Newscast", "Lion", "Mystery"), and
categories ("Crime drama", "Movie", "Series"). It can also provide
detailed information about actors, writers, editors, composers, etc.
We parse this information and allow it to be searched from the GUI.
We make this configurable since having 20+ actors per movie can
increase memory usage of the server and the clients to which we
send this information.
We also offer an option to append this information on to the
description. This allows people with old clients to see the
information.
We cache this information in to a csv string so users can search
across multiple actors such as "Douglas.*Stallone" to find movies
where both actors starred, rather than searching across each actor
individually.
The category is not currently searchable via regex since I think
that should probably be a search box similar to content type.
We currently only parse and store a few of the credits, viz., actor,
guest, presenter, writer, and director. If people really search for
films based on the composer or editor then we can add it in the future.
This information is stored on the epg_broadcast rather than the
epg_episode since theoretically a programme could have different
information for different showings of the same programme.
For example, my broadcaster shows the same film in the same week but
prefixes the description of some showings with a keyword (such as
"Frightfest") with other film of the same genre to create a pseudo-boxset.
Thus if we ever scrape keywords from EIT we'd probably tag the particular
films with this keyword as a tag on which people could search.
Similarly for credits, a daytime showing of a programme can contain edits
for violence, swearing that are not in the late night showing, thus potentially
changing the cast despite being the same "episode", or perhaps one showing
is dubbed.
We also parse a few more age ratings since a number of programmes
only have "word" ratings rather than age ratings (TV-14 instead of 14).
Also the existing age could underflow since one rating system
uses negative numbers which don't fit in our unsigned byte.
The _epg_object_set_lang_str has logic for setting lang_str_t.
We can re-use this logic for other objects in the future
(such as string_lists) so create a macro to abstract out
the destroy, compare, and copy functionality so the rest of
the logic can be re-used.
It's useful to be able to keep lists of sorted strings.
Although htsmsg allows us to keep lists, it is not
designed for easy sorted lists. So, build a simple
type on top of the existing RB trees.
Jaroslav Kysela [Fri, 29 Sep 2017 07:31:52 +0000 (09:31 +0200)]
prop: prop_sbuf / prop_ptr cleanups
The prop get string callbacks should use static buffers. This
patch is trying to fix the problematic parts (mostly in dvr_db.c)
and also introduces prop_ptr to not allocate static pointer
in each string get callback.
The problems in dvb_db.c were serious, so tvh could crash or
invalid data are written to the config files.
webui: Render upcoming recording duplicates with a line-through in grid. (#4632)
The advanced and expert views on the upcoming tab have a column for the
duplicate date. But on the basic view there is no indication of
duplicates unless you click information on the entry.
So we now render duplicates with a line-through. This makes it clear
they are duplicates that will not be recorded.
eit: Add drop-down list for selecting xmltv scraper configuration. (#4626)
Previously you had to enter text in the "scraper configuration to use"
drop-down on EPG grabber modules/Over the air module.
Now we populate a drop-down selection box by loading the configurations
and then getting their names. This is slight overhead compared to just
fb_scandir, but it is only used briefly during GUI configuration and
has the benefit of listing configurations that are valid in case the
user modified the configuration files and caused it to be invalid.
eit: Add simple test harness for scraping EIT data. (#4577)
This python script parses a scraper configuration file
from data/conf/epggrab/eit/scrape and a unit test file
from support/testdata/eitscrape.
The unit test contains numerous examples and the expected
scrape results, such as season and episode number.
The top of the test harness configuration file contains
some comments fields that are unparsed but help document
what environment the test harness is meant to be testing.
Daniel Becker [Thu, 21 Sep 2017 04:42:39 +0000 (21:42 -0700)]
fix storage space calculation
The f_bfree and f_bavail members in struct statvfs are expressed in
units of f_frsize, not f_bsize (see the statvfs(2) manpage, e.g. here:
<https://linux.die.net/man/2/statvfs>). The two can differ quite
substantially, e.g. on ZFS.
eit: Add extra eit episode scrape configurations (#4592)
We include the regex from the opentv configuration for scraping
episode informationfor Italy, Australia and New Zealand with
minor changes to allow parsing by the Python test harness.
Also added additional Italian regex from the bug report.
eit: Also scrape eit episode information from title and description (#4592)
Previously we only searched the summary for scraping episode information,
but several countries also put the information in the title or description.
So we search each one in turn with the same regex and merge the results.
Previously we only parsed numbers of the format "445" and
forced the minor number to be zero. However atsc numbers
have a major and a minor so we now parse "39.1" as
major=39 minor=1.
eit: Add additional documentation on EIT scraper. (#4578)
Add some more documentation on the EIT scraper to clarify
that it does not access the Internet and that we only ship
with a few configurations at the moment.
eit: Allow sub-title scraping to be configured on/off in the GUI. (#4578).
We need to be able to disable sub-title scraping since many people
have recording rules of "record if different subtitle". Since the
scraping alters the subtitle it means these people would get
re-records.
eit: Scrape sub-title from summary in OTA EIT. (#4578).
The Freeview/Freesat frequently have a subtitle as part of the
summary. So we have "Treehouse of Horror IX: Three scary stories."
from which we can deduce the subtitle as "Treehouse of Horror IX".
Other variants are "...title_continuation. Subtitle" (so the real
title of the program is split in to the summary), and
"x/y. Subtitle" where x/y is the episode number.
So allow scraping of this and use it as the subtitle. If we cannot
scrape a subtitle then we continue the existing practice of using
the summary buffer for the subtitle.
The subtitle is currently NOT removed from the summary.
projects / thirdparty / tvheadend.git / log
