Jaroslav Kysela [Fri, 29 Sep 2017 07:31:52 +0000 (09:31 +0200)]
prop: prop_sbuf / prop_ptr cleanups
The prop get string callbacks should use static buffers. This
patch is trying to fix the problematic parts (mostly in dvr_db.c)
and also introduces prop_ptr to not allocate static pointer
in each string get callback.
The problems in dvb_db.c were serious, so tvh could crash or
invalid data are written to the config files.
webui: Render upcoming recording duplicates with a line-through in grid. (#4632)
The advanced and expert views on the upcoming tab have a column for the
duplicate date. But on the basic view there is no indication of
duplicates unless you click information on the entry.
So we now render duplicates with a line-through. This makes it clear
they are duplicates that will not be recorded.
eit: Add drop-down list for selecting xmltv scraper configuration. (#4626)
Previously you had to enter text in the "scraper configuration to use"
drop-down on EPG grabber modules/Over the air module.
Now we populate a drop-down selection box by loading the configurations
and then getting their names. This is slight overhead compared to just
fb_scandir, but it is only used briefly during GUI configuration and
has the benefit of listing configurations that are valid in case the
user modified the configuration files and caused it to be invalid.
eit: Add simple test harness for scraping EIT data. (#4577)
This python script parses a scraper configuration file
from data/conf/epggrab/eit/scrape and a unit test file
from support/testdata/eitscrape.
The unit test contains numerous examples and the expected
scrape results, such as season and episode number.
The top of the test harness configuration file contains
some comments fields that are unparsed but help document
what environment the test harness is meant to be testing.
Daniel Becker [Thu, 21 Sep 2017 04:42:39 +0000 (21:42 -0700)]
fix storage space calculation
The f_bfree and f_bavail members in struct statvfs are expressed in
units of f_frsize, not f_bsize (see the statvfs(2) manpage, e.g. here:
<https://linux.die.net/man/2/statvfs>). The two can differ quite
substantially, e.g. on ZFS.
eit: Add extra eit episode scrape configurations (#4592)
We include the regex from the opentv configuration for scraping
episode informationfor Italy, Australia and New Zealand with
minor changes to allow parsing by the Python test harness.
Also added additional Italian regex from the bug report.
eit: Also scrape eit episode information from title and description (#4592)
Previously we only searched the summary for scraping episode information,
but several countries also put the information in the title or description.
So we search each one in turn with the same regex and merge the results.
Previously we only parsed numbers of the format "445" and
forced the minor number to be zero. However atsc numbers
have a major and a minor so we now parse "39.1" as
major=39 minor=1.
eit: Add additional documentation on EIT scraper. (#4578)
Add some more documentation on the EIT scraper to clarify
that it does not access the Internet and that we only ship
with a few configurations at the moment.
eit: Allow sub-title scraping to be configured on/off in the GUI. (#4578).
We need to be able to disable sub-title scraping since many people
have recording rules of "record if different subtitle". Since the
scraping alters the subtitle it means these people would get
re-records.
eit: Scrape sub-title from summary in OTA EIT. (#4578).
The Freeview/Freesat frequently have a subtitle as part of the
summary. So we have "Treehouse of Horror IX: Three scary stories."
from which we can deduce the subtitle as "Treehouse of Horror IX".
Other variants are "...title_continuation. Subtitle" (so the real
title of the program is split in to the summary), and
"x/y. Subtitle" where x/y is the episode number.
So allow scraping of this and use it as the subtitle. If we cannot
scrape a subtitle then we continue the existing practice of using
the summary buffer for the subtitle.
The subtitle is currently NOT removed from the summary.
DVR: Prefer autorec rule name if comment field is empty (#4500)
Currently the Upcoming recordings tab has a comment field that says
"Auto recording" or "Auto recording: <comment from autorec rule>".
This helps to identify why a recording is scheduled.
This patch ensures we continue to use the autorec comment field if
it is non-empty (keeping existing behaviour), but fallback to using
the recording rule name. If both are empty then we keep the
existing behaviour of fallback to an empty string.
This avoids the user having to duplicate the rule name in to
the comment string for manually created autorec rules.
So, in the above case it would be "Auto recording: <comment>",
"Auto recording: <rule name>", otherwise "Auto recording".
eit: Allow scraper configuration file to be configured at the GUI (#4287).
Previously the scraper was hard-coded based on the module name.
So "uk_freeview" module would check "uk_freeview" configuration file
and then the "uk" file.
However, this meant that the generic "eit" module (used by several
countries) had to be symlinked by the user to a specific configuration
for their country.
With this change, the user can simply enter "uk" in the GUI to read
that configuration.j
Also renamed "fixup" to be called "scrape" since we are scraping
data from the EIT rather than fixing it.
eit: Allow EIT scraping of season/episode to be disabled at GUI. (#4287).
We now have a tick box in the OTA configuration to enable/disable
the scraping of season/episode numbers from the eit grabbers.
This will allow us to add other scrapers and tidy-ups in the
future (such as removing "Also in HD" from the summary data
or "New:" from the title), and allow the user to disable ones
they do not want for very low-spec machines or due to their
duplicate rules relying on pre-tidy data.
To achieve this configuration, we now derive our eit grabbers
to be a "...scraper" type and hook in to the activate callback
to load/unload the regular expressions.
The loading of the config also had to be moved to the activate
rather than in the module create to allow us to access the
"scrape enabled" boolean.
eit: Extract season/episode numbers from OTA EIT. (#4287).
Broadcasters often include season and episode number in
the summary text in the OTA EIT.
For example, UK broadcasters often, but not always,
have a description of "Lorem ipsum (S5 Ep 8)" or
"Lorem ipsum (S3 Ep 4/9)" or "Lorem ipsum (Ep 4)".
From this we can use a regular expression match to
extract the season and episode data on a best effort
basis. This logic is based on the opentv extractor.
This is done via config files that are named after the
grabber module and exist in this directory:
data/conf/epggrab/eit/fixup/
Example names would be uk_freeview.
If the module-specific config file does not exist then we
fallback to trying the first component of the filename.
In the above example that would be "uk". This avoids having
duplicate files in the case where we have DVB-S and DVB-T
in the same country that share the same extraction regex.
The configuration file should contain season_num and
episode_num sections that can contain multiple regular
expressions to apply in sequence until one produces
a match.
For DVB-S, the configuration file normally needs to be copied to
a file named "eit" since data is broadcast via that mechanism.
This isn't done by default since the eit grabber is used by
multiple countries that may use different regular expressions.
eit: Move opentv pattern list functions to separate file. (#4287).
The pattern list functions are used for regular expression matching.
We move them to a separate file and rename them to have an
eit prefix instead of opentv prefix so they can be shared with
other eit modules.
E.Smith [Sat, 19 Aug 2017 09:26:44 +0000 (10:26 +0100)]
DVR: Record segmented programmes identified by EIT.
A broadcaster can split a programme in to multiple segments. These
are identified by the segments having a CRID containing an IMI
(a hash character followed by an ID). Segments have identical
values for this CRID and IMI and the segments start within three
hours of the end of the previous segment.
These rules are documented in this spec in section 7.1.7:
http://www.freeviewnz.tv/media/1055/freeview_dtt_transmission_rules_2_1.pdf
This document is based on the UK transmission specification.
For example, a movie may be broadcast as:
21:00--22:00 movie segment 1
22:00--22:05 five minute news
22:05--23:30 movie segment 2
The xmltv guides typically merges this segments in to one
programme such as:
21:00--23:30 movie (including news)
In theory, a programme can be split in to numerous segments.
In practice I have only seen a programme split in to two
segments as shown above.
To simplify recording these programmes, we identify segmented
programmes and extend the stop time. So, in the above case,
if the user records the 9pm showing then we will automatically
extend the stop time to be 23:30 instead of 22:00.
This patch explicitly disables "epg running state" for stopping
the recording. This is because the recording is tied to the first
showing and we don't want the recording to stop at 22:00 in the
above example.
We cache the calculated stop time to avoid any overheads, but
explicitly recalculate it at the start of the programme. This ensures
we detect any recent changes.
No modification is done of the actual EPG data to attempt to
merge the programme segments.
The consequence of this is that the EPG will only show a "recording"
marker against the first segment of the programme and not against
the second segment, which is unfortunate, however it is consistent
with recordings which have an extra stop time. The upcoming
recordings tab correctly shows the end time.
The duration of the finished recording is currently incorrectly
reported due to #3706. So the movie above would be reported as
60 minutes instead of 2h30.
Although the CRID processing is believed to be a global standard,
if other countries do not follow the UK/NZ specification then
the dvr_entry_get_segment_stop_extra could be updated to check a
(bitmask) config variable to enable/disable specific CRID processing.
I believe the overhead of the strcmp for the CRID check is minimal,
even on low-spec machines. If necessary we could cache to indicate
the CRID check has failed.
projects / thirdparty / tvheadend.git / log
