Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add metadata lookup by timestamp to tmdb3tv.py #700

Open
SteveErl opened this issue Jan 20, 2023 · 5 comments
Open

Add metadata lookup by timestamp to tmdb3tv.py #700

SteveErl opened this issue Jan 20, 2023 · 5 comments
Assignees

Comments

@SteveErl
Copy link
Contributor

Is your feature request related to a problem? Please describe.
When using "TheMovieDB.org V3 television" as the Television Metadata Source, and recording from a Manual Record rule or an automatic record rule based off of OTA broadcast information which does not specify a Subtitle field for TV shows, specific metadata for a TV episode is not available.

Describe the solution you'd like
In the tvmaze.py metadata script, there is an option to find a specific TV episode by providing the name of the show, and a timestamp for the initial airing. When recording the initial airing of a TV episode, use of this option allows us to retrieve specific episode metadata information, such as Season, Episode, Subtitle, and Description.

The same capability should be added to the TMDB metadata retrieval script for TV shows. Users who have chosen to use "TheMovieDB.org V3 television" will be able to collect more complete metadata information for first broadcast recordings.

Describe alternatives you've considered
The only alternative is for the user to select TvMaze instead of TMDB for TV show metadata collection.

SteveErl added a commit to SteveErl/mythtv that referenced this issue Jan 20, 2023
This change adds support for the following syntax options:

   tmdb3tv.py -N <title> <date time>
   tmdb3tv.py -N <inetref> <date time>

For example:

   tmdb3tv.py -N "The Blacklist" "2021-01-29 19:00:00"

<?xml version='1.0' encoding='UTF-8'?>
<metadata>
  <item>
    <title>The Blacklist</title>
    <subtitle>Elizabeth Keen</subtitle>
    <description>As Red and the task force search for Liz, she sets a new plan in motion that has catastrophic consequences.</description>
    <season>8</season>
    <episode>4</episode>
    <inetref>46952</inetref>
    <collectionref>46952</collectionref>
    <language>en</language>
    <releasedate>2021-01-29</releasedate>
...
  </item>
</metadata>

In the above example, by specifying the start of the
recording time, we were able to acquire the Season,
Episode, Subtitle, and Description. Without this
new capability, we would have only the Title and
a generic description of the TV series.

Please note that the TMDB database only stores the
date of the initial airing, so the match will not
be as precise as in the TvMaze lookup, which stores the
date, time, and timezone. If there are multiple
episodes aired on a single date, then TMDB provides
no way to descriminate between them. It will generate
a list with all the episodes matching that date.

Resolves: MythTV#700
SteveErl added a commit to SteveErl/mythtv that referenced this issue Jan 26, 2023
Special care should be taken when episodes air at or after midnight.
To ensure that the episode airdates on TMDB3 are in line with the
dates used by TV guides and listings worldwide, episodes that start
airing at or after midnight but before 5:00am should be considered
part of the previous day.

This adjustment allows us to retrieve the correct Season, Episode,
Subtitle, and Description during a tmdb3tv.py 'lookup by timestamp'.

After this commit, if a user records "Late Night With Seth Meyers"
in the Eastern timezone in the United States, it will return the
metadata for the 2023-01-25 episode. The timestamp specifies
2023-01-26, but the correct metadata has releasedate = 2023-01-25.

tmdb3tv.py -l en -a US -N "Late Night With Seth Meyers" "2023-01-26 00:37:00"
<?xml version='1.0' encoding='UTF-8'?>
<metadata>
  <item>
    <title>Late Night with Seth Meyers</title>
    <subtitle>Colin Quinn, Gabrielle Union</subtitle>
    <description>Colin Quinn, Gabrielle Union</description>
    <season>10</season>
    <episode>53</episode>
    <inetref>61818</inetref>
    <collectionref>61818</collectionref>
    <language>en</language>
    <releasedate>2023-01-25</releasedate>
    ...
  </item>
</metadata>

Resolves: MythTV#700
@rcrdnalor
Copy link
Contributor

The discussion on PR #702 led to the following open questions
which need to be resolved before MythTV can add this optional feature:

  • Exact definition of an air_date of an episode from given tv-series.
    The forum says in 2018, its is in UTC, which was revised by the same person 2020
    as "it's assumed to be 00:00 in the timezone of the original airing network."
    A final statement from themoviedb.org about the definition of air_date and
    how they handle falsely entered data is welcome.

  • Correctness of the data air_date provided by the API from themoviedb.org
    The mentioned PR found some discrepancies. This needs to be verified globally,
    not only on some examples from North America.
    If there are more discrepancies, MythTV needs to evaluate if this way of finding
    'an episode by given air_date' is of any use.

  • Finally, the script tmdb3tv.py which may interpret the provided data falsely.
    This needs to be checked by code review and tests using other metadata provider.

Note:
Currently, there is no way for MythTV to differentiate between wrong and correct metadata.
In other words, once the data is entered in the mythconverg database, one cannot overwrite
it easily, only by external scripts using MythTV's Services API or the MythTV Bindings.
In general, the users are very sensitive when it comes to the handling of air_date:
See issues #572 and #628.

@garybuhrmaster
Copy link
Contributor

FWIW, at least one guide provider defines all times/dates they provide to be in UTC in their metadata APIs, and provides both an origAirDate (date of the original TV airing), and a releaseDate (for TV shows that would typically be the same as origAirDate, but for theatrical movies, it may be different (the date of a movie release is separately provided by that guide provider)). If one is using the appropriate xmltv grabber the (guide providers) originalAirDate, if available, is inserted into the DB for the program. I do not know if that guide providers dates are highly aligned with other sources of date and metadata such as themoviedb.

@rcrdnalor
Copy link
Contributor

rcrdnalor commented Jan 29, 2023

@garybuhrmaster
Yes, that is true, but the organisations providing metadata need to verify that this event actually happen. Not all TV stations stick to their announced schedules. Some stations insert breaking news, e.g.: from the Ukraine War, and this might move the 'air_date' to the other day.
Edit: This might hit you when you record a new episode of a tv-show.

@SteveErl
Copy link
Contributor Author

SteveErl commented Feb 3, 2023

In the United States, there seems to be a convention for late night programs starting at midnight, or shortly after, of assigning the previous day to air_date. I tested this with "Late Night with Seth Meyers", which starts at 00:35:00 EST, and the TvMaze, TMDB3, and TTVDB4 databases all specify last night's episode as airing on 2023-02-02, even though, technically, it aired on 2023-02-03 in the Eastern timezone (America/New_York). It's the same for other late night talk shows originated from the US. I've tried looking for shows in other countries which either follow or break this convention, but it's been a difficult search. We want to look for countries with more than one timezone, where the crossover of midnight would cause differences in the date part of the timestamp for different timezones. According to
https://www.countries-ofthe-world.com/world-time-zones.html
The countries with multiple time zones in the mainland area (some of them also have the insular territories) are Russia, USA, Canada, Australia, Mexico, Brazil, Indonesia, Kazakhstan, Mongolia, the Democratic Republic of the Congo, Kiribati, Micronesia, Chile, Spain, Portugal, and Ecuador.

I checked TV listings in Australia and Canada, but found both of them primarily show late night talk shows from the US. I didn't find locally generated programs which would clarify behavior. There are very few TV shows whose episodes originate at midnight. There are some local news shows, but they didn't seem to have metadata to distinguish daily episodes. I lack the linguistic skills to find good TV listings for most of the other countries in the list.

@rcrdnalor
Copy link
Contributor

In the United States, there seems to be a convention for late night programs starting at midnight, or shortly after, of assigning the previous day to air_date.

Unless you have a confirmation from themoviedb.org, that they will stick always to that convention, the provided change is useless and subject for further errors.
Note: The grabbers for data provided by themoviedb.org are in use all over the world.

@rcrdnalor rcrdnalor self-assigned this Feb 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants