Importing Multiple M3U files issue ?

[dayvboy]

I noticed an anomaly when I'm importing m3u files ?  The initial first m3u file loads fine and channel.number = channel_mapping.channel_source_number where channel.oid = channel_mapping.channel_source_oid.   But any subsequent m3u file I import ends up with the new channels having channel_source_number being 1 more than it should be.  For example if the 2nd file started channel numbering at 300, then the first new row added would have channel.number = 300 but the matching channel_source_oid in channel_mapping would have channel_source_number = 301 and that pattern continues for all the newly added rows in the 2nd m3u file.

I presume NextPVR joins using the oid's so maybe this issue was never noticed before.  I noticed because I have some scripts that I use for reassigning tuning and epg columns and find it convenient to be able to query channel_mapping by channel_source_number.   I thought I was doing something wrong until I noticed that the channel numbers weren't aligning properly on the newly added rows.  So every time I import another m3u file, I end up running a script to correct the channel matching between channel & channel_mapping tables.

Both tables always have the same number of rows and as mentioined, the very first file always loads correctly.  Its the 2nd, 3rd, 4th, etc files that seem to hit this issue.

I get that in a purely "normalized" database, channel_source_number wouldn't even be there and that its likely there just for convenience, but in order to take advantage of this convenience, I need to correct it after each new m3u load.

[mvallevand]

Don't worry about the oid or database, NextPVR doesn't use pure SQL. What does the tvg-chno say in your m3u and are you adding multiple m3u file or trying to update one.

Martin

[dayvboy]

I'm loading new files.  The channel number assigned in the channel table is correct but the matching row in channel_mapping table (where channel.oid = channel_mapping.channel_source_oid) ends up with the right tuning value but the channel_source_number is one more than the matching entry in the channel table.  It should match the value in the corresponding channel table row (the oid's match, epg and tuning are correct, but the channel numbers are off by 1).  Initial file loads fine, its the rows added by any subsequent new m3u loads that are off.   Decrementing Channel_Mapping.Channel_source_number by 1 in all the newly added rows sets them correctly so that the matching row pairs in the 2 tables have the same channel numbers.

Example:  First file loads channels 1 to 499.  Oid's and channel numbers in the two tables match.   Second file loads starting at channel 500.   Channel number of first row assigned in channel is 500 but 1st row assigned in matching Channel_Mapping table is 501 and so on for each new row.    Next two new rows would have 501 vs 502, then 502 vs 503, etc.   And there is no row in Channel_Mapping with channel number 500 because it started numbering the new rows from 501 unlike channel which numbered the new rows starting from 500.

The Oid's match correctly in the newly added rows but the channel numbers are off by 1.

[mvallevand]

You didn't answer my question. When you load a new m3u file and do not update, channel numbers are supposed to be assigned the next high number unless you provide the tvg-chno in the m3u. Are you saying that NextPVR is not respecting tvg-chno ?

Or course if you are doing anything with sqlite3 you are on your own.

Martin

[dayvboy]

You didn't answer my question.  When you load a new m3u file and do not update, channel numbers are supposed to be assigned the next high number unless you provide the tvg-chno in the m3u.  Are you saying that NextPVR is not respecting tvg-chno ?

Or course if you are doing anything with sqlite3 you are on your own.

Martin

Yes I import a new m3u without update and channel table is assigned correctly starting at the next highest channel number but channel_mapping table is using next highest + 1 on the first row and continues being off by 1 for each row added thereafter.     For example when loading my new file the first channel row is assigned number 500 and so should the first channel_mapping row but the latter is assigned 501 instead of 500.  The OID's match correctly in each pair of rows added but the channel number in channel_mapping is always 1 more than the channel number in channel table.

[dayvboy]

I have been using sdl lite to maintain the db for years so maybe I corrupted the sequence number generation but that wouldn't explain why the oid's get generated matching correctly while the channel numbers are off by 1. I suspect its a problem in the import but because most of the system joins the rows by oid's it goes by unnoticed. It would appear Channel_Mapping.Channel_Source_Number is only in that table for convenience and never displayed or actually used for anything. If I don't repair by decrementing Channel_Source_Number, it has no effect as it doesn't cause any issue when they don't match with their Channel counterpart rows. If I don't patch by decrementing the value by 1 then if i query Channel_Mapping where Channel_source_number = 501, I'm actually going to get back the row that corresponds to channel 500 which isn't the correct pairing.

Assuming my first file had 499 channels, when I import the new file without update all imports fine and there is no noticeable effect that the channel numbers don't match i.e. it doesn't cause any issue aside from the fact that Channel_Source_Number column has the wrong value (1 more than it should be).

If I query my new rows (assuming they started importing at 500) as below joining by oid ....

Select name, number, channel_source_number, channel.oid, tuning, channel_mapping.channel_source_oid from channel, channel_mapping where channel.oid = channel_source_oid and number >= 500 order by number

then the newly added rows will come back correctly joined/paired except that channel_source_number will be 1 more than number. The value of tuning will be correct for the each paired row returned.

If I query the same row sets joining by channel number I get different results (should be the same) ...

Select name, number, channel_source_number, channel.oid, tuning, channel_mapping.channel_source_number from channel, channel_mapping where = number = channel_source_number and number >= 500 order by number

I get back a different set of paired rows and the value of tuning returned is wrong as its actually for the "next" channel (eg. for channel 501 instead of 500).

If channel_source_number was correctly assigned during the import, both queries would bring back the same results. Decrementing channel_source_number by 1 on all the newly added rows fixes it so that both queries then correctly return the same results.

[mvallevand]

You still didn't answer the question, but I can guess that you aren't using tvg-chno.

I just tried a test with just one m3u line and it worked as expect, ie the name was correct, the number was current and the stations plays but I think I see what you are try to say

Code:

select oid, name, number, channel_source_number from channel, channel_mapping where oid = channel_oid;

the channel_source_number is technically wrong but it is probably not used. Is it cause any trouble for you in practice? sub may simply have this undefined for IPTV

Martin

[dayvboy]

It causes no issues for me except that its convenient for me to use in scripts to check and/or patch the tuning column for a particular channel. My IPTV provider will often change a channel's http address or have several for the same channel but only one of them working. Its easier for me to patch the tuning column for a particular channel instead of reloading an entire m3u file. That way I don't have to reconfigure the epg either since it stays the same. Course I can always get the oid from the channel table first and use that to lookup channel_mapping but its quicker from me to lookup channel_mapping by channel_source_number since I can already know that before I begin the query. That's kind of what I meant by the field is irrelevant in the channel_mapping table except for the convenience of using it.

This issue is something I'm able to work-around by running a patch after the import to decrement source_channel_number by 1 on any newly added rows.

[sub]

It's not a bug. The channel_source_number isn't used with IPTV. It's only used with HDPVR and Analog tuners. For all other tuning types (including IPTV), it uses the 'tuning' column instead, with the more complex xml tuning strings.

From memory when scanning with an IPTV source it just uses the next available number, but for no real reason - it's not used.