Database locked error

[ceejayemm]

Martin

I think I understand what you are saying but you might want to review your last reply text to see what I mean.

I think you are asking me to delete the Catch-22 recordings but where (in the web app ?) or in the database (which table(s) and how to locate them). Catch-22 is a series of 4 recordings so do I need to delete them all?

I have the original recordings (.ts) files so once we get the problems sorted \I can reimport them PROVIDING we can sort out exactly what each relevant XML file (not the bulk recordings XML that can be exported from the web app) should contain.  We have had this conversation before , I am talking about the fact that the .XML file created at recording time, in the same folder as the .ts file, is COMPLETELY  the wrong format to be able to reuse it to reimport the recording file without undergoing significant editing work. This file format is different from the exported recordings .XML file and its about time this anomaly was corrected.

Chris

[mvallevand]

I would delete bad recording with a sqlite3 editor. The reason called Catch-22 bad is there was zero metadata other than the 0 OID.

Not sure if that rather demanding comment was meant for me, but I may have suggested my NextImport utility that imports NextPVR data recordings n either the backup format or .xml format and also from standard NFO, but it won't work with the mix of formats I saw you trying in your database.

Martin

[ceejayemm]

Martin

Which tables should I be looking at to find these recordings.  I am comfortable with SQL code but not sure of the database structure for the relevant data.

We have discussed the individual recordings XML file (with Sub) before see:

Import Recordings Version 5 (nextpvr.com)

With a properly configured individual .XML file you CAN import one recording at a time via the web app if you so desire. It is this that I would like to see in a future version, it would make my life a whole lot easier when a problem like this occurs.

The database might well be instate due to experimenting with differing XML file formats.  I would be happy try your NextImport utility for what I need to do. I have found a version here:

VidImport not working (nextpvr.com)

Is this the latest version and would it be worth adding this to the NextPVR Wiki so there is a easy point of reference to it ?

Thanks for all your help and support in heading towards a fix for this problem.

Chris

[ceejayemm]

Despite a successful manual update of the EPG earlier last night, when the automated version ran at 02:45 this morning I  ended up with the familiar 'Database is Locked' error. I have uploaded the logs for this 

I also installed Martin's NextImport utility to try and help sort out the 'bad' imports I have previously done. I installed the utility as detailed here:

Migrating from Windows v4.x to Linux v5.x (nextpvr.com)

and after few false starts I got it to run.  I thought rather than try it out on the bad recordings I thought I would try it first on a record not known to be bad so I ran the command:

sudo -u nextpvr /opt/dotnet/dotnet  /opt/nextpvr/system/NextImport.dll --export '/home/pi/nas_recordings/Arrival (2016)'

which ran and gave me the following output:

NEXTPVR_INSTALLDIR /opt/nextpvr/system/
NEXTPVR_DATADIR /opt/nextpvr/system/data/
NEXTPVR_DATADIR_USERDATA /var/opt/nextpvr/
Folder: Arrival (2016)
Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.
  at NextImport.Program.SearchDB(String filename, String table)
  at NextImport.Program.ProcessFile(String path)
  at NextImport.Program.ProcessDirectory(String targetDirectory, Int32 level)
  at NextImport.Program.GetFiles(String folder)
  at NextImport.Program.Main(String[] args)
  at NextImport.Program.<Main>(String[] args)
Aborted


Is this latest version and do I need to do something else to sort this out ? There was nothing useful in the NextImport.log file which I have also uploaded.

Chris

[mvallevand]

It is odd that you are getting errors on your recurring recordings, several are not the right channel but the reason behind the error is not clear. I figure it is related to the bad Lone Ranger recording with no OID. Restart the service and delete that record and see if the recurring recording errors go away when you do an EPG update.

I suspected that error in NextImport is because the database is locked when it goes to SearchDB.  However testing doesn't show that, see if the current version I have works better.

Martin

[ceejayemm]

Martin / Sub

All the 'bad' recordings seem to come items I have imported using modified XML files via the Web App. All these imported 'bad' recording seem to play correctly in the web app / Kodi, the problem appears to be in the metadata and what is or isn't there. The issues we are seeing have only started to appear since the upgrade to 5.1.2, so is this version more sensitive to incorrect metadata ?

The Channel OID for these 'bad' records is zero (because I don't what it was originally) and the Channel Name of 'Imported', 'Film4' or 'BBC One London'  was probably entered as text in the import .XML file. Over time I have tried several different entries in the XML file hence the various discrepancies. The format of the import XML file is not documented anywhere that I can find so I have made the XML file entries using the recordings-backup.xml file for guidance. I do need to import recording files from time to time and I haven't found a reliable way to do this.  This is why I have corresponded several times about getting the system to generate the XML file created when a recording is made by the app to be at least in the same format as the full recording-backup.xml file. I could then at least see what the minimum xml tags are when creating a manual XML file for imported recordings. If the NextImport utility can do this then so much the better - great. 

In respect of the NextImport errors reported above, this utility was run immediately after a system restart so as far as i can tell the database wasn't locked at this point in time. How / where can I get latest version of NextImport ?

What happens if I delete, as suggested by Martin, all these 'bad' records (apparently 8 in total, where the event_details field contains <OID>0</OID>) from the 'scheduled_recording' table ?  Will I still be able to play the actual recording files which in themselves seem to be ok ?

Chris

[mvallevand]

The Lone Ranger issue was <OID></OID> which didn't get cleaned out with the sqlite3 call I gave you which was to clean out records that don't have any OID markup. This is not the same as zero, but zero is fine to use if you don't know it.

NextImport.zip was attached. The -export folder option creates a file called NextImport.xml in the user data folder you can review and edit before importing. If you don't want to use NextPVR web to import it, there is another option -importxml filename.

The -import folder function brings them in directly should you want to bypass the xml file altogether. Both options use filename.xml file if it exists and tries to get some metadata from the filename itself.

Martin

[ceejayemm]

Martin

Sorry missed attachment of NextImport.  I have now downloaded it and installed this version. Unfortunately I get the following error (database is not locked):

pi@CMTV2:/opt/nextpvr/system $ sudo -u nextpvr /opt/dotnet/dotnet  /opt/nextpvr/system/NextImport.dll --export '/home/pi/nas_recordings/Arrival (2016)'
NEXTPVR_INSTALLDIR /opt/nextpvr/system/
NEXTPVR_DATADIR /opt/nextpvr/system/data/
NEXTPVR_DATADIR_USERDATA /var/opt/nextpvr/
Unhandled exception. System.TypeInitializationException: The type initializer for 'Microsoft.Data.Sqlite.SqliteConnection' threw an exception.
---> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
---> System.DllNotFoundException: Unable to load shared library 'e_sqlite3' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libe_sqlite3: cannot open shared object file: No such file or directory
  at System.Runtime.InteropServices.NativeLibrary.LoadLibraryByName(String libraryName, Assembly assembly, Nullable`1 searchPath, Boolean throwOnError)
  at System.Runtime.InteropServices.NativeLibrary.Load(String libraryName, Assembly assembly, Nullable`1 searchPath)
  at SQLitePCL.NativeLibrary.Load(String libraryName, Assembly assy, Int32 flags)
  at SQLitePCL.Batteries_V2.MakeDynamic(String name, Int32 flags)
  at SQLitePCL.Batteries_V2.DoDynamic_cdecl(String name, Int32 flags)
  at SQLitePCL.Batteries_V2.Init()
  --- End of inner exception stack trace ---
  at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
  at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
  at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
  at Microsoft.Data.Sqlite.Utilities.BundleInitializer.Initialize()
  at Microsoft.Data.Sqlite.SqliteConnection..cctor()
  --- End of inner exception stack trace ---
  at Microsoft.Data.Sqlite.SqliteConnection..ctor(String connectionString)
  at NUtility.DatabaseHelper.GetConnection()
  at NextImport.Program.Main(String[] args) in D:\v5\solutions\NextImport\Program.cs:line 91
  at NextImport.Program.<Main>(String[] args)
Aborted


Chris

[mvallevand]

Perhaps there is an NextImport.deps.json file in the folder from the old install that needs to be removed?

Martin

[ceejayemm]

Martin

Yes there was, I have removed it and restarted the Nextpvr service but still get the same error  message.

Chris