The update subscription emitted the full update status, which, depending on how big the status was, took forever because the graphql subscription does not support data loader batching, causing it to run into the n+1 problem
* Initial import of Kitsu tracker
Based on Mihon 6c6ea84509cc1bd859c880bebbc69067a241b358 because its
successor 9f99f03 relies on incompatible changes
* Kitsu: Avoid stupid long/int cast
* Optimize restoring manga chapters
* Streamline restoring manga data
* Optimize restoring manga trackers
* Simplify passing manga category restore data
* Properly prevent mangas from getting added to default category
76595233fc never actually worked...
* Extract logic to add manga to categories from gql mutation
* Optimize restoring manga categories
* Optimize restoring categories
coerceIn throws an error in case the max value is less than the min value ("Cannot coerce value to an empty range: maximum <max> is less than minimum <min>")
Regression from c8bd39b4bf
* Extract logic to restore manga chapters into function
* Extract logic to restore manga categories into function
* Extract logic to restore manga trackers into function
* Handle duplicated chapters in backup
In case a backup contained duplicated chapters for a manga, the manga failed to restore since the ChapterTable has a unique constraint to prevent multiple chapters with the same "url" and "mangaId"
* Añadiendo algunos cambios iniciales para probar OPDS
* Add suport to OPDS v1.2
* Added support for OPDS-PSE and reorganized controllers
* Rename chapterIndex to chapterId in the API and controller, and update descriptions in OPDS
* Refactor OPDS to use formatted timestamps and proxy thumbnail URLs
* Refactor OPDS to use formatted timestamps and proxy thumbnail URLs
* Update Manga API to download chapters cbz using only chapterId and improve chapter download query
* Optimize OPDS queries
* Update Manga API to download chapters cbz using only chapterId and improve chapter download query
* Optimize OPDS queries
* Use SourceDataClass to map sources and optimize thumbnail URL retrieval
* Kotlin lint errors in ChapterDownloadHelper and Opds
* Kotlin lint errors in ChapterDownloadHelper and Opds
It's possible that a manga is bound to a tracker while there is no search result.
This happens when e.g. restoring a backup which includes track bindings for which there was never a tracker search.
In that case when trying to e.g. copy the binding to another manga, the mutation would fail due to not finding a search result.
These cases can be handled by additionally checking the TrackRecordTable to get the necessary track info.
* Update to exposed-migrations v3.5.0
* Update to kotlin-logging v7.0.0
* Update to exposed v0.46.0
* Update to exposed v0.47.0
* Update to exposed v0.55.0
* Update to exposed v0.56.0
* Update to exposed v0.57.0
* Properly set download update type on exceptions
* Always send FINISHED download update to client for deprecated subscription
By the time the status was sent to the client, the finished download item was already removed from the queue, causing the client to never get the latest status, thus, having an outdated cache
Regression introduced with 168b76cb0c
* Validate setting values on mutation
* Handle invalid negative setting values
* Ensure at least one source is downloading at all times
* Prevent possible IllegalArgumentException
The "serverConfig.maxSourcesInParallel" value could have changed after the if-condition
* Emit only download changes instead of full status
The download subscription emitted the full download status, which, depending on how big the queue was, took forever because the graphql subscription does not support data loader batching, causing it to run into the n+1 problem
* Rename "DownloadManager#status" to "DownloadManager#updates"
* Add initial queue to download subscription type
Adds the current queue at the time of sending the initial message.
This field is null for all following messages after the initial one
* Optionally limit and omit download updates
To prevent the n+1 dataloader issue, the max number of updates included in the download subscription can be limited.
This way, the problem will be circumvented and instead, the latest download status should be (re-)fetched via the download status query, which does not run into this problem.
* Formatting
* feat(comicinfo): add date fields to comic info
This will be parsed by Komga, Kavita etc ... and any other library management to also have the date of the chapter.
* refactor: improve code readability
Co-authored-by: Mitchell Syer <Syer10@users.noreply.github.com>
---------
Co-authored-by: Mitchell Syer <Syer10@users.noreply.github.com>
* Launch missed auto backup task in background
* Launch missed auto global update task in background
* Launch missed auto webui update check task in background
In case a manga gets added to the library which has not been initialized yet, it should be tried to initialize it.
Since it's not an error to have uninitialized manga in the library, this can be done in the background via the updater and the client receives the updated data via the update subscription.
They were only initialized in case the setting to refresh manga metadata during an update was enabled.
However, this should always be done for uninitialized manga, regardless of the setting.
06bfc33e72 prevents uninitialized manga from getting filtered out, however, it did not ensure to initialize the manga
* Properly check for first page in cbz files
The download check for cbz files only checked if the archive existed but didn't check for the first page
* Streamline getImageImpl of ChapterDownloadProviders
* Exclude comic info file from page list
In case the download folder did not contain any page files, only the comic info file existed, which caused the download check to incorrectly detect the first page
* Add logging to ChapterForDownload#asDownloadReady
Manga can be added to the library while they have not been initialized yet.
In this case, depending on the manga exclusion setting, they will never be updated automatically unless they get refreshed once manually.
* Persist page count during chapter list update
In case a downloaded chapter gets deleted during a chapter list update, the download status was tried to be preserved.
However, in case the status could be preserved, the page count was lost and thus, the chapter now was marked as downloaded with a page count of -1.
* Mark downloaded chapters without page count as not downloaded
* Prevent adding duplicated chapters into the db
it's possible that the source returns a list containing chapters with the same url
once such duplicated chapters have been added, they aren't being removed anymore as long as there is
a chapter with the same url in the fetched chapter list, even if the duplicated chapter itself
does not exist anymore on the source
* Drop duplicated chapters from database table
* Add unique constraint to chapter table
This is to completely prevent duplicated chapters from being added to the database.
Since once a duplicated chapter has been added to the database, it does not get removed anymore as long as a chapter with the same url is included in the requested source chapter list
The automated backup cleanup just deleted every file (recursively in subfolders as well) in the set folder in case it was older than the set backup ttl.
This made it impossible to save the automated backups into a folder with different files.
Browsing a source loads only a minimal representation of a manga which does not include some metadata.
This metadata is only loaded when the specific manga gets fetched.
Thus, when the extra metadata of a manga was already loaded, it got removed when browsing the source and a page response included this manga
* Refresh track record only when logged in
In case one tracker was logged out, the refresh failed with an unauthenticated error and caused the other trackers to not get updated
* Prevent chapter track update from failing due to failure of other tracker
* Change level of log to "info"
* Properly prevent importing unsupported trackers from backup
Missed the early return in case no tracker record exists in the database in 2f362abb91be875e943b1364eb86d70a4144dd6f...
* Remove incorrect non null assertion
Prevented unbinding track records of unsupported trackers
* Prevent importing unsupported tracker from backup
This will lead to graphql field validation errors (non null declared field is null) once the track records get used, since they will point to trackers that do not exist
* Delete track records of unsupporter trackers
* Always return all track records of manga
Was already partially changed in 7df5f1c4c4 but this occurrence was missed
* Include tracking in validation of backup
* Always return track records
Not clear why an empty list should be returned in case no trackers are logged in
* Include tracking in backup creation
* Restore tracking from backup
In case the new chapters include duplicates from different scanlators, they would be included in the limit causing the auto download to potentially only download duplicated chapters while there might be more non duplicated chapters to download.
Instead, the limit should only consider unique chapters and then should include all duplicates of the chapters that should get downloaded
* Remove overrides of "ChapterFilesProvider::downloadImpl"
* Check final download folder for existing page on download
Downloads were changed to get downloaded to the system temp folder instead to directly into the final download folder.
This broke the check for existing pages, because now only the temp folder was checked instead of both the temp and the final download folder.
Regression introduced with 1c9a139006
* Properly check for already existing downloaded pages
The previous check was always false because the file ending of the page file is unknown and thus, missing from the created file path
* Cleanup cache download folder
* Update test/server-reference file
* Properly handle re-uploaded chapters in auto download of new chapters
In case of unhandable re-uploaded chapters (different chapter numbers) they potentially would have prevented auto downloads due being considered as unread.
Additionally, they would not have been considered to get downloaded due to not having a higher chapter number than the previous latest existing chapter before the chapter list fetch.
* Add option to ignore re-uploads for auto downloads
* Extract check for manga category download inclusion
* Extract logic to get new chapter ids to download
* Simplify manga category download inclusion check
In case the DEFAULT category does not exist, someone messed with the database and it is basically corrupted
* Add mutation to fetch the latest track data from the tracker
* Update Track.kt
---------
Co-authored-by: Mitchell Syer <Syer10@users.noreply.github.com>
