* Chapter fetch improvements
* Update previous date uploads
* Lint
* Fix backup inserts
* Remove extra maxSeenUploadDate
* Port downloaded over
* Make sure to set isDownloaded on all inserts
The function incorrectly exited early in case no latest read chapter was found.
This rendered disabling the setting "excludeEntryWithUnreadChapters" useless.
* Keep initial fetch date of existing chapters on a list fetch
The fetch at date should not get updated for existing chapters.
Updating this field makes it impossible to detect which chapters were actually newly fetched.
To get the last fetched timestamp of the chapters, the "chaptersLastFetchedAt" field of the manga should be used
got changed in 6d33d72663
* Get real chapter url safely
In case this causes an exception, it should not cause the whole list fetch to fail
was removed in 6d33d72663
* Reset backup status to idle in case of an exception
* Rename "performRestore" function
* Set backup status to failure on exception
Makes it possible to detect if the restore failed or not after the first status was received
* Set backup status to success on completion
Since the status is not provided over a subscription, but over a query that should be pulled, it is not really easily detectable if a restore finished or not, since both states will be indicated by "idle"
* Correctly wait for first new status when triggering backup import
The status is only "Idle" in case no backup import has ever run.
Once the first backup process finished it is either "Failure" or "Success"
* Rename "ProtoBackupImport::restore" function
* Add id to restore process
Makes it possible to differentiate between backup restore processes.
We do not allow any category to have the default categories name ("default" case-insensitive).
In case the backup includes a category with this name, it won't be matched to any category in our database and also won't insert a new category.
This will then lead to an exception, causing the backup to fail.
* Move running check to update function
* Move updating update status to process function
* Fail all source updates in case of update channel failure
In case the channel failed due to an exception, the update for the source failed completely.
This however was never handled and the pending updates for the source were never set to failed.
Due to this, the global updates running state was always true
* Remove completed update channel from available channels
* Always log specific update job failure
* Optionally fetch mangas during the update
The update only fetched the chapter list of a manga but never the manga itself.
Thus, e.g. unless the manga got online fetched via the ui, it would never get recognized if it is completed or not.
This would e.g. prevent the update setting, to not update completed mangas, from working as intended
* Make settings required
In case the database got deleted without deleting cached/downloaded thumbnails, the next time a manga gets inserted, it's possible that a thumbnail was already downloaded for its id.
This then causes mangas to be displayed with incorrect thumbnails due to using the outdated cached/downloaded thumbnails
In case download ahead is disabled, all new chapters should get downloaded.
Due to incorrectly calculating the index of the first new chapter to download, no new chapter was downloaded at all
* Address build warnings and cleanup
* Actual name of who defined the protocol
* Remove uneeded detekt supression
* GraphQL Before-After cleanup
* Lint
* Cleanup unused functions
* Fix some discrepancies with the 1.5 source api and fix lang exception
* Switch to new Ktlint plugin
* Add ktlintCheck to PR builds
* Run formatter
* Put ktlint version in libs toml
* Fix lint
* Use Zip4Java from libs.toml
* Add "download ahead" mutation
Checks if the specified number of unread chapters, that should be downloaded, are available.
In case not enough chapters are downloaded, the number of missing unread chapters will get downloaded
* Optionally pass the latest read chapter id of a manga
In case a chapter will get marked as read, which also triggered the download ahead call, it's possible, that by the time the download ahead logic gets triggered, the chapter hasn't been marked as read yet.
This could then cause this chapter to be included in the chapters to get downloaded.
By providing the chapter id, this chapter will be used as the latest read chapter instead, and thus, not be included inn the chapters to download.
In case a newer version of the extension is installed and the extension gets manually downgraded, the version in db is still the one of the newer version.
This will prevent detection of available updates, since it won't get recognized, that an older version is currently installed.
Chapters were added to the queue by database index order.
In case a chapters of different mangas got added to the queue, downloads got mingled instead of being group inserted per manga.
Also sort manga chapters by source order, to make sure, that, in case chapters of a manga are, for some reason, not in the correct order in the database, they will still get downloaded in the order of the source.
Since the number of chapters gets converted to be index based, 1 available chapter would result in 0.
Due to this, in case a manga had exactly one chapter before updating the chapters, it was incorrectly detected as the initial fetch and the new chapters did not get automatically downloaded.
Flow::stateIn has "Strong equality-based conflation" (see documentation).
Thus, it omits every value in case it's equal to the previous one.
Since the DownloadManger::getStatus function returns a status with a queue, that contains all current "DownloadChapters" by reference, the equality check was always true.
Thus, progress changes of downloads were never sent to subscribers.
Subscriber were only notified about finished downloads (size of queue changed) or downloader status changes
In case a download was finished, but the downloader got stopped before it was able to remove the finished download from the queue, the downloader got stuck in an endless loop of starting and pausing downloads.
This was caused by selecting the next chapter to download and then recognizing in "Downloader::step", that there is another chapter to download before the current one in the queue.
However, since this recognized chapter is already downloaded, the downloader selected the next queued chapter again.
It was then stuck in this loop until the finished chapter was manually removed from the queue.
* Rename "newChapters" to "updatedChapterList"
* Do not auto download new chapters of entries with unread chapters
Makes it possible to prevent unnecessary chapter downloads in case the entry hasn't yet been caught up
* Optionally limit auto new chapter downloads
* Prevent downloading new chapters for mangas not in the library
* Create manga download dir in case it's missing for cbz downloads
The directory, in which the cbz file should have been saved in, was never created.
* Correctly copy chapter download to final download location
"renameTo" does not include the content of a directory.
Thus, it just created an empty chapter folder int the final download directory
* Move chapter download logic to base class
* Do not reuse "FolderProvider" in "ArchiveProviders" download function
Due to reusing the "FolderProvider" to download a chapter as a cbz file, a normal chapter download folder was created.
In case the download was aborted before the cbz file got created and the folder deleted, the next time the chapter got downloaded, the wrong "FileProvider" was selected, causing the chapter not to get downloaded as a cbz file.
* Update chapter page refresh logic with logic from "ChapterMutation"
* Rename function to "getChapterDownloadReadyByIndex"
* Update "ChapterForDownload" to work with only "chapterId" being passed
* Return database chapter page list in case chapter is downloaded
In case the chapter is downloaded, fetching the chapter pages info should not be needed.
It should also currently break reading downloaded chapters while being offline, since the page request will always fail, since there is no internet connection
* Provide last global update timestamp
* Provide skipped mangas in update status
* Extract update status logic into function
* Rename update "statusMap" to "mangaStatusMap"
* Provide info about categories in update status
* Make server config value changes subscribable
* Make server config value changes subscribable - Update usage
* Add util functions to listen to server config value changes
* Listen to server config value changes - Auto backups
* Listen to server config value changes - Auto global update
* Listen to server config value changes - WebUI auto updates
* Listen to server config value changes - Javalin update ip and port
* Listen to server config value changes - Update socks proxy
* Listen to server config value changes - Update debug log level
* Listen to server config value changes - Update system tray icon
* Update config values one at a time
In case settings are changed in quick succession it's possible that each setting update reverts the change of the previous changed setting because the internal config hasn't been updated yet.
E.g.
1. settingA changed
2. settingB changed
3. settingA updates config file
4. settingB updates config file (internal config hasn't been updated yet with change from settingA)
5. settingA updates internal config (settingA updated)
6. settingB updates internal config (settingB updated, settingA outdated)
now settingA is unchanged because settingB reverted its change while updating the config with its new value
* Always add log interceptor to OkHttpClient
In case debug logs are disabled then the KotlinLogging log level will be set to level > debug and thus, these logs won't get logged
* Rename "maxParallelUpdateRequests" to "maxSourcesInParallel"
* Use server setting "maxSourcesInParallel" for downloads
* Listen to server config value changes - downloads
* Always use latest server settings - Browser
* Always use latest server settings - folders
* [Test] Fix type error
Gets already called by "Chapter::fetchChapterList", thus, this is unnecessary.
Additionally, "chapters.toList()" and "chapters.map()" have to be called in a transaction block, which they are not, and thus, cause an unhandled exception, breaking the mutation
There were cases where the last page read was greater than the max page count of a chapter.
This is not possible and is just invalid data, that is saved in the database, possible leading to other errors down the line.
This could happen in case the chapter was loaded at some point with e.g. 18 pages and after some time got fetched again from the source, now with fewer pages than before e.g. 15.
If the chapters last page was already read by that time, the last read page would have been 18, while the chapter now has only 15 pages.
* Rename "DownloadedFilesProvider" to "ChaptersFilesProvider"
* Move files into sub packages
* Further abstract "DownloadedFilesProvider"
* Rename "getCachedImageResponse" to "getImageResponse"
* Extract getting cached image response into new function
* Decouple thumbnail cache and download
* Download and delete permanent thumbnails
When adding/removing manga from/to the library make sure the permanent thumbnail files will get handled properly
* Move thumbnail cache to actual temp folder
* Rename "mangaDownloadsRoot" to "downloadRoot"
* Move manga downloads into "mangas" subfolder
* Clear downloaded thumbnail
* Get rid of multiple static "assets/" usage
* Correctly add new zip entry
The name of the entry has to be a "/" separated path, otherwise, the files can't be found.
* Extract reorder logic into function
* Save download queue everytime a download was finished
The download queue was never saved after a download was finished.
This caused finished download to be restored on a server start, which caused unnecessary "downloads" which most of the time would just finish immediately since the pages were still in the cache
* Wait for download queue save process to be finished
Since multiple downloaders could be finished at the same time, the download queue should be saved synchronously
* Remove unnecessary download queue save trigger
This gets called everytime a downloader finished downloading all chapters of its source.
Since the queue is now saved everytime a download is finished, this is trigger is not needed anymore
* Log extension load failure
In case the extension couldn't be loaded the error was never logged, making it impossible to analyse what was going on
* Log exception in "GetCatalogueSource:: getCatalogueSourceOrNull"
In case "GetCatalogueSource::getCatalogueSource" threw an error, this was never logged here
In case the file could not be retrieved, the page retrieve just failed and wasn't triggered again.
In case of the downloader, the chapter download just kept failing 3 times and was aborted
* Change log level of download error
* Change type of sourceId in Downloader
Unclear why it was converted to Long since it just got converted back to String anyway when it was used in the Downloader
* Only stop downloads from source of the Downloader
The downloader just changed the state of all downloads, ignoring if they are from the source the Downloader is for or not
* Remove unnecessary DownloadManager::start calls
In case chapters were added to the queue the DownloadManager will start itself
* Extract download filtering into property
* Improve Downloader logging
* Notify clients only in case Downloader was started
In case nothing was done there is nothing to notify about
* Do not start Downloaders for failed downloads
In case there were failed chapter downloads in the queue the DownloadManager still created a Downloader and started it.
This Downloader would than immediately call "onComplete", since there is no available download, which then would refresh the Downloaders again which created an infinite loop until the failed download got removed from the queue
* Retry download in case it failed it gets re-added to the queue
In case a failed downloaded that was still in the queue was tried to get added to the queue again, nothing happened.
Instead of doing nothing, the download should get retried.
Thus, it also provides the logic to easily retry a failed download by just "adding" the chapter to the queue again.
Currently, to retry a failed download, the download has to be removed from the queue and then get re-added.
* Rename function "unqueue" to "dequeue"
* Move "dequeue" function
* Extract dequeue logic into function
* Improve DownloadManager logging
* Override "toString" of DownloadChapter
See documentation (%/rem, mod) for differences.
Example for "issue" that occurred:
mathematical: -4 % 6 = 2 (expected)
kotlin: -4 % 6 = -4 (unexpected)
* Trigger missed auto global update immediately on server start
In case the last execution was missed, it was never immediately scheduled.
Thus, it had to be waited for the next scheduled execution to be executed.
* Schedule auto global updates at a later point during the startup
In case a global update was triggered immediately, the server setup wasn't far enough causing an error due to trying to use things (e.g. database) that weren't initialized yet
