* Validate setting values on mutation
* Handle invalid negative setting values
* Ensure at least one source is downloading at all times
* Prevent possible IllegalArgumentException
The "serverConfig.maxSourcesInParallel" value could have changed after the if-condition
* Persist page count during chapter list update
In case a downloaded chapter gets deleted during a chapter list update, the download status was tried to be preserved.
However, in case the status could be preserved, the page count was lost and thus, the chapter now was marked as downloaded with a page count of -1.
* Mark downloaded chapters without page count as not downloaded
* Prevent adding duplicated chapters into the db
it's possible that the source returns a list containing chapters with the same url
once such duplicated chapters have been added, they aren't being removed anymore as long as there is
a chapter with the same url in the fetched chapter list, even if the duplicated chapter itself
does not exist anymore on the source
* Drop duplicated chapters from database table
* Add unique constraint to chapter table
This is to completely prevent duplicated chapters from being added to the database.
Since once a duplicated chapter has been added to the database, it does not get removed anymore as long as a chapter with the same url is included in the requested source chapter list
In case the new chapters include duplicates from different scanlators, they would be included in the limit causing the auto download to potentially only download duplicated chapters while there might be more non duplicated chapters to download.
Instead, the limit should only consider unique chapters and then should include all duplicates of the chapters that should get downloaded
* Update test/server-reference file
* Properly handle re-uploaded chapters in auto download of new chapters
In case of unhandable re-uploaded chapters (different chapter numbers) they potentially would have prevented auto downloads due being considered as unread.
Additionally, they would not have been considered to get downloaded due to not having a higher chapter number than the previous latest existing chapter before the chapter list fetch.
* Add option to ignore re-uploads for auto downloads
* Extract check for manga category download inclusion
* Extract logic to get new chapter ids to download
* Simplify manga category download inclusion check
In case the DEFAULT category does not exist, someone messed with the database and it is basically corrupted
* Remove download ahead logic
Unnecessary on server side, should just be done by the client
* Rename "autoDownloadAheadLimit" to "autoDownloadNewChaptersLimit"
* Deprecate the old field
* Update Stable WebUI
* Update Stable WebUI
---------
Co-authored-by: Syer10 <syer10@users.noreply.github.com>
* Rename IncludeInUpdate class to IncludeOrExclude
Signed-off-by: Chance Zibolski <chance.zibolski@gmail.com>
* Add support for configuring which categories are downloaded automatically
If a manga has no configured categories, behavior remains the same and
the automatic download functionality will download new chapters without
consulting the category includeInDownload configuration.
Signed-off-by: Chance Zibolski <chance.zibolski@gmail.com>
---------
Signed-off-by: Chance Zibolski <chance.zibolski@gmail.com>
* add trackers support
* Cleanup Tracker Code
* Add GraphQL support for Tracking
* Fix lint and deprecation errors
* remove password from logs
* Fixes after merge
* Disable tracking for now
* More disabled
---------
Co-authored-by: Syer10 <syer10@users.noreply.github.com>
Subtracting 1 from the first chapter to download index caused an additional chapter to get downloaded (e.g. limit 4 would download 5 chapters)
Regression was introduced with 05bf4f5525
* Fix automatic chapter download for initial chapter list fetch
The initial fetch wasn't correctly detected which caused chapters to get downloaded.
Using index based numbers also caused the first chapter to not get downloaded due to it being omitted in the "subList" call which excludes the "toIndex".
* [Logging] Update logs
* Chapter fetch improvements
* Update previous date uploads
* Lint
* Fix backup inserts
* Remove extra maxSeenUploadDate
* Port downloaded over
* Make sure to set isDownloaded on all inserts
The function incorrectly exited early in case no latest read chapter was found.
This rendered disabling the setting "excludeEntryWithUnreadChapters" useless.
* Keep initial fetch date of existing chapters on a list fetch
The fetch at date should not get updated for existing chapters.
Updating this field makes it impossible to detect which chapters were actually newly fetched.
To get the last fetched timestamp of the chapters, the "chaptersLastFetchedAt" field of the manga should be used
got changed in 6d33d72663
* Get real chapter url safely
In case this causes an exception, it should not cause the whole list fetch to fail
was removed in 6d33d72663
In case download ahead is disabled, all new chapters should get downloaded.
Due to incorrectly calculating the index of the first new chapter to download, no new chapter was downloaded at all
* Switch to new Ktlint plugin
* Add ktlintCheck to PR builds
* Run formatter
* Put ktlint version in libs toml
* Fix lint
* Use Zip4Java from libs.toml
Since the number of chapters gets converted to be index based, 1 available chapter would result in 0.
Due to this, in case a manga had exactly one chapter before updating the chapters, it was incorrectly detected as the initial fetch and the new chapters did not get automatically downloaded.
* Rename "newChapters" to "updatedChapterList"
* Do not auto download new chapters of entries with unread chapters
Makes it possible to prevent unnecessary chapter downloads in case the entry hasn't yet been caught up
* Optionally limit auto new chapter downloads
* Prevent downloading new chapters for mangas not in the library
* Make server config value changes subscribable
* Make server config value changes subscribable - Update usage
* Add util functions to listen to server config value changes
* Listen to server config value changes - Auto backups
* Listen to server config value changes - Auto global update
* Listen to server config value changes - WebUI auto updates
* Listen to server config value changes - Javalin update ip and port
* Listen to server config value changes - Update socks proxy
* Listen to server config value changes - Update debug log level
* Listen to server config value changes - Update system tray icon
* Update config values one at a time
In case settings are changed in quick succession it's possible that each setting update reverts the change of the previous changed setting because the internal config hasn't been updated yet.
E.g.
1. settingA changed
2. settingB changed
3. settingA updates config file
4. settingB updates config file (internal config hasn't been updated yet with change from settingA)
5. settingA updates internal config (settingA updated)
6. settingB updates internal config (settingB updated, settingA outdated)
now settingA is unchanged because settingB reverted its change while updating the config with its new value
* Always add log interceptor to OkHttpClient
In case debug logs are disabled then the KotlinLogging log level will be set to level > debug and thus, these logs won't get logged
* Rename "maxParallelUpdateRequests" to "maxSourcesInParallel"
* Use server setting "maxSourcesInParallel" for downloads
* Listen to server config value changes - downloads
* Always use latest server settings - Browser
* Always use latest server settings - folders
* [Test] Fix type error
Gets already called by "Chapter::fetchChapterList", thus, this is unnecessary.
Additionally, "chapters.toList()" and "chapters.map()" have to be called in a transaction block, which they are not, and thus, cause an unhandled exception, breaking the mutation
* Change log level of download error
* Change type of sourceId in Downloader
Unclear why it was converted to Long since it just got converted back to String anyway when it was used in the Downloader
* Only stop downloads from source of the Downloader
The downloader just changed the state of all downloads, ignoring if they are from the source the Downloader is for or not
* Remove unnecessary DownloadManager::start calls
In case chapters were added to the queue the DownloadManager will start itself
* Extract download filtering into property
* Improve Downloader logging
* Notify clients only in case Downloader was started
In case nothing was done there is nothing to notify about
* Do not start Downloaders for failed downloads
In case there were failed chapter downloads in the queue the DownloadManager still created a Downloader and started it.
This Downloader would than immediately call "onComplete", since there is no available download, which then would refresh the Downloaders again which created an infinite loop until the failed download got removed from the queue
* Retry download in case it failed it gets re-added to the queue
In case a failed downloaded that was still in the queue was tried to get added to the queue again, nothing happened.
Instead of doing nothing, the download should get retried.
Thus, it also provides the logic to easily retry a failed download by just "adding" the chapter to the queue again.
Currently, to retry a failed download, the download has to be removed from the queue and then get re-added.
* Rename function "unqueue" to "dequeue"
* Move "dequeue" function
* Extract dequeue logic into function
* Improve DownloadManager logging
* Override "toString" of DownloadChapter
* Separate cache dir from download dir
* Move downloader logic outside of caching/image download logic
* remove unnecessary method duplication
* moved download logic inside download provider
* optimize and handle partial downloads
* made code review changes
* Support extensions lib 1.4
* Fix build
* Support UpdateStrategy
* Update extension lib min/max to match Tachiyomi
* Use HttpSource.getMangaUrl and add Chapter.realUrl
* Add new endpoint for batch editing any chapter
* Add option to batch editing chapters to delete chapter (remove downloaded content)
* Rename the endpoint to match single manga batch endpoint
* Do not return early, in case there are other changes
* PR changes
* Add POST /downloads endpoint for creating multiple
* Fix review notes
* Add chapter id to API endpoints
* Rewrite batch chapter download to use chapter id instead of mangaId+chapterIndex combination
* Change EnqueueInput format to be more futureproof
* Change endpoint path
* Change endpoint path
* Add lastFetchedAt and chaptersLastFetchedAt columns to manga
* Update lastFetchedAt columns when data are fetched from source
* Add age and chaptersAge fields to MangaDataClass
* Replace two migrations with single migration
