* Implement SyncYomi
* Add ability to select what to sync
* Properly fix default category bug
* Add periodic sync
* Add PostgreSQL support
* Deschedule previous task
* Check if SyncYomi is enabled in syncData function
* Don't allow multiple syncs at the same time
* Convert SyncYomiSyncService to object
* Make startSync non-suspend
* Return a result from startSync
* Sync before library update
* Improvements
* Use NetworkHelper client
* Lint
* Use measureTime
* Database improvements
- Move entire sync operation into a single transaction
- Stop loading all manga to memory
* Revert "Database improvements"
This reverts commit bee8d214c3.
* Actual database improvements
* Remove runBlocking
* Remove title check
* Update updateNonFavorites function
* Update timeout code
* Improve PostgreSQL query
* Create lastSyncState variable
* Create lastSyncStatus query
* Convert lastSyncState to StateFlow
* Create lastSyncStatusChange subscription
* Replace backupRestoreStatus with backupRestoreId
* Add startDate and endDate
* Add logs for sync start and end
* Handle all errors in syncData
* Change category restore function to match Mihon's behavior
* Fix comment
* Remove duplicate BackupMangaHandler.backup call
* Remove duplicated log
* Rename subscription to syncStatusChanged
* Use same flags for restoring
* Update syncInterval config to use DurationSetting
* Update sync scheduling logic
* Reorder conditions to reduce database calls
* Prevent deleted ghost chapters from reappearing during sync
jobobby04/TachiyomiSY#1575
* Improve sync merging categories
jobobby04/TachiyomiSY#1559
* Make columns not null
* Improve H2 triggers
* Add documentation
* Cleanup graphql setting mutation
* Validate values read from config
* Generate server-reference.conf files from ServerConfig
* Remove unnecessary enum value handling in config value update
Commit df0078b725 introduced the usage of config4k, which handles enums automatically. Thus, this handling is outdated and not needed anymore
* Generate gql SettingsType from ServerConfig
* Extract settings backup logic
* Generate settings backup files
* Move "group" arg to second position
To make it easier to detect and have it at the same position consistently for all settings.
* Remove setting generation from compilation
* Extract setting generation code into new module
* Extract pure setting generation code into new module
* Remove generated settings files from src tree
* Force each setting to set a default value
* Fix setting initial global update delay
In case no update has run yet, and the "last automated update" defaulted to 0, the calculation always resulted in a multiple of the interval.
This resulted for e.g., interval 24, to always be scheduled at 00:00.
For e.g., interval 6, it was always one of the following times: 00:00, 06:00, 12:00, 18:00; depending on the current system time.
* Delete the existing "last automated update time"
So that the update will be triggered based on the time the server got started again after the update.
* Extract migrations into separate functions
* Cleanup migration execution logic
The update subscription emitted the full update status, which, depending on how big the status was, took forever because the graphql subscription does not support data loader batching, causing it to run into the n+1 problem
* Update to exposed-migrations v3.5.0
* Update to kotlin-logging v7.0.0
* Update to exposed v0.46.0
* Update to exposed v0.47.0
* Update to exposed v0.55.0
* Update to exposed v0.56.0
* Update to exposed v0.57.0
* Launch missed auto backup task in background
* Launch missed auto global update task in background
* Launch missed auto webui update check task in background
In case a manga gets added to the library which has not been initialized yet, it should be tried to initialize it.
Since it's not an error to have uninitialized manga in the library, this can be done in the background via the updater and the client receives the updated data via the update subscription.
They were only initialized in case the setting to refresh manga metadata during an update was enabled.
However, this should always be done for uninitialized manga, regardless of the setting.
06bfc33e72 prevents uninitialized manga from getting filtered out, however, it did not ensure to initialize the manga
Manga can be added to the library while they have not been initialized yet.
In this case, depending on the manga exclusion setting, they will never be updated automatically unless they get refreshed once manually.
* Set updater running flag to false only at the end of the update
For clearing the data loader cache properly, the update status subscription requires the update to be running.
For the last completed manga update the flag was immediately set to false which prevented the dataloader cache from getting cleared, returning outdated data for the last updated manga
* Correctly clear the "MangaForIdsDataLoader" cache
The cache keys for this dataloader are lists of manga ids.
Thus, it is not possible to clear only the cached data of the provided manga id and instead each cache entry that includes the manga id has to be cleared
* Ensure that manga dataloader caches gets cleared during global update
The "StateFlow" drops value updates in case the collector is too slow, which was the case for the "UpdateSubscription".
This caused the dataloader cache to not get properly cleared because the running state of the update was already set to false.
* Remove log of mangas to update
This logged the full manga data objects in the list with information that is not needed (e.g. description of a manga).
Once a manga gets updated via the updater, it gets logged, which should be enough
* Include manga id in updater log
* Use "toString" to log mangas
* Change "HttpLoggingInterceptor" level to "BASIC"
Was unintentionally merged with d658e07583
* Rename IncludeInUpdate class to IncludeOrExclude
Signed-off-by: Chance Zibolski <chance.zibolski@gmail.com>
* Add support for configuring which categories are downloaded automatically
If a manga has no configured categories, behavior remains the same and
the automatic download functionality will download new chapters without
consulting the category includeInDownload configuration.
Signed-off-by: Chance Zibolski <chance.zibolski@gmail.com>
---------
Signed-off-by: Chance Zibolski <chance.zibolski@gmail.com>
* Trigger initial auto backup in case server was not running
In case the server was not started (stopped, system shutdown - not in hibernation) during the scheduled auto backup time, the auto backup never got triggered.
* Update server util preferences
* Move running check to update function
* Move updating update status to process function
* Fail all source updates in case of update channel failure
In case the channel failed due to an exception, the update for the source failed completely.
This however was never handled and the pending updates for the source were never set to failed.
Due to this, the global updates running state was always true
* Remove completed update channel from available channels
* Always log specific update job failure
* Optionally fetch mangas during the update
The update only fetched the chapter list of a manga but never the manga itself.
Thus, e.g. unless the manga got online fetched via the ui, it would never get recognized if it is completed or not.
This would e.g. prevent the update setting, to not update completed mangas, from working as intended
* Make settings required
* Switch to new Ktlint plugin
* Add ktlintCheck to PR builds
* Run formatter
* Put ktlint version in libs toml
* Fix lint
* Use Zip4Java from libs.toml
* Provide last global update timestamp
* Provide skipped mangas in update status
* Extract update status logic into function
* Rename update "statusMap" to "mangaStatusMap"
* Provide info about categories in update status
* Make server config value changes subscribable
* Make server config value changes subscribable - Update usage
* Add util functions to listen to server config value changes
* Listen to server config value changes - Auto backups
* Listen to server config value changes - Auto global update
* Listen to server config value changes - WebUI auto updates
* Listen to server config value changes - Javalin update ip and port
* Listen to server config value changes - Update socks proxy
* Listen to server config value changes - Update debug log level
* Listen to server config value changes - Update system tray icon
* Update config values one at a time
In case settings are changed in quick succession it's possible that each setting update reverts the change of the previous changed setting because the internal config hasn't been updated yet.
E.g.
1. settingA changed
2. settingB changed
3. settingA updates config file
4. settingB updates config file (internal config hasn't been updated yet with change from settingA)
5. settingA updates internal config (settingA updated)
6. settingB updates internal config (settingB updated, settingA outdated)
now settingA is unchanged because settingB reverted its change while updating the config with its new value
* Always add log interceptor to OkHttpClient
In case debug logs are disabled then the KotlinLogging log level will be set to level > debug and thus, these logs won't get logged
* Rename "maxParallelUpdateRequests" to "maxSourcesInParallel"
* Use server setting "maxSourcesInParallel" for downloads
* Listen to server config value changes - downloads
* Always use latest server settings - Browser
* Always use latest server settings - folders
* [Test] Fix type error
See documentation (%/rem, mod) for differences.
Example for "issue" that occurred:
mathematical: -4 % 6 = 2 (expected)
kotlin: -4 % 6 = -4 (unexpected)
* Trigger missed auto global update immediately on server start
In case the last execution was missed, it was never immediately scheduled.
Thus, it had to be waited for the next scheduled execution to be executed.
* Schedule auto global updates at a later point during the startup
In case a global update was triggered immediately, the server setup wasn't far enough causing an error due to trying to use things (e.g. database) that weren't initialized yet
* Rename schedule functions
* Introduce Base task for "HATask"
* Support kotlin Timer repeated interval in HAScheduler
It's not possible to schedule a task via cron expression to run every x hours in case the set hours are greater than 23.
To be able to do this and still keep the functionality provided by the "HAScheduler" it has to also support repeated tasks scheduled via the default Timer
* Support global update interval greater 23 hours
* Use "globalUpdateInterval" to disable auto updates
Gets rid of an unnecessary setting
* Add option to disable cleanup of backups
* Ensure the minimum TTL of backups to 1 day
* Schedule the automated backup on a specific time of the day
* Introduce scheduler that takes system hibernation time into account
In case the system was hibernating/suspended scheduled task that should have been executed during that time would not get triggered and thus, miss an execution.
To prevent this, this new scheduler periodically checks if the system was suspended and in case it was, triggers any task that missed its last execution
* Use new scheduler
Makes it possible to only update specific categories.
In case a manga is in an excluded category it will be excluded even if it is also in an included category.
