Exposes the existing push and pull functionality from the KoreaderSyncService via the GraphQL API.
This change introduces two new mutations:
- `pushKoSyncProgress`: Manually sends the current chapter's reading progress to the KOReader sync server.
- `pullKoSyncProgress`: Manually fetches and applies the latest reading progress from the KOReader sync server.
These mutations enable clients and WebUIs to implement manual sync triggers, providing users with more direct control over their reading progress synchronization, similar to the functionality offered by the official KOReader plugin and other clients like Readest.
* Support PostgreSQL Databases
* Set the database Schema
* See if we can test postgres
* Another test
* Disable node container
* Update database when changed
* Simplify test workflow
* Only exit on failed migrations
* Run the first databaseUp sync
* Map the port
* Use absolute path for LD_PRELOAD
* Timeout after 1m
* Open the server in both database configurations
* Only exit on migration failed in ci
* Lint
* Use new ServerConfig configuration
* Cleanup graphql setting mutation
* Validate values read from config
* Generate server-reference.conf files from ServerConfig
* Remove unnecessary enum value handling in config value update
Commit df0078b725 introduced the usage of config4k, which handles enums automatically. Thus, this handling is outdated and not needed anymore
* Generate gql SettingsType from ServerConfig
* Extract settings backup logic
* Generate settings backup files
* Move "group" arg to second position
To make it easier to detect and have it at the same position consistently for all settings.
* Remove setting generation from compilation
* Extract setting generation code into new module
* Extract pure setting generation code into new module
* Remove generated settings files from src tree
* Force each setting to set a default value
* Basic JWT implementation
* Move JWT to UI_LOGIN mode and bring back SIMPLE_LOGIN as before
* Update server/src/main/kotlin/suwayomi/tachidesk/global/impl/util/Jwt.kt
Co-authored-by: Mitchell Syer <Syer10@users.noreply.github.com>
* Refresh: Update only access token
Co-authored-by: Mitchell Syer <Syer10@users.noreply.github.com>
* Implement JWT Audience
* Store JWT key
Generates the key on startup if not set
* Handle invalid Base64
* Make JWT expiry configurable
* Missing value parse
* Update server/src/main/kotlin/suwayomi/tachidesk/global/impl/util/Jwt.kt
Co-authored-by: Mitchell Syer <Syer10@users.noreply.github.com>
* Simplify Duration parsing
* JWT Protect Mutations
* JWT Protect Queries and Subscriptions
* JWT Protect v1 WebSockets
* WebSockets allow sending token via protocol header
* Also respect the `suwayomi-server-token` cookie
* JWT reduce default token expiry
* JWT Support cookie on WebSocket as well
* Lint
* Authenticate graphql subscription via connection_init payload
* WebView: Prefer explicit token over cookie
This hack was implemented because WebView sent `"null"` if no token was
supplied, just don't send a bad token, then we can do this properly
* WebView: Implement basic login dialog if no token supplied
---------
Co-authored-by: Mitchell Syer <Syer10@users.noreply.github.com>
Co-authored-by: schroda <50052685+schroda@users.noreply.github.com>
* feat(sync/koreader): implement reading progress synchronization
This commit introduces a comprehensive integration with KOReader Sync Server to enable two-way synchronization of reading progress.
The core logic is encapsulated in a new `KoreaderSyncService`, which handles authentication, registration, and progress pushing/pulling based on user-defined strategies (LATEST, KOSYNC, SUWAYOMI).
Key changes include:
- A new GraphQL API is added to manage the KOReader Sync connection:
- `connectKoSyncAccount` mutation provides a simplified flow that attempts to log in and, if the user doesn't exist, automatically registers them.
- `logoutKoSyncAccount` mutation to clear credentials.
- `koSyncStatus` query to check the current connection status.
- Reading progress is now synchronized at key points:
- The `fetchChapterPages` mutation pulls the latest progress from the sync server before loading the reader. It respects the configured sync strategy and updates the local database if necessary.
- The `updateChapters` and other progress-updating methods now push changes to the sync server automatically.
- OPDS chapter entries also pull the latest progress, ensuring clients receive up-to-date reading status.
- Supporting backend changes have been made:
- The `Chapter` table is extended with a `koreader_hash` column to uniquely identify documents. A database migration is included.
- New configuration options are added to `server.conf` to manage the feature (enable, server URL, credentials, strategy, etc.).
* perf(opds): defer KOReader sync to improve chapter feed performance
Removes the KOReader Sync progress-pulling logic from the `createChapterListEntry` function.
The previous implementation triggered a network request to the sync server for every single chapter when generating a list, leading to severe performance issues and slow load times on feeds with many entries.
This change reverts to the more performant approach of always linking to the chapter's metadata feed. The expensive sync operation will be handled within the metadata entry generation instead, ensuring it's only triggered on-demand for a single chapter. This restores the responsiveness of browsing chapter feeds.
* refactor(koreader): Use enums for sync settings and correct OPDS logic
Refactor Koreader Sync settings to use enums instead of raw strings for `checksumMethod` and `strategy`. This improves type safety, prevents typos, and makes the configuration handling more robust.
The changes include:
- Introducing `KoreaderSyncChecksumMethod` and `KoreaderSyncStrategy` enums.
- Updating `ServerConfig`, GraphQL types, and backup models to use these new enums.
- Refactoring `KoreaderSyncService` to work with the enum types.
Additionally, this commit fixes an issue in `OpdsEntryBuilder` where the logic for determining which sync progress to use (local vs. remote) was duplicated. The builder now correctly delegates this decision to `KoreaderSyncService.pullProgress`, which already contains the necessary strategy logic. This centralizes the logic and ensures consistent behavior.
* refactor(koreader): Improve config handling and remove redundant update
This commit combines several refactoring and cleanup tasks:
- **Koreader Sync:** The sync service is updated to use the modern `serverConfig` provider instead of the legacy `GlobalConfigManager`. This aligns it with the current configuration management approach in the project.
- **Download Provider:** A redundant `pageCount` database update is removed from `ChaptersFilesProvider`. This operation was unnecessary because the `getChapterDownloadReady` function, which is called earlier in the download process, already verifies and corrects the page count. This change eliminates a superfluous database write and fixes a related import issue.
* feat(sync/koreader)!: enhance sync strategy and add progress tolerance
This commit overhauls the KOReader synchronization feature to provide more granular control and robustness. The simple on/off toggle has been replaced with a more flexible strategy-based system.
Key changes include:
- Replaced `koreaderSyncEnabled` with a more powerful `koreaderSyncStrategy` enum.
- Introduced new sync strategies: `PROMPT`, `SILENT`, `SEND`, `RECEIVE`, and `DISABLE`, allowing for fine-grained control over the sync direction and conflict resolution.
- Added a `koreaderSyncPercentageTolerance` setting. This prevents unnecessary sync updates for minor progress differences between Suwayomi and KOReader.
- Refactored the `KoreaderSyncService` to implement the new strategies and use the configurable tolerance.
- Updated GraphQL schemas, mutations, and server configuration to remove the old setting and incorporate the new ones.
- Adjusted the backup and restore process to correctly handle the new configuration parameters.
- Modified API endpoints and internal logic to check and apply remote progress based on the selected strategy.
BREAKING CHANGE: The `koreaderSyncEnabled` setting is removed and replaced by a more granular `koreaderSyncStrategy`. The enum values for the strategy have been completely changed, making previous configurations incompatible.
* fix: remove unused imports
* feat(opds, sync): enhance Koreader sync and OPDS conflict handling
This commit introduces significant improvements to the Koreader synchronization feature, focusing on providing a better user experience for handling progress conflicts in both OPDS and GraphQL clients.
Key changes include:
- **OPDS Conflict Resolution:** When a reading progress conflict is detected, the OPDS feed for a chapter now provides two distinct "Read Online" links: one to continue from the local progress and another to continue from the synced progress from the remote device (e.g., "Continue Reading Online (Synced from KOReader)"). This empowers users to choose which progress to follow.
- **GraphQL Sync Conflict Information:** The `fetchChapterPages` GraphQL mutation now includes a `syncConflict` field in its payload. This field provides the remote device name and page number, allowing GraphQL clients to implement a user-facing prompt to resolve sync conflicts.
- **Improved Sync Strategy Handling:**
- The `connectKoSyncAccount` mutation no longer unconditionally sets the sync strategy to `PROMPT`. It now respects the user's existing setting, preventing accidental configuration changes upon re-login.
- The default `koreaderSyncStrategy` in the configuration is changed to `DISABLED`, providing a safer and more intuitive default for new users.
- **Refinements & Fixes:**
- The fallback for the remote device name is now centralized within the KoreaderSyncService, defaulting to "KOReader" if not provided.
- Renamed `KoreaderSyncStrategy.DISABLE` to `DISABLED` for consistency.
- Updated i18n strings for OPDS links to be more descriptive and user-friendly.
* refactor(kosync): rename stream page link titles for consistency
* refactor(kosync): return SettingsType in auth mutation payloads
The `connectKoSyncAccount` and `logoutKoSyncAccount` mutations modify server settings (username and userkey) but did not previously return the updated configuration. This forced client applications to manually refetch settings to avoid a stale cache.
This change modifies the payloads for both mutations to include the full `SettingsType`.
By returning the updated settings directly, GraphQL clients like Apollo Client can automatically update their cache, simplifying client-side state management and ensuring the UI always reflects the current server configuration.
Additionally, `clientMutationId` has been added to `KoSyncConnectPayload` for consistency with GraphQL practices, aligning it with the logout mutation.
Refs: #1560
* refactor(kosync): replace KoSyncConnectPayload with ConnectResult in connect method
* fix(kosync): add koreaderSyncPercentageTolerance default setting
* [#1496] First conversion attempt
* [#1496] Configurable conversion
* Fix: allow nested configs (map)
* [#1496] Support explicit `none` conversion
* Use MimeUtils for provided download
* [1496] Support image conversion on load for downloaded images
* Lint
* [#1496] Support conversion on fresh download as well
Previous commit was only for already downloaded images, now also for
fresh and cached
* [#1496] Refactor: Move where conversion for download happens
* Rewrite config handling, improve custom types
* Lint
* Add format to pages mutation
* Lint
* Standardize url encode
* Lint
* Config: Allow additional conversion parameters
* Implement conversion quality parameter
* Lint
* Implement a conversion util to allow fallback readers
* Add downloadConversions to api and backup, fix updateValue issues
* Lint
* Minor cleanup
* Update libs.versions.toml
---------
Co-authored-by: Syer10 <syer10@users.noreply.github.com>
* [#1349] Stub basic cookie authentication
* [#1349] Basic login page
Also adjusts WebView header color and shadow to match WebUI. WebUI uses
a background-image gradient to change the perceived color, which was not
noticed originally.
* [#1349] Handle login post
* [#1349] Redirect to previous URL
* [#1349] Return a basic 401 for api endpoints
Instead of redirecting to a visual login page, API should just indicate
the bad state
* Use more appropriate 303 redirect
* Update server/src/main/kotlin/suwayomi/tachidesk/server/JavalinSetup.kt
Co-authored-by: Mitchell Syer <Syer10@users.noreply.github.com>
* Update server/src/main/kotlin/suwayomi/tachidesk/server/JavalinSetup.kt
Co-authored-by: Mitchell Syer <Syer10@users.noreply.github.com>
* Lint
* Transition to AuthMode enum with migration path
* Make basicAuthEnabled auto property, Lint
* ConfigManager: Make sure to re-parse the config after migration
* basicAuth{Username,Password} -> auth{Username,Password}
* Lint
* Update server settings backup model
* Update comment
* Minor cleanup
* Improve backup legacy settings fix
* Lint
* Simplify config value migration
---------
Co-authored-by: Mitchell Syer <Syer10@users.noreply.github.com>
* Export meta data
* Import meta data
* Add missing "opdsUseBinaryFileSize" setting to gql
* Export server settings
* Import server settings
* Streamline server config enum handling
* Use "restore amount" in backup import progress
* Remove existing installations with msi installer
* Remove unused x86 wxs file
* Uninstall old msi versions with different upgrade code
* Progress but error 2721 happens on install
* Remove added uninstall previous version wxs stuff
* Use revision as patch number
MSI only uninstalls previous versions in case the version number changed (it only checks the first three numbers (major, minor, patch)).
Thus, to prevent each preview install to result in it getting registered as a new "app" and for it to uninstall the old versions, we have to change the version on each release.
* Deprecate "BuildConfig.REVISION"
* Remove outdated env vars
---------
Co-authored-by: Syer10 <syer10@users.noreply.github.com>
* Log exceptions during graphql execution
Exceptions got swallowed by graphql
* Add stack trace to error in graphql response
Depending on the exceptions error message, the error in the response might be quite useless (e.g. "Stub!" error in android classes)
The update subscription emitted the full update status, which, depending on how big the status was, took forever because the graphql subscription does not support data loader batching, causing it to run into the n+1 problem
* Optimize restoring manga chapters
* Streamline restoring manga data
* Optimize restoring manga trackers
* Simplify passing manga category restore data
* Properly prevent mangas from getting added to default category
76595233fc never actually worked...
* Extract logic to add manga to categories from gql mutation
* Optimize restoring manga categories
* Optimize restoring categories
coerceIn throws an error in case the max value is less than the min value ("Cannot coerce value to an empty range: maximum <max> is less than minimum <min>")
Regression from c8bd39b4bf
* Update to exposed-migrations v3.5.0
* Update to kotlin-logging v7.0.0
* Update to exposed v0.46.0
* Update to exposed v0.47.0
* Update to exposed v0.55.0
* Update to exposed v0.56.0
* Update to exposed v0.57.0
* Update graphqlkotlin to v8
* Go back to JsonMapper
* Add context to data loaders
* Compile fixes
---------
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Syer10 <syer10@users.noreply.github.com>
* Validate setting values on mutation
* Handle invalid negative setting values
* Ensure at least one source is downloading at all times
* Prevent possible IllegalArgumentException
The "serverConfig.maxSourcesInParallel" value could have changed after the if-condition
* Emit only download changes instead of full status
The download subscription emitted the full download status, which, depending on how big the queue was, took forever because the graphql subscription does not support data loader batching, causing it to run into the n+1 problem
* Rename "DownloadManager#status" to "DownloadManager#updates"
* Add initial queue to download subscription type
Adds the current queue at the time of sending the initial message.
This field is null for all following messages after the initial one
* Optionally limit and omit download updates
To prevent the n+1 dataloader issue, the max number of updates included in the download subscription can be limited.
This way, the problem will be circumvented and instead, the latest download status should be (re-)fetched via the download status query, which does not run into this problem.
* Formatting
* Update graphqlkotlin to v6.8.5
* Replace Jackson with Kotlinx.Serialization where possible
---------
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Syer10 <syer10@users.noreply.github.com>
* Keep up to 31 log files
On average one log file per day gets created, thus, increasing to 31 files will store log files for one month
* Decrease total log files size to 100mb
* Make log appender settings configurable
In case a manga gets added to the library which has not been initialized yet, it should be tried to initialize it.
Since it's not an error to have uninitialized manga in the library, this can be done in the background via the updater and the client receives the updated data via the update subscription.
* Remove code duplication
* Remove unnecessary functions
* Simplify filtering for multiple values in queries
Makes it easier to filter for multiple values at ones without having to nest filters with multiple "and".
e.g.
```gql
query MyQuery {
mangas(
filter: {genre: {includesInsensitive: "action"}, and: {genre: {includesInsensitive: "adventure"}, and: { ... }}}
) {
nodes {
id
}
}
}
```
can be simplified to
```gql
query MyQuery {
mangas(
filter: {genre: {includesInsensitive: ["action", "adventure", ...]}}
) {
nodes {
id
}
}
}
```
* Add filter for matching "any" value in list
Makes it easier to filter for entries that match any value without having to nest filters with multiple "or".
e.g.
```gql
query MyQuery {
mangas(
filter: {genre: {includesInsensitiveAny: ["action", "adventure", ...]}}
) {
nodes {
id
}
}
}
```
instead of
```gql
query MyQuery {
mangas(
filter: {genre: {includesInsensitive: "action", or: {genre: includesInsensitive: "adventure", or: {...}}}}
) {
nodes {
id
}
}
}
```
* Add util function to apply "andWhere/All/Any"
