feat(sync/koreader): Add KOReader reading progress synchronization (#1560)

* feat(sync/koreader): implement reading progress synchronization

This commit introduces a comprehensive integration with KOReader Sync Server to enable two-way synchronization of reading progress.

The core logic is encapsulated in a new `KoreaderSyncService`, which handles authentication, registration, and progress pushing/pulling based on user-defined strategies (LATEST, KOSYNC, SUWAYOMI).

Key changes include:

- A new GraphQL API is added to manage the KOReader Sync connection:
  - `connectKoSyncAccount` mutation provides a simplified flow that attempts to log in and, if the user doesn't exist, automatically registers them.
  - `logoutKoSyncAccount` mutation to clear credentials.
  - `koSyncStatus` query to check the current connection status.

- Reading progress is now synchronized at key points:
  - The `fetchChapterPages` mutation pulls the latest progress from the sync server before loading the reader. It respects the configured sync strategy and updates the local database if necessary.
  - The `updateChapters` and other progress-updating methods now push changes to the sync server automatically.
- OPDS chapter entries also pull the latest progress, ensuring clients receive up-to-date reading status.

- Supporting backend changes have been made:
  - The `Chapter` table is extended with a `koreader_hash` column to uniquely identify documents. A database migration is included.
  - New configuration options are added to `server.conf` to manage the feature (enable, server URL, credentials, strategy, etc.).

* perf(opds): defer KOReader sync to improve chapter feed performance

Removes the KOReader Sync progress-pulling logic from the `createChapterListEntry` function.

The previous implementation triggered a network request to the sync server for every single chapter when generating a list, leading to severe performance issues and slow load times on feeds with many entries.

This change reverts to the more performant approach of always linking to the chapter's metadata feed. The expensive sync operation will be handled within the metadata entry generation instead, ensuring it's only triggered on-demand for a single chapter. This restores the responsiveness of browsing chapter feeds.

* refactor(koreader): Use enums for sync settings and correct OPDS logic

Refactor Koreader Sync settings to use enums instead of raw strings for `checksumMethod` and `strategy`. This improves type safety, prevents typos, and makes the configuration handling more robust.

The changes include:
- Introducing `KoreaderSyncChecksumMethod` and `KoreaderSyncStrategy` enums.
- Updating `ServerConfig`, GraphQL types, and backup models to use these new enums.
- Refactoring `KoreaderSyncService` to work with the enum types.

Additionally, this commit fixes an issue in `OpdsEntryBuilder` where the logic for determining which sync progress to use (local vs. remote) was duplicated. The builder now correctly delegates this decision to `KoreaderSyncService.pullProgress`, which already contains the necessary strategy logic. This centralizes the logic and ensures consistent behavior.

* refactor(koreader): Improve config handling and remove redundant update

This commit combines several refactoring and cleanup tasks:

- **Koreader Sync:** The sync service is updated to use the modern `serverConfig` provider instead of the legacy `GlobalConfigManager`. This aligns it with the current configuration management approach in the project.

- **Download Provider:** A redundant `pageCount` database update is removed from `ChaptersFilesProvider`. This operation was unnecessary because the `getChapterDownloadReady` function, which is called earlier in the download process, already verifies and corrects the page count. This change eliminates a superfluous database write and fixes a related import issue.

* feat(sync/koreader)!: enhance sync strategy and add progress tolerance

This commit overhauls the KOReader synchronization feature to provide more granular control and robustness. The simple on/off toggle has been replaced with a more flexible strategy-based system.

Key changes include:
- Replaced `koreaderSyncEnabled` with a more powerful `koreaderSyncStrategy` enum.
- Introduced new sync strategies: `PROMPT`, `SILENT`, `SEND`, `RECEIVE`, and `DISABLE`, allowing for fine-grained control over the sync direction and conflict resolution.
- Added a `koreaderSyncPercentageTolerance` setting. This prevents unnecessary sync updates for minor progress differences between Suwayomi and KOReader.
- Refactored the `KoreaderSyncService` to implement the new strategies and use the configurable tolerance.
- Updated GraphQL schemas, mutations, and server configuration to remove the old setting and incorporate the new ones.
- Adjusted the backup and restore process to correctly handle the new configuration parameters.
- Modified API endpoints and internal logic to check and apply remote progress based on the selected strategy.

BREAKING CHANGE: The `koreaderSyncEnabled` setting is removed and replaced by a more granular `koreaderSyncStrategy`. The enum values for the strategy have been completely changed, making previous configurations incompatible.

* fix: remove unused imports

* feat(opds, sync): enhance Koreader sync and OPDS conflict handling

This commit introduces significant improvements to the Koreader synchronization feature, focusing on providing a better user experience for handling progress conflicts in both OPDS and GraphQL clients.

Key changes include:

- **OPDS Conflict Resolution:** When a reading progress conflict is detected, the OPDS feed for a chapter now provides two distinct "Read Online" links: one to continue from the local progress and another to continue from the synced progress from the remote device (e.g., "Continue Reading Online (Synced from KOReader)"). This empowers users to choose which progress to follow.

- **GraphQL Sync Conflict Information:** The `fetchChapterPages` GraphQL mutation now includes a `syncConflict` field in its payload. This field provides the remote device name and page number, allowing GraphQL clients to implement a user-facing prompt to resolve sync conflicts.

- **Improved Sync Strategy Handling:**
  - The `connectKoSyncAccount` mutation no longer unconditionally sets the sync strategy to `PROMPT`. It now respects the user's existing setting, preventing accidental configuration changes upon re-login.
  - The default `koreaderSyncStrategy` in the configuration is changed to `DISABLED`, providing a safer and more intuitive default for new users.

- **Refinements & Fixes:**
  - The fallback for the remote device name is now centralized within the KoreaderSyncService, defaulting to "KOReader" if not provided.
  - Renamed `KoreaderSyncStrategy.DISABLE` to `DISABLED` for consistency.
  - Updated i18n strings for OPDS links to be more descriptive and user-friendly.

* refactor(kosync): rename stream page link titles for consistency

* refactor(kosync): return SettingsType in auth mutation payloads

The `connectKoSyncAccount` and `logoutKoSyncAccount` mutations modify server settings (username and userkey) but did not previously return the updated configuration. This forced client applications to manually refetch settings to avoid a stale cache.

This change modifies the payloads for both mutations to include the full `SettingsType`.

By returning the updated settings directly, GraphQL clients like Apollo Client can automatically update their cache, simplifying client-side state management and ensuring the UI always reflects the current server configuration.

Additionally, `clientMutationId` has been added to `KoSyncConnectPayload` for consistency with GraphQL practices, aligning it with the logout mutation.

Refs: #1560

* refactor(kosync): replace KoSyncConnectPayload with ConnectResult in connect method

* fix(kosync): add koreaderSyncPercentageTolerance default setting
This commit is contained in:
Zeedif
2025-08-19 13:00:19 -06:00
committed by GitHub
parent 16e9c0b19a
commit 590e43c827
23 changed files with 926 additions and 50 deletions

View File

@@ -95,7 +95,12 @@
<string name="opds_linktitle_next_page">Next Page</string> <string name="opds_linktitle_next_page">Next Page</string>
<string name="opds_linktitle_self_feed">Current Feed</string> <string name="opds_linktitle_self_feed">Current Feed</string>
<string name="opds_linktitle_view_on_web">View on Web</string> <string name="opds_linktitle_view_on_web">View on Web</string>
<string name="opds_linktitle_stream_pages">View Pages (Streaming)</string> <string name="opds_linktitle_stream_pages_start">Read Online</string>
<string name="opds_linktitle_stream_pages_continue">Continue Reading Online</string>
<string name="opds_linktitle_stream_pages_start_local">Read Online (Local Progress)</string>
<string name="opds_linktitle_stream_pages_continue_local">Continue Reading Online (Local Progress)</string>
<string name="opds_linktitle_stream_pages_start_remote">Read Online (Synced from %1$s)</string>
<string name="opds_linktitle_stream_pages_continue_remote">Continue Reading Online (Synced from %1$s)</string>
<string name="opds_linktitle_download_cbz">Download CBZ</string> <string name="opds_linktitle_download_cbz">Download CBZ</string>
<string name="opds_linktitle_chapter_cover">Chapter Cover</string> <string name="opds_linktitle_chapter_cover">Chapter Cover</string>
<string name="opds_linktitle_view_chapter_details">View Chapter Details &amp; Get Pages</string> <string name="opds_linktitle_view_chapter_details">View Chapter Details &amp; Get Pages</string>
@@ -106,7 +111,7 @@
<string name="opds_chapter_status_unread"></string> <string name="opds_chapter_status_unread"></string>
<string name="opds_chapter_status_downloaded">⬇️ </string> <string name="opds_chapter_status_downloaded">⬇️ </string>
<string name="opds_chapter_details_base">%1$s | %2$s</string> <string name="opds_chapter_details_base">Series: %1$s | %2$s</string>
<string name="opds_chapter_details_scanlator"> | By %1$s</string> <string name="opds_chapter_details_scanlator"> | By %1$s</string>
<string name="opds_chapter_details_progress"> | Progress: %1$d of %2$d</string> <string name="opds_chapter_details_progress"> | Progress: %1$d of %2$d</string>

View File

@@ -1,6 +1,8 @@
package suwayomi.tachidesk.graphql.mutations package suwayomi.tachidesk.graphql.mutations
import graphql.execution.DataFetcherResult import graphql.execution.DataFetcherResult
import kotlinx.coroutines.GlobalScope
import kotlinx.coroutines.launch
import org.jetbrains.exposed.dao.id.EntityID import org.jetbrains.exposed.dao.id.EntityID
import org.jetbrains.exposed.sql.SqlExpressionBuilder.eq import org.jetbrains.exposed.sql.SqlExpressionBuilder.eq
import org.jetbrains.exposed.sql.and import org.jetbrains.exposed.sql.and
@@ -8,11 +10,14 @@ import org.jetbrains.exposed.sql.deleteWhere
import org.jetbrains.exposed.sql.selectAll import org.jetbrains.exposed.sql.selectAll
import org.jetbrains.exposed.sql.statements.BatchUpdateStatement import org.jetbrains.exposed.sql.statements.BatchUpdateStatement
import org.jetbrains.exposed.sql.transactions.transaction import org.jetbrains.exposed.sql.transactions.transaction
import org.jetbrains.exposed.sql.update
import suwayomi.tachidesk.graphql.asDataFetcherResult import suwayomi.tachidesk.graphql.asDataFetcherResult
import suwayomi.tachidesk.graphql.types.ChapterMetaType import suwayomi.tachidesk.graphql.types.ChapterMetaType
import suwayomi.tachidesk.graphql.types.ChapterType import suwayomi.tachidesk.graphql.types.ChapterType
import suwayomi.tachidesk.graphql.types.SyncConflictInfoType
import suwayomi.tachidesk.manga.impl.Chapter import suwayomi.tachidesk.manga.impl.Chapter
import suwayomi.tachidesk.manga.impl.chapter.getChapterDownloadReadyById import suwayomi.tachidesk.manga.impl.chapter.getChapterDownloadReadyById
import suwayomi.tachidesk.manga.impl.sync.KoreaderSyncService
import suwayomi.tachidesk.manga.model.table.ChapterMetaTable import suwayomi.tachidesk.manga.model.table.ChapterMetaTable
import suwayomi.tachidesk.manga.model.table.ChapterTable import suwayomi.tachidesk.manga.model.table.ChapterTable
import suwayomi.tachidesk.server.JavalinSetup.future import suwayomi.tachidesk.server.JavalinSetup.future
@@ -68,8 +73,10 @@ class ChapterMutation {
ChapterTable ChapterTable
.select(ChapterTable.id, ChapterTable.pageCount) .select(ChapterTable.id, ChapterTable.pageCount)
.where { ChapterTable.id inList ids } .where { ChapterTable.id inList ids }
.groupBy { it[ChapterTable.id].value } .associateBy(
.mapValues { it.value.firstOrNull()?.let { it[ChapterTable.pageCount] } } { it[ChapterTable.id].value },
{ it[ChapterTable.pageCount] },
)
} else { } else {
emptyMap() emptyMap()
} }
@@ -94,6 +101,15 @@ class ChapterMutation {
} }
} }
} }
// Sync with KoreaderSync when progress is updated
if (patch.lastPageRead != null || patch.isRead == true) {
GlobalScope.launch {
ids.forEach { chapterId ->
KoreaderSyncService.pushProgress(chapterId)
}
}
}
} }
fun updateChapter(input: UpdateChapterInput): DataFetcherResult<UpdateChapterPayload?> = fun updateChapter(input: UpdateChapterInput): DataFetcherResult<UpdateChapterPayload?> =
@@ -241,6 +257,7 @@ class ChapterMutation {
val clientMutationId: String?, val clientMutationId: String?,
val pages: List<String>, val pages: List<String>,
val chapter: ChapterType, val chapter: ChapterType,
val syncConflict: SyncConflictInfoType?,
) )
fun fetchChapterPages(input: FetchChapterPagesInput): CompletableFuture<DataFetcherResult<FetchChapterPagesPayload?>> { fun fetchChapterPages(input: FetchChapterPagesInput): CompletableFuture<DataFetcherResult<FetchChapterPagesPayload?>> {
@@ -249,7 +266,35 @@ class ChapterMutation {
return future { return future {
asDataFetcherResult { asDataFetcherResult {
val chapter = getChapterDownloadReadyById(chapterId) var chapter = getChapterDownloadReadyById(chapterId)
val syncResult = KoreaderSyncService.checkAndPullProgress(chapter.id)
var syncConflictInfo: SyncConflictInfoType? = null
if (syncResult != null) {
if (syncResult.isConflict) {
syncConflictInfo =
SyncConflictInfoType(
deviceName = syncResult.device,
remotePage = syncResult.pageRead,
)
}
if (syncResult.shouldUpdate) {
// Update DB for SILENT and RECEIVE
transaction {
ChapterTable.update({ ChapterTable.id eq chapter.id }) {
it[lastPageRead] = syncResult.pageRead
it[lastReadAt] = syncResult.timestamp
}
}
}
// For PROMPT, SILENT, and RECEIVE, return the remote progress
chapter =
chapter.copy(
lastPageRead = if (syncResult.shouldUpdate) syncResult.pageRead else chapter.lastPageRead,
lastReadAt = if (syncResult.shouldUpdate) syncResult.timestamp else chapter.lastReadAt,
)
}
val params = val params =
buildString { buildString {
@@ -273,6 +318,7 @@ class ChapterMutation {
"/api/v1/manga/${chapter.mangaId}/chapter/${chapter.index}/page/${index}$params" "/api/v1/manga/${chapter.mangaId}/chapter/${chapter.index}/page/${index}$params"
}, },
chapter = ChapterType(chapter), chapter = ChapterType(chapter),
syncConflict = syncConflictInfo,
) )
} }
} }

View File

@@ -0,0 +1,43 @@
package suwayomi.tachidesk.graphql.mutations
import suwayomi.tachidesk.graphql.types.KoSyncConnectPayload
import suwayomi.tachidesk.graphql.types.LogoutKoSyncAccountPayload
import suwayomi.tachidesk.graphql.types.SettingsType
import suwayomi.tachidesk.manga.impl.sync.KoreaderSyncService
import suwayomi.tachidesk.server.JavalinSetup.future
import java.util.concurrent.CompletableFuture
class KoreaderSyncMutation {
data class ConnectKoSyncAccountInput(
val clientMutationId: String? = null,
val username: String,
val password: String,
)
fun connectKoSyncAccount(input: ConnectKoSyncAccountInput): CompletableFuture<KoSyncConnectPayload> =
future {
val result = KoreaderSyncService.connect(input.username, input.password)
KoSyncConnectPayload(
clientMutationId = input.clientMutationId,
success = result.success,
message = result.message,
username = result.username,
settings = SettingsType(),
)
}
data class LogoutKoSyncAccountInput(
val clientMutationId: String? = null,
)
fun logoutKoSyncAccount(input: LogoutKoSyncAccountInput): CompletableFuture<LogoutKoSyncAccountPayload> =
future {
KoreaderSyncService.logout()
LogoutKoSyncAccountPayload(
clientMutationId = input.clientMutationId,
success = true,
settings = SettingsType(),
)
}
}

View File

@@ -215,6 +215,15 @@ class SettingsMutation {
updateSetting(settings.opdsShowOnlyUnreadChapters, serverConfig.opdsShowOnlyUnreadChapters) updateSetting(settings.opdsShowOnlyUnreadChapters, serverConfig.opdsShowOnlyUnreadChapters)
updateSetting(settings.opdsShowOnlyDownloadedChapters, serverConfig.opdsShowOnlyDownloadedChapters) updateSetting(settings.opdsShowOnlyDownloadedChapters, serverConfig.opdsShowOnlyDownloadedChapters)
updateSetting(settings.opdsChapterSortOrder, serverConfig.opdsChapterSortOrder) updateSetting(settings.opdsChapterSortOrder, serverConfig.opdsChapterSortOrder)
// koreader sync
updateSetting(settings.koreaderSyncServerUrl, serverConfig.koreaderSyncServerUrl)
updateSetting(settings.koreaderSyncUsername, serverConfig.koreaderSyncUsername)
updateSetting(settings.koreaderSyncUserkey, serverConfig.koreaderSyncUserkey)
updateSetting(settings.koreaderSyncDeviceId, serverConfig.koreaderSyncDeviceId)
updateSetting(settings.koreaderSyncChecksumMethod, serverConfig.koreaderSyncChecksumMethod)
updateSetting(settings.koreaderSyncStrategy, serverConfig.koreaderSyncStrategy)
updateSetting(settings.koreaderSyncPercentageTolerance, serverConfig.koreaderSyncPercentageTolerance)
} }
fun setSettings(input: SetSettingsInput): SetSettingsPayload { fun setSettings(input: SetSettingsInput): SetSettingsPayload {

View File

@@ -0,0 +1,13 @@
package suwayomi.tachidesk.graphql.queries
import suwayomi.tachidesk.graphql.types.KoSyncStatusPayload
import suwayomi.tachidesk.manga.impl.sync.KoreaderSyncService
import suwayomi.tachidesk.server.JavalinSetup.future
import java.util.concurrent.CompletableFuture
class KoreaderSyncQuery {
fun koSyncStatus(): CompletableFuture<KoSyncStatusPayload> =
future {
KoreaderSyncService.getStatus()
}
}

View File

@@ -20,6 +20,7 @@ import suwayomi.tachidesk.graphql.mutations.DownloadMutation
import suwayomi.tachidesk.graphql.mutations.ExtensionMutation import suwayomi.tachidesk.graphql.mutations.ExtensionMutation
import suwayomi.tachidesk.graphql.mutations.ImageMutation import suwayomi.tachidesk.graphql.mutations.ImageMutation
import suwayomi.tachidesk.graphql.mutations.InfoMutation import suwayomi.tachidesk.graphql.mutations.InfoMutation
import suwayomi.tachidesk.graphql.mutations.KoreaderSyncMutation
import suwayomi.tachidesk.graphql.mutations.MangaMutation import suwayomi.tachidesk.graphql.mutations.MangaMutation
import suwayomi.tachidesk.graphql.mutations.MetaMutation import suwayomi.tachidesk.graphql.mutations.MetaMutation
import suwayomi.tachidesk.graphql.mutations.SettingsMutation import suwayomi.tachidesk.graphql.mutations.SettingsMutation
@@ -32,6 +33,7 @@ import suwayomi.tachidesk.graphql.queries.ChapterQuery
import suwayomi.tachidesk.graphql.queries.DownloadQuery import suwayomi.tachidesk.graphql.queries.DownloadQuery
import suwayomi.tachidesk.graphql.queries.ExtensionQuery import suwayomi.tachidesk.graphql.queries.ExtensionQuery
import suwayomi.tachidesk.graphql.queries.InfoQuery import suwayomi.tachidesk.graphql.queries.InfoQuery
import suwayomi.tachidesk.graphql.queries.KoreaderSyncQuery
import suwayomi.tachidesk.graphql.queries.MangaQuery import suwayomi.tachidesk.graphql.queries.MangaQuery
import suwayomi.tachidesk.graphql.queries.MetaQuery import suwayomi.tachidesk.graphql.queries.MetaQuery
import suwayomi.tachidesk.graphql.queries.SettingsQuery import suwayomi.tachidesk.graphql.queries.SettingsQuery
@@ -74,6 +76,7 @@ val schema =
TopLevelObject(DownloadQuery()), TopLevelObject(DownloadQuery()),
TopLevelObject(ExtensionQuery()), TopLevelObject(ExtensionQuery()),
TopLevelObject(InfoQuery()), TopLevelObject(InfoQuery()),
TopLevelObject(KoreaderSyncQuery()),
TopLevelObject(MangaQuery()), TopLevelObject(MangaQuery()),
TopLevelObject(MetaQuery()), TopLevelObject(MetaQuery()),
TopLevelObject(SettingsQuery()), TopLevelObject(SettingsQuery()),
@@ -90,6 +93,7 @@ val schema =
TopLevelObject(ExtensionMutation()), TopLevelObject(ExtensionMutation()),
TopLevelObject(ImageMutation()), TopLevelObject(ImageMutation()),
TopLevelObject(InfoMutation()), TopLevelObject(InfoMutation()),
TopLevelObject(KoreaderSyncMutation()),
TopLevelObject(MangaMutation()), TopLevelObject(MangaMutation()),
TopLevelObject(MetaMutation()), TopLevelObject(MetaMutation()),
TopLevelObject(SettingsMutation()), TopLevelObject(SettingsMutation()),

View File

@@ -19,6 +19,11 @@ import suwayomi.tachidesk.manga.model.dataclass.ChapterDataClass
import suwayomi.tachidesk.manga.model.table.ChapterTable import suwayomi.tachidesk.manga.model.table.ChapterTable
import java.util.concurrent.CompletableFuture import java.util.concurrent.CompletableFuture
data class SyncConflictInfoType(
val deviceName: String,
val remotePage: Int,
)
class ChapterType( class ChapterType(
val id: Int, val id: Int,
val url: String, val url: String,

View File

@@ -0,0 +1,33 @@
package suwayomi.tachidesk.graphql.types
enum class KoreaderSyncChecksumMethod {
BINARY,
FILENAME,
}
enum class KoreaderSyncStrategy {
PROMPT, // Ask on conflict
SILENT, // Always use latest
SEND, // Send changes only
RECEIVE, // Receive changes only
DISABLED,
}
data class KoSyncStatusPayload(
val isLoggedIn: Boolean,
val username: String?,
)
data class KoSyncConnectPayload(
val clientMutationId: String?,
val success: Boolean,
val message: String?,
val username: String?,
val settings: SettingsType,
)
data class LogoutKoSyncAccountPayload(
val clientMutationId: String?,
val success: Boolean,
val settings: SettingsType,
)

View File

@@ -112,6 +112,15 @@ interface Settings : Node {
val opdsShowOnlyUnreadChapters: Boolean? val opdsShowOnlyUnreadChapters: Boolean?
val opdsShowOnlyDownloadedChapters: Boolean? val opdsShowOnlyDownloadedChapters: Boolean?
val opdsChapterSortOrder: SortOrder? val opdsChapterSortOrder: SortOrder?
// koreader sync
val koreaderSyncServerUrl: String?
val koreaderSyncUsername: String?
val koreaderSyncUserkey: String?
val koreaderSyncDeviceId: String?
val koreaderSyncChecksumMethod: KoreaderSyncChecksumMethod?
val koreaderSyncStrategy: KoreaderSyncStrategy?
val koreaderSyncPercentageTolerance: Double?
} }
interface SettingsDownloadConversion { interface SettingsDownloadConversion {
@@ -206,6 +215,14 @@ data class PartialSettingsType(
override val opdsShowOnlyUnreadChapters: Boolean?, override val opdsShowOnlyUnreadChapters: Boolean?,
override val opdsShowOnlyDownloadedChapters: Boolean?, override val opdsShowOnlyDownloadedChapters: Boolean?,
override val opdsChapterSortOrder: SortOrder?, override val opdsChapterSortOrder: SortOrder?,
// koreader sync
override val koreaderSyncServerUrl: String?,
override val koreaderSyncUsername: String?,
override val koreaderSyncUserkey: String?,
override val koreaderSyncDeviceId: String?,
override val koreaderSyncChecksumMethod: KoreaderSyncChecksumMethod?,
override val koreaderSyncStrategy: KoreaderSyncStrategy?,
override val koreaderSyncPercentageTolerance: Double?,
) : Settings ) : Settings
class SettingsType( class SettingsType(
@@ -288,6 +305,14 @@ class SettingsType(
override val opdsShowOnlyUnreadChapters: Boolean, override val opdsShowOnlyUnreadChapters: Boolean,
override val opdsShowOnlyDownloadedChapters: Boolean, override val opdsShowOnlyDownloadedChapters: Boolean,
override val opdsChapterSortOrder: SortOrder, override val opdsChapterSortOrder: SortOrder,
// koreader sync
override val koreaderSyncServerUrl: String,
override val koreaderSyncUsername: String,
override val koreaderSyncUserkey: String,
override val koreaderSyncDeviceId: String,
override val koreaderSyncChecksumMethod: KoreaderSyncChecksumMethod,
override val koreaderSyncStrategy: KoreaderSyncStrategy,
override val koreaderSyncPercentageTolerance: Double,
) : Settings { ) : Settings {
constructor(config: ServerConfig = serverConfig) : this( constructor(config: ServerConfig = serverConfig) : this(
config.ip.value, config.ip.value,
@@ -367,5 +392,13 @@ class SettingsType(
config.opdsShowOnlyUnreadChapters.value, config.opdsShowOnlyUnreadChapters.value,
config.opdsShowOnlyDownloadedChapters.value, config.opdsShowOnlyDownloadedChapters.value,
config.opdsChapterSortOrder.value, config.opdsChapterSortOrder.value,
// koreader sync
config.koreaderSyncServerUrl.value,
config.koreaderSyncUsername.value,
config.koreaderSyncUserkey.value,
config.koreaderSyncDeviceId.value,
config.koreaderSyncChecksumMethod.value,
config.koreaderSyncStrategy.value,
config.koreaderSyncPercentageTolerance.value,
) )
} }

View File

@@ -9,7 +9,11 @@ package suwayomi.tachidesk.manga.controller
import io.javalin.http.HandlerType import io.javalin.http.HandlerType
import io.javalin.http.HttpStatus import io.javalin.http.HttpStatus
import kotlinx.coroutines.GlobalScope
import kotlinx.coroutines.launch
import kotlinx.serialization.json.Json import kotlinx.serialization.json.Json
import org.jetbrains.exposed.sql.transactions.transaction
import org.jetbrains.exposed.sql.update
import suwayomi.tachidesk.manga.impl.CategoryManga import suwayomi.tachidesk.manga.impl.CategoryManga
import suwayomi.tachidesk.manga.impl.Chapter import suwayomi.tachidesk.manga.impl.Chapter
import suwayomi.tachidesk.manga.impl.ChapterDownloadHelper import suwayomi.tachidesk.manga.impl.ChapterDownloadHelper
@@ -17,9 +21,11 @@ import suwayomi.tachidesk.manga.impl.Library
import suwayomi.tachidesk.manga.impl.Manga import suwayomi.tachidesk.manga.impl.Manga
import suwayomi.tachidesk.manga.impl.Page import suwayomi.tachidesk.manga.impl.Page
import suwayomi.tachidesk.manga.impl.chapter.getChapterDownloadReadyByIndex import suwayomi.tachidesk.manga.impl.chapter.getChapterDownloadReadyByIndex
import suwayomi.tachidesk.manga.impl.sync.KoreaderSyncService
import suwayomi.tachidesk.manga.model.dataclass.CategoryDataClass import suwayomi.tachidesk.manga.model.dataclass.CategoryDataClass
import suwayomi.tachidesk.manga.model.dataclass.ChapterDataClass import suwayomi.tachidesk.manga.model.dataclass.ChapterDataClass
import suwayomi.tachidesk.manga.model.dataclass.MangaDataClass import suwayomi.tachidesk.manga.model.dataclass.MangaDataClass
import suwayomi.tachidesk.manga.model.table.ChapterTable
import suwayomi.tachidesk.server.JavalinSetup.future import suwayomi.tachidesk.server.JavalinSetup.future
import suwayomi.tachidesk.server.util.formParam import suwayomi.tachidesk.server.util.formParam
import suwayomi.tachidesk.server.util.handler import suwayomi.tachidesk.server.util.handler
@@ -315,8 +321,29 @@ object MangaController {
}, },
behaviorOf = { ctx, mangaId, chapterIndex -> behaviorOf = { ctx, mangaId, chapterIndex ->
ctx.future { ctx.future {
future { getChapterDownloadReadyByIndex(chapterIndex, mangaId) } future {
.thenApply { ctx.json(it) } var chapter = getChapterDownloadReadyByIndex(chapterIndex, mangaId)
val syncResult = KoreaderSyncService.checkAndPullProgress(chapter.id)
if (syncResult != null) {
if (syncResult.shouldUpdate) {
// Update DB for SILENT and RECEIVE
transaction {
ChapterTable.update({ ChapterTable.id eq chapter.id }) {
it[lastPageRead] = syncResult.pageRead
it[lastReadAt] = syncResult.timestamp
}
}
}
// For PROMPT, SILENT, and RECEIVE, return the remote progress
chapter =
chapter.copy(
lastPageRead = syncResult.pageRead,
lastReadAt = syncResult.timestamp,
)
}
chapter
}.thenApply { ctx.json(it) }
} }
}, },
withResults = { withResults = {
@@ -341,7 +368,12 @@ object MangaController {
} }
}, },
behaviorOf = { ctx, mangaId, chapterIndex, read, bookmarked, markPrevRead, lastPageRead -> behaviorOf = { ctx, mangaId, chapterIndex, read, bookmarked, markPrevRead, lastPageRead ->
Chapter.modifyChapter(mangaId, chapterIndex, read, bookmarked, markPrevRead, lastPageRead) val chapterId = Chapter.modifyChapter(mangaId, chapterIndex, read, bookmarked, markPrevRead, lastPageRead)
// Sync with KoreaderSync when progress is updated
if (lastPageRead != null || read == true) {
GlobalScope.launch { KoreaderSyncService.pushProgress(chapterId) }
}
ctx.status(200) ctx.status(200)
}, },
@@ -422,7 +454,11 @@ object MangaController {
ctx.result(it.first) ctx.result(it.first)
if (updateProgress == true) { if (updateProgress == true) {
Chapter.updateChapterProgress(mangaId, chapterIndex, pageNo = index) val chapterId = Chapter.updateChapterProgress(mangaId, chapterIndex, pageNo = index)
// Sync progress with KoreaderSync if chapter update was successful
if (chapterId != -1) {
GlobalScope.launch { KoreaderSyncService.pushProgress(chapterId) }
}
} }
} }
} }

View File

@@ -431,8 +431,17 @@ object Chapter {
isBookmarked: Boolean?, isBookmarked: Boolean?,
markPrevRead: Boolean?, markPrevRead: Boolean?,
lastPageRead: Int?, lastPageRead: Int?,
) { ): Int {
val chapterId =
transaction { transaction {
val chapter =
ChapterTable
.selectAll()
.where { (ChapterTable.manga eq mangaId) and (ChapterTable.sourceOrder eq chapterIndex) }
.first()
val chapterIdValue = chapter[ChapterTable.id].value
if (listOf(isRead, isBookmarked, lastPageRead).any { it != null }) { if (listOf(isRead, isBookmarked, lastPageRead).any { it != null }) {
ChapterTable.update({ (ChapterTable.manga eq mangaId) and (ChapterTable.sourceOrder eq chapterIndex) }) { update -> ChapterTable.update({ (ChapterTable.manga eq mangaId) and (ChapterTable.sourceOrder eq chapterIndex) }) { update ->
isRead?.also { isRead?.also {
@@ -453,11 +462,15 @@ object Chapter {
it[ChapterTable.isRead] = markPrevRead it[ChapterTable.isRead] = markPrevRead
} }
} }
chapterIdValue
} }
if (isRead == true || markPrevRead == true) { if (isRead == true || markPrevRead == true) {
Track.asyncTrackChapter(setOf(mangaId)) Track.asyncTrackChapter(setOf(mangaId))
} }
return chapterId
} }
@Serializable @Serializable
@@ -733,7 +746,7 @@ object Chapter {
mangaId: Int, mangaId: Int,
chapterIndex: Int, chapterIndex: Int,
pageNo: Int, pageNo: Int,
) { ): Int {
val chapterData = val chapterData =
transaction { transaction {
ChapterTable ChapterTable
@@ -756,5 +769,7 @@ object Chapter {
isBookmarked = null, isBookmarked = null,
markPrevRead = null, markPrevRead = null,
) )
return chapterData.id
} }
} }

View File

@@ -463,6 +463,14 @@ object ProtoBackupExport : ProtoBackupBase() {
opdsShowOnlyUnreadChapters = serverConfig.opdsShowOnlyUnreadChapters.value, opdsShowOnlyUnreadChapters = serverConfig.opdsShowOnlyUnreadChapters.value,
opdsShowOnlyDownloadedChapters = serverConfig.opdsShowOnlyDownloadedChapters.value, opdsShowOnlyDownloadedChapters = serverConfig.opdsShowOnlyDownloadedChapters.value,
opdsChapterSortOrder = serverConfig.opdsChapterSortOrder.value, opdsChapterSortOrder = serverConfig.opdsChapterSortOrder.value,
// koreader sync
koreaderSyncServerUrl = serverConfig.koreaderSyncServerUrl.value,
koreaderSyncUsername = serverConfig.koreaderSyncUsername.value,
koreaderSyncUserkey = serverConfig.koreaderSyncUserkey.value,
koreaderSyncDeviceId = serverConfig.koreaderSyncDeviceId.value,
koreaderSyncChecksumMethod = serverConfig.koreaderSyncChecksumMethod.value,
koreaderSyncStrategy = serverConfig.koreaderSyncStrategy.value,
koreaderSyncPercentageTolerance = serverConfig.koreaderSyncPercentageTolerance.value,
) )
} }
} }

View File

@@ -4,6 +4,8 @@ import kotlinx.serialization.Serializable
import kotlinx.serialization.protobuf.ProtoNumber import kotlinx.serialization.protobuf.ProtoNumber
import org.jetbrains.exposed.sql.SortOrder import org.jetbrains.exposed.sql.SortOrder
import suwayomi.tachidesk.graphql.types.AuthMode import suwayomi.tachidesk.graphql.types.AuthMode
import suwayomi.tachidesk.graphql.types.KoreaderSyncChecksumMethod
import suwayomi.tachidesk.graphql.types.KoreaderSyncStrategy
import suwayomi.tachidesk.graphql.types.Settings import suwayomi.tachidesk.graphql.types.Settings
import suwayomi.tachidesk.graphql.types.SettingsDownloadConversion import suwayomi.tachidesk.graphql.types.SettingsDownloadConversion
import suwayomi.tachidesk.graphql.types.WebUIChannel import suwayomi.tachidesk.graphql.types.WebUIChannel
@@ -84,6 +86,14 @@ data class BackupServerSettings(
@ProtoNumber(53) override var opdsShowOnlyUnreadChapters: Boolean, @ProtoNumber(53) override var opdsShowOnlyUnreadChapters: Boolean,
@ProtoNumber(54) override var opdsShowOnlyDownloadedChapters: Boolean, @ProtoNumber(54) override var opdsShowOnlyDownloadedChapters: Boolean,
@ProtoNumber(55) override var opdsChapterSortOrder: SortOrder, @ProtoNumber(55) override var opdsChapterSortOrder: SortOrder,
// koreader sync
@ProtoNumber(59) override var koreaderSyncServerUrl: String,
@ProtoNumber(60) override var koreaderSyncUsername: String,
@ProtoNumber(61) override var koreaderSyncUserkey: String,
@ProtoNumber(62) override var koreaderSyncDeviceId: String,
@ProtoNumber(63) override var koreaderSyncChecksumMethod: KoreaderSyncChecksumMethod,
@ProtoNumber(64) override var koreaderSyncStrategy: KoreaderSyncStrategy,
@ProtoNumber(65) override var koreaderSyncPercentageTolerance: Double,
) : Settings { ) : Settings {
@Serializable @Serializable
class BackupSettingsDownloadConversionType( class BackupSettingsDownloadConversionType(

View File

@@ -13,11 +13,14 @@ import libcore.net.MimeUtils
import org.apache.commons.compress.archivers.zip.ZipArchiveEntry import org.apache.commons.compress.archivers.zip.ZipArchiveEntry
import org.jetbrains.exposed.sql.selectAll import org.jetbrains.exposed.sql.selectAll
import org.jetbrains.exposed.sql.transactions.transaction import org.jetbrains.exposed.sql.transactions.transaction
import org.jetbrains.exposed.sql.update
import suwayomi.tachidesk.manga.impl.Page import suwayomi.tachidesk.manga.impl.Page
import suwayomi.tachidesk.manga.impl.chapter.getChapterDownloadReady import suwayomi.tachidesk.manga.impl.chapter.getChapterDownloadReady
import suwayomi.tachidesk.manga.impl.download.model.DownloadChapter import suwayomi.tachidesk.manga.impl.download.model.DownloadChapter
import suwayomi.tachidesk.manga.impl.util.KoreaderHelper
import suwayomi.tachidesk.manga.impl.util.createComicInfoFile import suwayomi.tachidesk.manga.impl.util.createComicInfoFile
import suwayomi.tachidesk.manga.impl.util.getChapterCachePath import suwayomi.tachidesk.manga.impl.util.getChapterCachePath
import suwayomi.tachidesk.manga.impl.util.getChapterCbzPath
import suwayomi.tachidesk.manga.impl.util.getChapterDownloadPath import suwayomi.tachidesk.manga.impl.util.getChapterDownloadPath
import suwayomi.tachidesk.manga.impl.util.storage.ImageResponse import suwayomi.tachidesk.manga.impl.util.storage.ImageResponse
import suwayomi.tachidesk.manga.model.table.ChapterTable import suwayomi.tachidesk.manga.model.table.ChapterTable
@@ -43,20 +46,20 @@ sealed class FileType {
fun getName(): String = fun getName(): String =
when (this) { when (this) {
is FileType.RegularFile -> { is RegularFile -> {
this.file.name this.file.name
} }
is FileType.ZipFile -> { is ZipFile -> {
this.entry.name this.entry.name
} }
} }
fun getExtension(): String = fun getExtension(): String =
when (this) { when (this) {
is FileType.RegularFile -> { is RegularFile -> {
this.file.extension this.file.extension
} }
is FileType.ZipFile -> { is ZipFile -> {
this.entry.name.substringAfterLast(".") this.entry.name.substringAfterLast(".")
} }
} }
@@ -189,6 +192,19 @@ abstract class ChaptersFilesProvider<Type : FileType>(
handleSuccessfulDownload() handleSuccessfulDownload()
// Calculate and save Koreader hash for CBZ files
val chapterFile = File(getChapterCbzPath(mangaId, chapterId))
if (chapterFile.exists()) {
val koreaderHash = KoreaderHelper.hashContents(chapterFile)
if (koreaderHash != null) {
transaction {
ChapterTable.update({ ChapterTable.id eq chapterId }) {
it[ChapterTable.koreaderHash] = koreaderHash
}
}
}
}
File(cacheChapterDir).deleteRecursively() File(cacheChapterDir).deleteRecursively()
return true return true

View File

@@ -6,6 +6,8 @@ import org.apache.commons.compress.archivers.zip.ZipArchiveEntry
import org.apache.commons.compress.archivers.zip.ZipArchiveInputStream import org.apache.commons.compress.archivers.zip.ZipArchiveInputStream
import org.apache.commons.compress.archivers.zip.ZipArchiveOutputStream import org.apache.commons.compress.archivers.zip.ZipArchiveOutputStream
import org.apache.commons.compress.archivers.zip.ZipFile import org.apache.commons.compress.archivers.zip.ZipFile
import org.jetbrains.exposed.sql.transactions.transaction
import org.jetbrains.exposed.sql.update
import suwayomi.tachidesk.manga.impl.download.fileProvider.ChaptersFilesProvider import suwayomi.tachidesk.manga.impl.download.fileProvider.ChaptersFilesProvider
import suwayomi.tachidesk.manga.impl.download.fileProvider.FileType import suwayomi.tachidesk.manga.impl.download.fileProvider.FileType
import suwayomi.tachidesk.manga.impl.util.getChapterCachePath import suwayomi.tachidesk.manga.impl.util.getChapterCachePath
@@ -13,6 +15,7 @@ import suwayomi.tachidesk.manga.impl.util.getChapterCbzPath
import suwayomi.tachidesk.manga.impl.util.getChapterDownloadPath import suwayomi.tachidesk.manga.impl.util.getChapterDownloadPath
import suwayomi.tachidesk.manga.impl.util.getMangaDownloadDir import suwayomi.tachidesk.manga.impl.util.getMangaDownloadDir
import suwayomi.tachidesk.manga.impl.util.storage.FileDeletionHelper import suwayomi.tachidesk.manga.impl.util.storage.FileDeletionHelper
import suwayomi.tachidesk.manga.model.table.ChapterTable
import suwayomi.tachidesk.server.ApplicationDirs import suwayomi.tachidesk.server.ApplicationDirs
import uy.kohesive.injekt.injectLazy import uy.kohesive.injekt.injectLazy
import java.io.File import java.io.File
@@ -85,6 +88,13 @@ class ArchiveProvider(
} }
val cbzDeleted = cbzFile.delete() val cbzDeleted = cbzFile.delete()
if (cbzDeleted) {
transaction {
ChapterTable.update({ ChapterTable.id eq chapterId }) {
it[koreaderHash] = null
}
}
}
FileDeletionHelper.cleanupParentFoldersFor(cbzFile, applicationDirs.mangaDownloadsRoot) FileDeletionHelper.cleanupParentFoldersFor(cbzFile, applicationDirs.mangaDownloadsRoot)
return cbzDeleted return cbzDeleted
} }

View File

@@ -1,10 +1,13 @@
package suwayomi.tachidesk.manga.impl.download.fileProvider.impl package suwayomi.tachidesk.manga.impl.download.fileProvider.impl
import org.jetbrains.exposed.sql.transactions.transaction
import org.jetbrains.exposed.sql.update
import suwayomi.tachidesk.manga.impl.download.fileProvider.ChaptersFilesProvider import suwayomi.tachidesk.manga.impl.download.fileProvider.ChaptersFilesProvider
import suwayomi.tachidesk.manga.impl.download.fileProvider.FileType.RegularFile import suwayomi.tachidesk.manga.impl.download.fileProvider.FileType.RegularFile
import suwayomi.tachidesk.manga.impl.util.getChapterCachePath import suwayomi.tachidesk.manga.impl.util.getChapterCachePath
import suwayomi.tachidesk.manga.impl.util.getChapterDownloadPath import suwayomi.tachidesk.manga.impl.util.getChapterDownloadPath
import suwayomi.tachidesk.manga.impl.util.storage.FileDeletionHelper import suwayomi.tachidesk.manga.impl.util.storage.FileDeletionHelper
import suwayomi.tachidesk.manga.model.table.ChapterTable
import suwayomi.tachidesk.server.ApplicationDirs import suwayomi.tachidesk.server.ApplicationDirs
import uy.kohesive.injekt.injectLazy import uy.kohesive.injekt.injectLazy
import java.io.BufferedOutputStream import java.io.BufferedOutputStream
@@ -61,6 +64,13 @@ class FolderProvider(
} }
val chapterDirDeleted = chapterDir.deleteRecursively() val chapterDirDeleted = chapterDir.deleteRecursively()
if (chapterDirDeleted) {
transaction {
ChapterTable.update({ ChapterTable.id eq chapterId }) {
it[koreaderHash] = null
}
}
}
FileDeletionHelper.cleanupParentFoldersFor(chapterDir, applicationDirs.mangaDownloadsRoot) FileDeletionHelper.cleanupParentFoldersFor(chapterDir, applicationDirs.mangaDownloadsRoot)
return chapterDirDeleted return chapterDirDeleted
} }

View File

@@ -0,0 +1,425 @@
package suwayomi.tachidesk.manga.impl.sync
import eu.kanade.tachiyomi.network.NetworkHelper
import eu.kanade.tachiyomi.network.await
import eu.kanade.tachiyomi.util.lang.Hash
import io.github.oshai.kotlinlogging.KotlinLogging
import io.javalin.json.JsonMapper
import kotlinx.serialization.Serializable
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.buildJsonObject
import kotlinx.serialization.json.put
import okhttp3.MediaType.Companion.toMediaType
import okhttp3.Request
import okhttp3.RequestBody.Companion.toRequestBody
import org.jetbrains.exposed.sql.transactions.transaction
import org.jetbrains.exposed.sql.update
import suwayomi.tachidesk.graphql.types.KoSyncStatusPayload
import suwayomi.tachidesk.graphql.types.KoreaderSyncChecksumMethod
import suwayomi.tachidesk.graphql.types.KoreaderSyncStrategy
import suwayomi.tachidesk.manga.impl.util.KoreaderHelper
import suwayomi.tachidesk.manga.impl.util.getChapterCbzPath
import suwayomi.tachidesk.manga.model.table.ChapterTable
import suwayomi.tachidesk.manga.model.table.MangaTable
import suwayomi.tachidesk.server.serverConfig
import uy.kohesive.injekt.injectLazy
import java.io.File
import java.util.UUID
import kotlin.math.abs
object KoreaderSyncService {
private val logger = KotlinLogging.logger {}
private val network: NetworkHelper by injectLazy()
private val json: Json by injectLazy()
private val jsonMapper: JsonMapper by injectLazy()
@Serializable
private data class KoreaderProgressPayload(
val document: String,
val progress: String,
val percentage: Float,
val device: String,
val device_id: String,
)
@Serializable
private data class KoreaderProgressResponse(
val document: String? = null,
val progress: String? = null,
val percentage: Float? = null,
val timestamp: Long? = null,
val device: String? = null,
val device_id: String? = null,
)
@Serializable
data class SyncResult(
val pageRead: Int,
val timestamp: Long, // Unix timestamp in seconds
val device: String,
val shouldUpdate: Boolean = false,
val isConflict: Boolean = false,
)
data class ConnectResult(
val success: Boolean,
val message: String? = null,
val username: String? = null,
)
private data class AuthResult(
val success: Boolean,
val message: String? = null,
val isUserNotFoundError: Boolean = false,
)
private fun buildRequest(
url: String,
block: Request.Builder.() -> Unit,
): Request =
Request
.Builder()
.url(url)
.addHeader("Accept", "application/vnd.koreader.v1+json")
.apply(block)
.build()
private suspend fun getOrGenerateDeviceId(): String {
var deviceId = serverConfig.koreaderSyncDeviceId.value
if (deviceId.isBlank()) {
deviceId =
UUID
.randomUUID()
.toString()
.replace("-", "")
.uppercase()
logger.info { "[KOSYNC] Generated new KOSync Device ID: $deviceId" }
serverConfig.koreaderSyncDeviceId.value = deviceId
}
return deviceId
}
private fun getOrGenerateChapterHash(chapterId: Int): String? {
return transaction {
val existingHash =
ChapterTable
.select(ChapterTable.koreaderHash)
.where { ChapterTable.id eq chapterId }
.firstOrNull()
?.get(ChapterTable.koreaderHash)
if (!existingHash.isNullOrBlank()) {
return@transaction existingHash
}
val checksumMethod = serverConfig.koreaderSyncChecksumMethod.value
val newHash =
when (checksumMethod) {
KoreaderSyncChecksumMethod.BINARY -> {
logger.info { "[KOSYNC HASH] No hash for chapterId=$chapterId. Generating from CBZ content." }
val mangaId =
ChapterTable
.select(ChapterTable.manga)
.where { ChapterTable.id eq chapterId }
.firstOrNull()
?.get(ChapterTable.manga)
?.value ?: return@transaction null
val cbzFile = File(getChapterCbzPath(mangaId, chapterId))
if (!cbzFile.exists()) {
logger.info { "[KOSYNC HASH] Could not generate hash for chapterId=$chapterId. CBZ not found." }
return@transaction null
}
KoreaderHelper.hashContents(cbzFile)
}
KoreaderSyncChecksumMethod.FILENAME -> {
logger.info { "[KOSYNC HASH] No hash for chapterId=$chapterId. Generating from filename." }
(ChapterTable innerJoin MangaTable)
.select(ChapterTable.name, MangaTable.title)
.where { ChapterTable.id eq chapterId }
.firstOrNull()
?.let {
val chapterName = it[ChapterTable.name]
val mangaTitle = it[MangaTable.title]
val baseFilename = "$mangaTitle - $chapterName".split('.').dropLast(1).joinToString(".")
Hash.md5(baseFilename)
}
}
}
if (newHash != null) {
ChapterTable.update({ ChapterTable.id eq chapterId }) {
it[koreaderHash] = newHash
}
logger.info { "[KOSYNC HASH] Generated and saved new hash for chapterId=$chapterId" }
} else {
logger.warn { "[KOSYNC HASH] Hashing failed for chapterId=$chapterId." }
}
newHash
}
}
private suspend fun register(
username: String,
userkey: String,
): AuthResult {
val payload =
buildJsonObject {
put("username", username)
put("password", userkey)
}
val request =
buildRequest("${serverConfig.koreaderSyncServerUrl.value.removeSuffix("/")}/users/create") {
post(payload.toString().toRequestBody("application/json".toMediaType()))
}
return try {
network.client.newCall(request).await().use { response ->
if (response.isSuccessful) {
AuthResult(true, "Registration successful.")
} else {
val errorBody = response.body.string()
val errorMessage =
runCatching {
jsonMapper.fromJsonString<Map<String, String>>(
errorBody,
Map::class.java,
)["message"]
}.getOrNull()
val finalMessage = errorMessage ?: "Registration failed with code ${response.code}"
AuthResult(false, finalMessage)
}
}
} catch (e: Exception) {
logger.error(e) { "[KOSYNC REGISTER] Exception" }
AuthResult(false, e.message)
}
}
private suspend fun authorize(
username: String,
userkey: String,
): AuthResult {
val request =
buildRequest("${serverConfig.koreaderSyncServerUrl.value.removeSuffix("/")}/users/auth") {
get()
addHeader("x-auth-user", username)
addHeader("x-auth-key", userkey)
}
return try {
network.client.newCall(request).await().use { response ->
if (response.isSuccessful) {
AuthResult(true)
} else {
val isUserNotFound = response.code == 401 // Unauthorized often means user/pass combo is wrong
AuthResult(false, "Authorization failed with code ${response.code}", isUserNotFoundError = isUserNotFound)
}
}
} catch (e: Exception) {
logger.error(e) { "[KOSYNC AUTHORIZE] Exception" }
AuthResult(false, e.message)
}
}
suspend fun connect(
username: String,
password: String,
): ConnectResult {
val userkey = Hash.md5(password)
val authResult = authorize(username, userkey)
if (authResult.success) {
serverConfig.koreaderSyncUsername.value = username
serverConfig.koreaderSyncUserkey.value = userkey
return ConnectResult(true, "Login successful.", username)
}
if (authResult.isUserNotFoundError) {
logger.info { "[KOSYNC CONNECT] Authorization failed, attempting to register new user." }
val registerResult = register(username, userkey)
return if (registerResult.success) {
serverConfig.koreaderSyncUsername.value = username
serverConfig.koreaderSyncUserkey.value = userkey
ConnectResult(true, "Registration successful.", username)
} else {
ConnectResult(false, registerResult.message ?: "Registration failed.", null)
}
}
return ConnectResult(false, authResult.message ?: "Authentication failed.", null)
}
suspend fun logout() {
serverConfig.koreaderSyncUsername.value = ""
serverConfig.koreaderSyncUserkey.value = ""
}
suspend fun getStatus(): KoSyncStatusPayload {
val username = serverConfig.koreaderSyncUsername.value
val userkey = serverConfig.koreaderSyncUserkey.value
if (username.isBlank() || userkey.isBlank()) {
return KoSyncStatusPayload(isLoggedIn = false, username = null)
}
val authResult = authorize(username, userkey)
return KoSyncStatusPayload(isLoggedIn = authResult.success, username = if (authResult.success) username else null)
}
suspend fun pushProgress(chapterId: Int) {
val strategy = serverConfig.koreaderSyncStrategy.value
if (strategy == KoreaderSyncStrategy.DISABLED || strategy == KoreaderSyncStrategy.RECEIVE) return
val username = serverConfig.koreaderSyncUsername.value
val userkey = serverConfig.koreaderSyncUserkey.value
if (username.isBlank() || userkey.isBlank()) return
logger.info { "[KOSYNC PUSH] Init." }
val chapterHash = getOrGenerateChapterHash(chapterId)
if (chapterHash.isNullOrBlank()) {
logger.info { "[KOSYNC PUSH] Aborted for chapterId=$chapterId: No hash." }
return
}
val chapterInfo =
transaction {
ChapterTable
.select(ChapterTable.lastPageRead, ChapterTable.pageCount)
.where { ChapterTable.id eq chapterId }
.firstOrNull()
?.let {
object {
val lastPageRead = it[ChapterTable.lastPageRead]
val pageCount = it[ChapterTable.pageCount]
}
}
} ?: return
if (chapterInfo.pageCount <= 0) {
logger.warn { "[KOSYNC PUSH] Aborted for chapterId=$chapterId: Invalid pageCount." }
return
}
try {
val deviceId = getOrGenerateDeviceId()
val payload =
KoreaderProgressPayload(
document = chapterHash,
progress = (chapterInfo.lastPageRead + 1).toString(),
percentage = (chapterInfo.lastPageRead + 1).toFloat() / chapterInfo.pageCount.toFloat(),
device = "Suwayomi-Server (${System.getProperty("os.name")})",
device_id = deviceId,
)
val requestBody = json.encodeToString(KoreaderProgressPayload.serializer(), payload)
val request =
buildRequest("${serverConfig.koreaderSyncServerUrl.value.removeSuffix("/")}/syncs/progress") {
put(requestBody.toRequestBody("application/json".toMediaType()))
addHeader("x-auth-user", username)
addHeader("x-auth-key", userkey)
}
network.client.newCall(request).await().use { response ->
if (!response.isSuccessful) {
logger.warn { "[KOSYNC PUSH] Failed for chapterId=$chapterId: ${response.code}" }
} else {
logger.info { "[KOSYNC PUSH] Success for chapterId=$chapterId" }
}
}
} catch (e: Exception) {
logger.error(e) { "[KOSYNC PUSH] Exception for chapterId=$chapterId" }
}
}
suspend fun checkAndPullProgress(chapterId: Int): SyncResult? {
val strategy = serverConfig.koreaderSyncStrategy.value
if (strategy == KoreaderSyncStrategy.DISABLED || strategy == KoreaderSyncStrategy.SEND) return null
val username = serverConfig.koreaderSyncUsername.value
val userkey = serverConfig.koreaderSyncUserkey.value
if (username.isBlank() || userkey.isBlank()) return null
val chapterHash = getOrGenerateChapterHash(chapterId)
if (chapterHash.isNullOrBlank()) {
logger.info { "[KOSYNC PULL] Aborted for chapterId=$chapterId: No hash." }
return null
}
try {
val request =
buildRequest("${serverConfig.koreaderSyncServerUrl.value.removeSuffix("/")}/syncs/progress/$chapterHash") {
get()
addHeader("x-auth-user", username)
addHeader("x-auth-key", userkey)
}
network.client.newCall(request).await().use { response ->
if (response.isSuccessful) {
val body = response.body.string()
if (body.isBlank() || body == "{}") return null
val progressResponse = json.decodeFromString(KoreaderProgressResponse.serializer(), body)
val pageRead = progressResponse.progress?.toIntOrNull()?.minus(1)
val timestamp = progressResponse.timestamp
val device = progressResponse.device ?: "KOReader"
val localProgress =
transaction {
ChapterTable
.select(ChapterTable.lastReadAt, ChapterTable.lastPageRead, ChapterTable.pageCount)
.where { ChapterTable.id eq chapterId }
.firstOrNull()
?.let {
object {
val lastReadAt = it[ChapterTable.lastReadAt]
val lastPageRead = it[ChapterTable.lastPageRead]
val pageCount = it[ChapterTable.pageCount]
}
}
}
if (pageRead != null && timestamp != null) {
// Ignore XPath progress for now as we only support paginated files
if (progressResponse.progress?.startsWith("/") == true) {
return null
}
val localPercentage =
if (localProgress?.pageCount ?: 0 >
0
) {
(localProgress!!.lastPageRead + 1).toFloat() / localProgress.pageCount
} else {
0f
}
val percentageDifference = abs(localPercentage - (progressResponse.percentage ?: 0f))
// Progress is within tolerance, no sync needed
if (percentageDifference < serverConfig.koreaderSyncPercentageTolerance.value) {
return null
}
when (strategy) {
KoreaderSyncStrategy.RECEIVE -> {
return SyncResult(pageRead, timestamp, device, shouldUpdate = true)
}
KoreaderSyncStrategy.SILENT -> {
if (timestamp > (localProgress?.lastReadAt ?: 0L)) {
return SyncResult(pageRead, timestamp, device, shouldUpdate = true)
}
}
KoreaderSyncStrategy.PROMPT -> {
return SyncResult(pageRead, timestamp, device, isConflict = true)
}
else -> {} // SEND and DISABLED already handled at the start of the function
}
}
} else {
logger.warn { "[KOSYNC PULL] Failed for chapterId=$chapterId: ${response.code}" }
}
}
} catch (e: Exception) {
logger.error(e) { "[KOSYNC PULL] Exception for chapterId=$chapterId" }
}
return null
}
}

View File

@@ -0,0 +1,48 @@
package suwayomi.tachidesk.manga.impl.util
import java.io.File
import java.security.MessageDigest
object KoreaderHelper {
// Helper function to convert ByteArray to Hex String
private fun ByteArray.toHexString() = joinToString("") { "%02x".format(it) }
/**
* Hashes the document according to a custom Koreader hashing algorithm.
* https://github.com/koreader/koreader/blob/master/frontend/util.lua#L1040
* Only applies to epub and cbz files.
* @param file The file object to hash.
* @return The lowercase MD5 hash or null if hashing is not possible.
*/
fun hashContents(file: File): String? {
val extension = file.extension.lowercase()
if (!file.exists() || (extension != "epub" && extension != "cbz")) {
return null
}
try {
file.inputStream().use { fs ->
val step = 1024
val size = 1024
val md5 = MessageDigest.getInstance("MD5")
val buffer = ByteArray(size)
for (i in -1 until 10) {
val position = (step shl (2 * i)).toLong()
if (position >= file.length()) break // Avoid seeking past the end of small files
fs.channel.position(position)
val bytesRead = fs.read(buffer, 0, size)
if (bytesRead > 0) {
md5.update(buffer, 0, bytesRead)
} else {
break
}
}
return md5.digest().toHexString().lowercase()
}
} catch (e: Exception) {
// TODO: Should we log this error?
return null
}
}
}

View File

@@ -39,6 +39,8 @@ object ChapterTable : IntIdTable() {
val pageCount = integer("page_count").default(-1) val pageCount = integer("page_count").default(-1)
val manga = reference("manga", MangaTable, ReferenceOption.CASCADE) val manga = reference("manga", MangaTable, ReferenceOption.CASCADE)
val koreaderHash = varchar("koreader_hash", 32).nullable()
} }
fun ChapterTable.toDataClass( fun ChapterTable.toDataClass(

View File

@@ -6,6 +6,7 @@ import kotlinx.coroutines.withContext
import suwayomi.tachidesk.i18n.MR import suwayomi.tachidesk.i18n.MR
import suwayomi.tachidesk.manga.impl.ChapterDownloadHelper import suwayomi.tachidesk.manga.impl.ChapterDownloadHelper
import suwayomi.tachidesk.manga.impl.MangaList.proxyThumbnailUrl import suwayomi.tachidesk.manga.impl.MangaList.proxyThumbnailUrl
import suwayomi.tachidesk.manga.impl.sync.KoreaderSyncService
import suwayomi.tachidesk.manga.model.table.MangaStatus import suwayomi.tachidesk.manga.model.table.MangaStatus
import suwayomi.tachidesk.opds.constants.OpdsConstants import suwayomi.tachidesk.opds.constants.OpdsConstants
import suwayomi.tachidesk.opds.dto.OpdsChapterListAcqEntry import suwayomi.tachidesk.opds.dto.OpdsChapterListAcqEntry
@@ -185,7 +186,6 @@ object OpdsEntryBuilder {
append(MR.strings.opds_chapter_details_progress.localized(locale, chapter.lastPageRead, chapter.pageCount)) append(MR.strings.opds_chapter_details_progress.localized(locale, chapter.lastPageRead, chapter.pageCount))
} }
} }
return OpdsEntryXml( return OpdsEntryXml(
id = "urn:suwayomi:chapter:${chapter.id}", id = "urn:suwayomi:chapter:${chapter.id}",
title = entryTitle, title = entryTitle,
@@ -222,11 +222,23 @@ object OpdsEntryBuilder {
baseUrl: String, baseUrl: String,
locale: Locale, locale: Locale,
): OpdsEntryXml { ): OpdsEntryXml {
// Check remote progress before building the entry
val syncResult = KoreaderSyncService.checkAndPullProgress(chapter.id)
val localLastPageRead = chapter.lastPageRead
val remoteLastPageRead = syncResult?.pageRead
val remoteLastReadAt = syncResult?.timestamp
val showConflict = syncResult?.isConflict == true && remoteLastPageRead != null && localLastPageRead != remoteLastPageRead
val finalLastPageRead = if (syncResult?.shouldUpdate == true) remoteLastPageRead ?: localLastPageRead else localLastPageRead
val finalLastReadAt = if (syncResult?.shouldUpdate == true) remoteLastReadAt ?: chapter.lastReadAt else chapter.lastReadAt
val statusKey = val statusKey =
when { when {
chapter.downloaded -> MR.strings.opds_chapter_status_downloaded chapter.downloaded -> MR.strings.opds_chapter_status_downloaded
chapter.read -> MR.strings.opds_chapter_status_read chapter.read -> MR.strings.opds_chapter_status_read
chapter.lastPageRead > 0 -> MR.strings.opds_chapter_status_in_progress finalLastPageRead > 0 -> MR.strings.opds_chapter_status_in_progress
else -> MR.strings.opds_chapter_status_unread else -> MR.strings.opds_chapter_status_unread
} }
val titlePrefix = statusKey.localized(locale) val titlePrefix = statusKey.localized(locale)
@@ -238,7 +250,7 @@ object OpdsEntryBuilder {
append(MR.strings.opds_chapter_details_scanlator.localized(locale, it)) append(MR.strings.opds_chapter_details_scanlator.localized(locale, it))
} }
val pageCountDisplay = chapter.pageCount.takeIf { it > 0 } ?: "?" val pageCountDisplay = chapter.pageCount.takeIf { it > 0 } ?: "?"
append(MR.strings.opds_chapter_details_progress.localized(locale, chapter.lastPageRead, pageCountDisplay)) append(MR.strings.opds_chapter_details_progress.localized(locale, finalLastPageRead, pageCountDisplay))
} }
val links = mutableListOf<OpdsLinkXml>() val links = mutableListOf<OpdsLinkXml>()
var cbzFileSize: Long? = null var cbzFileSize: Long? = null
@@ -270,19 +282,73 @@ object OpdsEntryBuilder {
} }
} }
if (chapter.pageCount > 0) { if (chapter.pageCount > 0) {
val basePageHref =
"/api/v1/manga/${manga.id}/chapter/${chapter.sourceOrder}/page/{pageNumber}" +
"?updateProgress=${serverConfig.opdsEnablePageReadProgress.value}"
if (showConflict) {
// Option 1: Local progress
val localTitleRes =
if (localLastPageRead >
0
) {
MR.strings.opds_linktitle_stream_pages_continue_local
} else {
MR.strings.opds_linktitle_stream_pages_start_local
}
links.add( links.add(
OpdsLinkXml( OpdsLinkXml(
rel = OpdsConstants.LINK_REL_PSE_STREAM, rel = OpdsConstants.LINK_REL_PSE_STREAM,
href = href = basePageHref,
"/api/v1/manga/${manga.id}/chapter/${chapter.sourceOrder}/page/{pageNumber}" +
"?updateProgress=${serverConfig.opdsEnablePageReadProgress.value}",
type = OpdsConstants.TYPE_IMAGE_JPEG, type = OpdsConstants.TYPE_IMAGE_JPEG,
title = MR.strings.opds_linktitle_stream_pages.localized(locale), title = localTitleRes.localized(locale),
pseCount = chapter.pageCount, pseCount = chapter.pageCount,
pseLastRead = chapter.lastPageRead.takeIf { it > 0 }, pseLastRead = localLastPageRead.takeIf { it >= 0 },
pseLastReadDate = chapter.lastReadAt.takeIf { it > 0 }?.let { OpdsDateUtil.formatEpochMillisForOpds(it * 1000) }, pseLastReadDate = chapter.lastReadAt.takeIf { it > 0 }?.let { OpdsDateUtil.formatEpochMillisForOpds(it * 1000) },
), ),
) )
// Option 2: Remote progress
val remoteTitleRes =
if (remoteLastPageRead >
0
) {
MR.strings.opds_linktitle_stream_pages_continue_remote
} else {
MR.strings.opds_linktitle_stream_pages_start_remote
}
links.add(
OpdsLinkXml(
rel = OpdsConstants.LINK_REL_PSE_STREAM,
href = basePageHref,
type = OpdsConstants.TYPE_IMAGE_JPEG,
title = remoteTitleRes.localized(locale, syncResult.device),
pseCount = chapter.pageCount,
pseLastRead = remoteLastPageRead,
pseLastReadDate = remoteLastReadAt?.takeIf { it > 0 }?.let { OpdsDateUtil.formatEpochMillisForOpds(it * 1000) },
),
)
} else {
// Normal behavior: single progress link
val titleRes =
if (finalLastPageRead >
0
) {
MR.strings.opds_linktitle_stream_pages_continue
} else {
MR.strings.opds_linktitle_stream_pages_start
}
links.add(
OpdsLinkXml(
rel = OpdsConstants.LINK_REL_PSE_STREAM,
href = basePageHref,
type = OpdsConstants.TYPE_IMAGE_JPEG,
title = titleRes.localized(locale),
pseCount = chapter.pageCount,
pseLastRead = finalLastPageRead.takeIf { it > 0 },
pseLastReadDate = finalLastReadAt.takeIf { it > 0 }?.let { OpdsDateUtil.formatEpochMillisForOpds(it * 1000) },
),
)
}
links.add( links.add(
OpdsLinkXml( OpdsLinkXml(
rel = OpdsConstants.LINK_REL_IMAGE, rel = OpdsConstants.LINK_REL_IMAGE,

View File

@@ -26,6 +26,8 @@ import kotlinx.coroutines.flow.mapLatest
import kotlinx.coroutines.flow.onEach import kotlinx.coroutines.flow.onEach
import org.jetbrains.exposed.sql.SortOrder import org.jetbrains.exposed.sql.SortOrder
import suwayomi.tachidesk.graphql.types.AuthMode import suwayomi.tachidesk.graphql.types.AuthMode
import suwayomi.tachidesk.graphql.types.KoreaderSyncChecksumMethod
import suwayomi.tachidesk.graphql.types.KoreaderSyncStrategy
import suwayomi.tachidesk.graphql.types.WebUIChannel import suwayomi.tachidesk.graphql.types.WebUIChannel
import suwayomi.tachidesk.graphql.types.WebUIFlavor import suwayomi.tachidesk.graphql.types.WebUIFlavor
import suwayomi.tachidesk.graphql.types.WebUIInterface import suwayomi.tachidesk.graphql.types.WebUIInterface
@@ -195,6 +197,15 @@ class ServerConfig(
val opdsShowOnlyDownloadedChapters: MutableStateFlow<Boolean> by OverrideConfigValue() val opdsShowOnlyDownloadedChapters: MutableStateFlow<Boolean> by OverrideConfigValue()
val opdsChapterSortOrder: MutableStateFlow<SortOrder> by OverrideConfigValue() val opdsChapterSortOrder: MutableStateFlow<SortOrder> by OverrideConfigValue()
// koreader sync
val koreaderSyncServerUrl: MutableStateFlow<String> by OverrideConfigValue()
val koreaderSyncUsername: MutableStateFlow<String> by OverrideConfigValue()
val koreaderSyncUserkey: MutableStateFlow<String> by OverrideConfigValue()
val koreaderSyncDeviceId: MutableStateFlow<String> by OverrideConfigValue()
val koreaderSyncChecksumMethod: MutableStateFlow<KoreaderSyncChecksumMethod> by OverrideConfigValue()
val koreaderSyncStrategy: MutableStateFlow<KoreaderSyncStrategy> by OverrideConfigValue()
val koreaderSyncPercentageTolerance: MutableStateFlow<Double> by OverrideConfigValue()
@OptIn(ExperimentalCoroutinesApi::class) @OptIn(ExperimentalCoroutinesApi::class)
fun <T> subscribeTo( fun <T> subscribeTo(
flow: Flow<T>, flow: Flow<T>,

View File

@@ -0,0 +1,19 @@
package suwayomi.tachidesk.server.database.migration
/*
* Copyright (C) Contributors to the Suwayomi project
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at https://mozilla.org/MPL/2.0/. */
import de.neonew.exposed.migrations.helpers.AddColumnMigration
@Suppress("ClassName", "unused")
class M0051_AddKoreaderHashToChapterTable :
AddColumnMigration(
"Chapter",
"koreader_hash",
"VARCHAR(32)",
"NULL",
)

View File

@@ -85,3 +85,12 @@ server.opdsMarkAsReadOnDownload = false
server.opdsShowOnlyUnreadChapters = false server.opdsShowOnlyUnreadChapters = false
server.opdsShowOnlyDownloadedChapters = false server.opdsShowOnlyDownloadedChapters = false
server.opdsChapterSortOrder = "DESC" # "ASC", "DESC" server.opdsChapterSortOrder = "DESC" # "ASC", "DESC"
# Koreader Sync
server.koreaderSyncServerUrl = "http://localhost:17200"
server.koreaderSyncUsername = ""
server.koreaderSyncUserkey = ""
server.koreaderSyncDeviceId = ""
server.koreaderSyncChecksumMethod = "binary" # "binary" or "filename"
server.koreaderSyncStrategy = "disabled" # "prompt", "silent", "send", "receive", "disabled"
server.koreaderSyncPercentageTolerance = 0.00000000000001 # absolute tolerance for progress comparison from 1 (widest) to 1e-15 (strict)