feat(sync/koreader): Add KOReader reading progress synchronization (#1560)

* feat(sync/koreader): implement reading progress synchronization

This commit introduces a comprehensive integration with KOReader Sync Server to enable two-way synchronization of reading progress.

The core logic is encapsulated in a new `KoreaderSyncService`, which handles authentication, registration, and progress pushing/pulling based on user-defined strategies (LATEST, KOSYNC, SUWAYOMI).

Key changes include:

- A new GraphQL API is added to manage the KOReader Sync connection:
  - `connectKoSyncAccount` mutation provides a simplified flow that attempts to log in and, if the user doesn't exist, automatically registers them.
  - `logoutKoSyncAccount` mutation to clear credentials.
  - `koSyncStatus` query to check the current connection status.

- Reading progress is now synchronized at key points:
  - The `fetchChapterPages` mutation pulls the latest progress from the sync server before loading the reader. It respects the configured sync strategy and updates the local database if necessary.
  - The `updateChapters` and other progress-updating methods now push changes to the sync server automatically.
- OPDS chapter entries also pull the latest progress, ensuring clients receive up-to-date reading status.

- Supporting backend changes have been made:
  - The `Chapter` table is extended with a `koreader_hash` column to uniquely identify documents. A database migration is included.
  - New configuration options are added to `server.conf` to manage the feature (enable, server URL, credentials, strategy, etc.).

* perf(opds): defer KOReader sync to improve chapter feed performance

Removes the KOReader Sync progress-pulling logic from the `createChapterListEntry` function.

The previous implementation triggered a network request to the sync server for every single chapter when generating a list, leading to severe performance issues and slow load times on feeds with many entries.

This change reverts to the more performant approach of always linking to the chapter's metadata feed. The expensive sync operation will be handled within the metadata entry generation instead, ensuring it's only triggered on-demand for a single chapter. This restores the responsiveness of browsing chapter feeds.

* refactor(koreader): Use enums for sync settings and correct OPDS logic

Refactor Koreader Sync settings to use enums instead of raw strings for `checksumMethod` and `strategy`. This improves type safety, prevents typos, and makes the configuration handling more robust.

The changes include:
- Introducing `KoreaderSyncChecksumMethod` and `KoreaderSyncStrategy` enums.
- Updating `ServerConfig`, GraphQL types, and backup models to use these new enums.
- Refactoring `KoreaderSyncService` to work with the enum types.

Additionally, this commit fixes an issue in `OpdsEntryBuilder` where the logic for determining which sync progress to use (local vs. remote) was duplicated. The builder now correctly delegates this decision to `KoreaderSyncService.pullProgress`, which already contains the necessary strategy logic. This centralizes the logic and ensures consistent behavior.

* refactor(koreader): Improve config handling and remove redundant update

This commit combines several refactoring and cleanup tasks:

- **Koreader Sync:** The sync service is updated to use the modern `serverConfig` provider instead of the legacy `GlobalConfigManager`. This aligns it with the current configuration management approach in the project.

- **Download Provider:** A redundant `pageCount` database update is removed from `ChaptersFilesProvider`. This operation was unnecessary because the `getChapterDownloadReady` function, which is called earlier in the download process, already verifies and corrects the page count. This change eliminates a superfluous database write and fixes a related import issue.

* feat(sync/koreader)!: enhance sync strategy and add progress tolerance

This commit overhauls the KOReader synchronization feature to provide more granular control and robustness. The simple on/off toggle has been replaced with a more flexible strategy-based system.

Key changes include:
- Replaced `koreaderSyncEnabled` with a more powerful `koreaderSyncStrategy` enum.
- Introduced new sync strategies: `PROMPT`, `SILENT`, `SEND`, `RECEIVE`, and `DISABLE`, allowing for fine-grained control over the sync direction and conflict resolution.
- Added a `koreaderSyncPercentageTolerance` setting. This prevents unnecessary sync updates for minor progress differences between Suwayomi and KOReader.
- Refactored the `KoreaderSyncService` to implement the new strategies and use the configurable tolerance.
- Updated GraphQL schemas, mutations, and server configuration to remove the old setting and incorporate the new ones.
- Adjusted the backup and restore process to correctly handle the new configuration parameters.
- Modified API endpoints and internal logic to check and apply remote progress based on the selected strategy.

BREAKING CHANGE: The `koreaderSyncEnabled` setting is removed and replaced by a more granular `koreaderSyncStrategy`. The enum values for the strategy have been completely changed, making previous configurations incompatible.

* fix: remove unused imports

* feat(opds, sync): enhance Koreader sync and OPDS conflict handling

This commit introduces significant improvements to the Koreader synchronization feature, focusing on providing a better user experience for handling progress conflicts in both OPDS and GraphQL clients.

Key changes include:

- **OPDS Conflict Resolution:** When a reading progress conflict is detected, the OPDS feed for a chapter now provides two distinct "Read Online" links: one to continue from the local progress and another to continue from the synced progress from the remote device (e.g., "Continue Reading Online (Synced from KOReader)"). This empowers users to choose which progress to follow.

- **GraphQL Sync Conflict Information:** The `fetchChapterPages` GraphQL mutation now includes a `syncConflict` field in its payload. This field provides the remote device name and page number, allowing GraphQL clients to implement a user-facing prompt to resolve sync conflicts.

- **Improved Sync Strategy Handling:**
  - The `connectKoSyncAccount` mutation no longer unconditionally sets the sync strategy to `PROMPT`. It now respects the user's existing setting, preventing accidental configuration changes upon re-login.
  - The default `koreaderSyncStrategy` in the configuration is changed to `DISABLED`, providing a safer and more intuitive default for new users.

- **Refinements & Fixes:**
  - The fallback for the remote device name is now centralized within the KoreaderSyncService, defaulting to "KOReader" if not provided.
  - Renamed `KoreaderSyncStrategy.DISABLE` to `DISABLED` for consistency.
  - Updated i18n strings for OPDS links to be more descriptive and user-friendly.

* refactor(kosync): rename stream page link titles for consistency

* refactor(kosync): return SettingsType in auth mutation payloads

The `connectKoSyncAccount` and `logoutKoSyncAccount` mutations modify server settings (username and userkey) but did not previously return the updated configuration. This forced client applications to manually refetch settings to avoid a stale cache.

This change modifies the payloads for both mutations to include the full `SettingsType`.

By returning the updated settings directly, GraphQL clients like Apollo Client can automatically update their cache, simplifying client-side state management and ensuring the UI always reflects the current server configuration.

Additionally, `clientMutationId` has been added to `KoSyncConnectPayload` for consistency with GraphQL practices, aligning it with the logout mutation.

Refs: #1560

* refactor(kosync): replace KoSyncConnectPayload with ConnectResult in connect method

* fix(kosync): add koreaderSyncPercentageTolerance default setting
This commit is contained in:
Zeedif
2025-08-19 13:00:19 -06:00
committed by GitHub
parent 16e9c0b19a
commit 590e43c827
23 changed files with 926 additions and 50 deletions

View File

@@ -431,33 +431,46 @@ object Chapter {
isBookmarked: Boolean?,
markPrevRead: Boolean?,
lastPageRead: Int?,
) {
transaction {
if (listOf(isRead, isBookmarked, lastPageRead).any { it != null }) {
ChapterTable.update({ (ChapterTable.manga eq mangaId) and (ChapterTable.sourceOrder eq chapterIndex) }) { update ->
isRead?.also {
update[ChapterTable.isRead] = it
}
isBookmarked?.also {
update[ChapterTable.isBookmarked] = it
}
lastPageRead?.also {
update[ChapterTable.lastPageRead] = it
update[lastReadAt] = Instant.now().epochSecond
}
}
}
): Int {
val chapterId =
transaction {
val chapter =
ChapterTable
.selectAll()
.where { (ChapterTable.manga eq mangaId) and (ChapterTable.sourceOrder eq chapterIndex) }
.first()
markPrevRead?.let {
ChapterTable.update({ (ChapterTable.manga eq mangaId) and (ChapterTable.sourceOrder less chapterIndex) }) {
it[ChapterTable.isRead] = markPrevRead
val chapterIdValue = chapter[ChapterTable.id].value
if (listOf(isRead, isBookmarked, lastPageRead).any { it != null }) {
ChapterTable.update({ (ChapterTable.manga eq mangaId) and (ChapterTable.sourceOrder eq chapterIndex) }) { update ->
isRead?.also {
update[ChapterTable.isRead] = it
}
isBookmarked?.also {
update[ChapterTable.isBookmarked] = it
}
lastPageRead?.also {
update[ChapterTable.lastPageRead] = it
update[lastReadAt] = Instant.now().epochSecond
}
}
}
markPrevRead?.let {
ChapterTable.update({ (ChapterTable.manga eq mangaId) and (ChapterTable.sourceOrder less chapterIndex) }) {
it[ChapterTable.isRead] = markPrevRead
}
}
chapterIdValue
}
}
if (isRead == true || markPrevRead == true) {
Track.asyncTrackChapter(setOf(mangaId))
}
return chapterId
}
@Serializable
@@ -733,7 +746,7 @@ object Chapter {
mangaId: Int,
chapterIndex: Int,
pageNo: Int,
) {
): Int {
val chapterData =
transaction {
ChapterTable
@@ -756,5 +769,7 @@ object Chapter {
isBookmarked = null,
markPrevRead = null,
)
return chapterData.id
}
}

View File

@@ -463,6 +463,14 @@ object ProtoBackupExport : ProtoBackupBase() {
opdsShowOnlyUnreadChapters = serverConfig.opdsShowOnlyUnreadChapters.value,
opdsShowOnlyDownloadedChapters = serverConfig.opdsShowOnlyDownloadedChapters.value,
opdsChapterSortOrder = serverConfig.opdsChapterSortOrder.value,
// koreader sync
koreaderSyncServerUrl = serverConfig.koreaderSyncServerUrl.value,
koreaderSyncUsername = serverConfig.koreaderSyncUsername.value,
koreaderSyncUserkey = serverConfig.koreaderSyncUserkey.value,
koreaderSyncDeviceId = serverConfig.koreaderSyncDeviceId.value,
koreaderSyncChecksumMethod = serverConfig.koreaderSyncChecksumMethod.value,
koreaderSyncStrategy = serverConfig.koreaderSyncStrategy.value,
koreaderSyncPercentageTolerance = serverConfig.koreaderSyncPercentageTolerance.value,
)
}
}

View File

@@ -4,6 +4,8 @@ import kotlinx.serialization.Serializable
import kotlinx.serialization.protobuf.ProtoNumber
import org.jetbrains.exposed.sql.SortOrder
import suwayomi.tachidesk.graphql.types.AuthMode
import suwayomi.tachidesk.graphql.types.KoreaderSyncChecksumMethod
import suwayomi.tachidesk.graphql.types.KoreaderSyncStrategy
import suwayomi.tachidesk.graphql.types.Settings
import suwayomi.tachidesk.graphql.types.SettingsDownloadConversion
import suwayomi.tachidesk.graphql.types.WebUIChannel
@@ -84,6 +86,14 @@ data class BackupServerSettings(
@ProtoNumber(53) override var opdsShowOnlyUnreadChapters: Boolean,
@ProtoNumber(54) override var opdsShowOnlyDownloadedChapters: Boolean,
@ProtoNumber(55) override var opdsChapterSortOrder: SortOrder,
// koreader sync
@ProtoNumber(59) override var koreaderSyncServerUrl: String,
@ProtoNumber(60) override var koreaderSyncUsername: String,
@ProtoNumber(61) override var koreaderSyncUserkey: String,
@ProtoNumber(62) override var koreaderSyncDeviceId: String,
@ProtoNumber(63) override var koreaderSyncChecksumMethod: KoreaderSyncChecksumMethod,
@ProtoNumber(64) override var koreaderSyncStrategy: KoreaderSyncStrategy,
@ProtoNumber(65) override var koreaderSyncPercentageTolerance: Double,
) : Settings {
@Serializable
class BackupSettingsDownloadConversionType(

View File

@@ -13,11 +13,14 @@ import libcore.net.MimeUtils
import org.apache.commons.compress.archivers.zip.ZipArchiveEntry
import org.jetbrains.exposed.sql.selectAll
import org.jetbrains.exposed.sql.transactions.transaction
import org.jetbrains.exposed.sql.update
import suwayomi.tachidesk.manga.impl.Page
import suwayomi.tachidesk.manga.impl.chapter.getChapterDownloadReady
import suwayomi.tachidesk.manga.impl.download.model.DownloadChapter
import suwayomi.tachidesk.manga.impl.util.KoreaderHelper
import suwayomi.tachidesk.manga.impl.util.createComicInfoFile
import suwayomi.tachidesk.manga.impl.util.getChapterCachePath
import suwayomi.tachidesk.manga.impl.util.getChapterCbzPath
import suwayomi.tachidesk.manga.impl.util.getChapterDownloadPath
import suwayomi.tachidesk.manga.impl.util.storage.ImageResponse
import suwayomi.tachidesk.manga.model.table.ChapterTable
@@ -43,20 +46,20 @@ sealed class FileType {
fun getName(): String =
when (this) {
is FileType.RegularFile -> {
is RegularFile -> {
this.file.name
}
is FileType.ZipFile -> {
is ZipFile -> {
this.entry.name
}
}
fun getExtension(): String =
when (this) {
is FileType.RegularFile -> {
is RegularFile -> {
this.file.extension
}
is FileType.ZipFile -> {
is ZipFile -> {
this.entry.name.substringAfterLast(".")
}
}
@@ -189,6 +192,19 @@ abstract class ChaptersFilesProvider<Type : FileType>(
handleSuccessfulDownload()
// Calculate and save Koreader hash for CBZ files
val chapterFile = File(getChapterCbzPath(mangaId, chapterId))
if (chapterFile.exists()) {
val koreaderHash = KoreaderHelper.hashContents(chapterFile)
if (koreaderHash != null) {
transaction {
ChapterTable.update({ ChapterTable.id eq chapterId }) {
it[ChapterTable.koreaderHash] = koreaderHash
}
}
}
}
File(cacheChapterDir).deleteRecursively()
return true

View File

@@ -6,6 +6,8 @@ import org.apache.commons.compress.archivers.zip.ZipArchiveEntry
import org.apache.commons.compress.archivers.zip.ZipArchiveInputStream
import org.apache.commons.compress.archivers.zip.ZipArchiveOutputStream
import org.apache.commons.compress.archivers.zip.ZipFile
import org.jetbrains.exposed.sql.transactions.transaction
import org.jetbrains.exposed.sql.update
import suwayomi.tachidesk.manga.impl.download.fileProvider.ChaptersFilesProvider
import suwayomi.tachidesk.manga.impl.download.fileProvider.FileType
import suwayomi.tachidesk.manga.impl.util.getChapterCachePath
@@ -13,6 +15,7 @@ import suwayomi.tachidesk.manga.impl.util.getChapterCbzPath
import suwayomi.tachidesk.manga.impl.util.getChapterDownloadPath
import suwayomi.tachidesk.manga.impl.util.getMangaDownloadDir
import suwayomi.tachidesk.manga.impl.util.storage.FileDeletionHelper
import suwayomi.tachidesk.manga.model.table.ChapterTable
import suwayomi.tachidesk.server.ApplicationDirs
import uy.kohesive.injekt.injectLazy
import java.io.File
@@ -85,6 +88,13 @@ class ArchiveProvider(
}
val cbzDeleted = cbzFile.delete()
if (cbzDeleted) {
transaction {
ChapterTable.update({ ChapterTable.id eq chapterId }) {
it[koreaderHash] = null
}
}
}
FileDeletionHelper.cleanupParentFoldersFor(cbzFile, applicationDirs.mangaDownloadsRoot)
return cbzDeleted
}

View File

@@ -1,10 +1,13 @@
package suwayomi.tachidesk.manga.impl.download.fileProvider.impl
import org.jetbrains.exposed.sql.transactions.transaction
import org.jetbrains.exposed.sql.update
import suwayomi.tachidesk.manga.impl.download.fileProvider.ChaptersFilesProvider
import suwayomi.tachidesk.manga.impl.download.fileProvider.FileType.RegularFile
import suwayomi.tachidesk.manga.impl.util.getChapterCachePath
import suwayomi.tachidesk.manga.impl.util.getChapterDownloadPath
import suwayomi.tachidesk.manga.impl.util.storage.FileDeletionHelper
import suwayomi.tachidesk.manga.model.table.ChapterTable
import suwayomi.tachidesk.server.ApplicationDirs
import uy.kohesive.injekt.injectLazy
import java.io.BufferedOutputStream
@@ -61,6 +64,13 @@ class FolderProvider(
}
val chapterDirDeleted = chapterDir.deleteRecursively()
if (chapterDirDeleted) {
transaction {
ChapterTable.update({ ChapterTable.id eq chapterId }) {
it[koreaderHash] = null
}
}
}
FileDeletionHelper.cleanupParentFoldersFor(chapterDir, applicationDirs.mangaDownloadsRoot)
return chapterDirDeleted
}

View File

@@ -0,0 +1,425 @@
package suwayomi.tachidesk.manga.impl.sync
import eu.kanade.tachiyomi.network.NetworkHelper
import eu.kanade.tachiyomi.network.await
import eu.kanade.tachiyomi.util.lang.Hash
import io.github.oshai.kotlinlogging.KotlinLogging
import io.javalin.json.JsonMapper
import kotlinx.serialization.Serializable
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.buildJsonObject
import kotlinx.serialization.json.put
import okhttp3.MediaType.Companion.toMediaType
import okhttp3.Request
import okhttp3.RequestBody.Companion.toRequestBody
import org.jetbrains.exposed.sql.transactions.transaction
import org.jetbrains.exposed.sql.update
import suwayomi.tachidesk.graphql.types.KoSyncStatusPayload
import suwayomi.tachidesk.graphql.types.KoreaderSyncChecksumMethod
import suwayomi.tachidesk.graphql.types.KoreaderSyncStrategy
import suwayomi.tachidesk.manga.impl.util.KoreaderHelper
import suwayomi.tachidesk.manga.impl.util.getChapterCbzPath
import suwayomi.tachidesk.manga.model.table.ChapterTable
import suwayomi.tachidesk.manga.model.table.MangaTable
import suwayomi.tachidesk.server.serverConfig
import uy.kohesive.injekt.injectLazy
import java.io.File
import java.util.UUID
import kotlin.math.abs
object KoreaderSyncService {
private val logger = KotlinLogging.logger {}
private val network: NetworkHelper by injectLazy()
private val json: Json by injectLazy()
private val jsonMapper: JsonMapper by injectLazy()
@Serializable
private data class KoreaderProgressPayload(
val document: String,
val progress: String,
val percentage: Float,
val device: String,
val device_id: String,
)
@Serializable
private data class KoreaderProgressResponse(
val document: String? = null,
val progress: String? = null,
val percentage: Float? = null,
val timestamp: Long? = null,
val device: String? = null,
val device_id: String? = null,
)
@Serializable
data class SyncResult(
val pageRead: Int,
val timestamp: Long, // Unix timestamp in seconds
val device: String,
val shouldUpdate: Boolean = false,
val isConflict: Boolean = false,
)
data class ConnectResult(
val success: Boolean,
val message: String? = null,
val username: String? = null,
)
private data class AuthResult(
val success: Boolean,
val message: String? = null,
val isUserNotFoundError: Boolean = false,
)
private fun buildRequest(
url: String,
block: Request.Builder.() -> Unit,
): Request =
Request
.Builder()
.url(url)
.addHeader("Accept", "application/vnd.koreader.v1+json")
.apply(block)
.build()
private suspend fun getOrGenerateDeviceId(): String {
var deviceId = serverConfig.koreaderSyncDeviceId.value
if (deviceId.isBlank()) {
deviceId =
UUID
.randomUUID()
.toString()
.replace("-", "")
.uppercase()
logger.info { "[KOSYNC] Generated new KOSync Device ID: $deviceId" }
serverConfig.koreaderSyncDeviceId.value = deviceId
}
return deviceId
}
private fun getOrGenerateChapterHash(chapterId: Int): String? {
return transaction {
val existingHash =
ChapterTable
.select(ChapterTable.koreaderHash)
.where { ChapterTable.id eq chapterId }
.firstOrNull()
?.get(ChapterTable.koreaderHash)
if (!existingHash.isNullOrBlank()) {
return@transaction existingHash
}
val checksumMethod = serverConfig.koreaderSyncChecksumMethod.value
val newHash =
when (checksumMethod) {
KoreaderSyncChecksumMethod.BINARY -> {
logger.info { "[KOSYNC HASH] No hash for chapterId=$chapterId. Generating from CBZ content." }
val mangaId =
ChapterTable
.select(ChapterTable.manga)
.where { ChapterTable.id eq chapterId }
.firstOrNull()
?.get(ChapterTable.manga)
?.value ?: return@transaction null
val cbzFile = File(getChapterCbzPath(mangaId, chapterId))
if (!cbzFile.exists()) {
logger.info { "[KOSYNC HASH] Could not generate hash for chapterId=$chapterId. CBZ not found." }
return@transaction null
}
KoreaderHelper.hashContents(cbzFile)
}
KoreaderSyncChecksumMethod.FILENAME -> {
logger.info { "[KOSYNC HASH] No hash for chapterId=$chapterId. Generating from filename." }
(ChapterTable innerJoin MangaTable)
.select(ChapterTable.name, MangaTable.title)
.where { ChapterTable.id eq chapterId }
.firstOrNull()
?.let {
val chapterName = it[ChapterTable.name]
val mangaTitle = it[MangaTable.title]
val baseFilename = "$mangaTitle - $chapterName".split('.').dropLast(1).joinToString(".")
Hash.md5(baseFilename)
}
}
}
if (newHash != null) {
ChapterTable.update({ ChapterTable.id eq chapterId }) {
it[koreaderHash] = newHash
}
logger.info { "[KOSYNC HASH] Generated and saved new hash for chapterId=$chapterId" }
} else {
logger.warn { "[KOSYNC HASH] Hashing failed for chapterId=$chapterId." }
}
newHash
}
}
private suspend fun register(
username: String,
userkey: String,
): AuthResult {
val payload =
buildJsonObject {
put("username", username)
put("password", userkey)
}
val request =
buildRequest("${serverConfig.koreaderSyncServerUrl.value.removeSuffix("/")}/users/create") {
post(payload.toString().toRequestBody("application/json".toMediaType()))
}
return try {
network.client.newCall(request).await().use { response ->
if (response.isSuccessful) {
AuthResult(true, "Registration successful.")
} else {
val errorBody = response.body.string()
val errorMessage =
runCatching {
jsonMapper.fromJsonString<Map<String, String>>(
errorBody,
Map::class.java,
)["message"]
}.getOrNull()
val finalMessage = errorMessage ?: "Registration failed with code ${response.code}"
AuthResult(false, finalMessage)
}
}
} catch (e: Exception) {
logger.error(e) { "[KOSYNC REGISTER] Exception" }
AuthResult(false, e.message)
}
}
private suspend fun authorize(
username: String,
userkey: String,
): AuthResult {
val request =
buildRequest("${serverConfig.koreaderSyncServerUrl.value.removeSuffix("/")}/users/auth") {
get()
addHeader("x-auth-user", username)
addHeader("x-auth-key", userkey)
}
return try {
network.client.newCall(request).await().use { response ->
if (response.isSuccessful) {
AuthResult(true)
} else {
val isUserNotFound = response.code == 401 // Unauthorized often means user/pass combo is wrong
AuthResult(false, "Authorization failed with code ${response.code}", isUserNotFoundError = isUserNotFound)
}
}
} catch (e: Exception) {
logger.error(e) { "[KOSYNC AUTHORIZE] Exception" }
AuthResult(false, e.message)
}
}
suspend fun connect(
username: String,
password: String,
): ConnectResult {
val userkey = Hash.md5(password)
val authResult = authorize(username, userkey)
if (authResult.success) {
serverConfig.koreaderSyncUsername.value = username
serverConfig.koreaderSyncUserkey.value = userkey
return ConnectResult(true, "Login successful.", username)
}
if (authResult.isUserNotFoundError) {
logger.info { "[KOSYNC CONNECT] Authorization failed, attempting to register new user." }
val registerResult = register(username, userkey)
return if (registerResult.success) {
serverConfig.koreaderSyncUsername.value = username
serverConfig.koreaderSyncUserkey.value = userkey
ConnectResult(true, "Registration successful.", username)
} else {
ConnectResult(false, registerResult.message ?: "Registration failed.", null)
}
}
return ConnectResult(false, authResult.message ?: "Authentication failed.", null)
}
suspend fun logout() {
serverConfig.koreaderSyncUsername.value = ""
serverConfig.koreaderSyncUserkey.value = ""
}
suspend fun getStatus(): KoSyncStatusPayload {
val username = serverConfig.koreaderSyncUsername.value
val userkey = serverConfig.koreaderSyncUserkey.value
if (username.isBlank() || userkey.isBlank()) {
return KoSyncStatusPayload(isLoggedIn = false, username = null)
}
val authResult = authorize(username, userkey)
return KoSyncStatusPayload(isLoggedIn = authResult.success, username = if (authResult.success) username else null)
}
suspend fun pushProgress(chapterId: Int) {
val strategy = serverConfig.koreaderSyncStrategy.value
if (strategy == KoreaderSyncStrategy.DISABLED || strategy == KoreaderSyncStrategy.RECEIVE) return
val username = serverConfig.koreaderSyncUsername.value
val userkey = serverConfig.koreaderSyncUserkey.value
if (username.isBlank() || userkey.isBlank()) return
logger.info { "[KOSYNC PUSH] Init." }
val chapterHash = getOrGenerateChapterHash(chapterId)
if (chapterHash.isNullOrBlank()) {
logger.info { "[KOSYNC PUSH] Aborted for chapterId=$chapterId: No hash." }
return
}
val chapterInfo =
transaction {
ChapterTable
.select(ChapterTable.lastPageRead, ChapterTable.pageCount)
.where { ChapterTable.id eq chapterId }
.firstOrNull()
?.let {
object {
val lastPageRead = it[ChapterTable.lastPageRead]
val pageCount = it[ChapterTable.pageCount]
}
}
} ?: return
if (chapterInfo.pageCount <= 0) {
logger.warn { "[KOSYNC PUSH] Aborted for chapterId=$chapterId: Invalid pageCount." }
return
}
try {
val deviceId = getOrGenerateDeviceId()
val payload =
KoreaderProgressPayload(
document = chapterHash,
progress = (chapterInfo.lastPageRead + 1).toString(),
percentage = (chapterInfo.lastPageRead + 1).toFloat() / chapterInfo.pageCount.toFloat(),
device = "Suwayomi-Server (${System.getProperty("os.name")})",
device_id = deviceId,
)
val requestBody = json.encodeToString(KoreaderProgressPayload.serializer(), payload)
val request =
buildRequest("${serverConfig.koreaderSyncServerUrl.value.removeSuffix("/")}/syncs/progress") {
put(requestBody.toRequestBody("application/json".toMediaType()))
addHeader("x-auth-user", username)
addHeader("x-auth-key", userkey)
}
network.client.newCall(request).await().use { response ->
if (!response.isSuccessful) {
logger.warn { "[KOSYNC PUSH] Failed for chapterId=$chapterId: ${response.code}" }
} else {
logger.info { "[KOSYNC PUSH] Success for chapterId=$chapterId" }
}
}
} catch (e: Exception) {
logger.error(e) { "[KOSYNC PUSH] Exception for chapterId=$chapterId" }
}
}
suspend fun checkAndPullProgress(chapterId: Int): SyncResult? {
val strategy = serverConfig.koreaderSyncStrategy.value
if (strategy == KoreaderSyncStrategy.DISABLED || strategy == KoreaderSyncStrategy.SEND) return null
val username = serverConfig.koreaderSyncUsername.value
val userkey = serverConfig.koreaderSyncUserkey.value
if (username.isBlank() || userkey.isBlank()) return null
val chapterHash = getOrGenerateChapterHash(chapterId)
if (chapterHash.isNullOrBlank()) {
logger.info { "[KOSYNC PULL] Aborted for chapterId=$chapterId: No hash." }
return null
}
try {
val request =
buildRequest("${serverConfig.koreaderSyncServerUrl.value.removeSuffix("/")}/syncs/progress/$chapterHash") {
get()
addHeader("x-auth-user", username)
addHeader("x-auth-key", userkey)
}
network.client.newCall(request).await().use { response ->
if (response.isSuccessful) {
val body = response.body.string()
if (body.isBlank() || body == "{}") return null
val progressResponse = json.decodeFromString(KoreaderProgressResponse.serializer(), body)
val pageRead = progressResponse.progress?.toIntOrNull()?.minus(1)
val timestamp = progressResponse.timestamp
val device = progressResponse.device ?: "KOReader"
val localProgress =
transaction {
ChapterTable
.select(ChapterTable.lastReadAt, ChapterTable.lastPageRead, ChapterTable.pageCount)
.where { ChapterTable.id eq chapterId }
.firstOrNull()
?.let {
object {
val lastReadAt = it[ChapterTable.lastReadAt]
val lastPageRead = it[ChapterTable.lastPageRead]
val pageCount = it[ChapterTable.pageCount]
}
}
}
if (pageRead != null && timestamp != null) {
// Ignore XPath progress for now as we only support paginated files
if (progressResponse.progress?.startsWith("/") == true) {
return null
}
val localPercentage =
if (localProgress?.pageCount ?: 0 >
0
) {
(localProgress!!.lastPageRead + 1).toFloat() / localProgress.pageCount
} else {
0f
}
val percentageDifference = abs(localPercentage - (progressResponse.percentage ?: 0f))
// Progress is within tolerance, no sync needed
if (percentageDifference < serverConfig.koreaderSyncPercentageTolerance.value) {
return null
}
when (strategy) {
KoreaderSyncStrategy.RECEIVE -> {
return SyncResult(pageRead, timestamp, device, shouldUpdate = true)
}
KoreaderSyncStrategy.SILENT -> {
if (timestamp > (localProgress?.lastReadAt ?: 0L)) {
return SyncResult(pageRead, timestamp, device, shouldUpdate = true)
}
}
KoreaderSyncStrategy.PROMPT -> {
return SyncResult(pageRead, timestamp, device, isConflict = true)
}
else -> {} // SEND and DISABLED already handled at the start of the function
}
}
} else {
logger.warn { "[KOSYNC PULL] Failed for chapterId=$chapterId: ${response.code}" }
}
}
} catch (e: Exception) {
logger.error(e) { "[KOSYNC PULL] Exception for chapterId=$chapterId" }
}
return null
}
}

View File

@@ -0,0 +1,48 @@
package suwayomi.tachidesk.manga.impl.util
import java.io.File
import java.security.MessageDigest
object KoreaderHelper {
// Helper function to convert ByteArray to Hex String
private fun ByteArray.toHexString() = joinToString("") { "%02x".format(it) }
/**
* Hashes the document according to a custom Koreader hashing algorithm.
* https://github.com/koreader/koreader/blob/master/frontend/util.lua#L1040
* Only applies to epub and cbz files.
* @param file The file object to hash.
* @return The lowercase MD5 hash or null if hashing is not possible.
*/
fun hashContents(file: File): String? {
val extension = file.extension.lowercase()
if (!file.exists() || (extension != "epub" && extension != "cbz")) {
return null
}
try {
file.inputStream().use { fs ->
val step = 1024
val size = 1024
val md5 = MessageDigest.getInstance("MD5")
val buffer = ByteArray(size)
for (i in -1 until 10) {
val position = (step shl (2 * i)).toLong()
if (position >= file.length()) break // Avoid seeking past the end of small files
fs.channel.position(position)
val bytesRead = fs.read(buffer, 0, size)
if (bytesRead > 0) {
md5.update(buffer, 0, bytesRead)
} else {
break
}
}
return md5.digest().toHexString().lowercase()
}
} catch (e: Exception) {
// TODO: Should we log this error?
return null
}
}
}