Compare commits

...

38 Commits

Author SHA1 Message Date
renovate[bot]
97a9bc51b2 Update plugin shadowjar to v9 2026-06-26 06:36:48 +00:00
herowinb
c8f5d83e9c add reportSyncEvent for SyncYomi service (#2110)
* add reportSyncEvent

* Update SyncYomiSyncService.kt
2026-06-17 22:46:21 -04:00
Zeedif
be5e3f022e feat(download): improve chapter download filenames (#2100)
* feat(download): improve chapter download filenames

* refactor(download): use SafePath helper for filename sanitization
2026-06-17 22:41:31 -04:00
renovate[bot]
40a21fabca Update dex2jar to v2.4.37 (#2122)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-06-17 22:39:37 -04:00
Zeedif
b33069f107 fix(opds): resolve sql group by syntax error when filtering library (#2118) 2026-06-17 22:39:30 -04:00
Zeedif
14ab3aa9f4 fix(opds): handle dead sources and prevent kosync binary hash crashes (#2116) 2026-06-17 22:39:22 -04:00
Weblate (bot)
bfa0038f53 Weblate translations (#1903)
Translate-URL: https://hosted.weblate.org/projects/suwayomi/suwayomi-server/el/
Translate-URL: https://hosted.weblate.org/projects/suwayomi/suwayomi-server/ja/
Translate-URL: https://hosted.weblate.org/projects/suwayomi/suwayomi-server/pl/
Translate-URL: https://hosted.weblate.org/projects/suwayomi/suwayomi-server/ru/
Translation: Suwayomi/Suwayomi-Server

Co-authored-by: Micka149 <dr.mischutckin2017@yandex.ru>
Co-authored-by: Philip Prescott-Decie <presdec@gmail.com>
Co-authored-by: Roland Vezsenyi <miscogd5yf2paqvxvc@farvoid.com>
Co-authored-by: Syer10 <Mitchellptbo@gmail.com>
Co-authored-by: TheRay82 <raycoc1382@gmail.com>
Co-authored-by: UnknownSkyrimPasserby <f7022961@opayq.com>
Co-authored-by: 圭紫 <kaceykoo@gmail.com>
2026-06-17 22:39:15 -04:00
Mitchell Syer
934459f15f Dataclass cleanup and minor fixes (#2115) 2026-06-15 14:32:09 -04:00
Mitchell Syer
bab58daecc Fix Postgres Backups (#2113)
* Fix Postgres Backups

* Changelog

* Import

* More accurate changelog
2026-06-15 14:31:58 -04:00
renovate[bot]
61eb0630cb Update dependency com.github.usefulness:webp-imageio to v0.11.0 (#2114)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-06-15 14:31:05 -04:00
renovate[bot]
0d3afadfa3 Update koin to v4.2.2 (#2111)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-06-15 14:30:44 -04:00
renovate[bot]
f9e81c75b6 Update dependency com.zaxxer:HikariCP to v7.1.0 (#2108)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-06-15 14:30:28 -04:00
renovate[bot]
e123857399 Update JCEF + JetBrains Runtime to jbr-release-25.0.3b508.4 (#2104)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-06-15 14:30:19 -04:00
renovate[bot]
31817639bd Update jackson monorepo to v3.2.0 (#2102)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-06-15 14:28:51 -04:00
renovate[bot]
2b91ab755d Update okhttp monorepo to v5.4.0 (#2101)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-06-15 14:27:00 -04:00
Zeedif
98576c6e62 feat(opds): Add option to skip metadata feed for direct access (#1879)
* feat(opds): add option to skip chapter metadata feed

Introduces a new server configuration `server.opdsSkipChapterMetadataFeed` (default: false).

When enabled, the OPDS chapter feed generates direct acquisition (CBZ download) and streaming (OPDS-PSE) links within the chapter list entries, bypassing the intermediate metadata subsection. This streamlines the user experience and improves compatibility with OPDS clients like KOReader that rely on direct links for automated downloading features.

* fix: lint

* fix(opds): enrich chapter data and refine sync logic for skip-metadata mode

Refines the `opdsSkipChapterMetadataFeed` implementation to ensure necessary data is available for direct links and handles synchronization logic appropriate for a list view.

- **Refactor ChapterForDownload:** Extract `refreshChapterPageList` and `updateChapterPersistence` to allow reusing page count verification logic outside the download flow.
- **Enrich Chapter Repository:** When skipping metadata, asynchronously verify page counts and calculate CBZ file sizes for chapters in the list. This ensures direct stream/download links are valid even if the chapter wasn't previously fully indexed.
- **KoSync Logic:** Implement synchronization logic in `OpdsEntryBuilder`. Since the user cannot be prompted in the chapter list view, `PROMPT` conflicts are explicitly ignored (prioritizing local progress), while updates are applied if non-conflicting.
- **OPDS Attributes:** Add `length` (file size) to acquisition links and ensure download links only appear for actually downloaded chapters.
- **Documentation:** Update `server.conf` description to clarify KoSync behavior in this mode.

* feat(download): improve chapter download filenames

* feat(opds): append language to source names

* feat(opds): handle empty chapter titles

* fix import org.jetbrains.exposed.v1.core.inList

* refactor(opds): reorganize API routes and update facet count calculations based on active filters

- **API Routing & Controllers**: Reorganize OPDS v1.2 route paths into logical groups in `OpdsAPI`. Centralize request filter extraction into `OpdsMangaFilter.fromContext`.
- **Facet Counting**: Extract `Query.applyOpdsMangaFilter` to apply active filters to facet and navigation queries. Pass the active filters to `NavigationRepository` and `MangaRepository` count queries (using `excludeField` to calculate sibling counts). This ensures category, source, language, status, and genre counts (`thr:count`) are accurately computed based on active selections.
- **Pagination**: Add pagination support to computed navigation feeds in `NavigationRepository` ( statuses and content languages).
- **Builders**: Standardize parameter ordering in `FeedBuilderInternal` and `OpdsEntryBuilder` constructors. Simplify pagination and facet link URL generation.

* fix(opds): remove redundant filter logic to avoid duplicate HAVING clauses

Resolve IllegalStateException crash caused by applying content filters twice in MangaRepository. Filtering is now handled exclusively by `applyOpdsMangaFilter`, allowing `applyMangaLibrarySort` to focus solely on ordering operations.

* revert(download): restore original CBZ filename scheme

* refactor(opds): simplify persistence updates and clean up chapter mapping

- Simplify page count and download checks in ChapterForDownload
- Clean up enriched chapter mapping in ChapterRepository to improve readability

* fix(opds): retrieve chapter archive size without leaving stream open

* perf(opds): avoid redundant DB query when refreshing chapter page list
2026-06-15 14:26:16 -04:00
schroda
8fbc8fd3d4 Fix automatic chapter downloads (#2098)
The returned result rows of the inserted chapters did not have the up-to-date "last_modified_at".
This caused "downloadNewChapters" to not be able to correctly detect unread chapters. it included the newly inserted ones, leading to exiting early due to having unread chapters.

Regression 811e15162b

fixes #2097
2026-06-08 14:21:57 -04:00
Constantin Piber
c81020dbb1 CEF: Remove jogl and jogamp deps by implementing a no-op renderer (#2095)
* CEF: Remove jogl and jogamp deps by implementing a no-op renderer

* Update readme
2026-06-08 14:21:47 -04:00
renovate[bot]
348d525b00 Update dependency ch.qos.logback:logback-classic to v1.5.34 (#2085)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-06-05 16:15:00 -04:00
renovate[bot]
a9e3d4dc81 Update graphqlkotlin to v10.0.0 (#2086)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-06-05 16:14:46 -04:00
renovate[bot]
15af84e626 Update plugin buildconfig to v6.0.10 (#2088)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-06-05 16:14:31 -04:00
renovate[bot]
745b11d91a Update kotlin monorepo to v2.4.0 (#2092)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-06-05 16:14:16 -04:00
renovate[bot]
b9efbc1aa4 Update dependency com.typesafe:config to v1.4.9 (#2093)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-06-05 16:14:07 -04:00
Syer10
ba347e24a2 [skip ci] Changelog 2026-06-05 15:33:21 -04:00
Bartu Özen
811e15162b Implement SyncYomi (#1813)
* Implement SyncYomi

* Add ability to select what to sync

* Properly fix default category bug

* Add periodic sync

* Add PostgreSQL support

* Deschedule previous task

* Check if SyncYomi is enabled in syncData function

* Don't allow multiple syncs at the same time

* Convert SyncYomiSyncService to object

* Make startSync non-suspend

* Return a result from startSync

* Sync before library update

* Improvements

* Use NetworkHelper client

* Lint

* Use measureTime

* Database improvements
- Move entire sync operation into a single transaction
- Stop loading all manga to memory

* Revert "Database improvements"

This reverts commit bee8d214c3.

* Actual database improvements

* Remove runBlocking

* Remove title check

* Update updateNonFavorites function

* Update timeout code

* Improve PostgreSQL query

* Create lastSyncState variable

* Create lastSyncStatus query

* Convert lastSyncState to StateFlow

* Create lastSyncStatusChange subscription

* Replace backupRestoreStatus with backupRestoreId

* Add startDate and endDate

* Add logs for sync start and end

* Handle all errors in syncData

* Change category restore function to match Mihon's behavior

* Fix comment

* Remove duplicate BackupMangaHandler.backup call

* Remove duplicated log

* Rename subscription to syncStatusChanged

* Use same flags for restoring

* Update syncInterval config to use DurationSetting

* Update sync scheduling logic

* Reorder conditions to reduce database calls

* Prevent deleted ghost chapters from reappearing during sync
jobobby04/TachiyomiSY#1575

* Improve sync merging categories
jobobby04/TachiyomiSY#1559

* Make columns not null

* Improve H2 triggers

* Add documentation
2026-06-05 15:31:51 -04:00
renovate[bot]
a403b1c564 Update JCEF + JetBrains Runtime (#2083)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-05-31 19:28:16 -04:00
renovate[bot]
98833fa738 Update dependency zulu to v25.34.17_25.0.3 (#2084)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-05-31 19:28:11 -04:00
Syer10
f571b52740 [skip ci] Disable Android-Jar 2026-05-31 19:00:51 -04:00
Syer10
744c5189c4 [skip ci] Zulu Config 3 2026-05-31 18:55:45 -04:00
Syer10
96859f90f0 [skip ci] Zulu Config 2 2026-05-31 18:51:27 -04:00
Syer10
76ee471933 [skip ci] Try this Zulu config 2026-05-31 18:45:57 -04:00
Syer10
6fdc247ace Try this way 2026-05-31 18:09:37 -04:00
Syer10
d0d28a692a Update both JCEFs in one package 2026-05-31 18:03:04 -04:00
Syer10
7051a5d525 Add renovate config for JCEF 2026-05-31 17:57:44 -04:00
Constantin Piber
0e4de02b52 wiki: fetch all objects to determine tag (#2081) 2026-05-31 17:44:51 -04:00
renovate[bot]
1b9db6cb01 Update jackson monorepo (#2079)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-05-31 17:14:31 -04:00
renovate[bot]
af45cce641 Update dependency ch.qos.logback:logback-classic to v1.5.33 (#2072)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-05-31 17:14:20 -04:00
Constantin Piber
903a3d53b3 wiki: fix missing git repo change (#2080) 2026-05-31 17:14:11 -04:00
66 changed files with 3567 additions and 993 deletions

View File

@@ -11,4 +11,8 @@ ktlint_standard_if-else-wrapping=disabled
ktlint_standard_no-consecutive-comments=disabled
[**/generated/**]
ktlint=disabled
ktlint=disabled
[*.json]
indent_size=2
indent_style = space

View File

@@ -24,6 +24,8 @@ jobs:
with:
repository: ${{github.repository}}
path: ${{github.repository}}
fetch-depth: 0 # fetch history & tags to determine stable version
fetch-tags: true
- name: Checkout Wiki
uses: actions/checkout@v6
@@ -37,7 +39,7 @@ jobs:
cd $GITHUB_WORKSPACE/${{github.repository}}.wiki
cp -r $GITHUB_WORKSPACE/${{github.repository}}/docs/* .
stable="$(git describe --abbrev=0 --tags)"
stable="$(git -C $GITHUB_WORKSPACE/${{github.repository}} describe --abbrev=0 --tags)"
if ! git -C $GITHUB_WORKSPACE/${{github.repository}} log --exit-code --pretty= "$stable.." -- docs/Configuring-SuwayomiServer.md; then
echo "Changes to config detected, embedding link to stable"
sed -i '1s/^/> [!WARNING]\n> This document describes the settings available in the preview version. Please head to ['$stable'](https:\/\/github.com\/Suwayomi\/Suwayomi-Server\/blob\/'$stable'\/docs\/Configuring-Suwayomi%E2%80%90Server.md) for the current stable version\n\n/' Configuring-SuwayomiServer.md

View File

@@ -21,7 +21,7 @@ object CefHelper {
}
fun waitForInit() =
callbackFlow<CefApp> {
callbackFlow {
val app = cefApp.first { it.isFailure || it.getOrThrow() != null }.getOrThrow()!!
app.onInitialization {
logger.debug { "CEF: Initialization state $it" }

View File

@@ -68,6 +68,7 @@ import org.cef.handler.CefLoadHandler
import org.cef.handler.CefLoadHandlerAdapter
import org.cef.handler.CefMessageRouterHandlerAdapter
import org.cef.handler.CefPermissionHandler
import org.cef.handler.CefRenderHandlerAdapter
import org.cef.handler.CefRequestHandler
import org.cef.handler.CefRequestHandlerAdapter
import org.cef.handler.CefResourceHandler
@@ -82,10 +83,13 @@ import org.cef.network.CefPostDataElement
import org.cef.network.CefRequest
import org.cef.network.CefResponse
import org.koin.mp.KoinPlatformTools
import java.awt.Rectangle
import java.io.BufferedWriter
import java.io.File
import java.io.IOException
import java.nio.ByteBuffer
import java.util.concurrent.Executor
import javax.swing.JPanel
import kotlin.math.min
import kotlin.reflect.KFunction
import kotlin.reflect.full.declaredMemberFunctions
@@ -97,6 +101,7 @@ class KcefWebViewProvider(
private val settings = KcefWebSettings()
private var viewClient = WebViewClient()
private var chromeClient = WebChromeClient()
private val renderHandler = RenderHandler()
private val mappings: MutableList<FunctionMapping> = mutableListOf()
private val urlHttpMapping: MutableMap<String, String> = mutableMapOf()
private var initialRequestData: InitialRequestData? = null
@@ -522,6 +527,21 @@ class KcefWebViewProvider(
}
}
private class RenderHandler : CefRenderHandlerAdapter() {
override fun getViewRect(browser: CefBrowser): Rectangle = Rectangle(0, 0, 1280, 2856)
override fun onPaint(
browser: CefBrowser,
popup: Boolean,
dirtyRects: Array<Rectangle>,
buffer: ByteBuffer,
width: Int,
height: Int,
) {
// do nothing
}
}
override fun init(
javaScriptInterfaces: Map<String, Any>?,
privateBrowsing: Boolean,
@@ -617,7 +637,7 @@ class KcefWebViewProvider(
kcefClient!!
.createBrowser(
loadUrl,
CefRendering.OFFSCREEN,
CefRendering.CefRenderingWithHandler(renderHandler, JPanel()),
false,
).apply {
// NOTE: Without this, we don't seem to be receiving any events
@@ -642,7 +662,7 @@ class KcefWebViewProvider(
kcefClient!!
.createBrowser(
url,
CefRendering.OFFSCREEN,
CefRendering.CefRenderingWithHandler(renderHandler, JPanel()),
false,
).apply {
// NOTE: Without this, we don't seem to be receiving any events
@@ -676,7 +696,7 @@ class KcefWebViewProvider(
kcefClient!!
.createBrowser(
url,
CefRendering.OFFSCREEN,
CefRendering.CefRenderingWithHandler(renderHandler, JPanel()),
false,
).apply {
// NOTE: Without this, we don't seem to be receiving any events

View File

@@ -7,7 +7,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
## [Unreleased] (Preview)
### Added
- .
- (**Sync**) Added [SyncYomi](https://github.com/syncyomi/syncyomi) support
- (**OPDS**) Add option to skip chapter metadata feed providing direct stream/download links
### Changed
- (**Database/H2**) Use the latest H2 database engine
@@ -29,6 +30,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
- (**API**) Fix GraphQL `Filter` `notAll` and `notAny` being inversed
- (**API**) Fix GraphQL `Filter` causing an UnsupportedOperationException when passing an empty list as a `Any` filter value
- (**Build**) Fix CURL failing silently in builds
- (**Backup/Database**) Fix backup creation slowdown when mapping chapters
## [v2.2.2100] + [WebUI: v20260508.01] - 2026-05-08

View File

@@ -110,18 +110,13 @@ Download the latest `linux-x64`(x86_64) release from [the releases section](http
WebView support is implemented via [JCEF](https://github.com/JetBrains/jcef).
This is optional, and is only necessary to support some extensions.
To have a functional WebView, several dependencies are required; aside from X11 libraries necessary for rendering Chromium, some JNI bindings are necessary: gluegen and jogl (found in Ubuntu as `libgluegen2-jni` and `libjogl2-jni`).
Note that on some systems (e.g. Ubuntu), the JNI libraries are not automatically found, see below.
To have a functional WebView, some X11 dependencies are required for rendering Chromium.
These include `libxrender`, `libxcomposite` `libxdamage`, `libxkbcommon` and `libxtst`.
A CEF server is launched on startup, which loads the X11 libraries.
If those are missing, you should see "Could not load 'jcef' library".
If so, use `ldd ~/.local/share/Tachidesk/bin/kcef/libjcef.so | grep not` to figure out which libraries are not found on your system.
The JNI bindings are only loaded when a browser is actually launched.
This is done by extensions that rely on WebView, not by Suwayomi itself.
If there is a problem loading the JNI libraries, you should see a message indicating the library and the search path.
This search path includes the current working directory, if you do not want to modify system directories.
Refer to the [Dockerfile](https://github.com/Suwayomi/Suwayomi-Server-docker/blob/main/Dockerfile) for more details.
Note that it is required to have an X session active and available to Suwayomi (i.e. `DISPLAY` is set).

View File

@@ -14,7 +14,7 @@ val getTachideskVersion = { "v2.2.${getCommitCount()}" }
val webUIRevisionTag = "r3136"
val webviewJbrRelease = "jbr-release-25.0.3b475.60"
val webviewJbrRelease = "jbr-release-25.0.3b508.4"
private val getCommitCount = {
runCatching {

View File

@@ -232,6 +232,7 @@ server.opdsShowOnlyUnreadChapters = false
server.opdsShowOnlyDownloadedChapters = false
server.opdsChapterSortOrder = "DESC"
server.opdsCbzMimetype = "MODERN"
server.opdsSkipChapterMetadataFeed = false
```
- `server.opdsUseBinaryFileSizes = false` controls if Suwayomi should display file sizes in binary units (KiB, MiB, GiB) or decimal (KB, MB, GB) in OPDS listings.
- `server.opdsItemsPerPage = 50` sets the number of items per page in OPDS listings. Range: 10 <= n <= 5000.
@@ -241,6 +242,7 @@ server.opdsCbzMimetype = "MODERN"
- `server.opdsShowOnlyDownloadedChapters = false` controls if OPDS listings should only include downloaded chapters.
- `server.opdsChapterSortOrder = "DESC"` sets the default chapter sort order in OPDS listings, either `"ASC"` or `"DESC"`
- `server.opdsCbzMimetype = "MODERN"` controls which mimetype to use for CBZ downloads. This affects the offered link in OPDS, as well as the content type of the CBZ download. Allowed is MODERN (current IANA standard), LEGACY (deprecated mimetype for .cbz) and COMPATIBLE (deprecated mimetype for all comic archives). Use LEGACY or COMPATIBLE if older clients don't offer the chapter download (note that the chapter needs to first be downloaded in Suwayomi, before it is available in OPDS).
- `server.opdsSkipChapterMetadataFeed = false` controls if the metadata feed should be skipped. When enabled, download and streaming links are provided directly in the chapter list. This improves compatibility with automated downloaders (like KOReader). KoSync strategies are applied, but `PROMPT` conflicts are ignored (treating local progress as priority).
### KOReader Sync
```
@@ -276,6 +278,28 @@ server.useHikariConnectionPool = true
- `server.databasePassword` the username with which to authenticate at the PostgreSQL instance.
- `server.useHikariConnectionPool` use Hikari Connection Pool to connect to the database.
### SyncYomi
```
server.syncYomiEnabled = false
server.syncYomiHost = ""
server.syncYomiApiKey = ""
server.syncDataManga = true
server.syncDataChapters = true
server.syncDataTracking = true
server.syncDataHistory = true
server.syncDataCategories = true
server.syncInterval = "0s"
```
- `server.syncYomiEnabled` controls whether SyncYomi is enabled.
- `server.syncYomiHost` base URL of the SyncYomi server instance. e.g. `http://localhost:8282`
- `server.syncYomiApiKey` API key to authenticate with SyncYomi. You must use the same API key in both Suwayomi and SyncYomi.
- `server.syncDataManga` enables syncing manga.
- `server.syncDataChapters` enables syncing chapters.
- `server.syncDataTracking` enables syncing tracking data.
- `server.syncDataHistory` enables syncing reading history.
- `server.syncDataCategories` enables syncing categories.
- `server.syncInterval` interval between automatic sync operations. Use `0s` to disable.
**Note:** The example [docker-compose.yml file](https://github.com/Suwayomi/Suwayomi-Server-docker/blob/main/docker-compose.yml) contains everything you need to get started with Suwayomi+PostgreSQL. Please be aware that PostgreSQL support is currently still in beta.
**Note:** These settings are excluded from backups, so a backup can be used to easily switch database installations by setting up the connection first, then restoring the backup.

View File

@@ -1,23 +1,23 @@
[versions]
kotlin = "2.3.21"
kotlin = "2.4.0"
coroutines = "1.11.0"
serialization = "1.11.0"
jvmTarget = "21"
okhttp = "5.3.2" # Major version is locked by Tachiyomi extensions
okhttp = "5.4.0" # Major version is locked by Tachiyomi extensions
javalin = "7.2.2"
jte = "3.2.4"
jackson = "3.1.3" # jackson version locked by javalin, ref: `io.javalin.core.util.OptionalDependency`
jackson = "3.2.0" # jackson version locked by javalin, ref: `io.javalin.core.util.OptionalDependency`
exposed = "1.2.0"
dex2jar = "2.4.36"
dex2jar = "2.4.37"
polyglot = "25.0.3"
settings = "1.3.0"
twelvemonkeys = "3.13.1"
graphqlkotlin = "10.0.0-alpha.4"
graphqlkotlin = "10.0.0"
xmlserialization = "0.91.3"
ktlint = "1.8.0"
koin = "4.2.1"
koin = "4.2.2"
moko = "0.26.4"
jcef = "144.0.15-g72717cf-chromium-144.0.7559.172-api-1.21-262-b34"
jcef = "144.0.15-g72717cf-chromium-144.0.7559.172-api-1.21-262-b37"
[libraries]
# Kotlin
@@ -39,7 +39,7 @@ serialization-xml = { module = "io.github.pdvrieze.xmlutil:serialization-jvm", v
# Logging
slf4japi = "org.slf4j:slf4j-api:2.0.18"
logback = "ch.qos.logback:logback-classic:1.5.32"
logback = "ch.qos.logback:logback-classic:1.5.34"
kotlinlogging = "io.github.oshai:kotlin-logging-jvm:8.0.4"
# OkHttp
@@ -55,7 +55,7 @@ javalin-openapi = { module = "io.javalin:javalin-openapi", version.ref = "javali
javalin-rendering = { module = "io.javalin:javalin-rendering-jte", version.ref = "javalin" }
jackson-databind = { module = "tools.jackson.core:jackson-databind", version.ref = "jackson" }
jackson-kotlin = { module = "tools.jackson.module:jackson-module-kotlin", version.ref = "jackson" }
jackson-annotations = "com.fasterxml.jackson.core:jackson-annotations:2.21"
jackson-annotations = "com.fasterxml.jackson.core:jackson-annotations:2.22"
jte = { module = "gg.jte:jte", version.ref = "jte" }
kte = { module = "gg.jte:jte-kotlin", version.ref = "jte" }
@@ -72,7 +72,7 @@ exposed-javatime = { module = "org.jetbrains.exposed:exposed-java-time", version
exposed-kotlintime = { module = "org.jetbrains.exposed:exposed-kotlin-datetime", version.ref = "exposed" }
postgres = "org.postgresql:postgresql:42.7.11"
h2 = "com.h2database:h2:2.4.240"
hikaricp = "com.zaxxer:HikariCP:7.0.2"
hikaricp = "com.zaxxer:HikariCP:7.1.0"
# Exposed Migrations
exposed-migrations = "com.github.Suwayomi:exposed-migrations:3.10.1"
@@ -91,7 +91,7 @@ rxjava = "io.reactivex:rxjava:1.3.8"
jsoup = "org.jsoup:jsoup:1.22.2"
# Config
config = "com.typesafe:config:1.4.8"
config = "com.typesafe:config:1.4.9"
config4k = "io.github.config4k:config4k:0.7.0"
# Sort
@@ -145,7 +145,7 @@ twelvemonkeys-imageio-metadata = { module = "com.twelvemonkeys.imageio:imageio-m
twelvemonkeys-imageio-jpeg = { module = "com.twelvemonkeys.imageio:imageio-jpeg", version.ref = "twelvemonkeys" }
twelvemonkeys-imageio-webp = { module = "com.twelvemonkeys.imageio:imageio-webp", version.ref = "twelvemonkeys" }
imageio-webp = "com.github.usefulness:webp-imageio:0.10.2"
imageio-webp = "com.github.usefulness:webp-imageio:0.11.0"
# Testing
mockk = "io.mockk:mockk:1.14.11"
@@ -158,8 +158,6 @@ cronUtils = "com.cronutils:cron-utils:9.2.1"
# Webview
jcef = { module = "org.jetbrains.intellij.deps.jcef:jcef", version.ref = "jcef" }
gluegen = "org.jogamp.gluegen:gluegen-rt:2.5.0"
jogl = "org.jogamp.jogl:jogl-all:2.5.0"
# User
jwt = "com.auth0:java-jwt:4.5.2"
@@ -180,13 +178,13 @@ kotlin-multiplatform = { id = "org.jetbrains.kotlin.multiplatform", version.ref
ktlint = { id = "org.jlleitschuh.gradle.ktlint", version = "14.2.0"}
# Build config
buildconfig = { id = "com.github.gmazzo.buildconfig", version = "6.0.9"}
buildconfig = { id = "com.github.gmazzo.buildconfig", version = "6.0.10"}
# Download
download = { id = "de.undercouch.download", version = "5.7.0"}
# ShadowJar
shadowjar = { id = "com.gradleup.shadow", version = "8.3.11"}
shadowjar = { id = "com.gradleup.shadow", version = "9.4.3"}
# Moko
moko = { id = "dev.icerock.mobile.multiplatform-resources", version.ref = "moko" }

View File

@@ -16,14 +16,43 @@
"depNameTemplate": "zulu",
"datasourceTemplate": "custom.zulu",
"versioningTemplate": "regex:^(?<major>\\d+)\\.(?<minor>\\d+)\\.(?<patch>\\d+).*$"
},
{
"customType": "regex",
"managerFilePatterns": [
"/buildSrc/src/main/kotlin/Constants.kt/"
],
"matchStrings": [
"val\\s+webviewJbrRelease\\s*=\\s*\"(?<currentValue>[^\"]+)\""
],
"depNameTemplate": "JetBrainsRuntime",
"packageNameTemplate": "JetBrains/JetBrainsRuntime",
"datasourceTemplate": "github-releases",
"versioningTemplate": "regex:^jbr-release-(?<major>\\d+)\\.(?<minor>\\d+)\\.(?<patch>\\d+)b(?<build>\\d+)\\.(?<revision>\\d+)$"
}
],
"customDatasources": {
"zulu": {
"defaultRegistryUrlTemplate": "https://api.azul.com/metadata/v1/zulu/packages?availability_types=ca&release_status=both&java_package_type=jre&crac_supported=false&javafx_bundled=false&support_term=lts&arch=x86&os=linux&archive_type=zip&page_size=1000&include_fields=java_package_features,release_status,support_term,os,arch,hw_bitness,abi,java_package_type,javafx_bundled,sha256_hash,cpu_gen,size,archive_type,certifications,lib_c_type,crac_supported&page=1&azul_com=true",
"transformTemplates": [
"{\"releases\": $$.$join([$join($map(distro_version[[0..2]], $string), \".\"), \"_\", $join($map(java_version, $string), \".\")])}"
"{ \"releases\": $map($, function($v) { { \"version\": $join([$string($v.distro_version[0]), \".\", $string($v.distro_version[1]), \".\", $string($v.distro_version[2]), \"_\", $string($v.java_version[0]), \".\", $string($v.java_version[1]), \".\", $string($v.java_version[2])]) } }) }"
]
}
}
},
"packageRules": [
{
"matchPackageNames": [
"org.jetbrains.intellij.deps.jcef:jcef",
"JetBrains/JetBrainsRuntime"
],
"groupName": "JCEF + JetBrains Runtime",
"groupSlug": "jcef-jbr"
},
{
"matchPackageNames": [
"com.github.Suwayomi:android-jar"
],
"enabled": false
}
]
}

View File

@@ -42,7 +42,7 @@ main() {
gcc -fPIC -shared scripts/resources/catch_abort.c -lpthread -o scripts/resources/catch_abort.so
fi
JRE_ZULU="25.30.17_25.0.1"
JRE_ZULU="25.34.17_25.0.3"
JRE_RELEASE="jre${JRE_ZULU#*_}" # e.g. jre25.0.1
ZULU_RELEASE="zulu${JRE_ZULU%_*}" # e.g. zulu25.30.17

View File

@@ -37,10 +37,6 @@ dependencies {
implementation(libs.bundles.shared)
testImplementation(libs.bundles.sharedTest)
// WebView
implementation(libs.gluegen)
implementation(libs.jogl)
// OkHttp
implementation(libs.bundles.okhttp)
implementation(libs.okio)

View File

@@ -50,6 +50,7 @@
<!-- OPDS errors -->
<string name="opds_error_manga_not_found">Series with ID %1$d not found.</string>
<string name="opds_error_chapter_not_found">Chapter with index %1$d not found.</string>
<string name="opds_error_chapters_not_found">No chapters found or the source is unreachable on page %1$d.</string>
<!-- OPDS facets (Filters and Sorting) -->
<string name="opds_facetgroup_sort_order">Sort Order</string>
@@ -153,4 +154,7 @@
<string name="login_label_login">Log In</string>
<string name="login_placeholder_username">Type username...</string>
<string name="login_placeholder_password">Secret...</string>
<string name="opds_chapter_title_oneshot">Oneshot</string>
<string name="opds_chapter_title_fallback">Chapter %1$s</string>
</resources>

View File

@@ -0,0 +1,125 @@
<?xml version="1.0" encoding="utf-8"?>
<resources xmlns:xliff="urn:oasis:names:tc:xliff:document:1.2">
<string name="opds_feeds_root">Suwayomi OPDS Κατάλογος</string>
<string name="opds_feeds_manga_chapters">Κεφάλαια %1$s</string>
<string name="opds_feeds_chapter_details">%1$s | %2$s | Λεπτομέρειες</string>
<string name="opds_feeds_explore_title">Εξερεύνηση</string>
<string name="opds_feeds_explore_entry_content">Εξερεύνησε νέες σειρές από τις πηγές σου</string>
<string name="opds_feeds_history_title">Ιστορικό</string>
<string name="opds_feeds_history_entry_content">Πρόσφατα διαβασμένα κεφάλαια</string>
<string name="opds_feeds_all_series_in_library_title">Όλες οι σειρές</string>
<string name="opds_feeds_all_series_in_library_entry_content">Περιήγηση σε όλες τις σειρές της βιβλιοθήκης σου</string>
<string name="opds_feeds_library_sources_title">Πηγές</string>
<string name="opds_feeds_library_sources_entry_content">Περιήγηση σε σειρές της βιβλιοθήκης σου φιλτραρισμένες κατά πηγή</string>
<string name="opds_feeds_categories_title">Κατηγορίες</string>
<string name="opds_feeds_categories_entry_content">Περιήγηση σε σειρές οργανωμένες κατά κατηγορία</string>
<string name="opds_feeds_genres_title">Είδη</string>
<string name="opds_feeds_genres_entry_content">Περιήγηση σε σειρές κατά ετικέτες είδους</string>
<string name="opds_feeds_status_title">Κατάσταση</string>
<string name="opds_feeds_status_entry_content">Περιήγηση σε σειρές κατά κατάσταση δημοσίευσης</string>
<string name="opds_feeds_languages_title">Γλώσσες</string>
<string name="opds_feeds_languages_entry_content">Περιήγηση σε σειρές κατά γλώσσα περιεχομένου</string>
<string name="opds_feeds_library_updates_title">Ιστορικό ενημερώσεων βιβλιοθήκης</string>
<string name="opds_feeds_library_updates_entry_content">Πρόσφατα ενημερωμένα κεφάλαια από τη βιβλιοθήκη σου</string>
<string name="opds_feeds_search_results_title">Αποτελέσματα αναζήτησης</string>
<string name="opds_feeds_sources_title">Όλες οι πηγές</string>
<string name="opds_feeds_category_specific_title">Κατηγορία: %1$s</string>
<string name="opds_feeds_genre_specific_title">Είδος: %1$s</string>
<string name="opds_feeds_status_specific_title">Κατάσταση: %1$s</string>
<string name="opds_feeds_language_specific_title">Γλώσσα: %1$s</string>
<string name="opds_feeds_source_specific_title">Πηγή: %1$s</string>
<string name="opds_feeds_library_source_specific_title">Βιβλιοθήκη - Πηγή: %1$s</string>
<string name="opds_feeds_source_specific_popular_title">Πηγή: %1$s - Δημοφιλές</string>
<string name="opds_feeds_source_specific_latest_title">Πηγή: %1$s - Τελευταίο</string>
<string name="opds_search_shortname">Suwayomi OPDS Αναζήτηση</string>
<string name="opds_search_description">Αναζήτηση σειρών στον κατάλογο.</string>
<string name="opds_error_manga_not_found">Η σειρά με ID %1$d δεν βρέθηκε.</string>
<string name="opds_error_chapter_not_found">Το κεφάλαιο με δείκτη %1$d δεν βρέθηκε.</string>
<string name="opds_facetgroup_sort_order">Σειρά ταξινόμησης</string>
<string name="opds_facetgroup_filter_read_status">Φίλτρο κατά κατάσταση ανάγνωσης</string>
<string name="opds_facetgroup_filter_content">Φίλτρο περιεχομένου</string>
<string name="opds_facetgroup_filter_source">Φίλτρο κατά πηγή</string>
<string name="opds_facetgroup_filter_category">Φίλτρο κατά κατηγορία</string>
<string name="opds_facetgroup_filter_status">Φίλτρο κατά κατάσταση</string>
<string name="opds_facetgroup_filter_language">Φίλτρο κατά γλώσσα</string>
<string name="opds_facetgroup_filter_genre">Φίλτρο κατά είδος</string>
<string name="opds_facet_sort_oldest_first">Παλαιότερα πρώτα</string>
<string name="opds_facet_sort_newest_first">Νεότερα πρώτα</string>
<string name="opds_facet_sort_date_asc">Ημερομηνία αύξουσα</string>
<string name="opds_facet_sort_date_desc">Ημερομηνία φθίνουσα</string>
<string name="opds_facet_sort_popular">Δημοφιλές</string>
<string name="opds_facet_sort_latest">Τελευταίο</string>
<string name="opds_facet_sort_alpha_asc">Αλφαβητικά Α</string>
<string name="opds_facet_sort_alpha_desc">Αλφαβητικά Ω-Α</string>
<string name="opds_facet_sort_last_read_desc">Τελευταία ανάγνωση</string>
<string name="opds_facet_sort_latest_chapter_desc">Τελευταίο κεφάλαιο</string>
<string name="opds_facet_sort_date_added_desc">Ημερομηνία προσθήκης</string>
<string name="opds_facet_sort_unread_desc">Αδιάβαστα κεφάλαια</string>
<string name="opds_facet_filter_all">Όλα</string>
<string name="opds_facet_filter_all_chapters">Όλα τα κεφάλαια</string>
<string name="opds_facet_filter_unread_only">Αδιάβαστα</string>
<string name="opds_facet_filter_read_only">Διαβασμένα</string>
<string name="opds_facet_filter_downloaded">Ληφθέντα</string>
<string name="opds_facet_filter_ongoing">Σε εξέλιξη</string>
<string name="opds_facet_filter_completed">Ολοκληρωμένα</string>
<string name="opds_facet_all_sources">Όλες οι πηγές</string>
<string name="opds_facet_all_categories">Όλες οι κατηγορίες</string>
<string name="opds_facet_all_statuses">Όλες οι καταστάσεις</string>
<string name="opds_facet_all_languages">Όλες οι γλώσσες</string>
<string name="opds_facet_all_genres">Όλα τα είδη</string>
<string name="opds_linktitle_catalog_root">Ρίζα καταλόγου</string>
<string name="opds_linktitle_search_catalog">Αναζήτηση καταλόγου</string>
<string name="opds_linktitle_first_page">Πρώτη σελίδα</string>
<string name="opds_linktitle_previous_page">Προηγούμενη σελίδα</string>
<string name="opds_linktitle_next_page">Επόμενη σελίδα</string>
<string name="opds_linktitle_last_page">Τελευταία σελίδα</string>
<string name="opds_linktitle_self_feed">Τρέχουσα ροή</string>
<string name="opds_linktitle_view_on_web">Προβολή στο Web</string>
<string name="opds_linktitle_stream_pages_start">Διάβασε Online</string>
<string name="opds_linktitle_stream_pages_continue">Συνέχισε να διαβάζεις Online</string>
<string name="opds_linktitle_stream_pages_start_local">Διάβασε Online (Τοπική πρόοδος)</string>
<string name="opds_linktitle_stream_pages_continue_local">Συνέχισε να διαβάζεις Online (Τοπική πρόοδος)</string>
<string name="opds_linktitle_stream_pages_start_remote">Διάβασε Online (Συγχρονισμένο από %1$s)</string>
<string name="opds_linktitle_stream_pages_continue_remote">Συνέχισε να διαβάζεις Online (Συγχρονισμένο από %1$s)</string>
<string name="opds_linktitle_download_cbz">Λήψη CBZ</string>
<string name="opds_linktitle_chapter_cover">Εξώφυλλο κεφαλαίου</string>
<string name="opds_linktitle_view_chapter_details">Λεπτομέρειες κεφαλαίου &amp; Σελίδες</string>
<string name="opds_chapter_status_read"></string>
<string name="opds_chapter_status_in_progress"></string>
<string name="opds_chapter_status_unread"></string>
<string name="opds_chapter_status_downloaded">⬇️</string>
<string name="opds_chapter_status_synced">🌐</string>
<string name="opds_chapter_details_base">Σειρά: %1$s | %2$s</string>
<string name="opds_chapter_details_scanlator">| Από %1$s</string>
<string name="opds_chapter_details_progress">| Πρόοδος: %1$d από %2$d</string>
<string name="opds_manga_summary_status">Κατάσταση: %1$s</string>
<string name="opds_manga_summary_source">Πηγή: %1$s</string>
<string name="opds_manga_summary_language">Γλώσσα: %1$s</string>
<string name="manga_status_ongoing">Σε εξέλιξη</string>
<string name="manga_status_completed">Ολοκληρωμένο</string>
<string name="manga_status_licensed">Υπό άδεια</string>
<string name="manga_status_publishing_finished">Δημοσίευση ολοκληρωμένη</string>
<string name="manga_status_cancelled">Ακυρωμένο</string>
<string name="manga_status_on_hiatus">Σε παύση</string>
<string name="manga_status_unknown">Άγνωστο</string>
<string name="label_error">Σφάλμα</string>
<string name="label_version">Έκδοση <xliff:g id="version" example="v2.0.1833">%1$s</xliff:g></string>
<string name="label_close">Κλείσε</string>
<string name="webview_label_title">Suwayomi WebView</string>
<string name="webview_label_disconnected">Αποσυνδεδεμένο, κάνε ανανέωση</string>
<string name="webview_label_reversescroll">Αντίστροφη κύλιση</string>
<string name="webview_label_bindingshint">Σημείωση: Όταν η εστίαση είναι στο τμήμα WebView, καμία συντόμευση πληκτρολογίου, συμπεριλαμβανομένης της ανανέωσης, δεν θα αντιμετωπίζεται από το πρόγραμμα περιήγησης.</string>
<string name="webview_label_init">Αρχικοποίηση... Παρακαλώ περίμενε</string>
<string name="webview_label_getstarted">Εισήγαγε μια διεύθυνση URL για να ξεκινήσεις</string>
<string name="webview_label_loading">Φόρτωση σελίδας...</string>
<string name="webview_label_copy">Αντιγραφή στο πρόχειρο</string>
<string name="webview_label_copy_description">Η αυτόματη αντιγραφή στο πρόχειρο απέτυχε, χρησιμοποίησε το παρακάτω πεδίο για να αντιγράψεις χειροκίνητα την τιμή.</string>
<string name="webview_label_login_required">Η διαμόρφωσή σου απαιτεί σύνδεση. Εισήγαγε όνομα χρήστη και κωδικό.</string>
<string name="webview_placeholder_url">Εισήγαγε URL...</string>
<string name="login_label_title">Suwayomi Σύνδεση</string>
<string name="login_label_username">Όνομα χρήστη</string>
<string name="login_label_password">Κωδικός</string>
<string name="login_label_login">Σύνδεση</string>
<string name="login_placeholder_username">Πληκτρολόγησε όνομα χρήστη...</string>
<string name="login_placeholder_password">Μυστικό...</string>
</resources>

View File

@@ -15,16 +15,13 @@ import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.ExperimentalCoroutinesApi
import kotlinx.coroutines.SupervisorJob
import kotlinx.coroutines.channels.BufferOverflow
import kotlinx.coroutines.flow.Flow
import kotlinx.coroutines.flow.MutableSharedFlow
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.flow.conflate
import kotlinx.coroutines.flow.distinctUntilChanged
import kotlinx.coroutines.flow.drop
import kotlinx.coroutines.flow.launchIn
import kotlinx.coroutines.flow.mapLatest
import kotlinx.coroutines.flow.onEach
import org.jetbrains.exposed.v1.core.SortOrder
import suwayomi.tachidesk.graphql.types.AuthMode
@@ -1038,7 +1035,77 @@ class ServerConfig(
description = "Enable the WebView via CEF (Chromium)"
)
val syncYomiEnabled: MutableStateFlow<Boolean> by BooleanSetting(
protoNumber = 87,
defaultValue = false,
group = SettingGroup.SYNCYOMI,
privacySafe = true
)
val syncYomiHost: MutableStateFlow<String> by StringSetting(
protoNumber = 88,
defaultValue = "",
group = SettingGroup.SYNCYOMI,
privacySafe = true,
)
val syncYomiApiKey: MutableStateFlow<String> by StringSetting(
protoNumber = 89,
defaultValue = "",
group = SettingGroup.SYNCYOMI,
privacySafe = false,
)
val syncDataManga: MutableStateFlow<Boolean> by BooleanSetting(
protoNumber = 90,
defaultValue = true,
group = SettingGroup.SYNCYOMI,
privacySafe = true,
)
val syncDataChapters: MutableStateFlow<Boolean> by BooleanSetting(
protoNumber = 91,
defaultValue = true,
group = SettingGroup.SYNCYOMI,
privacySafe = true,
)
val syncDataTracking: MutableStateFlow<Boolean> by BooleanSetting(
protoNumber = 92,
defaultValue = true,
group = SettingGroup.SYNCYOMI,
privacySafe = true,
)
val syncDataHistory: MutableStateFlow<Boolean> by BooleanSetting(
protoNumber = 93,
defaultValue = true,
group = SettingGroup.SYNCYOMI,
privacySafe = true,
)
val syncDataCategories: MutableStateFlow<Boolean> by BooleanSetting(
protoNumber = 94,
defaultValue = true,
group = SettingGroup.SYNCYOMI,
privacySafe = true,
)
val syncInterval: MutableStateFlow<Duration> by DurationSetting(
protoNumber = 95,
defaultValue = 0.seconds,
group = SettingGroup.SYNCYOMI,
privacySafe = true,
)
val opdsSkipChapterMetadataFeed: MutableStateFlow<Boolean> by BooleanSetting(
protoNumber = 96,
group = SettingGroup.OPDS,
privacySafe = true,
defaultValue = false,
description = "Skips the metadata feed and provides download/stream links directly in the chapter list. Improves compatibility with KOReader auto-downloader. KoSync strategies are applied, but PROMPT conflicts are ignored (treating local progress as priority)."
)
/** ****************************************************************** **/
/** **/

View File

@@ -18,6 +18,7 @@ enum class SettingGroup(
OPDS("OPDS"),
KOREADER_SYNC("KOReader sync"),
WEB_VIEW("WebView"),
SYNCYOMI("SyncYomi")
;
override fun toString(): String = value

View File

@@ -0,0 +1,519 @@
package suwayomi.tachidesk.global.impl.sync
import android.app.Application
import android.content.Context
import io.github.oshai.kotlinlogging.KotlinLogging
import kotlinx.coroutines.DelicateCoroutinesApi
import kotlinx.coroutines.GlobalScope
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.flow.combine
import kotlinx.coroutines.flow.first
import kotlinx.coroutines.launch
import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.withLock
import kotlinx.serialization.Serializable
import kotlinx.serialization.protobuf.ProtoBuf
import org.jetbrains.exposed.v1.core.and
import org.jetbrains.exposed.v1.core.eq
import org.jetbrains.exposed.v1.jdbc.selectAll
import org.jetbrains.exposed.v1.jdbc.transactions.transaction
import org.jetbrains.exposed.v1.jdbc.update
import suwayomi.tachidesk.graphql.types.StartSyncResult
import suwayomi.tachidesk.manga.impl.Category
import suwayomi.tachidesk.manga.impl.Library.handleMangaThumbnail
import suwayomi.tachidesk.manga.impl.backup.BackupFlags
import suwayomi.tachidesk.manga.impl.backup.proto.ProtoBackupImport
import suwayomi.tachidesk.manga.impl.backup.proto.handlers.BackupCategoryHandler
import suwayomi.tachidesk.manga.impl.backup.proto.handlers.BackupMangaHandler
import suwayomi.tachidesk.manga.impl.backup.proto.handlers.BackupSourceHandler
import suwayomi.tachidesk.manga.impl.backup.proto.models.Backup
import suwayomi.tachidesk.manga.impl.backup.proto.models.BackupChapter
import suwayomi.tachidesk.manga.impl.backup.proto.models.BackupManga
import suwayomi.tachidesk.manga.model.dataclass.ChapterDataClass
import suwayomi.tachidesk.manga.model.dataclass.MangaDataClass
import suwayomi.tachidesk.manga.model.table.CategoryMangaTable
import suwayomi.tachidesk.manga.model.table.CategoryTable
import suwayomi.tachidesk.manga.model.table.ChapterTable
import suwayomi.tachidesk.manga.model.table.MangaTable
import suwayomi.tachidesk.manga.model.table.toDataClass
import suwayomi.tachidesk.server.serverConfig
import suwayomi.tachidesk.util.HAScheduler
import uy.kohesive.injekt.Injekt
import uy.kohesive.injekt.api.get
import kotlin.time.Clock
import kotlin.time.Duration.Companion.seconds
import kotlin.time.Instant
import kotlin.time.measureTime
@Serializable
data class SyncData(
val backup: Backup? = null,
)
object SyncManager {
private val syncPreferences = Injekt.get<Application>().getSharedPreferences("sync", Context.MODE_PRIVATE)
private val logger = KotlinLogging.logger {}
private var currentTaskId: String? = null
private val syncMutex = Mutex()
private val _lastSyncState: MutableStateFlow<SyncState?> = MutableStateFlow(null)
val lastSyncState: StateFlow<SyncState?> = _lastSyncState.asStateFlow()
@OptIn(DelicateCoroutinesApi::class)
fun scheduleSyncTask() {
serverConfig.subscribeTo(
combine(
serverConfig.syncYomiEnabled,
serverConfig.syncInterval,
) { enabled, interval -> Pair(enabled, interval) },
{ (enabled, interval) ->
currentTaskId?.let { HAScheduler.deschedule(it) }
currentTaskId =
if (enabled && interval > 0.seconds) {
val lastSyncDate =
syncPreferences
.getLong("last_scheduled_sync", 0L)
.takeIf { it != 0L }
?.let { Instant.fromEpochMilliseconds(it) }
if (lastSyncDate == null) {
syncPreferences
.edit()
.putLong("last_scheduled_sync", Clock.System.now().toEpochMilliseconds())
.apply()
}
val delay =
if (lastSyncDate != null) {
((interval) - (Clock.System.now() - lastSyncDate)).coerceAtLeast(0.seconds)
} else {
interval
}
HAScheduler.schedule(
{
startSync(periodic = true)
syncPreferences
.edit()
.putLong("last_scheduled_sync", Clock.System.now().toEpochMilliseconds())
.apply()
},
interval = interval.inWholeMilliseconds,
delay = delay.inWholeMilliseconds,
name = "sync",
)
} else {
syncPreferences
.edit()
.remove("last_scheduled_sync")
.apply()
null
}
},
ignoreInitialValue = false,
)
}
@OptIn(DelicateCoroutinesApi::class)
fun startSync(periodic: Boolean = false): StartSyncResult {
if (!serverConfig.syncYomiEnabled.value) {
return StartSyncResult.SYNC_DISABLED
}
if (!syncMutex.tryLock()) {
return StartSyncResult.SYNC_IN_PROGRESS
}
GlobalScope.launch {
try {
syncData(periodic)
} finally {
syncMutex.unlock()
}
}
return StartSyncResult.SUCCESS
}
suspend fun ensureSync() {
if (!serverConfig.syncYomiEnabled.value) {
return
}
if (syncMutex.tryLock()) {
// there is no ongoing sync, so start one
try {
syncData()
} finally {
syncMutex.unlock()
}
} else {
// wait for the ongoing sync to finish
syncMutex.withLock {}
}
}
private suspend fun syncData(periodic: Boolean = false) {
val startInstant = Clock.System.now()
_lastSyncState.value = SyncState.Started(startInstant)
try {
logger.info {
if (periodic) {
"Starting periodic sync"
} else {
"Starting manual sync"
}
}
transaction {
MangaTable.update({ MangaTable.isSyncing eq true }) {
it[isSyncing] = false
}
ChapterTable.update({ ChapterTable.isSyncing eq true }) {
it[isSyncing] = false
}
CategoryTable.update({ CategoryTable.isSyncing eq true }) {
it[isSyncing] = false
}
}
val backupFlags =
BackupFlags(
includeManga = serverConfig.syncDataManga.value,
includeCategories = serverConfig.syncDataCategories.value,
includeChapters = serverConfig.syncDataChapters.value,
includeTracking = serverConfig.syncDataTracking.value,
includeHistory = serverConfig.syncDataHistory.value,
includeClientData = false,
includeServerSettings = false,
)
_lastSyncState.value = SyncState.CreatingBackup(startInstant)
val backupMangas = BackupMangaHandler.backup(backupFlags)
val backup =
Backup(
backupMangas,
BackupCategoryHandler.backup(backupFlags).filter { it.name != Category.DEFAULT_CATEGORY_NAME },
BackupSourceHandler.backup(backupMangas, backupFlags),
emptyMap(),
null,
)
val syncData =
SyncData(
backup = backup,
)
val remoteBackup =
SyncYomiSyncService.doSync(syncData, startInstant) {
_lastSyncState.value = it
}
if (remoteBackup == null) {
logger.debug { "Skip restore due to network issues" }
finishWithError(startInstant, "Network error", periodic)
return
}
if (remoteBackup === syncData.backup) {
// nothing changed
logger.debug { "Skip restore due to remote was overwrite from local" }
finishWithSuccess(startInstant, periodic)
return
}
// Stop the sync early if the remote backup is null or empty
if (remoteBackup.backupManga.isEmpty() && remoteBackup.backupCategories.isEmpty() && remoteBackup.backupSources.isEmpty()) {
logger.error { "No data found on remote server." }
finishWithError(startInstant, "No data found on remote server.", periodic)
return
}
val isLibraryEmpty =
transaction {
MangaTable
.selectAll()
.where { MangaTable.inLibrary eq true }
.empty()
}
// Check if it's first sync based on lastSyncTimestamp
if (syncPreferences.getLong("last_sync_timestamp", 0) == 0L && !isLibraryEmpty) {
// It's first sync no need to restore data. (just update remote data)
finishWithSuccess(startInstant, periodic)
return
}
val (filteredFavorites, nonFavorites) = filterFavoritesAndNonFavorites(remoteBackup)
updateNonFavorites(nonFavorites)
val newSyncData =
backup.copy(
backupManga = filteredFavorites,
backupCategories = remoteBackup.backupCategories,
backupSources = remoteBackup.backupSources,
)
val hasMangaChanges = filteredFavorites.isNotEmpty()
val hasCategoryChanges = remoteBackup.backupCategories != backup.backupCategories
val hasSourceChanges = remoteBackup.backupSources != backup.backupSources
if (!hasMangaChanges && !hasCategoryChanges && !hasSourceChanges) {
// update the sync timestamp
finishWithSuccess(startInstant, periodic)
return
}
if (serverConfig.syncDataCategories.value) {
val mergedUids = newSyncData.backupCategories.map { it.uid }.toSet()
val mergedNames = newSyncData.backupCategories.map { it.name }.toSet()
val localCategories = Category.getCategoryList().filterNot { it.default } // Exclude system category
val categoriesToDelete =
localCategories.filter {
it.uid !in mergedUids && it.name !in mergedNames
}
if (categoriesToDelete.isNotEmpty()) {
transaction {
categoriesToDelete.forEach {
Category.removeCategory(it.id)
}
}
}
}
val backupStream = ProtoBuf.encodeToByteArray(Backup.serializer(), newSyncData).inputStream()
val restoreId =
ProtoBackupImport.restore(
sourceStream = backupStream,
flags = backupFlags,
isSync = true,
)
_lastSyncState.value = SyncState.Restoring(startInstant, restoreId)
ProtoBackupImport.notifyFlow.first {
val restoreState = ProtoBackupImport.getRestoreState(restoreId)
restoreState == ProtoBackupImport.BackupRestoreState.Success ||
restoreState == ProtoBackupImport.BackupRestoreState.Failure
}
// update the sync timestamp
finishWithSuccess(startInstant, periodic)
} catch (e: Throwable) {
logger.error { "Error syncing: ${e.message}" }
finishWithError(startInstant, "${e::class.qualifiedName}: ${e.message}", periodic)
}
}
private fun finishWithSuccess(
startInstant: Instant,
periodic: Boolean,
) {
syncPreferences
.edit()
.putLong("last_sync_timestamp", Clock.System.now().toEpochMilliseconds())
.apply()
_lastSyncState.value = SyncState.Success(startInstant)
logger.info {
if (periodic) {
"Periodic sync completed successfully"
} else {
"Manual sync completed successfully"
}
}
}
private fun finishWithError(
startInstant: Instant,
message: String,
periodic: Boolean,
) {
_lastSyncState.value = SyncState.Error(startInstant, message)
logger.info {
if (periodic) {
"Periodic sync failed: $message"
} else {
"Manual sync failed: $message"
}
}
}
private fun isMangaDifferent(
localManga: MangaDataClass,
remoteManga: BackupManga,
): Boolean {
if (localManga.version != remoteManga.version) {
return true
}
val localChapters =
transaction {
ChapterTable
.selectAll()
.where { ChapterTable.manga eq localManga.id }
.map { ChapterTable.toDataClass(it) }
}
if (areChaptersDifferent(localChapters, remoteManga.chapters)) {
return true
}
val localCategories =
transaction {
CategoryMangaTable
.innerJoin(CategoryTable)
.selectAll()
.where { CategoryMangaTable.manga eq localManga.id }
.map { it[CategoryTable.order] }
}
return localCategories.toSet() != remoteManga.categories.toSet()
}
private fun areChaptersDifferent(
localChapters: List<ChapterDataClass>,
remoteChapters: List<BackupChapter>,
): Boolean {
val localChapterMap = localChapters.associateBy { it.url }
val remoteChapterMap = remoteChapters.associateBy { it.url }
if (localChapterMap.size != remoteChapterMap.size) {
return true
}
for ((url, localChapter) in localChapterMap) {
val remoteChapter = remoteChapterMap[url]
// If a matching remote chapter doesn't exist, or the version numbers are different, consider them different
if (remoteChapter == null || localChapter.version != remoteChapter.version) {
return true
}
}
return false
}
private fun filterFavoritesAndNonFavorites(backup: Backup): Pair<List<BackupManga>, List<BackupManga>> {
val favorites = mutableListOf<BackupManga>()
val nonFavorites = mutableListOf<BackupManga>()
val elapsedTime =
measureTime {
logger.debug { "Starting to filter favorites and non-favorites from backup data." }
backup.backupManga.forEach { remoteManga ->
val localManga =
transaction {
MangaTable
.selectAll()
.where {
(MangaTable.sourceReference eq remoteManga.source) and
(MangaTable.url eq remoteManga.url)
}.limit(1)
.map { MangaTable.toDataClass(it) }
.firstOrNull()
}
when {
// Checks if the manga is in favorites and needs updating or adding
remoteManga.favorite -> {
if (localManga == null || isMangaDifferent(localManga, remoteManga)) {
logger.debug { "Adding to favorites: ${remoteManga.title}" }
favorites.add(remoteManga)
} else {
logger.debug { "Already up-to-date favorite: ${remoteManga.title}" }
}
}
// Handle non-favorites
!remoteManga.favorite -> {
logger.debug { "Adding to non-favorites: ${remoteManga.title}" }
nonFavorites.add(remoteManga)
}
}
}
}
logger.debug {
"Filtering completed in $elapsedTime. Favorites found: ${favorites.size}, Non-favorites found: ${nonFavorites.size}"
}
return Pair(favorites, nonFavorites)
}
private fun updateNonFavorites(nonFavorites: List<BackupManga>) {
nonFavorites.forEach { nonFavorite ->
val localManga =
transaction {
MangaTable
.selectAll()
.where {
(MangaTable.sourceReference eq nonFavorite.source) and
(MangaTable.url eq nonFavorite.url)
}.limit(1)
.map { MangaTable.toDataClass(it) }
.firstOrNull()
}
if (localManga != null) {
if (localManga.inLibrary != nonFavorite.favorite) {
transaction {
MangaTable.update({ MangaTable.id eq localManga.id }) {
it[inLibrary] = nonFavorite.favorite
}
}.apply {
handleMangaThumbnail(localManga.id, nonFavorite.favorite)
}
}
}
}
}
sealed class SyncState(
open val startDate: Instant,
) {
data class Started(
override val startDate: Instant,
) : SyncState(startDate)
data class CreatingBackup(
override val startDate: Instant,
) : SyncState(startDate)
data class Downloading(
override val startDate: Instant,
) : SyncState(startDate)
data class Merging(
override val startDate: Instant,
) : SyncState(startDate)
data class Uploading(
override val startDate: Instant,
) : SyncState(startDate)
data class Restoring(
override val startDate: Instant,
val restoreId: String,
) : SyncState(startDate)
data class Success(
override val startDate: Instant,
val endDate: Instant = Clock.System.now(),
) : SyncState(startDate)
data class Error(
override val startDate: Instant,
val message: String,
val endDate: Instant = Clock.System.now(),
) : SyncState(startDate)
}
}

View File

@@ -0,0 +1,597 @@
package suwayomi.tachidesk.global.impl.sync
import android.app.Application
import android.content.Context
import eu.kanade.tachiyomi.network.GET
import eu.kanade.tachiyomi.network.NetworkHelper
import eu.kanade.tachiyomi.network.POST
import eu.kanade.tachiyomi.network.PUT
import eu.kanade.tachiyomi.network.await
import io.github.oshai.kotlinlogging.KotlinLogging
import io.javalin.http.HttpStatus
import kotlinx.coroutines.CancellationException
import kotlinx.serialization.Serializable
import kotlinx.serialization.SerializationException
import kotlinx.serialization.json.Json
import kotlinx.serialization.protobuf.ProtoBuf
import okhttp3.Headers
import okhttp3.MediaType.Companion.toMediaType
import okhttp3.RequestBody.Companion.toRequestBody
import suwayomi.tachidesk.manga.impl.backup.proto.models.Backup
import suwayomi.tachidesk.manga.impl.backup.proto.models.BackupCategory
import suwayomi.tachidesk.manga.impl.backup.proto.models.BackupChapter
import suwayomi.tachidesk.manga.impl.backup.proto.models.BackupManga
import suwayomi.tachidesk.manga.impl.backup.proto.models.BackupSource
import suwayomi.tachidesk.server.serverConfig
import uy.kohesive.injekt.Injekt
import uy.kohesive.injekt.api.get
import uy.kohesive.injekt.injectLazy
import kotlin.time.Duration.Companion.milliseconds
import kotlin.time.Duration.Companion.seconds
import kotlin.time.Instant
object SyncYomiSyncService {
private val syncPreferences = Injekt.get<Application>().getSharedPreferences("sync", Context.MODE_PRIVATE)
private val network: NetworkHelper by injectLazy()
private val logger = KotlinLogging.logger {}
private class SyncYomiException(
message: String?,
) : Exception(message)
@Serializable
private data class SyncEvent(
val event: SyncEventStatus,
val device_Name: String? = null,
val message: String? = null,
)
@Serializable
private enum class SyncEventStatus {
SYNC_STARTED,
SYNC_SUCCESS,
SYNC_FAILED,
SYNC_ERROR,
SYNC_CANCELLED,
}
suspend fun doSync(
syncData: SyncData,
startDate: Instant,
setSyncState: (SyncManager.SyncState) -> Unit,
): Backup? {
reportSyncEvent(SyncEventStatus.SYNC_STARTED)
setSyncState(SyncManager.SyncState.Downloading(startDate))
return try {
val (remoteData, etag) = pullSyncData()
val finalSyncData =
if (remoteData != null) {
require(etag.isNotEmpty()) { "ETag should never be empty if remote data is not null" }
logger.debug { "Try update remote data with ETag($etag)" }
setSyncState(SyncManager.SyncState.Merging(startDate))
mergeSyncData(syncData, remoteData)
} else {
// init or overwrite remote data
logger.debug { "Try overwrite remote data with ETag($etag)" }
syncData
}
if (finalSyncData.backup != null) {
setSyncState(SyncManager.SyncState.Uploading(startDate))
}
val success = pushSyncData(finalSyncData, etag)
if (success) {
reportSyncEvent(SyncEventStatus.SYNC_SUCCESS)
} else {
reportSyncEvent(SyncEventStatus.SYNC_FAILED, "Failed to push sync data")
}
finalSyncData.backup
} catch (e: Exception) {
if (e is CancellationException) {
reportSyncEvent(SyncEventStatus.SYNC_CANCELLED, e.message)
throw e
}
logger.error { "Error syncing: ${e.message}" }
reportSyncEvent(SyncEventStatus.SYNC_ERROR, e.message)
throw e
}
}
private suspend fun pullSyncData(): Pair<SyncData?, String> {
val host = serverConfig.syncYomiHost.value
val apiKey = serverConfig.syncYomiApiKey.value
val downloadUrl = "$host/api/sync/content"
val headersBuilder = Headers.Builder().add("X-API-Token", apiKey)
val lastETag = syncPreferences.getString("last_sync_etag", "") ?: ""
if (lastETag != "") {
headersBuilder.add("If-None-Match", lastETag)
}
val headers = headersBuilder.build()
val downloadRequest =
GET(
url = downloadUrl,
headers = headers,
)
val response = network.client.newCall(downloadRequest).await()
if (response.code == HttpStatus.NOT_MODIFIED.code) {
// not modified
require(lastETag.isNotEmpty())
logger.info { "Remote server not modified" }
return Pair(null, lastETag)
} else if (response.code == HttpStatus.NOT_FOUND.code) {
// maybe got deleted from remote
return Pair(null, "")
}
if (response.isSuccessful) {
val newETag =
response.headers["ETag"]
?.takeIf { it.isNotEmpty() } ?: throw SyncYomiException("Missing ETag")
val byteArray =
response.body.byteStream().use {
return@use it.readBytes()
}
return try {
val backup = ProtoBuf.decodeFromByteArray(Backup.serializer(), byteArray)
return Pair(SyncData(backup = backup), newETag)
} catch (_: SerializationException) {
logger.info { "Bad content responsed from server" }
// the body is invalid
// return default value so we can overwrite it
Pair(null, "")
}
} else {
val responseBody = response.body.string()
logger.error { "SyncError: $responseBody" }
throw SyncYomiException("Failed to download sync data: $responseBody")
}
}
private suspend fun pushSyncData(
syncData: SyncData,
eTag: String,
): Boolean {
val backup = syncData.backup ?: return true
val host = serverConfig.syncYomiHost.value
val apiKey = serverConfig.syncYomiApiKey.value
val uploadUrl = "$host/api/sync/content"
val headersBuilder = Headers.Builder().add("X-API-Token", apiKey)
if (eTag.isNotEmpty()) {
headersBuilder.add("If-Match", eTag)
}
val headers = headersBuilder.build()
// Set timeout to 30 seconds
val timeout = 30.seconds
val client =
network.client
.newBuilder()
.connectTimeout(timeout)
.readTimeout(timeout)
.writeTimeout(timeout)
.build()
val byteArray = ProtoBuf.encodeToByteArray(Backup.serializer(), backup)
if (byteArray.isEmpty()) {
throw IllegalStateException("Empty backup error")
}
val body = byteArray.toRequestBody("application/octet-stream".toMediaType())
val uploadRequest =
PUT(
url = uploadUrl,
headers = headers,
body = body,
)
val response = client.newCall(uploadRequest).await()
return if (response.isSuccessful) {
val newETag =
response.headers["ETag"]
?.takeIf { it.isNotEmpty() } ?: throw SyncYomiException("Missing ETag")
syncPreferences
.edit()
.putString("last_sync_etag", newETag)
.apply()
logger.debug { "SyncYomi sync completed" }
true
} else if (response.code == HttpStatus.PRECONDITION_FAILED.code) {
// other clients updated remote data, will try next time
logger.debug { "SyncYomi sync failed with 412" }
false
} else {
val responseBody = response.body.string()
logger.error { "SyncError: $responseBody" }
false
}
}
private suspend fun reportSyncEvent(
event: SyncEventStatus,
message: String? = null,
) {
try {
val host = serverConfig.syncYomiHost.value
val apiKey = serverConfig.syncYomiApiKey.value
val url = "$host/api/sync/event"
val headers = Headers.Builder().add("X-API-Token", apiKey).build()
// Use a fixed server name.
val bodyObj =
SyncEvent(
event = event,
device_Name = "Suwayomi Server",
message = message,
)
val jsonBody = Json.encodeToString(SyncEvent.serializer(), bodyObj)
val requestBody = jsonBody.toRequestBody("application/json; charset=utf-8".toMediaType())
val request =
POST(
url = url,
headers = headers,
body = requestBody,
)
network.client
.newCall(request)
.await()
.close()
} catch (e: Exception) {
logger.error { "Failed to report sync event: ${e.message}" }
}
}
fun mergeSyncData(
localSyncData: SyncData,
remoteSyncData: SyncData,
): SyncData {
val mergedCategoriesList =
mergeCategoriesLists(localSyncData.backup?.backupCategories, remoteSyncData.backup?.backupCategories)
val mergedMangaList =
mergeMangaLists(
localSyncData.backup?.backupManga,
remoteSyncData.backup?.backupManga,
localSyncData.backup?.backupCategories ?: emptyList(),
remoteSyncData.backup?.backupCategories ?: emptyList(),
mergedCategoriesList,
)
val mergedSourcesList =
mergeSourcesLists(localSyncData.backup?.backupSources, remoteSyncData.backup?.backupSources)
// Create the merged Backup object
val mergedBackup =
Backup(
backupManga = mergedMangaList,
backupCategories = mergedCategoriesList,
backupSources = mergedSourcesList,
meta = emptyMap(),
serverSettings = null,
)
// Create the merged SData object
return SyncData(
backup = mergedBackup,
)
}
private fun mergeMangaLists(
localMangaList: List<BackupManga>?,
remoteMangaList: List<BackupManga>?,
localCategories: List<BackupCategory>,
remoteCategories: List<BackupCategory>,
mergedCategories: List<BackupCategory>,
): List<BackupManga> {
val localMangaListSafe = localMangaList.orEmpty()
val remoteMangaListSafe = remoteMangaList.orEmpty()
logger.debug { "Starting merge. Local list size: ${localMangaListSafe.size}, Remote list size: ${remoteMangaListSafe.size}" }
fun mangaCompositeKey(manga: BackupManga): String =
"${manga.source}|${manga.url}|${manga.title.lowercase().trim()}|${manga.author?.lowercase()?.trim()}"
// Create maps using composite keys
val localMangaMap = localMangaListSafe.associateBy { mangaCompositeKey(it) }
val remoteMangaMap = remoteMangaListSafe.associateBy { mangaCompositeKey(it) }
val localCategoriesMapByOrder = localCategories.associateBy { it.order }
val remoteCategoriesMapByOrder = remoteCategories.associateBy { it.order }
val mergedCategoriesMapByName = mergedCategories.associateBy { it.name }
fun updateCategories(
theManga: BackupManga,
theMap: Map<Int, BackupCategory>,
): BackupManga =
theManga.copy(
categories =
theManga.categories.mapNotNull {
theMap[it]?.let { category ->
mergedCategoriesMapByName[category.name]?.order
}
},
)
val lastSyncTime = syncPreferences.getLong("last_sync_timestamp", 0).milliseconds.inWholeSeconds
val mergedList =
(localMangaMap.keys + remoteMangaMap.keys).distinct().mapNotNull { compositeKey ->
val local = localMangaMap[compositeKey]
val remote = remoteMangaMap[compositeKey]
// New version comparison logic
when {
local != null && remote == null -> {
if (lastSyncTime == 0L || local.lastModifiedAt > lastSyncTime) {
updateCategories(local, localCategoriesMapByOrder)
} else {
logger.debug { "Dropping local manga deleted on remote: ${local.title}." }
null
}
}
local == null && remote != null -> {
if (lastSyncTime == 0L || remote.lastModifiedAt > lastSyncTime) {
updateCategories(remote, remoteCategoriesMapByOrder)
} else {
logger.debug { "Dropping deleted remote manga: ${remote.title}." }
null
}
}
local != null && remote != null -> {
// Compare versions to decide which manga to keep
if (local.version >= remote.version) {
logger.debug { "Keeping local version of ${local.title} with merged chapters." }
updateCategories(
local.copy(
chapters =
mergeChapters(
local.chapters,
remote.chapters,
lastSyncTime,
serverConfig.syncDataChapters.value,
),
),
localCategoriesMapByOrder,
)
} else {
logger.debug { "Keeping remote version of ${remote.title} with merged chapters." }
updateCategories(
remote.copy(
chapters =
mergeChapters(
local.chapters,
remote.chapters,
lastSyncTime,
serverConfig.syncDataChapters.value,
),
),
remoteCategoriesMapByOrder,
)
}
}
else -> {
null
} // No manga found for key
}
}
// Counting favorites and non-favorites
val (favorites, nonFavorites) = mergedList.partition { it.favorite }
logger.debug {
"Merge completed. Total merged manga: ${mergedList.size}, Favorites: ${favorites.size}, Non-Favorites: ${nonFavorites.size}"
}
return mergedList
}
private fun mergeChapters(
localChapters: List<BackupChapter>,
remoteChapters: List<BackupChapter>,
lastSyncTime: Long,
syncingChapters: Boolean,
): List<BackupChapter> {
if (!syncingChapters) {
return remoteChapters // If not syncing chapters, keep remote untouched
}
fun chapterCompositeKey(chapter: BackupChapter): String = "${chapter.url}|${chapter.name}|${chapter.chapterNumber}"
val localChapterMap = localChapters.associateBy { chapterCompositeKey(it) }
val remoteChapterMap = remoteChapters.associateBy { chapterCompositeKey(it) }
logger.debug { "Starting chapter merge. Local chapters: ${localChapters.size}, Remote chapters: ${remoteChapters.size}" }
// Merge both chapter maps based on version numbers
val mergedChapters =
(localChapterMap.keys + remoteChapterMap.keys).distinct().mapNotNull { compositeKey ->
val localChapter = localChapterMap[compositeKey]
val remoteChapter = remoteChapterMap[compositeKey]
logger.debug {
"Processing chapter key: $compositeKey. Local chapter: ${localChapter != null}, Remote chapter: ${remoteChapter != null}"
}
when {
localChapter != null && remoteChapter == null -> {
if (lastSyncTime == 0L || localChapter.lastModifiedAt > lastSyncTime) {
logger.debug { "Keeping local chapter: ${localChapter.name}." }
localChapter
} else {
logger.debug { "Dropping local chapter deleted on remote: ${localChapter.name}." }
null
}
}
localChapter == null && remoteChapter != null -> {
if (lastSyncTime == 0L || remoteChapter.lastModifiedAt > lastSyncTime) {
logger.debug { "Taking remote chapter: ${remoteChapter.name}." }
remoteChapter
} else {
logger.debug { "Dropping deleted remote chapter: ${remoteChapter.name}." }
null
}
}
localChapter != null && remoteChapter != null -> {
// Use version number to decide which chapter to keep
val chosenChapter =
if (localChapter.version >= remoteChapter.version) {
// If there are more chapter on remote, local sourceOrder will need to be updated to maintain correct source order.
if (localChapters.size < remoteChapters.size) {
localChapter.copy(sourceOrder = remoteChapter.sourceOrder)
} else {
localChapter
}
} else {
remoteChapter
}
logger.debug {
"Merging chapter: ${chosenChapter.name}. Chosen version from: ${if (localChapter.version >= remoteChapter.version) "Local" else "Remote"}, Local version: ${localChapter.version}, Remote version: ${remoteChapter.version}."
}
chosenChapter
}
else -> {
logger.debug { "No chapter found for composite key: $compositeKey. Skipping." }
null
}
}
}
logger.debug { "Chapter merge completed. Total merged chapters: ${mergedChapters.size}" }
return mergedChapters
}
private fun mergeCategoriesLists(
localCategoriesList: List<BackupCategory>?,
remoteCategoriesList: List<BackupCategory>?,
): List<BackupCategory> {
if (localCategoriesList == null) return remoteCategoriesList ?: emptyList()
if (remoteCategoriesList == null) return localCategoriesList
val result = mutableListOf<BackupCategory>()
val processedLocals = mutableSetOf<BackupCategory>()
val localMapByUid = localCategoriesList.filter { it.uid != 0L }.associateBy { it.uid }
val localMapByName = localCategoriesList.associateBy { it.name }
val lastSyncTime = syncPreferences.getLong("last_sync_timestamp", 0)
remoteCategoriesList.forEach { remote ->
var localMatch: BackupCategory? = null
// 1. Try match by UID
if (remote.uid != 0L) {
localMatch = localMapByUid[remote.uid]
}
// 2. Try match by Name (fallback)
if (localMatch == null) {
localMatch = localMapByName[remote.name]
}
if (localMatch != null) {
processedLocals.add(localMatch)
// Conflict resolution
if (localMatch.version >= remote.version) {
logger.debug { "Keeping local category: ${localMatch.name} (UID: ${localMatch.uid})" }
result.add(localMatch)
} else {
logger.debug { "Keeping remote category: ${remote.name} (UID: ${remote.uid})" }
// Preserve Local UID if Remote was 0
if (remote.uid == 0L) {
remote.uid = localMatch.uid
}
result.add(remote)
}
} else {
val remoteModifiedTimeMillis = remote.lastModifiedAt.seconds.inWholeMilliseconds
if (lastSyncTime == 0L || remoteModifiedTimeMillis > lastSyncTime) {
logger.debug { "Adding new remote category: ${remote.name} (UID: ${remote.uid})" }
result.add(remote)
} else {
logger.debug { "Dropping deleted remote category: ${remote.name} (UID: ${remote.uid})" }
}
}
}
// Add remaining Local Categories
localCategoriesList.forEach { local ->
if (local !in processedLocals) {
val localModifiedTimeMillis = local.lastModifiedAt.seconds.inWholeMilliseconds
if (lastSyncTime == 0L || localModifiedTimeMillis > lastSyncTime) {
logger.debug { "Keeping local only category: ${local.name} (UID: ${local.uid})" }
result.add(local)
} else {
logger.debug { "Dropping local category deleted on remote: ${local.name} (UID: ${local.uid})" }
}
}
}
return result.sortedBy { it.order }
}
private fun mergeSourcesLists(
localSources: List<BackupSource>?,
remoteSources: List<BackupSource>?,
): List<BackupSource> {
// Create maps using sourceId as key
val localSourceMap = localSources?.associateBy { it.sourceId } ?: emptyMap()
val remoteSourceMap = remoteSources?.associateBy { it.sourceId } ?: emptyMap()
logger.debug { "Starting source merge. Local sources: ${localSources?.size}, Remote sources: ${remoteSources?.size}" }
// Merge both source maps
val mergedSources =
(localSourceMap.keys + remoteSourceMap.keys).distinct().mapNotNull { sourceId ->
val localSource = localSourceMap[sourceId]
val remoteSource = remoteSourceMap[sourceId]
logger.debug {
"Processing source ID: $sourceId. Local source: ${localSource != null}, Remote source: ${remoteSource != null}"
}
when {
localSource != null && remoteSource == null -> {
logger.debug { "Using local source: ${localSource.name}." }
localSource
}
remoteSource != null && localSource == null -> {
logger.debug { "Using remote source: ${remoteSource.name}." }
remoteSource
}
else -> {
logger.debug { "Remote and local have the same source ID: $sourceId. Keeping local." }
localSource
}
}
}
logger.debug { "Source merge completed. Total merged sources: ${mergedSources.size}" }
return mergedSources
}
}

View File

@@ -0,0 +1,28 @@
package suwayomi.tachidesk.graphql.mutations
import suwayomi.tachidesk.global.impl.sync.SyncManager
import suwayomi.tachidesk.graphql.directives.RequireAuth
import suwayomi.tachidesk.graphql.types.StartSyncResult
class SyncMutation {
data class StartSyncInput(
val clientMutationId: String? = null,
)
data class StartSyncPayload(
val clientMutationId: String? = null,
val result: StartSyncResult,
)
@RequireAuth
fun startSync(input: StartSyncInput): StartSyncPayload {
val (clientMutationId) = input
val result = SyncManager.startSync()
return StartSyncPayload(
clientMutationId = clientMutationId,
result = result,
)
}
}

View File

@@ -0,0 +1,11 @@
package suwayomi.tachidesk.graphql.queries
import suwayomi.tachidesk.global.impl.sync.SyncManager
import suwayomi.tachidesk.graphql.directives.RequireAuth
import suwayomi.tachidesk.graphql.types.SyncStatus
import suwayomi.tachidesk.graphql.types.toStatus
class SyncQuery {
@RequireAuth
fun lastSyncStatus(): SyncStatus? = SyncManager.lastSyncState.value?.toStatus()
}

View File

@@ -27,6 +27,7 @@ import suwayomi.tachidesk.graphql.mutations.MangaMutation
import suwayomi.tachidesk.graphql.mutations.MetaMutation
import suwayomi.tachidesk.graphql.mutations.SettingsMutation
import suwayomi.tachidesk.graphql.mutations.SourceMutation
import suwayomi.tachidesk.graphql.mutations.SyncMutation
import suwayomi.tachidesk.graphql.mutations.TrackMutation
import suwayomi.tachidesk.graphql.mutations.UpdateMutation
import suwayomi.tachidesk.graphql.mutations.UserMutation
@@ -41,6 +42,7 @@ import suwayomi.tachidesk.graphql.queries.MangaQuery
import suwayomi.tachidesk.graphql.queries.MetaQuery
import suwayomi.tachidesk.graphql.queries.SettingsQuery
import suwayomi.tachidesk.graphql.queries.SourceQuery
import suwayomi.tachidesk.graphql.queries.SyncQuery
import suwayomi.tachidesk.graphql.queries.TrackQuery
import suwayomi.tachidesk.graphql.queries.UpdateQuery
import suwayomi.tachidesk.graphql.server.primitives.Cursor
@@ -50,6 +52,7 @@ import suwayomi.tachidesk.graphql.server.primitives.GraphQLLongAsString
import suwayomi.tachidesk.graphql.server.primitives.GraphQLUpload
import suwayomi.tachidesk.graphql.subscriptions.DownloadSubscription
import suwayomi.tachidesk.graphql.subscriptions.InfoSubscription
import suwayomi.tachidesk.graphql.subscriptions.SyncSubscription
import suwayomi.tachidesk.graphql.subscriptions.UpdateSubscription
import kotlin.reflect.KClass
import kotlin.reflect.KType
@@ -98,6 +101,7 @@ val schema =
TopLevelObject(MetaQuery()),
TopLevelObject(SettingsQuery()),
TopLevelObject(SourceQuery()),
TopLevelObject(SyncQuery()),
TopLevelObject(TrackQuery()),
TopLevelObject(UpdateQuery()),
),
@@ -114,6 +118,7 @@ val schema =
TopLevelObject(MangaMutation()),
TopLevelObject(MetaMutation()),
TopLevelObject(SettingsMutation()),
TopLevelObject(SyncMutation()),
TopLevelObject(SourceMutation()),
TopLevelObject(TrackMutation()),
TopLevelObject(UpdateMutation()),
@@ -123,6 +128,7 @@ val schema =
listOf(
TopLevelObject(DownloadSubscription()),
TopLevelObject(InfoSubscription()),
TopLevelObject(SyncSubscription()),
TopLevelObject(UpdateSubscription()),
),
)

View File

@@ -0,0 +1,17 @@
package suwayomi.tachidesk.graphql.subscriptions
import kotlinx.coroutines.flow.Flow
import kotlinx.coroutines.flow.filterNotNull
import kotlinx.coroutines.flow.map
import suwayomi.tachidesk.global.impl.sync.SyncManager
import suwayomi.tachidesk.graphql.directives.RequireAuth
import suwayomi.tachidesk.graphql.types.SyncStatus
import suwayomi.tachidesk.graphql.types.toStatus
class SyncSubscription {
@RequireAuth
fun syncStatusChanged(): Flow<SyncStatus> =
SyncManager.lastSyncState
.filterNotNull()
.map { it.toStatus() }
}

View File

@@ -0,0 +1,91 @@
package suwayomi.tachidesk.graphql.types
import suwayomi.tachidesk.global.impl.sync.SyncManager
enum class StartSyncResult {
SUCCESS,
SYNC_IN_PROGRESS,
SYNC_DISABLED,
}
enum class SyncState {
STARTED,
CREATING_BACKUP,
DOWNLOADING,
MERGING,
UPLOADING,
RESTORING,
SUCCESS,
ERROR,
}
data class SyncStatus(
val state: SyncState,
val startDate: Long,
val endDate: Long? = null,
val backupRestoreId: String? = null,
val errorMessage: String? = null,
)
fun SyncManager.SyncState.toStatus(): SyncStatus =
when (this) {
is SyncManager.SyncState.Started -> {
SyncStatus(
state = SyncState.STARTED,
startDate = startDate.toEpochMilliseconds(),
)
}
is SyncManager.SyncState.CreatingBackup -> {
SyncStatus(
state = SyncState.CREATING_BACKUP,
startDate = startDate.toEpochMilliseconds(),
)
}
is SyncManager.SyncState.Downloading -> {
SyncStatus(
state = SyncState.DOWNLOADING,
startDate = startDate.toEpochMilliseconds(),
)
}
is SyncManager.SyncState.Merging -> {
SyncStatus(
state = SyncState.MERGING,
startDate = startDate.toEpochMilliseconds(),
)
}
is SyncManager.SyncState.Uploading -> {
SyncStatus(
state = SyncState.UPLOADING,
startDate = startDate.toEpochMilliseconds(),
)
}
is SyncManager.SyncState.Restoring -> {
SyncStatus(
state = SyncState.RESTORING,
startDate = startDate.toEpochMilliseconds(),
backupRestoreId = restoreId,
)
}
is SyncManager.SyncState.Success -> {
SyncStatus(
state = SyncState.SUCCESS,
startDate = startDate.toEpochMilliseconds(),
endDate = endDate.toEpochMilliseconds(),
)
}
is SyncManager.SyncState.Error -> {
SyncStatus(
state = SyncState.ERROR,
startDate = startDate.toEpochMilliseconds(),
endDate = endDate.toEpochMilliseconds(),
errorMessage = message,
)
}
}

View File

@@ -85,6 +85,12 @@ object CategoryManga {
}
}
fun removeMangaFromAllCategories(mangaId: Int) {
transaction {
CategoryMangaTable.deleteWhere { CategoryMangaTable.manga eq mangaId }
}
}
/**
* list of mangas that belong to a category
*/
@@ -111,12 +117,14 @@ object CategoryManga {
val transform: (ResultRow) -> MangaDataClass = {
// Map the data from the result row to the MangaDataClass
val dataClass = MangaTable.toDataClass(it)
dataClass.lastReadAt = it[lastReadAt]
dataClass.unreadCount = it[unreadCount]
dataClass.downloadCount = it[downloadedCount]
dataClass.chapterCount = it[chapterCount]
dataClass
MangaTable
.toDataClass(it)
.copy(
lastReadAt = it[lastReadAt],
unreadCount = it[unreadCount],
downloadCount = it[downloadedCount],
chapterCount = it[chapterCount],
)
}
return transaction {

View File

@@ -104,9 +104,6 @@ object Chapter {
.associateBy({ it[ChapterTable.url] }, { it })
}
val chapterIds = chapterList.map { dbChapterMap.getValue(it.url)[ChapterTable.id] }
val chapterMetas = getChaptersMetaMaps(chapterIds.map { it.value })
return chapterList.mapIndexed { index, it ->
val dbChapter = dbChapterMap.getValue(it.url)
@@ -128,8 +125,8 @@ object Chapter {
realUrl = dbChapter[ChapterTable.realUrl],
downloaded = dbChapter[ChapterTable.isDownloaded],
pageCount = dbChapter[ChapterTable.pageCount],
chapterCount = chapterList.size,
meta = chapterMetas.getValue(dbChapter[ChapterTable.id].value),
lastModifiedAt = dbChapter[ChapterTable.lastModifiedAt],
version = dbChapter[ChapterTable.version],
)
}
}
@@ -192,7 +189,7 @@ object Chapter {
}
// new chapters after they have been added to the database for auto downloads
val insertedChapters = mutableListOf<ChapterDataClass>()
val insertedChapterIds = mutableListOf<Int>()
val chaptersToInsert = mutableListOf<ChapterDataClass>() // do not yet have an ID from the database
val chaptersToUpdate = mutableListOf<ChapterDataClass>()
@@ -279,6 +276,8 @@ object Chapter {
this[ChapterTable.isRead] = false
this[ChapterTable.isBookmarked] = false
this[ChapterTable.isDownloaded] = false
this[ChapterTable.lastModifiedAt] = chapter.lastModifiedAt
this[ChapterTable.version] = chapter.version
this[ChapterTable.pageCount] = -1
// is recognized chapter number
@@ -305,7 +304,7 @@ object Chapter {
}
}
}
}.forEach { insertedChapters.add(ChapterTable.toDataClass(it)) }
}.forEach { insertedChapterIds.add(it[ChapterTable.id].value) }
}
if (chaptersToUpdate.isNotEmpty()) {
@@ -322,6 +321,8 @@ object Chapter {
this[ChapterTable.scanlator] = it.scanlator
this[ChapterTable.sourceOrder] = it.index
this[ChapterTable.realUrl] = it.realUrl
this[ChapterTable.lastModifiedAt] = it.lastModifiedAt
this[ChapterTable.version] = it.version
this[ChapterTable.isDownloaded] = currentChapter.downloaded
this[ChapterTable.pageCount] = currentChapter.pageCount
@@ -348,6 +349,13 @@ object Chapter {
}
if (manga.inLibrary) {
// We have to query the inserted chapters to get the up-to-date data. I.e. "last_modified_at" is not returned by the insert statement, due to being set by a DB trigger
val insertedChapters =
transaction {
ChapterTable.selectAll().where { ChapterTable.id inList insertedChapterIds }.map(
ChapterTable::toDataClass,
)
}
downloadNewChapters(mangaId, currentLatestChapterNumber, numberOfCurrentChapters, insertedChapters)
}
@@ -605,7 +613,7 @@ object Chapter {
.withDefault { emptyMap() }
}
fun getChapterMetaMap(chapter: EntityID<Int>): Map<String, String> =
fun getChapterMetaMap(chapter: Int): Map<String, String> =
transaction {
ChapterMetaTable
.selectAll()

View File

@@ -16,6 +16,7 @@ import suwayomi.tachidesk.manga.model.table.ChapterTable
import suwayomi.tachidesk.manga.model.table.MangaTable
import suwayomi.tachidesk.manga.model.table.toDataClass
import suwayomi.tachidesk.server.serverConfig
import xyz.nulldev.androidcompat.util.SafePath
import java.io.File
import java.io.InputStream
@@ -64,6 +65,11 @@ object ChapterDownloadHelper {
chapterId: Int,
): Pair<InputStream, Long> = provider(mangaId, chapterId).getAsArchiveStream()
fun getChapterArchiveSize(
mangaId: Int,
chapterId: Int,
): Long = provider(mangaId, chapterId).getArchiveSize()
private fun getChapterWithCbzFileName(chapterId: Int): Pair<ChapterDataClass, String> =
transaction {
val row =
@@ -71,13 +77,46 @@ object ChapterDownloadHelper {
.select(ChapterTable.columns + MangaTable.columns)
.where { ChapterTable.id eq chapterId }
.firstOrNull() ?: throw IllegalArgumentException("ChapterId $chapterId not found")
val chapter = ChapterTable.toDataClass(row)
val mangaTitle = row[MangaTable.title]
val mangaTitle = row[MangaTable.title].trim()
val scanlatorPart = chapter.scanlator?.let { "[$it] " } ?: ""
val fileName = "$mangaTitle - $scanlatorPart${chapter.name}.cbz"
val scanlatorName = chapter.scanlator?.trim()?.takeIf { it.isNotEmpty() }
val chapterName = chapter.name.trim().takeIf { it.isNotEmpty() }
Pair(chapter, fileName)
val fileName =
buildString {
append(mangaTitle)
append(" - ")
if (chapterName != null) {
append(chapterName)
} else if (chapter.chapterNumber >= 0f) {
// chapterNumber is stored as Float, drop .0 for whole numbers.
val formatNumber =
if (chapter.chapterNumber % 1 == 0f) {
chapter.chapterNumber.toInt().toString()
} else {
chapter.chapterNumber.toString()
}
append("#$formatNumber")
} else {
// Fallback when neither name nor valid chapter number exists
append("#${chapter.index}")
}
if (scanlatorName != null) {
append(" [")
append(scanlatorName)
append("]")
}
append(".cbz")
}
// Sanitize filename for OS compatibility
val safeFileName = SafePath.buildValidFilename(fileName)
Pair(chapter, safeFileName)
}
fun getCbzForDownload(

View File

@@ -92,13 +92,14 @@ object Manga {
inLibrary = mangaEntry[MangaTable.inLibrary],
inLibraryAt = mangaEntry[MangaTable.inLibraryAt],
source = getSource(mangaEntry[MangaTable.sourceReference]),
meta = getMangaMetaMap(mangaId),
realUrl = mangaEntry[MangaTable.realUrl],
lastFetchedAt = mangaEntry[MangaTable.lastFetchedAt],
chaptersLastFetchedAt = mangaEntry[MangaTable.chaptersLastFetchedAt],
updateStrategy = UpdateStrategy.valueOf(mangaEntry[MangaTable.updateStrategy]),
freshData = true,
trackers = Track.getTrackRecordsByMangaId(mangaId),
lastModifiedAt = mangaEntry[MangaTable.lastModifiedAt],
version = mangaEntry[MangaTable.version],
)
}
}
@@ -211,12 +212,12 @@ object Manga {
.orderBy(ChapterTable.sourceOrder to SortOrder.DESC)
.firstOrNull { it[ChapterTable.isRead] }
mangaDaaClass.unreadCount = unreadCount
mangaDaaClass.downloadCount = downloadCount
mangaDaaClass.chapterCount = chapterCount
mangaDaaClass.lastChapterRead = lastChapterRead?.let { ChapterTable.toDataClass(it) }
mangaDaaClass
mangaDaaClass.copy(
unreadCount = unreadCount,
downloadCount = downloadCount,
chapterCount = chapterCount,
lastChapterRead = lastChapterRead?.let { ChapterTable.toDataClass(it) },
)
}
}
@@ -239,13 +240,14 @@ object Manga {
inLibrary = mangaEntry[MangaTable.inLibrary],
inLibraryAt = mangaEntry[MangaTable.inLibraryAt],
source = getSource(mangaEntry[MangaTable.sourceReference]),
meta = getMangaMetaMap(mangaId),
realUrl = mangaEntry[MangaTable.realUrl],
lastFetchedAt = mangaEntry[MangaTable.lastFetchedAt],
chaptersLastFetchedAt = mangaEntry[MangaTable.chaptersLastFetchedAt],
updateStrategy = UpdateStrategy.valueOf(mangaEntry[MangaTable.updateStrategy]),
freshData = false,
trackers = Track.getTrackRecordsByMangaId(mangaId),
lastModifiedAt = mangaEntry[MangaTable.lastModifiedAt],
version = mangaEntry[MangaTable.version],
)
fun getMangaMetaMap(mangaId: Int): Map<String, String> =

View File

@@ -148,13 +148,11 @@ object ProtoBackupExport : ProtoBackupBase() {
fun createBackup(flags: BackupFlags): InputStream {
// Create root object
val backupMangas = BackupMangaHandler.backup(flags)
val backup: Backup =
transaction {
val backupMangas = BackupMangaHandler.backup(flags)
Backup(
BackupMangaHandler.backup(flags),
backupMangas,
BackupCategoryHandler.backup(flags),
BackupSourceHandler.backup(backupMangas, flags),
BackupGlobalMetaHandler.backup(flags),

View File

@@ -21,6 +21,9 @@ import kotlinx.coroutines.sync.withLock
import okio.buffer
import okio.gzip
import okio.source
import org.jetbrains.exposed.v1.core.eq
import org.jetbrains.exposed.v1.jdbc.transactions.transaction
import org.jetbrains.exposed.v1.jdbc.update
import suwayomi.tachidesk.graphql.types.toStatus
import suwayomi.tachidesk.manga.impl.backup.BackupFlags
import suwayomi.tachidesk.manga.impl.backup.proto.ProtoBackupValidator.ValidationResult
@@ -31,6 +34,8 @@ import suwayomi.tachidesk.manga.impl.backup.proto.handlers.BackupMangaHandler
import suwayomi.tachidesk.manga.impl.backup.proto.handlers.BackupSettingsHandler
import suwayomi.tachidesk.manga.impl.backup.proto.handlers.BackupSourceHandler
import suwayomi.tachidesk.manga.impl.backup.proto.models.Backup
import suwayomi.tachidesk.manga.model.table.ChapterTable
import suwayomi.tachidesk.manga.model.table.MangaTable
import java.io.InputStream
import java.util.Date
import java.util.Timer
@@ -109,6 +114,7 @@ object ProtoBackupImport : ProtoBackupBase() {
fun restore(
sourceStream: InputStream,
flags: BackupFlags,
isSync: Boolean = false,
): String {
val restoreId = System.currentTimeMillis().toString()
@@ -117,7 +123,7 @@ object ProtoBackupImport : ProtoBackupBase() {
updateRestoreState(restoreId, BackupRestoreState.Idle)
GlobalScope.launch {
restoreLegacy(sourceStream, restoreId, flags)
restoreLegacy(sourceStream, restoreId, flags, isSync)
}
return restoreId
@@ -127,11 +133,12 @@ object ProtoBackupImport : ProtoBackupBase() {
sourceStream: InputStream,
restoreId: String = "legacy",
flags: BackupFlags = BackupFlags.DEFAULT,
isSync: Boolean = false,
): ValidationResult =
backupMutex.withLock {
try {
logger.info { "restore($restoreId): restoring..." }
performRestore(restoreId, sourceStream, flags)
performRestore(restoreId, sourceStream, flags, isSync)
} catch (e: Exception) {
logger.error(e) { "restore($restoreId): failed due to" }
@@ -152,12 +159,14 @@ object ProtoBackupImport : ProtoBackupBase() {
id: String,
sourceStream: InputStream,
flags: BackupFlags,
isSync: Boolean,
): ValidationResult {
val backupString =
sourceStream
.source()
.gzip()
.buffer()
.run {
if (!isSync) gzip() else this
}.buffer()
.use { it.readByteArray() }
val backup = parser.decodeFromByteArray(Backup.serializer(), backupString)
@@ -235,6 +244,17 @@ object ProtoBackupImport : ProtoBackupBase() {
""".trimIndent()
}
if (isSync) {
transaction {
MangaTable.update({ MangaTable.isSyncing eq true }) {
it[isSyncing] = false
}
ChapterTable.update({ ChapterTable.isSyncing eq true }) {
it[isSyncing] = false
}
}
}
updateRestoreState(id, BackupRestoreState.Success)
return validationResult

View File

@@ -8,7 +8,11 @@ package suwayomi.tachidesk.manga.impl.backup.proto.handlers
* file, You can obtain one at https://mozilla.org/MPL/2.0/. */
import org.jetbrains.exposed.v1.core.SortOrder
import org.jetbrains.exposed.v1.core.eq
import org.jetbrains.exposed.v1.jdbc.insertAndGetId
import org.jetbrains.exposed.v1.jdbc.selectAll
import org.jetbrains.exposed.v1.jdbc.transactions.transaction
import org.jetbrains.exposed.v1.jdbc.update
import suwayomi.tachidesk.manga.impl.Category
import suwayomi.tachidesk.manga.impl.Category.modifyCategoriesMetas
import suwayomi.tachidesk.manga.impl.backup.BackupFlags
@@ -38,6 +42,9 @@ object BackupCategoryHandler {
it.name,
it.order,
0, // not supported in Tachidesk
it.version,
it.uid,
it.lastModifiedAt,
).apply {
this.meta = categoryToMeta[it.id] ?: emptyMap()
}
@@ -45,7 +52,56 @@ object BackupCategoryHandler {
}
fun restore(backupCategories: List<BackupCategory>): Map<Int, Int> {
val categoryIds = Category.createCategories(backupCategories.map { it.name })
val dbCategories = Category.getCategoryList()
val dbCategoriesByName = dbCategories.associateBy { it.name }
val dbCategoriesByUid = dbCategories.associateBy { it.uid }
var nextOrder = dbCategories.maxOfOrNull { it.order }?.plus(1) ?: 0
val categoryIds =
transaction {
backupCategories
.map { backupCategory ->
var dbCategory =
if (backupCategory.uid != 0L) {
dbCategoriesByUid[backupCategory.uid]
} else {
null
}
if (dbCategory == null) {
dbCategory = dbCategoriesByName[backupCategory.name]
}
if (dbCategory != null) {
CategoryTable.update({ CategoryTable.id eq dbCategory.id }) {
it[name] = backupCategory.name
it[order] = backupCategory.order
it[version] = backupCategory.version
it[uid] = if (backupCategory.uid != 0L) backupCategory.uid else dbCategory.uid
it[lastModifiedAt] = backupCategory.lastModifiedAt
it[isSyncing] = true
}
return@map dbCategory.id
}
val currentOrder = nextOrder++
CategoryTable
.insertAndGetId {
it[name] = backupCategory.name
it[order] = currentOrder
it[version] = backupCategory.version
it[uid] = backupCategory.uid
it[lastModifiedAt] = backupCategory.lastModifiedAt
}.value
}
}
transaction {
CategoryTable.update({ CategoryTable.isSyncing eq true }) {
it[isSyncing] = false
}
}
val metaEntryByCategoryId =
categoryIds

View File

@@ -38,7 +38,6 @@ import suwayomi.tachidesk.manga.model.dataclass.TrackRecordDataClass
import suwayomi.tachidesk.manga.model.table.ChapterTable
import suwayomi.tachidesk.manga.model.table.MangaStatus
import suwayomi.tachidesk.manga.model.table.MangaTable
import suwayomi.tachidesk.manga.model.table.toDataClass
import suwayomi.tachidesk.server.database.dbTransaction
import java.util.Date
import kotlin.math.max
@@ -75,6 +74,8 @@ object BackupMangaHandler {
dateAdded = mangaRow[MangaTable.inLibraryAt].seconds.inWholeMilliseconds,
viewer = 0, // not supported in Tachidesk
updateStrategy = UpdateStrategy.valueOf(mangaRow[MangaTable.updateStrategy]),
lastModifiedAt = mangaRow[MangaTable.lastModifiedAt],
version = mangaRow[MangaTable.version],
)
val mangaId = mangaRow[MangaTable.id].value
@@ -90,29 +91,31 @@ object BackupMangaHandler {
.selectAll()
.where { ChapterTable.manga eq mangaId }
.orderBy(ChapterTable.sourceOrder to SortOrder.DESC)
.map {
ChapterTable.toDataClass(it)
}
.toList()
}
if (flags.includeChapters) {
val chapterToMeta = Chapter.getChaptersMetaMaps(chapters.map { it.id })
val chapterToMeta =
Chapter.getChaptersMetaMaps(chapters.map { it[ChapterTable.id].value })
backupManga.chapters =
chapters.map {
BackupChapter(
it.url,
it.name,
it.scanlator,
it.read,
it.bookmarked,
it.lastPageRead,
it.fetchedAt.seconds.inWholeMilliseconds,
it.uploadDate,
it.chapterNumber,
chapters.size - it.index,
url = it[ChapterTable.url],
name = it[ChapterTable.name],
scanlator = it[ChapterTable.scanlator],
read = it[ChapterTable.isRead],
bookmark = it[ChapterTable.isBookmarked],
lastPageRead = it[ChapterTable.lastPageRead],
dateFetch = it[ChapterTable.fetchedAt].seconds.inWholeMilliseconds,
dateUpload = it[ChapterTable.date_upload],
chapterNumber = it[ChapterTable.chapter_number],
sourceOrder = chapters.size - it[ChapterTable.sourceOrder],
lastModifiedAt = it[ChapterTable.lastModifiedAt],
version = it[ChapterTable.version],
).apply {
if (flags.includeClientData) {
this.meta = chapterToMeta[it.id] ?: emptyMap()
this.meta = chapterToMeta[it[ChapterTable.id].value] ?: emptyMap()
}
}
}
@@ -120,10 +123,10 @@ object BackupMangaHandler {
if (flags.includeHistory) {
backupManga.history =
chapters.mapNotNull {
if (it.lastReadAt > 0) {
if (it[ChapterTable.lastReadAt] > 0) {
BackupHistory(
url = it.url,
lastRead = it.lastReadAt.seconds.inWholeMilliseconds,
url = it[ChapterTable.url],
lastRead = it[ChapterTable.lastReadAt].seconds.inWholeMilliseconds,
)
} else {
null
@@ -232,6 +235,9 @@ object BackupMangaHandler {
it[inLibrary] = manga.favorite
it[inLibraryAt] = manga.dateAdded.milliseconds.inWholeSeconds
it[lastModifiedAt] = manga.lastModifiedAt
it[version] = manga.version
}.value
} else {
val dbMangaId = dbManga[MangaTable.id].value
@@ -251,6 +257,9 @@ object BackupMangaHandler {
it[inLibrary] = manga.favorite || dbManga[inLibrary]
it[inLibraryAt] = manga.dateAdded.milliseconds.inWholeSeconds
it[lastModifiedAt] = manga.lastModifiedAt
it[version] = manga.version
}
dbMangaId
@@ -268,7 +277,7 @@ object BackupMangaHandler {
restoreMangaChapterData(mangaId, restoreMode, chapters, history, flags)
}
// merge categories
// update categories
if (flags.includeCategories) {
restoreMangaCategoryData(mangaId, categoryIds)
}
@@ -339,6 +348,9 @@ object BackupMangaHandler {
this[ChapterTable.lastReadAt] =
historyByChapter[chapter.url]?.maxOrNull()?.milliseconds?.inWholeSeconds ?: 0
}
this[ChapterTable.lastModifiedAt] = chapter.lastModifiedAt
this[ChapterTable.version] = chapter.version
}.map { it[ChapterTable.id].value }
} else {
emptyList()
@@ -387,6 +399,7 @@ object BackupMangaHandler {
mangaId: Int,
categoryIds: List<Int>,
) {
CategoryManga.removeMangaFromAllCategories(mangaId)
CategoryManga.addMangaToCategories(mangaId, categoryIds)
}

View File

@@ -10,6 +10,10 @@ class BackupCategory(
// @ProtoNumber(3) val updateInterval: Int = 0, 1.x value not used in 0.x
// Bump by 100 to specify this is a 0.x value
@ProtoNumber(100) var flags: Int = 0,
// syncyomi
@ProtoNumber(601) var version: Long = 0,
@ProtoNumber(602) var uid: Long = 0,
@ProtoNumber(603) var lastModifiedAt: Long = 0,
// suwayomi
@ProtoNumber(9000) var meta: Map<String, String> = emptyMap(),
)

View File

@@ -19,6 +19,9 @@ data class BackupChapter(
// chapterNumber is called number is 1.x
@ProtoNumber(9) var chapterNumber: Float = 0F,
@ProtoNumber(10) var sourceOrder: Int = 0,
// syncyomi
@ProtoNumber(11) var lastModifiedAt: Long = 0,
@ProtoNumber(12) var version: Long = 0,
// suwayomi
@ProtoNumber(9000) var meta: Map<String, String> = emptyMap(),
)

View File

@@ -34,6 +34,9 @@ data class BackupManga(
@ProtoNumber(103) var viewer_flags: Int? = null,
@ProtoNumber(104) var history: List<BackupHistory> = emptyList(),
@ProtoNumber(105) var updateStrategy: UpdateStrategy = UpdateStrategy.ALWAYS_UPDATE,
// syncyomi
@ProtoNumber(106) var lastModifiedAt: Long = 0,
@ProtoNumber(109) var version: Long = 0,
// suwayomi
@ProtoNumber(9000) var meta: Map<String, String> = emptyMap(),
)

View File

@@ -31,6 +31,88 @@ import suwayomi.tachidesk.manga.model.table.PageTable
import suwayomi.tachidesk.manga.model.table.toDataClass
import kotlin.time.Duration.Companion.minutes
/**
* Updates chapter download status and page count in the database if they differ from the file system.
*/
fun updateChapterPersistence(
chapterId: Int,
isMarkedAsDownloaded: Boolean,
dbPageCount: Int,
downloadPageCount: Int,
lastPageRead: Int,
logger: KLogger,
): Boolean {
if (isMarkedAsDownloaded && dbPageCount == downloadPageCount) {
return false
}
return transaction {
var needsUpdate = false
if (!isMarkedAsDownloaded) {
logger.debug { "mark as downloaded" }
ChapterTable.update({ ChapterTable.id eq chapterId }) {
it[isDownloaded] = true
}
needsUpdate = true
}
if (dbPageCount != downloadPageCount) {
logger.debug { "use page count of downloaded chapter" }
ChapterTable.update({ ChapterTable.id eq chapterId }) {
it[pageCount] = downloadPageCount
it[ChapterTable.lastPageRead] = lastPageRead.coerceAtMost(downloadPageCount - 1).coerceAtLeast(0)
}
needsUpdate = true
}
needsUpdate
}
}
suspend fun refreshChapterPageList(
mangaId: Int,
chapterId: Int,
existingChapterEntry: ResultRow? = null,
): Int {
val mutex = mutexByChapterId.get(chapterId) { Mutex() }
return mutex.withLock {
val chapterEntry = existingChapterEntry ?: transaction { ChapterTable.selectAll().where { ChapterTable.id eq chapterId }.first() }
val mangaEntry = transaction { MangaTable.selectAll().where { MangaTable.id eq mangaId }.first() }
val source = getCatalogueSourceOrStub(mangaEntry[MangaTable.sourceReference])
val pageList =
source
.getPageList(
SChapter.create().apply {
url = chapterEntry[ChapterTable.url]
name = chapterEntry[ChapterTable.name]
scanlator = chapterEntry[ChapterTable.scanlator]
chapter_number = chapterEntry[ChapterTable.chapter_number]
date_upload = chapterEntry[ChapterTable.date_upload]
},
).mapIndexed { index, page -> Page(index, page.url, page.imageUrl, page.uri) }
transaction {
ChapterTable.update({ ChapterTable.id eq chapterId }) {
it[isDownloaded] = false
}
PageTable.deleteWhere { PageTable.chapter eq chapterId }
PageTable.batchInsert(pageList) { page ->
this[PageTable.index] = page.index
this[PageTable.url] = page.url
this[PageTable.imageUrl] = page.imageUrl
this[PageTable.chapter] = chapterId
}
ChapterTable.update({ ChapterTable.id eq chapterId }) {
it[pageCount] = pageList.size
it[lastPageRead] = chapterEntry[ChapterTable.lastPageRead].coerceAtMost(pageList.size - 1).coerceAtLeast(0)
}
}
pageList.size
}
}
suspend fun getChapterDownloadReady(
chapterId: Int? = null,
chapterIndex: Int? = null,
@@ -68,56 +150,31 @@ private class ChapterForDownload(
suspend fun asDownloadReady(): ChapterDataClass {
val log = KotlinLogging.logger("${logger.name}::asDownloadReady")
val downloadPageCount =
try {
ChapterDownloadHelper.getImageCount(mangaId, chapterId)
} catch (_: Exception) {
0
}
val downloadPageCount = runCatching { ChapterDownloadHelper.getImageCount(mangaId, chapterId) }.getOrDefault(0)
val isMarkedAsDownloaded = chapterEntry[ChapterTable.isDownloaded]
val dbPageCount = chapterEntry[ChapterTable.pageCount]
val doesDownloadExist = downloadPageCount != 0
val doPageCountsMatch = dbPageCount == downloadPageCount
log.debug { "isMarkedAsDownloaded= $isMarkedAsDownloaded, dbPageCount= $dbPageCount, downloadPageCount= $downloadPageCount" }
return if (!doesDownloadExist) {
log.debug { "reset download status and fetch page list" }
updateDownloadStatusAndPageList(false)
refreshChapterPageList(mangaId, chapterId, chapterEntry)
chapterEntry = freshChapterEntry(optChapterId = chapterId)
ChapterTable.toDataClass(chapterEntry)
} else {
transaction {
var needsUpdate = false
if (!isMarkedAsDownloaded) {
log.debug { "mark as downloaded" }
ChapterTable.update({ ChapterTable.id eq chapterId }) {
it[isDownloaded] = true
}
needsUpdate = true
}
if (!doPageCountsMatch) {
log.debug { "use page count of downloaded chapter" }
ChapterTable.update({ ChapterTable.id eq chapterId }) {
it[pageCount] = downloadPageCount
it[lastPageRead] = chapterEntry[ChapterTable.lastPageRead].coerceAtMost(downloadPageCount - 1).coerceAtLeast(0)
}
needsUpdate = true
}
// Return updated chapter data
val updatedRow =
ChapterTable
.selectAll()
.where { ChapterTable.id eq chapterId }
.first()
if (needsUpdate) {
chapterEntry = updatedRow
}
ChapterTable.toDataClass(updatedRow)
if (updateChapterPersistence(
chapterId,
isMarkedAsDownloaded,
dbPageCount,
downloadPageCount,
chapterEntry[ChapterTable.lastPageRead],
log,
)
) {
chapterEntry = freshChapterEntry(optChapterId = chapterId)
}
ChapterTable.toDataClass(chapterEntry)
}
}
@@ -150,59 +207,4 @@ private class ChapterForDownload(
}
}.first()
}
private suspend fun updateDownloadStatusAndPageList(downloaded: Boolean): ChapterDataClass {
val mutex = mutexByChapterId.get(chapterId) { Mutex() }
return mutex.withLock {
val pageList = fetchPageList()
transaction {
// Update download status
ChapterTable.update({ ChapterTable.id eq chapterId }) {
it[isDownloaded] = downloaded
}
// Clear existing pages and insert new ones
PageTable.deleteWhere { PageTable.chapter eq chapterId }
PageTable.batchInsert(pageList) { page ->
this[PageTable.index] = page.index
this[PageTable.url] = page.url
this[PageTable.imageUrl] = page.imageUrl
this[PageTable.chapter] = chapterId
}
// Update page count
ChapterTable.update({ ChapterTable.id eq chapterId }) {
it[pageCount] = pageList.size
it[lastPageRead] = chapterEntry[ChapterTable.lastPageRead].coerceAtMost(pageList.size - 1).coerceAtLeast(0)
}
// Get updated chapter data
val updatedRow =
ChapterTable
.selectAll()
.where { ChapterTable.id eq chapterId }
.first()
chapterEntry = updatedRow
ChapterTable.toDataClass(updatedRow)
}
}
}
private suspend fun fetchPageList(): List<Page> {
val mangaEntry = transaction { MangaTable.selectAll().where { MangaTable.id eq mangaId }.first() }
val source = getCatalogueSourceOrStub(mangaEntry[MangaTable.sourceReference])
return source
.getPageList(
SChapter.create().apply {
url = chapterEntry[ChapterTable.url]
name = chapterEntry[ChapterTable.name]
scanlator = chapterEntry[ChapterTable.scanlator]
chapter_number = chapterEntry[ChapterTable.chapter_number]
date_upload = chapterEntry[ChapterTable.date_upload]
},
).mapIndexed { index, page -> Page(index, page.url, page.imageUrl, page.uri) }
}
}

View File

@@ -127,29 +127,36 @@ object KoreaderSyncService {
}
val mangaId = chapterRow[ChapterTable.manga].value
val isDownloaded = chapterRow[ChapterTable.isDownloaded]
val checksumMethod = serverConfig.koreaderSyncChecksumMethod.value
val newHash =
when (checksumMethod) {
KoreaderSyncChecksumMethod.BINARY -> {
logger.debug { "[KOSYNC HASH] No hash for chapterId=$chapterId. Generating from downloaded content." }
try {
// Always create a CBZ in memory if it doesn't exist
val (stream, _) = ChapterDownloadHelper.getArchiveStreamWithSize(mangaId, chapterId)
// Write the stream to a temp file for partial hashing
val tempFile = File.createTempFile("kosync-hash-", ".cbz")
// Only generate binary hash if the chapter is downloaded to avoid fetching missing files
if (isDownloaded) {
logger.debug { "[KOSYNC HASH] No hash for chapterId=$chapterId. Generating from downloaded content." }
try {
tempFile.outputStream().use { fos ->
stream.use { it.copyTo(fos) }
// Always create a CBZ in memory if it doesn't exist
val (stream, _) = ChapterDownloadHelper.getArchiveStreamWithSize(mangaId, chapterId)
// Write the stream to a temp file for partial hashing
val tempFile = File.createTempFile("kosync-hash-", ".cbz")
try {
tempFile.outputStream().use { fos ->
stream.use { it.copyTo(fos) }
}
// Use the same hashing method as for downloads
KoreaderHelper.hashContents(tempFile)
} finally {
// Always delete the temp file
tempFile.delete()
}
// Use the same hashing method as for downloads
KoreaderHelper.hashContents(tempFile)
} finally {
// Always delete the temp file
tempFile.delete()
} catch (e: Exception) {
logger.warn(e) { "[KOSYNC HASH] Failed to generate archive stream for chapterId=$chapterId." }
null
}
} catch (e: Exception) {
logger.warn(e) { "[KOSYNC HASH] Failed to generate archive stream for chapterId=$chapterId." }
} else {
logger.debug { "[KOSYNC HASH] Skipping binary hash for chapterId=$chapterId because it is not downloaded." }
null
}
}
@@ -175,7 +182,7 @@ object KoreaderSyncService {
}
logger.info { "[KOSYNC HASH] Generated and saved new hash for chapterId=$chapterId" }
} else {
logger.warn { "[KOSYNC HASH] Hashing failed for chapterId=$chapterId." }
logger.warn { "[KOSYNC HASH] Hashing failed or skipped for chapterId=$chapterId." }
}
newHash
}

View File

@@ -28,6 +28,7 @@ import kotlinx.coroutines.flow.update
import kotlinx.coroutines.launch
import kotlinx.coroutines.sync.Semaphore
import kotlinx.coroutines.sync.withPermit
import suwayomi.tachidesk.global.impl.sync.SyncManager
import suwayomi.tachidesk.manga.impl.Category
import suwayomi.tachidesk.manga.impl.CategoryManga
import suwayomi.tachidesk.manga.impl.Chapter
@@ -337,90 +338,98 @@ class Updater : IUpdater {
clear: Boolean?,
forceAll: Boolean,
) {
saveLastUpdateTimestamp()
if (clear == true) {
reset()
}
val includeInUpdateStatusToCategoryMap = categories.groupBy { it.includeInUpdate }
val excludedCategories = includeInUpdateStatusToCategoryMap[IncludeOrExclude.EXCLUDE].orEmpty()
val includedCategories = includeInUpdateStatusToCategoryMap[IncludeOrExclude.INCLUDE].orEmpty()
val unsetCategories = includeInUpdateStatusToCategoryMap[IncludeOrExclude.UNSET].orEmpty()
val categoriesToUpdate =
if (forceAll) {
categories
} else {
includedCategories.ifEmpty { unsetCategories }
}
val skippedCategories = categories.subtract(categoriesToUpdate.toSet()).toList()
val updateStatusCategories =
mapOf(
Pair(CategoryUpdateStatus.UPDATING, categoriesToUpdate),
Pair(CategoryUpdateStatus.SKIPPED, skippedCategories),
)
logger.debug { "Updating categories: '${categoriesToUpdate.joinToString("', '") { it.name }}'" }
val categoriesToUpdateMangas =
categoriesToUpdate
.flatMap { CategoryManga.getCategoryMangaList(it.id) }
.distinctBy { it.id }
val mangasToCategoriesMap = CategoryManga.getMangasCategories(categoriesToUpdateMangas.map { it.id })
val mangasToUpdate =
categoriesToUpdateMangas
.asSequence()
.filter { it.updateStrategy == UpdateStrategy.ALWAYS_UPDATE }
.filter {
if (serverConfig.excludeUnreadChapters.value) {
(it.unreadCount ?: 0L) == 0L
} else {
true
}
}.filter {
if (it.initialized && serverConfig.excludeNotStarted.value) {
it.lastReadAt != null
} else {
true
}
}.filter {
if (serverConfig.excludeCompleted.value) {
it.status != MangaStatus.COMPLETED.name
} else {
true
}
}.filter { forceAll || !excludedCategories.any { category -> mangasToCategoriesMap[it.id]?.contains(category) == true } }
.toList()
val skippedMangas = categoriesToUpdateMangas.subtract(mangasToUpdate.toSet()).toList()
this.updateStatusCategories = updateStatusCategories
this.updateStatusSkippedMangas = skippedMangas
if (mangasToUpdate.isEmpty()) {
// In case no manga gets updated and no update job was running before, the client would never receive an info
// about its update request
scope.launch {
updateStatus(immediate = true)
}
return
}
scope.launch {
updateStatus(
categoryUpdates =
updateStatusCategories[CategoryUpdateStatus.UPDATING]
?.map {
CategoryUpdateJob(it, CategoryUpdateStatus.UPDATING)
}.orEmpty(),
mangaUpdates = mangasToUpdate.map { UpdateJob(it) },
isRunning = true,
SyncManager.ensureSync()
saveLastUpdateTimestamp()
if (clear == true) {
reset()
}
val includeInUpdateStatusToCategoryMap = categories.groupBy { it.includeInUpdate }
val excludedCategories = includeInUpdateStatusToCategoryMap[IncludeOrExclude.EXCLUDE].orEmpty()
val includedCategories = includeInUpdateStatusToCategoryMap[IncludeOrExclude.INCLUDE].orEmpty()
val unsetCategories = includeInUpdateStatusToCategoryMap[IncludeOrExclude.UNSET].orEmpty()
val categoriesToUpdate =
if (forceAll) {
categories
} else {
includedCategories.ifEmpty { unsetCategories }
}
val skippedCategories = categories.subtract(categoriesToUpdate.toSet()).toList()
val updateStatusCategories =
mapOf(
Pair(CategoryUpdateStatus.UPDATING, categoriesToUpdate),
Pair(CategoryUpdateStatus.SKIPPED, skippedCategories),
)
logger.debug { "Updating categories: '${categoriesToUpdate.joinToString("', '") { it.name }}'" }
val categoriesToUpdateMangas =
categoriesToUpdate
.flatMap { CategoryManga.getCategoryMangaList(it.id) }
.distinctBy { it.id }
val mangasToCategoriesMap = CategoryManga.getMangasCategories(categoriesToUpdateMangas.map { it.id })
val mangasToUpdate =
categoriesToUpdateMangas
.asSequence()
.filter { it.updateStrategy == UpdateStrategy.ALWAYS_UPDATE }
.filter {
if (serverConfig.excludeUnreadChapters.value) {
(it.unreadCount ?: 0L) == 0L
} else {
true
}
}.filter {
if (it.initialized && serverConfig.excludeNotStarted.value) {
it.lastReadAt != null
} else {
true
}
}.filter {
if (serverConfig.excludeCompleted.value) {
it.status != MangaStatus.COMPLETED.name
} else {
true
}
}.filter {
forceAll ||
!excludedCategories.any { category ->
mangasToCategoriesMap[it.id]?.contains(category) == true
}
}.toList()
val skippedMangas = categoriesToUpdateMangas.subtract(mangasToUpdate.toSet()).toList()
this@Updater.updateStatusCategories = updateStatusCategories
this@Updater.updateStatusSkippedMangas = skippedMangas
if (mangasToUpdate.isEmpty()) {
// In case no manga gets updated and no update job was running before, the client would never receive an info
// about its update request
scope.launch {
updateStatus(immediate = true)
}
return@launch
}
scope.launch {
updateStatus(
categoryUpdates =
updateStatusCategories[CategoryUpdateStatus.UPDATING]
?.map {
CategoryUpdateJob(it, CategoryUpdateStatus.UPDATING)
}.orEmpty(),
mangaUpdates = mangasToUpdate.map { UpdateJob(it) },
isRunning = true,
)
}
addMangasToQueue(
mangasToUpdate
.sortedWith(compareBy(String.CASE_INSENSITIVE_ORDER, MangaDataClass::title)),
)
}
addMangasToQueue(
mangasToUpdate
.sortedWith(compareBy(String.CASE_INSENSITIVE_ORDER, MangaDataClass::title)),
)
}
override fun addMangasToQueue(mangas: List<MangaDataClass>) {

View File

@@ -1,6 +1,7 @@
package suwayomi.tachidesk.manga.model.dataclass
import com.fasterxml.jackson.annotation.JsonValue
import suwayomi.tachidesk.manga.impl.Category
/*
* Copyright (C) Contributors to the Suwayomi project
@@ -18,7 +19,7 @@ enum class IncludeOrExclude(
;
companion object {
fun fromValue(value: Int) = IncludeOrExclude.values().find { it.value == value } ?: UNSET
fun fromValue(value: Int) = entries.find { it.value == value } ?: UNSET
}
}
@@ -27,8 +28,19 @@ data class CategoryDataClass(
val order: Int,
val name: String,
val default: Boolean,
val size: Int,
val includeInUpdate: IncludeOrExclude,
val includeInDownload: IncludeOrExclude,
val meta: Map<String, String> = emptyMap(),
)
val version: Long,
val uid: Long,
val lastModifiedAt: Long,
) {
@Deprecated("Remove with V1 Api")
val size: Int by lazy {
Category.getCategorySize(id)
}
@Deprecated("Remove with V1 Api")
val meta: Map<String, String> by lazy {
Category.getCategoryMetaMap(id)
}
}

View File

@@ -1,6 +1,11 @@
package suwayomi.tachidesk.manga.model.dataclass
import eu.kanade.tachiyomi.source.model.SChapter
import org.jetbrains.exposed.v1.core.eq
import org.jetbrains.exposed.v1.jdbc.selectAll
import org.jetbrains.exposed.v1.jdbc.transactions.transaction
import suwayomi.tachidesk.manga.impl.Chapter.getChapterMetaMap
import suwayomi.tachidesk.manga.model.table.ChapterTable
/*
* Copyright (C) Contributors to the Suwayomi project
@@ -36,10 +41,8 @@ data class ChapterDataClass(
val downloaded: Boolean,
/** used to construct pages in the front-end */
val pageCount: Int = -1,
/** total chapter count, used to calculate if there's a next and prev chapter */
val chapterCount: Int? = null,
/** used to store client specific values */
val meta: Map<String, String> = emptyMap(),
val lastModifiedAt: Long = 0,
val version: Long = 0,
) {
companion object {
fun fromSChapter(
@@ -68,4 +71,20 @@ data class ChapterDataClass(
downloaded = false,
)
}
@Deprecated("Remove with V1 Api")
val chapterCount: Int by lazy {
transaction {
ChapterTable
.selectAll()
.where { ChapterTable.manga eq mangaId }
.count()
.toInt()
}
}
@Deprecated("Remove with V1 Api")
val meta: Map<String, String> by lazy {
getChapterMetaMap(id)
}
}

View File

@@ -8,6 +8,7 @@ package suwayomi.tachidesk.manga.model.dataclass
* file, You can obtain one at https://mozilla.org/MPL/2.0/. */
import eu.kanade.tachiyomi.source.model.UpdateStrategy
import suwayomi.tachidesk.manga.impl.Manga.getMangaMetaMap
import suwayomi.tachidesk.manga.impl.util.lang.trimAll
import suwayomi.tachidesk.manga.model.table.MangaStatus
import java.time.Instant
@@ -28,23 +29,28 @@ data class MangaDataClass(
val inLibrary: Boolean = false,
val inLibraryAt: Long = 0,
val source: SourceDataClass? = null,
/** meta data for clients */
val meta: Map<String, String> = emptyMap(),
val realUrl: String? = null,
var lastFetchedAt: Long? = 0,
var chaptersLastFetchedAt: Long? = 0,
var updateStrategy: UpdateStrategy = UpdateStrategy.ALWAYS_UPDATE,
val lastFetchedAt: Long? = 0,
val chaptersLastFetchedAt: Long? = 0,
val updateStrategy: UpdateStrategy = UpdateStrategy.ALWAYS_UPDATE,
val freshData: Boolean = false,
var unreadCount: Long? = null,
var downloadCount: Long? = null,
var chapterCount: Long? = null,
var lastReadAt: Long? = null,
var lastChapterRead: ChapterDataClass? = null,
val unreadCount: Long? = null,
val downloadCount: Long? = null,
val chapterCount: Long? = null,
val lastReadAt: Long? = null,
val lastChapterRead: ChapterDataClass? = null,
val age: Long? = if (lastFetchedAt == null) 0 else Instant.now().epochSecond.minus(lastFetchedAt),
val chaptersAge: Long? = if (chaptersLastFetchedAt == null) null else Instant.now().epochSecond.minus(chaptersLastFetchedAt),
val trackers: List<MangaTrackerDataClass>? = null,
val lastModifiedAt: Long = 0,
val version: Long = 0,
) {
override fun toString(): String = "\"$title\" (id= $id) (sourceId= $sourceId)"
@Deprecated("Remove with V1 Api")
val meta: Map<String, String> by lazy {
getMangaMetaMap(id)
}
}
data class PagedMangaListDataClass(

View File

@@ -9,7 +9,6 @@ package suwayomi.tachidesk.manga.model.table
import org.jetbrains.exposed.v1.core.ResultRow
import org.jetbrains.exposed.v1.core.dao.id.IntIdTable
import suwayomi.tachidesk.manga.impl.Category
import suwayomi.tachidesk.manga.model.dataclass.CategoryDataClass
import suwayomi.tachidesk.manga.model.dataclass.IncludeOrExclude
@@ -19,16 +18,22 @@ object CategoryTable : IntIdTable() {
val isDefault = bool("is_default").default(false)
val includeInUpdate = integer("include_in_update").default(IncludeOrExclude.UNSET.value)
val includeInDownload = integer("include_in_download").default(IncludeOrExclude.UNSET.value)
val version = long("version").default(0)
val uid = long("uid").default(0)
val lastModifiedAt = long("last_modified_at").default(0)
val isSyncing = bool("is_syncing").default(false)
}
fun CategoryTable.toDataClass(categoryEntry: ResultRow) =
CategoryDataClass(
categoryEntry[id].value,
categoryEntry[order],
categoryEntry[name],
categoryEntry[isDefault],
Category.getCategorySize(categoryEntry[id].value),
IncludeOrExclude.fromValue(categoryEntry[includeInUpdate]),
IncludeOrExclude.fromValue(categoryEntry[includeInDownload]),
Category.getCategoryMetaMap(categoryEntry[id].value),
id = categoryEntry[id].value,
order = categoryEntry[order],
name = categoryEntry[name],
default = categoryEntry[isDefault],
includeInUpdate = IncludeOrExclude.fromValue(categoryEntry[includeInUpdate]),
includeInDownload = IncludeOrExclude.fromValue(categoryEntry[includeInDownload]),
version = categoryEntry[version],
uid = categoryEntry[uid],
lastModifiedAt = categoryEntry[lastModifiedAt],
)

View File

@@ -10,10 +10,6 @@ package suwayomi.tachidesk.manga.model.table
import org.jetbrains.exposed.v1.core.ReferenceOption
import org.jetbrains.exposed.v1.core.ResultRow
import org.jetbrains.exposed.v1.core.dao.id.IntIdTable
import org.jetbrains.exposed.v1.core.eq
import org.jetbrains.exposed.v1.jdbc.selectAll
import org.jetbrains.exposed.v1.jdbc.transactions.transaction
import suwayomi.tachidesk.manga.impl.Chapter.getChapterMetaMap
import suwayomi.tachidesk.manga.model.dataclass.ChapterDataClass
import suwayomi.tachidesk.manga.model.table.columns.truncatingVarchar
@@ -42,45 +38,30 @@ object ChapterTable : IntIdTable() {
val manga = reference("manga", MangaTable, ReferenceOption.CASCADE)
val koreaderHash = varchar("koreader_hash", 32).nullable()
val lastModifiedAt = long("last_modified_at").default(0)
val version = long("version").default(0)
val isSyncing = bool("is_syncing").default(false)
}
fun ChapterTable.toDataClass(
chapterEntry: ResultRow,
includeChapterCount: Boolean = true,
includeChapterMeta: Boolean = true,
) = ChapterDataClass(
id = chapterEntry[id].value,
url = chapterEntry[url],
name = chapterEntry[name],
uploadDate = chapterEntry[date_upload],
chapterNumber = chapterEntry[chapter_number],
scanlator = chapterEntry[scanlator],
mangaId = chapterEntry[manga].value,
read = chapterEntry[isRead],
bookmarked = chapterEntry[isBookmarked],
lastPageRead = chapterEntry[lastPageRead],
lastReadAt = chapterEntry[lastReadAt],
index = chapterEntry[sourceOrder],
fetchedAt = chapterEntry[fetchedAt],
realUrl = chapterEntry[realUrl],
downloaded = chapterEntry[isDownloaded],
pageCount = chapterEntry[pageCount],
chapterCount =
if (includeChapterCount) {
transaction {
ChapterTable
.selectAll()
.where { manga eq chapterEntry[manga].value }
.count()
.toInt()
}
} else {
null
},
meta =
if (includeChapterMeta) {
getChapterMetaMap(chapterEntry[id])
} else {
emptyMap()
},
)
fun ChapterTable.toDataClass(chapterEntry: ResultRow) =
ChapterDataClass(
id = chapterEntry[id].value,
url = chapterEntry[url],
name = chapterEntry[name],
uploadDate = chapterEntry[date_upload],
chapterNumber = chapterEntry[chapter_number],
scanlator = chapterEntry[scanlator],
mangaId = chapterEntry[manga].value,
read = chapterEntry[isRead],
bookmarked = chapterEntry[isBookmarked],
lastPageRead = chapterEntry[lastPageRead],
lastReadAt = chapterEntry[lastReadAt],
index = chapterEntry[sourceOrder],
fetchedAt = chapterEntry[fetchedAt],
realUrl = chapterEntry[realUrl],
downloaded = chapterEntry[isDownloaded],
pageCount = chapterEntry[pageCount],
lastModifiedAt = chapterEntry[lastModifiedAt],
version = chapterEntry[version],
)

View File

@@ -11,11 +11,9 @@ import eu.kanade.tachiyomi.source.model.SManga
import eu.kanade.tachiyomi.source.model.UpdateStrategy
import org.jetbrains.exposed.v1.core.ResultRow
import org.jetbrains.exposed.v1.core.dao.id.IntIdTable
import suwayomi.tachidesk.manga.impl.Manga.getMangaMetaMap
import suwayomi.tachidesk.manga.impl.MangaList.proxyThumbnailUrl
import suwayomi.tachidesk.manga.model.dataclass.MangaDataClass
import suwayomi.tachidesk.manga.model.dataclass.toGenreList
import suwayomi.tachidesk.manga.model.table.MangaStatus.Companion
import suwayomi.tachidesk.manga.model.table.columns.truncatingVarchar
import suwayomi.tachidesk.manga.model.table.columns.unlimitedVarchar
@@ -46,37 +44,35 @@ object MangaTable : IntIdTable() {
val chaptersLastFetchedAt = long("chapters_last_fetched_at").default(0)
val updateStrategy = varchar("update_strategy", 256).default(UpdateStrategy.ALWAYS_UPDATE.name)
val lastModifiedAt = long("last_modified_at").default(0)
val version = long("version").default(0)
val isSyncing = bool("is_syncing").default(false)
}
fun MangaTable.toDataClass(
mangaEntry: ResultRow,
includeMangaMeta: Boolean = true,
) = MangaDataClass(
id = mangaEntry[this.id].value,
sourceId = mangaEntry[sourceReference].toString(),
url = mangaEntry[url],
title = mangaEntry[title],
thumbnailUrl = proxyThumbnailUrl(mangaEntry[this.id].value),
thumbnailUrlLastFetched = mangaEntry[thumbnailUrlLastFetched],
initialized = mangaEntry[initialized],
artist = mangaEntry[artist],
author = mangaEntry[author],
description = mangaEntry[description],
genre = mangaEntry[genre].toGenreList(),
status = Companion.valueOf(mangaEntry[status]).name,
inLibrary = mangaEntry[inLibrary],
inLibraryAt = mangaEntry[inLibraryAt],
meta =
if (includeMangaMeta) {
getMangaMetaMap(mangaEntry[id].value)
} else {
emptyMap()
},
realUrl = mangaEntry[realUrl],
lastFetchedAt = mangaEntry[lastFetchedAt],
chaptersLastFetchedAt = mangaEntry[chaptersLastFetchedAt],
updateStrategy = UpdateStrategy.valueOf(mangaEntry[updateStrategy]),
)
fun MangaTable.toDataClass(mangaEntry: ResultRow) =
MangaDataClass(
id = mangaEntry[this.id].value,
sourceId = mangaEntry[sourceReference].toString(),
url = mangaEntry[url],
title = mangaEntry[title],
thumbnailUrl = proxyThumbnailUrl(mangaEntry[this.id].value),
thumbnailUrlLastFetched = mangaEntry[thumbnailUrlLastFetched],
initialized = mangaEntry[initialized],
artist = mangaEntry[artist],
author = mangaEntry[author],
description = mangaEntry[description],
genre = mangaEntry[genre].toGenreList(),
status = MangaStatus.valueOf(mangaEntry[status]).name,
inLibrary = mangaEntry[inLibrary],
inLibraryAt = mangaEntry[inLibraryAt],
realUrl = mangaEntry[realUrl],
lastFetchedAt = mangaEntry[lastFetchedAt],
chaptersLastFetchedAt = mangaEntry[chaptersLastFetchedAt],
updateStrategy = UpdateStrategy.valueOf(mangaEntry[updateStrategy]),
lastModifiedAt = mangaEntry[lastModifiedAt],
version = mangaEntry[version],
)
enum class MangaStatus(
val value: Int,

View File

@@ -13,18 +13,22 @@ object OpdsAPI {
// OPDS Search Description Feed
get("search", OpdsV1Controller.searchFeed)
// --- Main Navigation Feeds ---
// Explore Navigation Feed
get("explore", OpdsV1Controller.exploreSourcesFeed)
// Reading History Acquisition Feed
get("history", OpdsV1Controller.historyFeed)
// Library Updates Acquisition Feed
get("library-updates", OpdsV1Controller.libraryUpdatesFeed)
// --- Library-Specific Feeds ---
// --- Remote Catalog Exploration ---
// List of available online sources
get("explore", OpdsV1Controller.exploreSourcesFeed)
// Browse series from a specific online source
path("explore/source/{sourceId}") {
get(OpdsV1Controller.exploreSourceFeed)
}
// --- Library Navigation Feeds ---
path("library") {
// All Series in Library / Search Results Feed (Acquisition)
get("series", OpdsV1Controller.seriesFeed)
@@ -32,11 +36,6 @@ object OpdsAPI {
// Library Sources Navigation Feed
get("sources", OpdsV1Controller.librarySourcesFeed)
// Library Source-Specific Series Acquisition Feed
path("source/{sourceId}") {
get(OpdsV1Controller.librarySourceFeed)
}
// Library Categories Navigation Feed
get("categories", OpdsV1Controller.categoriesFeed)
@@ -50,26 +49,10 @@ object OpdsAPI {
get("languages", OpdsV1Controller.languagesFeed)
}
// --- Explore-Specific Feeds ---
// All Sources Navigation Feed (Explore)
get("sources", OpdsV1Controller.exploreSourcesFeed)
// Source-Specific Series Acquisition Feed (Explore)
// --- Library Series Filters ---
// Source-Specific Series Acquisition Feed (Library)
path("source/{sourceId}") {
get(OpdsV1Controller.exploreSourceFeed)
}
// --- Item-Specific Feeds (Apply to both Library and Explore contexts) ---
// Series Chapters Acquisition Feed
path("series/{seriesId}/chapters") {
get(OpdsV1Controller.seriesChaptersFeed)
}
// Chapter Metadata Acquisition Feed
path("series/{seriesId}/chapter/{chapterIndex}/metadata") {
get(OpdsV1Controller.chapterMetadataFeed)
get(OpdsV1Controller.librarySourceFeed)
}
// Category-Specific Series Acquisition Feed (Library)
@@ -91,6 +74,16 @@ object OpdsAPI {
path("language/{langCode}") {
get(OpdsV1Controller.languageFeed)
}
// --- Item Specific Feeds ---
// Series Chapters Acquisition Feed
path("series/{seriesId}/chapters") {
get(OpdsV1Controller.seriesChaptersFeed)
}
// Chapter Metadata Acquisition Feed
path("series/{seriesId}/chapter/{chapterIndex}/metadata") {
get(OpdsV1Controller.chapterMetadataFeed)
}
}
}
}

View File

@@ -32,21 +32,21 @@ object OpdsV1Controller {
*/
private fun getLibraryFeed(
ctx: Context,
pageNum: Int?,
locale: Locale,
criteria: OpdsMangaFilter,
isSearch: Boolean,
pageNum: Int,
) {
val locale: Locale = LocalizationHelper.ctxToLocale(ctx, ctx.queryParam("lang"))
ctx.future {
future {
OpdsFeedBuilder.getLibraryFeed(
criteria = criteria,
baseUrl = BASE_URL,
pageNum = pageNum ?: 1,
sort = criteria.sort,
filter = criteria.filter,
locale = locale,
isSearch = isSearch,
BASE_URL,
locale,
criteria,
isSearch,
pageNum,
criteria.sort,
criteria.filter,
)
}.thenApply { xml ->
ctx.contentType(OPDS_MIME).result(xml)
@@ -94,7 +94,7 @@ object OpdsV1Controller {
val locale: Locale = LocalizationHelper.ctxToLocale(ctx, lang)
ctx.future {
future {
OpdsFeedBuilder.getHistoryFeed(BASE_URL, pageNumber ?: 1, locale)
OpdsFeedBuilder.getHistoryFeed(BASE_URL, locale, pageNumber ?: 1)
}.thenApply { xml ->
ctx.contentType(OPDS_MIME).result(xml)
}
@@ -139,6 +139,7 @@ object OpdsV1Controller {
/**
* Serves an acquisition feed for all series in the library or search results.
* This endpoint handles both general library browsing and specific search queries.
* This is the ONLY feed that extracts all the cross-filters from the context.
*/
val seriesFeed =
handler(
@@ -157,29 +158,14 @@ object OpdsV1Controller {
val opdsSearchCriteria = OpdsSearchCriteria(query, author, title)
ctx.future {
future {
OpdsFeedBuilder.getSearchFeed(opdsSearchCriteria, BASE_URL, pageNumber ?: 1, locale)
OpdsFeedBuilder.getSearchFeed(BASE_URL, locale, opdsSearchCriteria, pageNumber ?: 1)
}.thenApply { xml ->
ctx.contentType(OPDS_MIME).result(xml)
}
}
} else {
val criteria =
OpdsMangaFilter(
sourceId = ctx.queryParam("source_id")?.toLongOrNull(),
categoryId = ctx.queryParam("category_id")?.toIntOrNull(),
statusId = ctx.queryParam("status_id")?.toIntOrNull(),
genre = ctx.queryParam("genre"),
langCode = ctx.queryParam("lang_code"),
sort = ctx.queryParam("sort"),
filter = ctx.queryParam("filter"),
primaryFilter = PrimaryFilterType.NONE,
)
getLibraryFeed(
ctx,
pageNumber,
criteria,
isSearch = false,
)
val criteria = OpdsMangaFilter.fromContext(ctx, PrimaryFilterType.NONE)
getLibraryFeed(ctx, locale, criteria, false, pageNumber ?: 1)
}
},
withResults = { httpCode(HttpStatus.OK) },
@@ -203,7 +189,7 @@ object OpdsV1Controller {
val locale: Locale = LocalizationHelper.ctxToLocale(ctx, lang)
ctx.future {
future {
OpdsFeedBuilder.getExploreSourcesFeed(BASE_URL, pageNumber ?: 1, locale)
OpdsFeedBuilder.getExploreSourcesFeed(BASE_URL, locale, pageNumber ?: 1)
}.thenApply { xml ->
ctx.contentType(OPDS_MIME).result(xml)
}
@@ -230,7 +216,7 @@ object OpdsV1Controller {
val locale: Locale = LocalizationHelper.ctxToLocale(ctx, lang)
ctx.future {
future {
OpdsFeedBuilder.getLibrarySourcesFeed(BASE_URL, pageNumber ?: 1, locale)
OpdsFeedBuilder.getLibrarySourcesFeed(BASE_URL, locale, pageNumber ?: 1)
}.thenApply { xml ->
ctx.contentType(OPDS_MIME).result(xml)
}
@@ -257,7 +243,7 @@ object OpdsV1Controller {
val locale: Locale = LocalizationHelper.ctxToLocale(ctx, lang)
ctx.future {
future {
OpdsFeedBuilder.getCategoriesFeed(BASE_URL, pageNumber ?: 1, locale)
OpdsFeedBuilder.getCategoriesFeed(BASE_URL, locale, pageNumber ?: 1)
}.thenApply { xml ->
ctx.contentType(OPDS_MIME).result(xml)
}
@@ -284,7 +270,7 @@ object OpdsV1Controller {
val locale: Locale = LocalizationHelper.ctxToLocale(ctx, lang)
ctx.future {
future {
OpdsFeedBuilder.getGenresFeed(BASE_URL, pageNumber ?: 1, locale)
OpdsFeedBuilder.getGenresFeed(BASE_URL, locale, pageNumber ?: 1)
}.thenApply { xml ->
ctx.contentType(OPDS_MIME).result(xml)
}
@@ -298,6 +284,7 @@ object OpdsV1Controller {
*/
val statusesFeed =
handler(
queryParam<Int?>("pageNumber"),
queryParam<String?>("lang"),
documentWith = {
withOperation {
@@ -305,12 +292,12 @@ object OpdsV1Controller {
description("Navigation feed listing series publication statuses for the library.")
}
},
behaviorOf = { ctx, lang ->
behaviorOf = { ctx, pageNumber, lang ->
ctx.getAttribute(Attribute.TachideskUser).requireUserWithBasicFallback(ctx)
val locale: Locale = LocalizationHelper.ctxToLocale(ctx, lang)
ctx.future {
future {
OpdsFeedBuilder.getStatusFeed(BASE_URL, 1, locale)
OpdsFeedBuilder.getStatusFeed(BASE_URL, locale, pageNumber ?: 1)
}.thenApply { xml ->
ctx.contentType(OPDS_MIME).result(xml)
}
@@ -324,6 +311,7 @@ object OpdsV1Controller {
*/
val languagesFeed =
handler(
queryParam<Int?>("pageNumber"),
queryParam<String?>("lang"),
documentWith = {
withOperation {
@@ -331,12 +319,12 @@ object OpdsV1Controller {
description("Navigation feed listing available content languages for series in the library.")
}
},
behaviorOf = { ctx, lang ->
behaviorOf = { ctx, pageNumber, lang ->
ctx.getAttribute(Attribute.TachideskUser).requireUserWithBasicFallback(ctx)
val locale: Locale = LocalizationHelper.ctxToLocale(ctx, lang)
ctx.future {
future {
OpdsFeedBuilder.getLanguagesFeed(BASE_URL, locale)
OpdsFeedBuilder.getLanguagesFeed(BASE_URL, locale, pageNumber ?: 1)
}.thenApply { xml ->
ctx.contentType(OPDS_MIME).result(xml)
}
@@ -363,7 +351,7 @@ object OpdsV1Controller {
val locale: Locale = LocalizationHelper.ctxToLocale(ctx, lang)
ctx.future {
future {
OpdsFeedBuilder.getLibraryUpdatesFeed(BASE_URL, pageNumber ?: 1, locale)
OpdsFeedBuilder.getLibraryUpdatesFeed(BASE_URL, locale, pageNumber ?: 1)
}.thenApply { xml ->
ctx.contentType(OPDS_MIME).result(xml)
}
@@ -392,7 +380,7 @@ object OpdsV1Controller {
val locale: Locale = LocalizationHelper.ctxToLocale(ctx, lang)
ctx.future {
future {
OpdsFeedBuilder.getExploreSourceFeed(sourceId, BASE_URL, pageNumber ?: 1, sort ?: "popular", locale)
OpdsFeedBuilder.getExploreSourceFeed(BASE_URL, locale, sourceId, pageNumber ?: 1, sort ?: "popular")
}.thenApply { xml ->
ctx.contentType(OPDS_MIME).result(xml)
}
@@ -425,8 +413,9 @@ object OpdsV1Controller {
documentWith = { withOperation { summary("OPDS Library Source Specific Series Feed") } },
behaviorOf = { ctx, sourceId ->
ctx.getAttribute(Attribute.TachideskUser).requireUserWithBasicFallback(ctx)
val locale: Locale = LocalizationHelper.ctxToLocale(ctx, ctx.queryParam("lang"))
val criteria = buildCriteriaFromContext(ctx, OpdsMangaFilter(sourceId = sourceId, primaryFilter = PrimaryFilterType.SOURCE))
getLibraryFeed(ctx, ctx.queryParam("pageNumber")?.toIntOrNull(), criteria, isSearch = false)
getLibraryFeed(ctx, locale, criteria, false, ctx.queryParam("pageNumber")?.toIntOrNull() ?: 1)
},
withResults = {
httpCode(HttpStatus.OK)
@@ -443,9 +432,10 @@ object OpdsV1Controller {
documentWith = { withOperation { summary("OPDS Category Specific Series Feed") } },
behaviorOf = { ctx, categoryId ->
ctx.getAttribute(Attribute.TachideskUser).requireUserWithBasicFallback(ctx)
val locale: Locale = LocalizationHelper.ctxToLocale(ctx, ctx.queryParam("lang"))
val criteria =
buildCriteriaFromContext(ctx, OpdsMangaFilter(categoryId = categoryId, primaryFilter = PrimaryFilterType.CATEGORY))
getLibraryFeed(ctx, ctx.queryParam("pageNumber")?.toIntOrNull(), criteria, isSearch = false)
getLibraryFeed(ctx, locale, criteria, false, ctx.queryParam("pageNumber")?.toIntOrNull() ?: 1)
},
withResults = {
httpCode(HttpStatus.OK)
@@ -462,8 +452,9 @@ object OpdsV1Controller {
documentWith = { withOperation { summary("OPDS Genre Specific Series Feed") } },
behaviorOf = { ctx, genre ->
ctx.getAttribute(Attribute.TachideskUser).requireUserWithBasicFallback(ctx)
val locale: Locale = LocalizationHelper.ctxToLocale(ctx, ctx.queryParam("lang"))
val criteria = buildCriteriaFromContext(ctx, OpdsMangaFilter(genre = genre, primaryFilter = PrimaryFilterType.GENRE))
getLibraryFeed(ctx, ctx.queryParam("pageNumber")?.toIntOrNull(), criteria, isSearch = false)
getLibraryFeed(ctx, locale, criteria, false, ctx.queryParam("pageNumber")?.toIntOrNull() ?: 1)
},
withResults = {
httpCode(HttpStatus.OK)
@@ -480,8 +471,9 @@ object OpdsV1Controller {
documentWith = { withOperation { summary("OPDS Status Specific Series Feed") } },
behaviorOf = { ctx, statusId ->
ctx.getAttribute(Attribute.TachideskUser).requireUserWithBasicFallback(ctx)
val locale: Locale = LocalizationHelper.ctxToLocale(ctx, ctx.queryParam("lang"))
val criteria = buildCriteriaFromContext(ctx, OpdsMangaFilter(statusId = statusId, primaryFilter = PrimaryFilterType.STATUS))
getLibraryFeed(ctx, ctx.queryParam("pageNumber")?.toIntOrNull(), criteria, isSearch = false)
getLibraryFeed(ctx, locale, criteria, false, ctx.queryParam("pageNumber")?.toIntOrNull() ?: 1)
},
withResults = {
httpCode(HttpStatus.OK)
@@ -503,9 +495,10 @@ object OpdsV1Controller {
},
behaviorOf = { ctx, langCode ->
ctx.getAttribute(Attribute.TachideskUser).requireUserWithBasicFallback(ctx)
val locale: Locale = LocalizationHelper.ctxToLocale(ctx, ctx.queryParam("lang"))
val criteria =
buildCriteriaFromContext(ctx, OpdsMangaFilter(langCode = langCode, primaryFilter = PrimaryFilterType.LANGUAGE))
getLibraryFeed(ctx, ctx.queryParam("pageNumber")?.toIntOrNull(), criteria, isSearch = false)
getLibraryFeed(ctx, locale, criteria, false, ctx.queryParam("pageNumber")?.toIntOrNull() ?: 1)
},
withResults = {
httpCode(HttpStatus.OK)
@@ -534,7 +527,7 @@ object OpdsV1Controller {
val locale: Locale = LocalizationHelper.ctxToLocale(ctx, lang)
ctx.future {
future {
OpdsFeedBuilder.getSeriesChaptersFeed(seriesId, BASE_URL, pageNumber ?: 1, sort, filter, locale)
OpdsFeedBuilder.getSeriesChaptersFeed(BASE_URL, locale, seriesId, pageNumber ?: 1, sort, filter)
}.thenApply { xml ->
ctx.contentType(OPDS_MIME).result(xml)
}
@@ -565,7 +558,7 @@ object OpdsV1Controller {
val locale: Locale = LocalizationHelper.ctxToLocale(ctx, lang)
ctx.future {
future {
OpdsFeedBuilder.getChapterMetadataFeed(seriesId, chapterIndex, BASE_URL, locale)
OpdsFeedBuilder.getChapterMetadataFeed(BASE_URL, locale, seriesId, chapterIndex)
}.thenApply { xml ->
ctx.contentType(OPDS_MIME).result(xml)
}

View File

@@ -12,4 +12,6 @@ data class OpdsChapterListAcqEntry(
val lastReadAt: Long,
val sourceOrder: Int,
val pageCount: Int, // Can be -1 if not known
val downloaded: Boolean,
val cbzFileSize: Long? = null,
)

View File

@@ -5,6 +5,7 @@ data class OpdsChapterMetadataAcqEntry(
val mangaId: Int,
val name: String,
val uploadDate: Long,
val chapterNumber: Float, // Added to fallback chapter titles
val scanlator: String?,
val read: Boolean,
val lastPageRead: Int,
@@ -13,4 +14,5 @@ data class OpdsChapterMetadataAcqEntry(
val downloaded: Boolean,
val pageCount: Int,
val url: String?,
val cbzFileSize: Long? = null,
)

View File

@@ -5,6 +5,7 @@ data class OpdsHistoryAcqEntry(
val mangaTitle: String,
val mangaAuthor: String?,
val mangaId: Int,
val mangaTotalChapters: Long,
val mangaSourceLang: String?,
val mangaThumbnailUrl: String?,
)

View File

@@ -5,6 +5,7 @@ data class OpdsLibraryUpdateAcqEntry(
val mangaTitle: String,
val mangaAuthor: String?,
val mangaId: Int,
val mangaTotalChapters: Long,
val mangaSourceLang: String?,
val mangaThumbnailUrl: String?,
)

View File

@@ -5,4 +5,5 @@ data class OpdsMangaDetails( // Kept name, it's specific enough
val title: String,
val thumbnailUrl: String?,
val author: String?, // Added for chapter entry authors
val totalChapters: Long, // Added to determine if it's a oneshot
)

View File

@@ -1,5 +1,6 @@
package suwayomi.tachidesk.opds.dto
import io.javalin.http.Context
import suwayomi.tachidesk.opds.util.OpdsStringUtil.encodeForOpdsURL
/**
@@ -69,4 +70,24 @@ data class OpdsMangaFilter(
"genre" -> this.copy(genre = value)
else -> this
}
companion object {
/**
* Creates an OpdsMangaFilter directly from the Javalin Context, capturing all cross-filters.
*/
fun fromContext(
ctx: Context,
primaryFilter: PrimaryFilterType = PrimaryFilterType.NONE,
): OpdsMangaFilter =
OpdsMangaFilter(
sourceId = ctx.queryParam("source_id")?.toLongOrNull(),
categoryId = ctx.queryParam("category_id")?.toIntOrNull(),
statusId = ctx.queryParam("status_id")?.toIntOrNull(),
genre = ctx.queryParam("genre"),
langCode = ctx.queryParam("lang_code"),
sort = ctx.queryParam("sort"),
filter = ctx.queryParam("filter"),
primaryFilter = primaryFilter,
)
}
}

View File

@@ -16,145 +16,131 @@ import kotlin.math.ceil
*/
class FeedBuilderInternal(
private val baseUrl: String,
private val locale: Locale,
private val idPath: String,
private val title: String,
private val locale: Locale,
private val feedType: String,
private val pageNum: Int? = 1,
private val pageNum: Int? = null,
private val explicitQueryParams: String? = null,
private val currentSort: String? = null,
private val currentFilter: String? = null,
private val isSearchFeed: Boolean = false,
) {
private val opdsItemsPerPageBounded: Int
get() = serverConfig.opdsItemsPerPage.value
private val feedAuthor = OpdsAuthorXml("Suwayomi", "https://suwayomi.org/")
private val feedGeneratedAt: String = OpdsDateUtil.formatCurrentInstantForOpds()
var totalResults: Long = 0
var icon: String? = null
val links = mutableListOf<OpdsLinkXml>()
val entries = mutableListOf<OpdsEntryXml>()
private fun buildUrlWithParams(
baseHrefPath: String,
page: Int?,
): String {
val sb = StringBuilder("$baseUrl/$baseHrefPath")
val queryParamsList = mutableListOf<String>()
private fun buildUrlWithParams(page: Int? = pageNum): String {
val queryParams =
listOfNotNull(
explicitQueryParams?.takeIf(String::isNotBlank),
page?.let { "pageNumber=$it" },
currentSort?.let { "sort=$it" },
currentFilter?.let { "filter=$it" },
"lang=${locale.toLanguageTag()}",
).joinToString("&")
explicitQueryParams?.takeIf { it.isNotBlank() }?.let { queryParamsList.add(it) }
page?.let { queryParamsList.add("pageNumber=$it") }
currentSort?.let { queryParamsList.add("sort=$it") }
currentFilter?.let { queryParamsList.add("filter=$it") }
queryParamsList.add("lang=${locale.toLanguageTag()}")
if (queryParamsList.isNotEmpty()) {
sb.append("?").append(queryParamsList.joinToString("&"))
}
return sb.toString()
return "$baseUrl/$idPath" + if (queryParams.isNotEmpty()) "?$queryParams" else ""
}
fun build(): OpdsFeedXml {
val selfLinkHref = buildUrlWithParams(idPath, if (pageNum != null) pageNum else null)
val feedLinks = mutableListOf<OpdsLinkXml>()
feedLinks.addAll(this.links)
feedLinks.add(
OpdsLinkXml(OpdsConstants.LINK_REL_SELF, selfLinkHref, feedType, MR.strings.opds_linktitle_self_feed.localized(locale)),
)
feedLinks.add(
OpdsLinkXml(
OpdsConstants.LINK_REL_START,
"$baseUrl?lang=${locale.toLanguageTag()}",
OpdsConstants.TYPE_ATOM_XML_FEED_NAVIGATION,
MR.strings.opds_linktitle_catalog_root.localized(locale),
),
)
feedLinks.add(
OpdsLinkXml(
OpdsConstants.LINK_REL_SEARCH,
"$baseUrl/search?lang=${locale.toLanguageTag()}",
OpdsConstants.TYPE_OPENSEARCH_DESCRIPTION,
MR.strings.opds_linktitle_search_catalog.localized(locale),
),
)
// Add pagination links if needed
if (pageNum != null) {
val totalPages = ceil(totalResults.toDouble() / opdsItemsPerPageBounded).toInt()
if (totalPages > 1) {
val currentPage = pageNum.coerceAtLeast(1)
// Always add 'first' link when there are multiple pages
feedLinks.add(
OpdsLinkXml(
OpdsConstants.LINK_REL_FIRST,
buildUrlWithParams(idPath, 1),
feedType,
MR.strings.opds_linktitle_first_page.localized(locale),
),
)
// Add 'prev' link if not on first page
if (currentPage > 1) {
feedLinks.add(
OpdsLinkXml(
OpdsConstants.LINK_REL_PREV,
buildUrlWithParams(idPath, currentPage - 1),
feedType,
MR.strings.opds_linktitle_previous_page.localized(locale),
),
)
}
// Add 'next' link if not on last page
if (currentPage < totalPages) {
feedLinks.add(
OpdsLinkXml(
OpdsConstants.LINK_REL_NEXT,
buildUrlWithParams(idPath, currentPage + 1),
feedType,
MR.strings.opds_linktitle_next_page.localized(locale),
),
)
}
// Always add 'last' link when there are multiple pages
feedLinks.add(
OpdsLinkXml(
OpdsConstants.LINK_REL_LAST,
buildUrlWithParams(idPath, totalPages),
feedType,
MR.strings.opds_linktitle_last_page.localized(locale),
),
)
}
}
val urnParams = mutableListOf<String>()
urnParams.add(locale.toLanguageTag())
pageNum?.let { urnParams.add("page$it") }
explicitQueryParams?.let { urnParams.add(it.replace("&", ":").replace("=", "_")) }
currentSort?.let { urnParams.add("sort_$it") }
currentFilter?.let { urnParams.add("filter_$it") }
val urnSuffix = if (urnParams.isNotEmpty()) ":${urnParams.joinToString(":")}" else ""
val showOpenSearchFields = isSearchFeed && pageNum != null && totalResults > 0
val itemsPerPage = serverConfig.opdsItemsPerPage.value
val showOpenSearch = isSearchFeed && pageNum != null && totalResults > 0
val urnSuffix =
listOfNotNull(
locale.toLanguageTag(),
pageNum?.let { "page$it" },
explicitQueryParams?.replace("&", ":")?.replace("=", "_"),
currentSort?.let { "sort_$it" },
currentFilter?.let { "filter_$it" },
).joinToString(":")
return OpdsFeedXml(
id = "urn:suwayomi:feed:${idPath.replace('/',':')}$urnSuffix",
id = "urn:suwayomi:feed:${idPath.replace('/', ':')}${if (urnSuffix.isNotEmpty()) ":$urnSuffix" else ""}",
title = title,
updated = feedGeneratedAt,
updated = OpdsDateUtil.formatCurrentInstantForOpds(),
icon = icon,
author = feedAuthor,
links = feedLinks,
author = OpdsAuthorXml("Suwayomi", "https://suwayomi.org/"),
links =
buildList {
addAll(this@FeedBuilderInternal.links)
add(
OpdsLinkXml(
OpdsConstants.LINK_REL_SELF,
buildUrlWithParams(),
feedType,
MR.strings.opds_linktitle_self_feed.localized(locale),
),
)
add(
OpdsLinkXml(
OpdsConstants.LINK_REL_START,
"$baseUrl?lang=${locale.toLanguageTag()}",
OpdsConstants.TYPE_ATOM_XML_FEED_NAVIGATION,
MR.strings.opds_linktitle_catalog_root.localized(locale),
),
)
add(
OpdsLinkXml(
OpdsConstants.LINK_REL_SEARCH,
"$baseUrl/search?lang=${locale.toLanguageTag()}",
OpdsConstants.TYPE_OPENSEARCH_DESCRIPTION,
MR.strings.opds_linktitle_search_catalog.localized(locale),
),
)
if (pageNum != null) {
val totalPages = ceil(totalResults.toDouble() / itemsPerPage).toInt()
if (totalPages > 1) {
val currentPage = pageNum.coerceAtLeast(1)
add(
OpdsLinkXml(
OpdsConstants.LINK_REL_FIRST,
buildUrlWithParams(1),
feedType,
MR.strings.opds_linktitle_first_page.localized(locale),
),
)
if (currentPage >
1
) {
add(
OpdsLinkXml(
OpdsConstants.LINK_REL_PREV,
buildUrlWithParams(currentPage - 1),
feedType,
MR.strings.opds_linktitle_previous_page.localized(locale),
),
)
}
if (currentPage <
totalPages
) {
add(
OpdsLinkXml(
OpdsConstants.LINK_REL_NEXT,
buildUrlWithParams(currentPage + 1),
feedType,
MR.strings.opds_linktitle_next_page.localized(locale),
),
)
}
add(
OpdsLinkXml(
OpdsConstants.LINK_REL_LAST,
buildUrlWithParams(totalPages),
feedType,
MR.strings.opds_linktitle_last_page.localized(locale),
),
)
}
}
},
entries = entries,
totalResults = totalResults.takeIf { showOpenSearchFields },
itemsPerPage = if (showOpenSearchFields) opdsItemsPerPageBounded else null,
startIndex = if (showOpenSearchFields) ((pageNum - 1) * opdsItemsPerPageBounded) + 1 else null,
totalResults = totalResults.takeIf { showOpenSearch },
itemsPerPage = itemsPerPage.takeIf { showOpenSearch },
startIndex = if (showOpenSearch && pageNum != null) ((pageNum - 1) * itemsPerPage) + 1 else null,
)
}
}

View File

@@ -1,10 +1,7 @@
package suwayomi.tachidesk.opds.impl
import dev.icerock.moko.resources.StringResource
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
import suwayomi.tachidesk.i18n.MR
import suwayomi.tachidesk.manga.impl.ChapterDownloadHelper
import suwayomi.tachidesk.manga.impl.MangaList.proxyThumbnailUrl
import suwayomi.tachidesk.manga.impl.sync.KoreaderSyncService
import suwayomi.tachidesk.manga.model.table.MangaStatus
@@ -93,15 +90,15 @@ object OpdsEntryBuilder {
/**
* Converts a manga data object into a full OPDS acquisition entry.
* @param entry The manga data object.
* @param baseUrl The base URL for constructing links.
* @param locale The locale for localization.
* @param entry The manga data object.
* @return An [OpdsEntryXml] object representing the manga.
*/
fun mangaAcqEntryToEntry(
entry: OpdsMangaAcqEntry,
baseUrl: String,
locale: Locale,
entry: OpdsMangaAcqEntry,
): OpdsEntryXml {
val displayThumbnailUrl = entry.thumbnailUrl?.let { proxyThumbnailUrl(entry.id) }
val categoryScheme = if (entry.inLibrary) "$baseUrl/library/genres" else "$baseUrl/genres"
@@ -152,40 +149,151 @@ object OpdsEntryBuilder {
)
}
/**
* Resolves the display title for a chapter, ensuring an empty name falls back to
* a generic "Chapter X" or "Oneshot" to keep the OPDS feed clean and usable.
*/
private fun resolveChapterTitle(
chapterName: String,
chapterNumber: Float,
sourceOrder: Int,
totalChapters: Long,
locale: Locale,
): String {
val trimmedName = chapterName.trim()
if (trimmedName.isNotEmpty()) {
return trimmedName
}
return if (totalChapters <= 1L) {
MR.strings.opds_chapter_title_oneshot.localized(locale)
} else {
val formatNumber =
if (chapterNumber >= 0f) {
if (chapterNumber % 1 == 0f) chapterNumber.toInt().toString() else chapterNumber.toString()
} else {
sourceOrder.toString()
}
MR.strings.opds_chapter_title_fallback.localized(locale, formatNumber)
}
}
/**
* Creates an OPDS entry for a chapter, including acquisition and streaming links.
* @param baseUrl The base URL for constructing links.
* @param locale The locale for localization.
* @param chapter The chapter data object.
* @param manga The parent manga's details.
* @param baseUrl The base URL for constructing links.
* @param addMangaTitle Whether to prepend the manga title to the entry title.
* @param locale The locale for localization.
* @param skipMetadataFeed Whether to skip the metadata feed.
* @return An [OpdsEntryXml] object for the chapter.
*/
suspend fun createChapterListEntry(
baseUrl: String,
locale: Locale,
chapter: OpdsChapterListAcqEntry,
manga: OpdsMangaDetails,
baseUrl: String,
addMangaTitle: Boolean,
locale: Locale,
skipMetadataFeed: Boolean,
): OpdsEntryXml {
var effectiveLastPageRead = chapter.lastPageRead
var effectiveLastReadAt = chapter.lastReadAt
if (skipMetadataFeed) {
val syncResult = KoreaderSyncService.checkAndPullProgress(chapter.id)
// If sync strategy dictates an update (e.g. KEEP_REMOTE), use remote data.
// If sync strategy is PROMPT (isConflict=true), we ignore it here (effectively KEEP_LOCAL/DISABLED)
// because we cannot show a prompt in the chapter list feed.
if (syncResult != null && syncResult.shouldUpdate) {
effectiveLastPageRead = syncResult.pageRead
effectiveLastReadAt = syncResult.timestamp
}
}
val statusKey =
when {
chapter.downloaded -> MR.strings.opds_chapter_status_downloaded
chapter.read -> MR.strings.opds_chapter_status_read
chapter.lastPageRead > 0 -> MR.strings.opds_chapter_status_in_progress
effectiveLastPageRead > 0 -> MR.strings.opds_chapter_status_in_progress
else -> MR.strings.opds_chapter_status_unread
}
val titlePrefix = statusKey.localized(locale)
val entryTitle = titlePrefix + (if (addMangaTitle) " ${manga.title}:" else "") + " ${chapter.name}"
val chapterName = resolveChapterTitle(chapter.name, chapter.chapterNumber, chapter.sourceOrder, manga.totalChapters, locale)
val entryTitle = titlePrefix + (if (addMangaTitle) " ${manga.title}:" else "") + " $chapterName"
val details =
buildString {
append(MR.strings.opds_chapter_details_base.localized(locale, manga.title, chapter.name))
append(MR.strings.opds_chapter_details_base.localized(locale, manga.title, chapterName))
chapter.scanlator?.takeIf { it.isNotBlank() }?.let {
append(MR.strings.opds_chapter_details_scanlator.localized(locale, it))
}
if (chapter.pageCount > 0) {
append(MR.strings.opds_chapter_details_progress.localized(locale, chapter.lastPageRead, chapter.pageCount))
append(MR.strings.opds_chapter_details_progress.localized(locale, effectiveLastPageRead, chapter.pageCount))
}
}
val links = mutableListOf<OpdsLinkXml>()
if (skipMetadataFeed) {
// Provide Acquisition Link (Download CBZ) if downloaded
if (chapter.downloaded) {
links.add(
OpdsLinkXml(
OpdsConstants.LINK_REL_ACQUISITION_OPEN_ACCESS,
"/api/v1/chapter/${chapter.id}/download?markAsRead=${serverConfig.opdsMarkAsReadOnDownload.value}",
serverConfig.opdsCbzMimetype.value.mediaType,
MR.strings.opds_linktitle_download_cbz.localized(locale),
length = chapter.cbzFileSize,
),
)
}
// Provide Stream Link (OPDS-PSE) if page count is known
if (chapter.pageCount > 0) {
val basePageHref =
"/api/v1/manga/${manga.id}/chapter/${chapter.sourceOrder}/page/{pageNumber}" +
"?updateProgress=${serverConfig.opdsEnablePageReadProgress.value}&opds=true"
val titleRes =
if (effectiveLastPageRead > 0) {
MR.strings.opds_linktitle_stream_pages_continue
} else {
MR.strings.opds_linktitle_stream_pages_start
}
links.add(
OpdsLinkXml(
rel = OpdsConstants.LINK_REL_PSE_STREAM,
href = basePageHref,
type = OpdsConstants.TYPE_IMAGE_JPEG,
title = titleRes.localized(locale),
pseCount = chapter.pageCount,
pseLastRead = effectiveLastPageRead.takeIf { it > 0 },
pseLastReadDate = effectiveLastReadAt.takeIf { it > 0 }?.let { OpdsDateUtil.formatEpochMillisForOpds(it * 1000) },
),
)
// Page 0 Cover
links.add(
OpdsLinkXml(
rel = OpdsConstants.LINK_REL_IMAGE,
href = "/api/v1/manga/${manga.id}/chapter/${chapter.sourceOrder}/page/0",
type = OpdsConstants.TYPE_IMAGE_JPEG,
title = MR.strings.opds_linktitle_chapter_cover.localized(locale),
),
)
}
} else {
// Link to Metadata Feed
links.add(
OpdsLinkXml(
rel = OpdsConstants.LINK_REL_SUBSECTION,
href = "$baseUrl/series/${manga.id}/chapter/${chapter.sourceOrder}/metadata?lang=${locale.toLanguageTag()}",
type = OpdsConstants.TYPE_ATOM_XML_ENTRY_PROFILE_OPDS,
title = MR.strings.opds_linktitle_view_chapter_details.localized(locale),
),
)
}
return OpdsEntryXml(
id = "urn:suwayomi:chapter:${chapter.id}",
title = entryTitle,
@@ -196,33 +304,25 @@ object OpdsEntryBuilder {
chapter.scanlator?.takeIf { it.isNotBlank() }?.let { OpdsAuthorXml(name = it) },
),
summary = OpdsSummaryXml(value = details),
link =
listOf(
OpdsLinkXml(
rel = OpdsConstants.LINK_REL_SUBSECTION,
href = "$baseUrl/series/${manga.id}/chapter/${chapter.sourceOrder}/metadata?lang=${locale.toLanguageTag()}",
type = OpdsConstants.TYPE_ATOM_XML_ENTRY_PROFILE_OPDS,
title = MR.strings.opds_linktitle_view_chapter_details.localized(locale),
),
),
link = links,
)
}
/**
* Creates one or two OPDS entries for a chapter, handling synchronization conflicts internally.
*
* @param chapter The chapter metadata object.
* @param manga The parent manga's details.
* @param baseUrl The base URL for constructing links.
* @param locale The locale for localization.
* @param chapter The chapter metadata object.
* @param manga The parent manga's details.
* @return A `Pair` where the first element is the primary entry (always present) and the
* second is an optional entry representing the remote progress in case of a conflict.
*/
suspend fun createChapterMetadataEntries(
chapter: OpdsChapterMetadataAcqEntry,
manga: OpdsMangaDetails,
baseUrl: String,
locale: Locale,
chapter: OpdsChapterMetadataAcqEntry,
manga: OpdsMangaDetails,
): Pair<OpdsEntryXml, OpdsEntryXml?> {
// Check remote progress before building the entry
val syncResult = KoreaderSyncService.checkAndPullProgress(chapter.id)
@@ -234,20 +334,20 @@ object OpdsEntryBuilder {
// Generate two entries: one for local progress and another for remote.
val localEntry =
buildSingleChapterMetadataEntry(
chapter,
manga,
baseUrl,
locale,
chapter,
manga,
progressSource = ProgressSource.Local(chapter.lastPageRead, chapter.lastReadAt),
isConflict = true,
)
val remoteEntry =
buildSingleChapterMetadataEntry(
chapter,
manga,
baseUrl,
locale,
chapter,
manga,
progressSource = ProgressSource.Remote(syncResult!!.pageRead, syncResult.timestamp, syncResult.device),
isConflict = true,
)
@@ -263,10 +363,10 @@ object OpdsEntryBuilder {
val mainEntry =
buildSingleChapterMetadataEntry(
chapter,
manga,
baseUrl,
locale,
chapter,
manga,
progressSource = progressSource,
isConflict = false,
)
@@ -297,10 +397,10 @@ object OpdsEntryBuilder {
* Helper function to build a single OpdsEntryXml for a chapter.
*/
private suspend fun buildSingleChapterMetadataEntry(
chapter: OpdsChapterMetadataAcqEntry,
manga: OpdsMangaDetails,
baseUrl: String,
locale: Locale,
chapter: OpdsChapterMetadataAcqEntry,
manga: OpdsMangaDetails,
progressSource: ProgressSource,
isConflict: Boolean,
): OpdsEntryXml {
@@ -337,14 +437,7 @@ object OpdsEntryBuilder {
}
val entryTitle = "$titlePrefix ${chapter.name}"
val cbzFileSize =
if (chapter.downloaded) {
withContext(Dispatchers.IO) {
runCatching { ChapterDownloadHelper.getArchiveStreamWithSize(manga.id, chapter.id).second }.getOrNull()
}
} else {
null
}
val cbzFileSize = chapter.cbzFileSize
val links = mutableListOf<OpdsLinkXml>()
chapter.url?.let {
@@ -449,8 +542,8 @@ object OpdsEntryBuilder {
fun addSourceSortFacets(
feedBuilder: FeedBuilderInternal,
baseUrl: String,
currentSort: String,
locale: Locale,
currentSort: String,
) {
val sortGroup = MR.strings.opds_facetgroup_sort_order.localized(locale)
val addFacet = { href: String, titleKey: StringResource, isActive: Boolean ->
@@ -475,9 +568,9 @@ object OpdsEntryBuilder {
fun addChapterSortAndFilterFacets(
feedBuilder: FeedBuilderInternal,
baseUrl: String,
locale: Locale,
currentSort: String,
currentFilter: String,
locale: Locale,
filterCounts: Map<String, Long>? = null,
) {
val sortGroup = MR.strings.opds_facetgroup_sort_order.localized(locale)
@@ -554,15 +647,15 @@ object OpdsEntryBuilder {
fun addLibraryFacets(
feedBuilder: FeedBuilderInternal,
baseUrl: String,
activeFilters: OpdsMangaFilter,
locale: Locale,
activeFilters: OpdsMangaFilter,
) {
val currentSort = activeFilters.sort ?: "alpha_asc"
val currentFilter = activeFilters.filter ?: "all"
val sortGroup = MR.strings.opds_facetgroup_sort_order.localized(locale)
val filterGroup = MR.strings.opds_facetgroup_filter_content.localized(locale)
val filterCounts = MangaRepository.getLibraryFilterCounts()
val filterCounts = MangaRepository.getLibraryFilterCounts(activeFilters)
val buildUrl = { newFilters: OpdsMangaFilter, newSort: String, newFilter: String ->
val crossFilterParams = newFilters.toCrossFilterQueryParameters()
@@ -667,7 +760,7 @@ object OpdsEntryBuilder {
// --- Cross-Filter Facets ---
if (activeFilters.primaryFilter != PrimaryFilterType.SOURCE) {
val sources = NavigationRepository.getLibrarySources(1).first
val sources = NavigationRepository.getLibrarySources(pageNum = null, activeFilters = activeFilters).first
addFacet(
feedBuilder,
buildUrl(activeFilters.without("source_id"), currentSort, currentFilter),
@@ -688,7 +781,7 @@ object OpdsEntryBuilder {
}
}
if (activeFilters.primaryFilter != PrimaryFilterType.CATEGORY) {
val categories = NavigationRepository.getCategories(1).first
val categories = NavigationRepository.getCategories(pageNum = null, activeFilters = activeFilters).first
addFacet(
feedBuilder,
buildUrl(activeFilters.without("category_id"), currentSort, currentFilter),
@@ -709,7 +802,7 @@ object OpdsEntryBuilder {
}
}
if (activeFilters.primaryFilter != PrimaryFilterType.STATUS) {
val statuses = NavigationRepository.getStatuses(locale)
val statuses = NavigationRepository.getStatuses(locale, pageNum = null, activeFilters = activeFilters).first
addFacet(
feedBuilder,
buildUrl(activeFilters.without("status_id"), currentSort, currentFilter),
@@ -730,7 +823,7 @@ object OpdsEntryBuilder {
}
}
if (activeFilters.primaryFilter != PrimaryFilterType.LANGUAGE) {
val languages = NavigationRepository.getContentLanguages(locale)
val languages = NavigationRepository.getContentLanguages(locale, pageNum = null, activeFilters = activeFilters).first
addFacet(
feedBuilder,
buildUrl(activeFilters.without("lang_code"), currentSort, currentFilter),
@@ -751,7 +844,7 @@ object OpdsEntryBuilder {
}
}
if (activeFilters.primaryFilter != PrimaryFilterType.GENRE) {
val genres = NavigationRepository.getGenres(1, locale).first
val genres = NavigationRepository.getGenres(locale, pageNum = null, activeFilters = activeFilters).first
addFacet(
feedBuilder,
buildUrl(activeFilters.without("genre"), currentSort, currentFilter),

View File

@@ -17,6 +17,7 @@ import suwayomi.tachidesk.opds.repository.ChapterRepository
import suwayomi.tachidesk.opds.repository.MangaRepository
import suwayomi.tachidesk.opds.repository.NavigationRepository
import suwayomi.tachidesk.opds.util.OpdsDateUtil
import suwayomi.tachidesk.opds.util.OpdsStringUtil
import suwayomi.tachidesk.opds.util.OpdsXmlUtil
import suwayomi.tachidesk.server.serverConfig
import java.util.Locale
@@ -40,12 +41,11 @@ object OpdsFeedBuilder {
val navItems = NavigationRepository.getRootNavigationItems(locale)
val builder =
FeedBuilderInternal(
baseUrl,
"", // Root path is empty
MR.strings.opds_feeds_root.localized(locale),
locale,
OpdsConstants.TYPE_ATOM_XML_FEED_NAVIGATION,
null,
baseUrl = baseUrl,
locale = locale,
idPath = "", // Root path is empty
title = MR.strings.opds_feeds_root.localized(locale),
feedType = OpdsConstants.TYPE_ATOM_XML_FEED_NAVIGATION,
)
builder.totalResults = navItems.size.toLong()
builder.entries.addAll(
@@ -73,30 +73,39 @@ object OpdsFeedBuilder {
/**
* Generates the history feed showing recently read chapters.
* @param baseUrl The base URL for constructing links.
* @param pageNum The page number for pagination.
* @param locale The locale for localization.
* @param pageNum The page number for pagination.
* @return An XML string representing the history feed.
*/
suspend fun getHistoryFeed(
baseUrl: String,
pageNum: Int,
locale: Locale,
pageNum: Int,
): String {
val (historyItems, total) = ChapterRepository.getHistory(pageNum)
val builder =
FeedBuilderInternal(
baseUrl,
"history",
MR.strings.opds_feeds_history_title.localized(locale),
locale,
OpdsConstants.TYPE_ATOM_XML_FEED_ACQUISITION,
pageNum,
baseUrl = baseUrl,
locale = locale,
idPath = "history",
title = MR.strings.opds_feeds_history_title.localized(locale),
feedType = OpdsConstants.TYPE_ATOM_XML_FEED_ACQUISITION,
pageNum = pageNum,
)
builder.totalResults = total
val skipMetadata = serverConfig.opdsSkipChapterMetadataFeed.value
builder.entries.addAll(
historyItems.map { item ->
val mangaDetails = OpdsMangaDetails(item.mangaId, item.mangaTitle, item.mangaThumbnailUrl, item.mangaAuthor)
OpdsEntryBuilder.createChapterListEntry(item.chapter, mangaDetails, baseUrl, true, locale)
val mangaDetails =
OpdsMangaDetails(item.mangaId, item.mangaTitle, item.mangaThumbnailUrl, item.mangaAuthor, item.mangaTotalChapters)
OpdsEntryBuilder.createChapterListEntry(
baseUrl,
locale,
item.chapter,
mangaDetails,
true,
skipMetadata,
)
},
)
return OpdsXmlUtil.serializeFeedToString(builder.build())
@@ -104,54 +113,55 @@ object OpdsFeedBuilder {
/**
* Generates a feed for search results based on the provided criteria.
* @param criteria The search criteria.
* @param baseUrl The base URL for constructing links.
* @param pageNum The page number for pagination.
* @param locale The locale for localization.
* @param criteria The search criteria.
* @param pageNum The page number for pagination.
* @return An XML string representing the search results feed.
*/
fun getSearchFeed(
criteria: OpdsSearchCriteria,
baseUrl: String,
pageNum: Int,
locale: Locale,
criteria: OpdsSearchCriteria,
pageNum: Int,
): String {
val (mangaEntries, total) = MangaRepository.findMangaByCriteria(criteria)
val builder =
FeedBuilderInternal(
baseUrl,
"library/series",
MR.strings.opds_feeds_search_results_title.localized(locale),
locale,
OpdsConstants.TYPE_ATOM_XML_FEED_ACQUISITION,
pageNum,
baseUrl = baseUrl,
locale = locale,
idPath = "library/series",
title = MR.strings.opds_feeds_search_results_title.localized(locale),
feedType = OpdsConstants.TYPE_ATOM_XML_FEED_ACQUISITION,
pageNum = pageNum,
isSearchFeed = true,
)
builder.totalResults = total
builder.entries.addAll(mangaEntries.map { OpdsEntryBuilder.mangaAcqEntryToEntry(it, baseUrl, locale) })
builder.entries.addAll(mangaEntries.map { OpdsEntryBuilder.mangaAcqEntryToEntry(baseUrl, locale, it) })
return OpdsXmlUtil.serializeFeedToString(builder.build())
}
/**
* Generates a generic library feed based on various filtering and sorting criteria.
* @param criteria The filtering criteria.
* @param baseUrl The base URL for constructing links.
* @param locale The locale for localization.
* @param criteria The filtering criteria.
* @param isSearch Indicates if it's a search feed.
* @param pageNum The page number for pagination.
* @param sort The sorting parameter.
* @param filter The filtering parameter.
* @param locale The locale for localization.
* @return An XML string representing the library feed.
*/
fun getLibraryFeed(
criteria: OpdsMangaFilter,
baseUrl: String,
locale: Locale,
criteria: OpdsMangaFilter,
isSearch: Boolean,
pageNum: Int,
sort: String?,
filter: String?,
locale: Locale,
isSearch: Boolean,
): String {
val result = MangaRepository.getLibraryManga(pageNum, sort, filter, criteria)
val result = MangaRepository.getLibraryManga(criteria, pageNum, sort, filter)
val feedTitle =
when (criteria.primaryFilter) {
@@ -177,7 +187,12 @@ object OpdsFeedBuilder {
}
PrimaryFilterType.STATUS -> {
val statusName = NavigationRepository.getStatuses(locale).find { it.id == criteria.statusId }?.title
val statusName =
NavigationRepository
.getStatuses(locale, pageNum = null, activeFilters = criteria)
.first
.find { it.id == criteria.statusId }
?.title
MR.strings.opds_feeds_status_specific_title.localized(locale, statusName ?: criteria.statusId.toString())
}
@@ -193,7 +208,7 @@ object OpdsFeedBuilder {
val feedUrl =
when (criteria.primaryFilter) {
PrimaryFilterType.SOURCE -> "library/source/${criteria.sourceId}"
PrimaryFilterType.SOURCE -> "source/${criteria.sourceId}"
PrimaryFilterType.CATEGORY -> "category/${criteria.categoryId}"
PrimaryFilterType.GENRE -> "genre/${criteria.genre}"
PrimaryFilterType.STATUS -> "status/${criteria.statusId}"
@@ -203,23 +218,23 @@ object OpdsFeedBuilder {
val builder =
FeedBuilderInternal(
baseUrl,
feedUrl,
feedTitle,
locale,
OpdsConstants.TYPE_ATOM_XML_FEED_ACQUISITION,
pageNum,
baseUrl = baseUrl,
locale = locale,
idPath = feedUrl,
title = feedTitle,
feedType = OpdsConstants.TYPE_ATOM_XML_FEED_ACQUISITION,
pageNum = pageNum,
explicitQueryParams = criteria.toCrossFilterQueryParameters(),
currentSort = criteria.sort,
currentFilter = criteria.filter,
explicitQueryParams = criteria.toCrossFilterQueryParameters(),
isSearchFeed = isSearch,
)
builder.totalResults = result.totalCount
// Add all library facets (sort, filter, and cross-filtering)
OpdsEntryBuilder.addLibraryFacets(builder, baseUrl, criteria, locale)
OpdsEntryBuilder.addLibraryFacets(builder, baseUrl, locale, criteria)
builder.entries.addAll(result.mangaEntries.map { OpdsEntryBuilder.mangaAcqEntryToEntry(it, baseUrl, locale) })
builder.entries.addAll(result.mangaEntries.map { OpdsEntryBuilder.mangaAcqEntryToEntry(baseUrl, locale, it) })
return OpdsXmlUtil.serializeFeedToString(builder.build())
}
@@ -227,24 +242,24 @@ object OpdsFeedBuilder {
/**
* Generates a navigation feed listing all available sources for exploration.
* @param baseUrl The base URL for constructing links.
* @param pageNum The page number for pagination.
* @param locale The locale for localization.
* @param pageNum The page number for pagination.
* @return An XML string representing the explore sources feed.
*/
fun getExploreSourcesFeed(
baseUrl: String,
pageNum: Int,
locale: Locale,
pageNum: Int,
): String {
val (sourceNavEntries, total) = NavigationRepository.getExploreSources(pageNum)
val builder =
FeedBuilderInternal(
baseUrl,
"sources",
MR.strings.opds_feeds_sources_title.localized(locale),
locale,
OpdsConstants.TYPE_ATOM_XML_FEED_NAVIGATION,
pageNum,
baseUrl = baseUrl,
locale = locale,
idPath = "sources",
title = MR.strings.opds_feeds_sources_title.localized(locale),
feedType = OpdsConstants.TYPE_ATOM_XML_FEED_NAVIGATION,
pageNum = pageNum,
)
builder.totalResults = total
builder.entries.addAll(
@@ -257,7 +272,7 @@ object OpdsFeedBuilder {
listOf(
OpdsLinkXml(
OpdsConstants.LINK_REL_SUBSECTION,
"$baseUrl/source/${entry.id}?sort=popular&lang=${locale.toLanguageTag()}",
"$baseUrl/explore/source/${entry.id}?sort=popular&lang=${locale.toLanguageTag()}",
OpdsConstants.TYPE_ATOM_XML_FEED_ACQUISITION,
),
),
@@ -270,24 +285,24 @@ object OpdsFeedBuilder {
/**
* Generates a navigation feed listing sources for series present in the library.
* @param baseUrl The base URL for constructing links.
* @param pageNum The page number for pagination.
* @param locale The locale for localization.
* @param pageNum The page number for pagination.
* @return An XML string representing the library sources feed.
*/
fun getLibrarySourcesFeed(
baseUrl: String,
pageNum: Int,
locale: Locale,
pageNum: Int,
): String {
val (sourceNavEntries, total) = NavigationRepository.getLibrarySources(pageNum)
val builder =
FeedBuilderInternal(
baseUrl,
"library/sources",
MR.strings.opds_feeds_library_sources_title.localized(locale),
locale,
OpdsConstants.TYPE_ATOM_XML_FEED_NAVIGATION,
pageNum,
baseUrl = baseUrl,
locale = locale,
idPath = "library/sources",
title = MR.strings.opds_feeds_library_sources_title.localized(locale),
feedType = OpdsConstants.TYPE_ATOM_XML_FEED_NAVIGATION,
pageNum = pageNum,
)
builder.totalResults = total
builder.entries.addAll(
@@ -300,7 +315,7 @@ object OpdsFeedBuilder {
listOf(
OpdsLinkXml(
OpdsConstants.LINK_REL_SUBSECTION,
"$baseUrl/library/source/${entry.id}?lang=${locale.toLanguageTag()}",
"$baseUrl/source/${entry.id}?lang=${locale.toLanguageTag()}",
OpdsConstants.TYPE_ATOM_XML_FEED_ACQUISITION,
entry.name,
thrCount = entry.mangaCount?.toInt(),
@@ -314,78 +329,75 @@ object OpdsFeedBuilder {
/**
* Generates an acquisition feed for manga from a specific source (explore context).
* @param sourceId The ID of the source.
* @param baseUrl The base URL for constructing links.
* @param locale The locale for localization.
* @param sourceId The ID of the source.
* @param pageNum The page number for pagination.
* @param sort The sorting parameter ('popular' or 'latest').
* @param locale The locale for localization.
* @return An XML string representing the source-specific feed.
*/
suspend fun getExploreSourceFeed(
sourceId: Long,
baseUrl: String,
locale: Locale,
sourceId: Long,
pageNum: Int,
sort: String,
locale: Locale,
): String {
val (mangaEntries, hasNextPage) = MangaRepository.getMangaBySource(sourceId, pageNum, sort)
val sourceNavEntry = NavigationRepository.getExploreSources(1).first.find { it.id == sourceId }
val sourceNameOrId = sourceNavEntry?.name ?: sourceId.toString()
val sourceInfo = NavigationRepository.getSourceDetails(sourceId)
val sourceName = sourceInfo?.first ?: sourceId.toString()
val titleRes =
if (sort == "latest") {
if (sort ==
"latest"
) {
MR.strings.opds_feeds_source_specific_latest_title
} else {
MR.strings.opds_feeds_source_specific_popular_title
}
val feedTitle = titleRes.localized(locale, sourceNameOrId)
val feedUrl = "source/$sourceId"
val feedTitle = titleRes.localized(locale, sourceName)
val builder =
FeedBuilderInternal(
baseUrl,
feedUrl,
feedTitle,
locale,
OpdsConstants.TYPE_ATOM_XML_FEED_ACQUISITION,
pageNum,
baseUrl = baseUrl,
locale = locale,
idPath = "explore/source/$sourceId",
title = feedTitle,
feedType = OpdsConstants.TYPE_ATOM_XML_FEED_ACQUISITION,
pageNum = pageNum,
currentSort = sort,
)
builder.totalResults =
if (hasNextPage) {
(pageNum * serverConfig.opdsItemsPerPage.value + 1).toLong()
} else {
(
(pageNum - 1) *
serverConfig.opdsItemsPerPage.value +
mangaEntries.size
).toLong()
((pageNum - 1) * serverConfig.opdsItemsPerPage.value + mangaEntries.size).toLong()
}
builder.icon = sourceNavEntry?.iconUrl
OpdsEntryBuilder.addSourceSortFacets(builder, "$baseUrl/$feedUrl", sort, locale)
builder.entries.addAll(mangaEntries.map { OpdsEntryBuilder.mangaAcqEntryToEntry(it, baseUrl, locale) })
builder.icon = sourceInfo?.second
OpdsEntryBuilder.addSourceSortFacets(builder, "$baseUrl/explore/source/$sourceId", locale, sort)
builder.entries.addAll(mangaEntries.map { OpdsEntryBuilder.mangaAcqEntryToEntry(baseUrl, locale, it) })
return OpdsXmlUtil.serializeFeedToString(builder.build())
}
/**
* Generates a navigation feed for library categories.
* @param baseUrl The base URL for constructing links.
* @param pageNum The page number for pagination.
* @param locale The locale for localization.
* @param pageNum The page number for pagination.
* @return An XML string representing the categories navigation feed.
*/
fun getCategoriesFeed(
baseUrl: String,
pageNum: Int,
locale: Locale,
pageNum: Int,
): String {
val (categoryNavEntries, total) = NavigationRepository.getCategories(pageNum)
val builder =
FeedBuilderInternal(
baseUrl,
"library/categories",
MR.strings.opds_feeds_categories_title.localized(locale),
locale,
OpdsConstants.TYPE_ATOM_XML_FEED_NAVIGATION,
pageNum,
baseUrl = baseUrl,
locale = locale,
idPath = "library/categories",
title = MR.strings.opds_feeds_categories_title.localized(locale),
feedType = OpdsConstants.TYPE_ATOM_XML_FEED_NAVIGATION,
pageNum = pageNum,
)
builder.totalResults = total
builder.entries.addAll(
@@ -413,24 +425,24 @@ object OpdsFeedBuilder {
/**
* Generates a navigation feed for library genres.
* @param baseUrl The base URL for constructing links.
* @param pageNum The page number for pagination.
* @param locale The locale for localization.
* @param pageNum The page number for pagination.
* @return An XML string representing the genres navigation feed.
*/
fun getGenresFeed(
baseUrl: String,
pageNum: Int,
locale: Locale,
pageNum: Int,
): String {
val (genreNavEntries, total) = NavigationRepository.getGenres(pageNum, locale)
val (genreNavEntries, total) = NavigationRepository.getGenres(locale, pageNum)
val builder =
FeedBuilderInternal(
baseUrl,
"library/genres",
MR.strings.opds_feeds_genres_title.localized(locale),
locale,
OpdsConstants.TYPE_ATOM_XML_FEED_NAVIGATION,
pageNum,
baseUrl = baseUrl,
locale = locale,
idPath = "library/genres",
title = MR.strings.opds_feeds_genres_title.localized(locale),
feedType = OpdsConstants.TYPE_ATOM_XML_FEED_NAVIGATION,
pageNum = pageNum,
)
builder.totalResults = total
builder.entries.addAll(
@@ -458,26 +470,26 @@ object OpdsFeedBuilder {
/**
* Generates a navigation feed for manga publication statuses.
* @param baseUrl The base URL for constructing links.
* @param pageNum The page number (currently unused).
* @param locale The locale for localization.
* @param pageNum The page number (currently unused).
* @return An XML string representing the status navigation feed.
*/
fun getStatusFeed(
baseUrl: String,
@Suppress("UNUSED_PARAMETER") pageNum: Int,
locale: Locale,
pageNum: Int,
): String {
val statuses = NavigationRepository.getStatuses(locale)
val (statuses, total) = NavigationRepository.getStatuses(locale, pageNum)
val builder =
FeedBuilderInternal(
baseUrl,
"library/statuses",
MR.strings.opds_feeds_status_title.localized(locale),
locale,
OpdsConstants.TYPE_ATOM_XML_FEED_NAVIGATION,
null,
baseUrl = baseUrl,
locale = locale,
idPath = "library/statuses",
title = MR.strings.opds_feeds_status_title.localized(locale),
feedType = OpdsConstants.TYPE_ATOM_XML_FEED_NAVIGATION,
pageNum = pageNum,
)
builder.totalResults = statuses.size.toLong()
builder.totalResults = total
builder.entries.addAll(
statuses.map { entry ->
OpdsEntryXml(
@@ -503,24 +515,26 @@ object OpdsFeedBuilder {
/**
* Generates a navigation feed for content languages available in the library.
* @param baseUrl The base URL for constructing links.
* @param uiLocale The locale for the user interface.
* @param locale The locale for the user interface.
* @param pageNum The page number for pagination.
* @return An XML string representing the languages navigation feed.
*/
fun getLanguagesFeed(
baseUrl: String,
uiLocale: Locale,
locale: Locale,
pageNum: Int,
): String {
val languages = NavigationRepository.getContentLanguages(uiLocale)
val (languages, total) = NavigationRepository.getContentLanguages(locale, pageNum)
val builder =
FeedBuilderInternal(
baseUrl,
"library/languages",
MR.strings.opds_feeds_languages_title.localized(uiLocale),
uiLocale,
OpdsConstants.TYPE_ATOM_XML_FEED_NAVIGATION,
null,
baseUrl = baseUrl,
locale = locale,
idPath = "library/languages",
title = MR.strings.opds_feeds_languages_title.localized(locale),
feedType = OpdsConstants.TYPE_ATOM_XML_FEED_NAVIGATION,
pageNum = pageNum,
)
builder.totalResults = languages.size.toLong()
builder.totalResults = total
builder.entries.addAll(
languages.map { entry ->
OpdsEntryXml(
@@ -531,7 +545,7 @@ object OpdsFeedBuilder {
listOf(
OpdsLinkXml(
OpdsConstants.LINK_REL_SUBSECTION,
"$baseUrl/language/${entry.id}?lang=${uiLocale.toLanguageTag()}",
"$baseUrl/language/${entry.id}?lang=${locale.toLanguageTag()}",
OpdsConstants.TYPE_ATOM_XML_FEED_ACQUISITION,
entry.title,
thrCount = entry.mangaCount.toInt(),
@@ -544,32 +558,41 @@ object OpdsFeedBuilder {
}
/**
* Generates an acquisition feed for recent chapter updates in the library.
* Generates an acquisition feed of recent chapter updates for series in the library.
* @param baseUrl The base URL for constructing links.
* @param pageNum The page number for pagination.
* @param locale The locale for localization.
* @param pageNum The page number for pagination.
* @return An XML string representing the library updates feed.
*/
suspend fun getLibraryUpdatesFeed(
baseUrl: String,
pageNum: Int,
locale: Locale,
pageNum: Int,
): String {
val (updateItems, total) = ChapterRepository.getLibraryUpdates(pageNum)
val builder =
FeedBuilderInternal(
baseUrl,
"library-updates",
MR.strings.opds_feeds_library_updates_title.localized(locale),
locale,
OpdsConstants.TYPE_ATOM_XML_FEED_ACQUISITION,
pageNum,
baseUrl = baseUrl,
locale = locale,
idPath = "library-updates",
title = MR.strings.opds_feeds_library_updates_title.localized(locale),
feedType = OpdsConstants.TYPE_ATOM_XML_FEED_ACQUISITION,
pageNum = pageNum,
)
builder.totalResults = total
val skipMetadata = serverConfig.opdsSkipChapterMetadataFeed.value
builder.entries.addAll(
updateItems.map { item ->
val mangaDetails = OpdsMangaDetails(item.mangaId, item.mangaTitle, item.mangaThumbnailUrl, item.mangaAuthor)
OpdsEntryBuilder.createChapterListEntry(item.chapter, mangaDetails, baseUrl, true, locale)
val mangaDetails =
OpdsMangaDetails(item.mangaId, item.mangaTitle, item.mangaThumbnailUrl, item.mangaAuthor, item.mangaTotalChapters)
OpdsEntryBuilder.createChapterListEntry(
baseUrl,
locale,
item.chapter,
mangaDetails,
true,
skipMetadata,
)
},
)
return OpdsXmlUtil.serializeFeedToString(builder.build())
@@ -577,29 +600,29 @@ object OpdsFeedBuilder {
/**
* Generates an acquisition feed for all chapters of a specific manga.
* @param mangaId The ID of the manga.
* @param baseUrl The base URL for constructing links.
* @param locale The locale for localization.
* @param mangaId The ID of the manga.
* @param pageNum The page number for pagination.
* @param sortParam The sorting parameter for chapters.
* @param filterParam The filtering parameter for chapters.
* @param locale The locale for localization.
* @return An XML string representing the series' chapters feed.
*/
suspend fun getSeriesChaptersFeed(
mangaId: Int,
baseUrl: String,
locale: Locale,
mangaId: Int,
pageNum: Int,
sortParam: String?,
filterParam: String?,
locale: Locale,
): String {
val mangaDetails =
MangaRepository.getMangaDetails(mangaId)
?: return buildNotFoundFeed(
baseUrl,
locale,
"series/$mangaId/chapters",
MR.strings.opds_error_manga_not_found.localized(locale, mangaId),
locale,
)
val (sortColumn, currentSortOrder) =
when (sortParam?.lowercase()) {
@@ -610,15 +633,27 @@ object OpdsFeedBuilder {
else -> ChapterTable.sourceOrder to (serverConfig.opdsChapterSortOrder.value)
}
val currentFilter = filterParam?.lowercase() ?: if (serverConfig.opdsShowOnlyUnreadChapters.value) "unread" else "all"
val skipMetadata = serverConfig.opdsSkipChapterMetadataFeed.value
var (chapterEntries, totalChapters) =
ChapterRepository.getChaptersForManga(
mangaId,
pageNum,
sortColumn,
currentSortOrder,
currentFilter,
pageNum,
skipMetadata,
)
// Return a not-found feed if all available chapters are filtered out as unreachable
if (skipMetadata && chapterEntries.isEmpty() && totalChapters > 0L) {
return buildNotFoundFeed(
baseUrl = baseUrl,
locale = locale,
idPath = "series/$mangaId/chapters",
title = MR.strings.opds_error_chapters_not_found.localized(locale, pageNum),
)
}
// If no chapters are found in the database, attempt to fetch them from the source.
if (chapterEntries.isEmpty() && totalChapters == 0L) {
try {
@@ -629,10 +664,11 @@ object OpdsFeedBuilder {
val (refetchedChapters, refetchedTotal) =
ChapterRepository.getChaptersForManga(
mangaId,
pageNum,
sortColumn,
currentSortOrder,
currentFilter,
pageNum,
skipMetadata,
)
chapterEntries = refetchedChapters
totalChapters = refetchedTotal
@@ -651,12 +687,12 @@ object OpdsFeedBuilder {
val feedUrl = "series/$mangaId/chapters"
val builder =
FeedBuilderInternal(
baseUrl,
feedUrl,
MR.strings.opds_feeds_manga_chapters.localized(locale, mangaDetails.title),
locale,
OpdsConstants.TYPE_ATOM_XML_FEED_ACQUISITION,
pageNum,
baseUrl = baseUrl,
locale = locale,
idPath = feedUrl,
title = MR.strings.opds_feeds_manga_chapters.localized(locale, mangaDetails.title),
feedType = OpdsConstants.TYPE_ATOM_XML_FEED_ACQUISITION,
pageNum = pageNum,
currentSort = actualSortParamForLinks,
currentFilter = currentFilter,
)
@@ -669,14 +705,21 @@ object OpdsFeedBuilder {
OpdsEntryBuilder.addChapterSortAndFilterFacets(
builder,
"$baseUrl/$feedUrl",
locale,
actualSortParamForLinks,
currentFilter,
locale,
filterCounts,
)
builder.entries.addAll(
chapterEntries.map { chapter ->
OpdsEntryBuilder.createChapterListEntry(chapter, mangaDetails, baseUrl, false, locale)
OpdsEntryBuilder.createChapterListEntry(
baseUrl,
locale,
chapter,
mangaDetails,
false,
skipMetadata,
)
},
)
return OpdsXmlUtil.serializeFeedToString(builder.build())
@@ -684,43 +727,43 @@ object OpdsFeedBuilder {
/**
* Generates an acquisition feed with detailed metadata for a single chapter.
* @param mangaId The ID of the manga.
* @param chapterSourceOrder The source order index of the chapter.
* @param baseUrl The base URL for constructing links.
* @param locale The locale for localization.
* @param mangaId The ID of the manga.
* @param chapterSourceOrder The source order index of the chapter.
* @return An XML string representing the chapter's metadata feed.
*/
suspend fun getChapterMetadataFeed(
mangaId: Int,
chapterSourceOrder: Int,
baseUrl: String,
locale: Locale,
mangaId: Int,
chapterSourceOrder: Int,
): String {
val mangaDetails =
MangaRepository.getMangaDetails(mangaId)
?: return buildNotFoundFeed(
baseUrl,
locale,
"series/$mangaId/chapter/$chapterSourceOrder/metadata",
MR.strings.opds_error_manga_not_found.localized(locale, mangaId),
locale,
)
val chapterMetadata =
ChapterRepository.getChapterDetailsForMetadataFeed(mangaId, chapterSourceOrder)
?: return buildNotFoundFeed(
baseUrl,
locale,
"series/$mangaId/chapter/$chapterSourceOrder/metadata",
MR.strings.opds_error_chapter_not_found.localized(locale, chapterSourceOrder),
locale,
)
val builder =
FeedBuilderInternal(
baseUrl,
"series/$mangaId/chapter/${chapterMetadata.sourceOrder}/metadata",
MR.strings.opds_feeds_chapter_details.localized(locale, mangaDetails.title, chapterMetadata.name),
locale,
OpdsConstants.TYPE_ATOM_XML_FEED_ACQUISITION,
null,
baseUrl = baseUrl,
locale = locale,
idPath = "series/$mangaId/chapter/${chapterMetadata.sourceOrder}/metadata",
title = MR.strings.opds_feeds_chapter_details.localized(locale, mangaDetails.title, chapterMetadata.name),
feedType = OpdsConstants.TYPE_ATOM_XML_FEED_ACQUISITION,
pageNum = null,
)
mangaDetails.thumbnailUrl?.let { proxyThumbnailUrl(mangaDetails.id) }?.also {
@@ -731,10 +774,10 @@ object OpdsFeedBuilder {
val (primaryEntry, conflictEntry) =
OpdsEntryBuilder.createChapterMetadataEntries(
chapter = chapterMetadata,
manga = mangaDetails,
baseUrl = baseUrl,
locale = locale,
chapter = chapterMetadata,
manga = mangaDetails,
)
builder.entries.add(primaryEntry)
@@ -751,19 +794,25 @@ object OpdsFeedBuilder {
/**
* Builds a simple OPDS feed to indicate that a resource was not found.
* @param baseUrl The base URL.
* @param locale The locale for localization.
* @param idPath The path that was not found.
* @param title The title for the feed (e.g., an error message).
* @param locale The locale for localization.
* @return An XML string representing the 'not found' feed.
*/
fun buildNotFoundFeed(
baseUrl: String,
locale: Locale,
idPath: String,
title: String,
locale: Locale,
): String =
FeedBuilderInternal(baseUrl, idPath, title, locale, feedType = OpdsConstants.TYPE_ATOM_XML_FEED_ACQUISITION, pageNum = null)
.apply { totalResults = 0L }
FeedBuilderInternal(
baseUrl = baseUrl,
locale = locale,
idPath = idPath,
title = title,
feedType = OpdsConstants.TYPE_ATOM_XML_FEED_ACQUISITION,
pageNum = null,
).apply { totalResults = 0L }
.build()
.let(OpdsXmlUtil::serializeFeedToString)
}

View File

@@ -1,17 +1,29 @@
package suwayomi.tachidesk.opds.repository
import io.github.oshai.kotlinlogging.KotlinLogging
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.async
import kotlinx.coroutines.awaitAll
import kotlinx.coroutines.coroutineScope
import kotlinx.coroutines.withContext
import org.jetbrains.exposed.v1.core.Column
import org.jetbrains.exposed.v1.core.JoinType
import org.jetbrains.exposed.v1.core.Op
import org.jetbrains.exposed.v1.core.ResultRow
import org.jetbrains.exposed.v1.core.SortOrder
import org.jetbrains.exposed.v1.core.and
import org.jetbrains.exposed.v1.core.count
import org.jetbrains.exposed.v1.core.eq
import org.jetbrains.exposed.v1.core.greater
import org.jetbrains.exposed.v1.core.inList
import org.jetbrains.exposed.v1.jdbc.andWhere
import org.jetbrains.exposed.v1.jdbc.select
import org.jetbrains.exposed.v1.jdbc.transactions.transaction
import org.jetbrains.exposed.v1.jdbc.update
import suwayomi.tachidesk.manga.impl.ChapterDownloadHelper
import suwayomi.tachidesk.manga.impl.chapter.getChapterDownloadReady
import suwayomi.tachidesk.manga.impl.chapter.refreshChapterPageList
import suwayomi.tachidesk.manga.impl.chapter.updateChapterPersistence
import suwayomi.tachidesk.manga.model.table.ChapterTable
import suwayomi.tachidesk.manga.model.table.MangaTable
import suwayomi.tachidesk.manga.model.table.SourceTable
@@ -24,6 +36,7 @@ import suwayomi.tachidesk.server.serverConfig
object ChapterRepository {
private val opdsItemsPerPageBounded: Int
get() = serverConfig.opdsItemsPerPage.value
private val logger = KotlinLogging.logger {}
private fun ResultRow.toOpdsChapterListAcqEntry(): OpdsChapterListAcqEntry =
OpdsChapterListAcqEntry(
@@ -38,69 +51,165 @@ object ChapterRepository {
lastReadAt = this[ChapterTable.lastReadAt],
sourceOrder = this[ChapterTable.sourceOrder],
pageCount = this[ChapterTable.pageCount],
downloaded = this[ChapterTable.isDownloaded],
)
fun getChaptersForManga(
suspend fun getChaptersForManga(
mangaId: Int,
pageNum: Int,
sortColumn: Column<*>,
sortOrder: SortOrder,
filter: String,
): Pair<List<OpdsChapterListAcqEntry>, Long> =
transaction {
val conditions = mutableListOf<Op<Boolean>>()
conditions.add(ChapterTable.manga eq mangaId)
pageNum: Int,
skipMetadata: Boolean,
): Pair<List<OpdsChapterListAcqEntry>, Long> {
val (rawChapters, totalCount) =
transaction {
val conditions = mutableListOf<Op<Boolean>>()
conditions.add(ChapterTable.manga eq mangaId)
when (filter) {
"unread" -> conditions.add(ChapterTable.isRead eq false)
"read" -> conditions.add(ChapterTable.isRead eq true)
}
if (serverConfig.opdsShowOnlyDownloadedChapters.value) {
conditions.add(ChapterTable.isDownloaded eq true)
when (filter) {
"unread" -> conditions.add(ChapterTable.isRead eq false)
"read" -> conditions.add(ChapterTable.isRead eq true)
}
if (serverConfig.opdsShowOnlyDownloadedChapters.value) {
conditions.add(ChapterTable.isDownloaded eq true)
}
val finalCondition = conditions.reduceOrNull { acc, op -> acc and op } ?: Op.TRUE
val baseQuery =
ChapterTable
.select(ChapterTable.columns)
.where(finalCondition)
val totalCount = baseQuery.count()
val chapters =
baseQuery
.orderBy(sortColumn to sortOrder)
.limit(opdsItemsPerPageBounded)
.offset(((pageNum - 1) * opdsItemsPerPageBounded).toLong())
.map { it.toOpdsChapterListAcqEntry() }
Pair(chapters, totalCount)
}
val finalCondition = conditions.reduceOrNull { acc, op -> acc and op } ?: Op.TRUE
val baseQuery =
ChapterTable
.select(ChapterTable.columns)
.where(finalCondition)
val totalCount = baseQuery.count()
val chapters =
baseQuery
.orderBy(sortColumn to sortOrder)
.limit(opdsItemsPerPageBounded)
.offset(((pageNum - 1) * opdsItemsPerPageBounded).toLong())
.map { it.toOpdsChapterListAcqEntry() }
Pair(chapters, totalCount)
// If not skipping metadata, return basic DTOs
if (!skipMetadata) {
return Pair(rawChapters, totalCount)
}
// If skipping metadata, enrich DTOs with page count and file size
val enrichedChapters =
coroutineScope {
rawChapters.map { entry ->
async(Dispatchers.IO) {
var pageCount = entry.pageCount
var isDownloaded = entry.downloaded
// Verify physical files if page count is unknown or the DB marks it as downloaded
if (pageCount <= 0 || isDownloaded) {
val physicalPageCount =
runCatching {
ChapterDownloadHelper.getImageCount(entry.mangaId, entry.id)
}.getOrDefault(0)
if (physicalPageCount > 0) {
// Files exist! Sync DB if needed
if (updateChapterPersistence(
chapterId = entry.id,
isMarkedAsDownloaded = isDownloaded,
dbPageCount = pageCount,
downloadPageCount = physicalPageCount,
lastPageRead = entry.lastPageRead,
logger = logger,
)
) {
pageCount = physicalPageCount
isDownloaded = true
}
} else {
if (isDownloaded) {
// Fix DB state if marked as downloaded but physical files are missing
transaction {
ChapterTable.update({ ChapterTable.id eq entry.id }) {
it[ChapterTable.isDownloaded] = false
}
}
isDownloaded = false
}
if (pageCount <= 0) {
// No files, and DB has no page count. Fetch from network
pageCount =
runCatching {
refreshChapterPageList(entry.mangaId, entry.id)
}.onFailure {
logger.warn(it) { "Failed to fetch page count for chapter ${entry.id}" }
}.getOrDefault(0)
}
}
}
// Calculate CBZ size if downloaded
val cbzFileSize =
if (isDownloaded) {
runCatching {
ChapterDownloadHelper.getChapterArchiveSize(entry.mangaId, entry.id)
}.getOrNull()
} else {
null
}
entry.copy(
pageCount = pageCount,
downloaded = isDownloaded,
cbzFileSize = cbzFileSize,
)
}
}
}.awaitAll()
// Exclude unreachable chapters that are not downloaded and have no page count
.filter { it.downloaded || it.pageCount > 0 }
return Pair(enrichedChapters, totalCount)
}
suspend fun getChapterDetailsForMetadataFeed(
mangaId: Int,
chapterSourceOrder: Int,
): OpdsChapterMetadataAcqEntry? =
try {
val chapterDataClass = getChapterDownloadReady(chapterIndex = chapterSourceOrder, mangaId = mangaId)
OpdsChapterMetadataAcqEntry(
id = chapterDataClass.id,
mangaId = chapterDataClass.mangaId,
name = chapterDataClass.name,
uploadDate = chapterDataClass.uploadDate,
scanlator = chapterDataClass.scanlator,
read = chapterDataClass.read,
lastPageRead = chapterDataClass.lastPageRead,
lastReadAt = chapterDataClass.lastReadAt,
sourceOrder = chapterDataClass.index,
downloaded = chapterDataClass.downloaded,
pageCount = chapterDataClass.pageCount,
url = chapterDataClass.realUrl,
)
} catch (e: Exception) {
null
}
): OpdsChapterMetadataAcqEntry? {
val chapterDataClass =
try {
getChapterDownloadReady(chapterIndex = chapterSourceOrder, mangaId = mangaId)
} catch (e: Exception) {
return null
}
return OpdsChapterMetadataAcqEntry(
id = chapterDataClass.id,
mangaId = chapterDataClass.mangaId,
name = chapterDataClass.name,
uploadDate = chapterDataClass.uploadDate,
chapterNumber = chapterDataClass.chapterNumber,
scanlator = chapterDataClass.scanlator,
read = chapterDataClass.read,
lastPageRead = chapterDataClass.lastPageRead,
lastReadAt = chapterDataClass.lastReadAt,
sourceOrder = chapterDataClass.index,
downloaded = chapterDataClass.downloaded,
pageCount = chapterDataClass.pageCount,
url = chapterDataClass.realUrl,
cbzFileSize =
if (chapterDataClass.downloaded) {
withContext(Dispatchers.IO) {
runCatching { ChapterDownloadHelper.getChapterArchiveSize(mangaId, chapterDataClass.id) }.getOrNull()
}
} else {
null
},
)
}
fun getLibraryUpdates(pageNum: Int): Pair<List<OpdsLibraryUpdateAcqEntry>, Long> =
transaction {
@@ -115,21 +224,38 @@ object ChapterRepository {
val totalCount = query.count()
val items =
val rawItems =
query
.orderBy(ChapterTable.fetchedAt to SortOrder.DESC, ChapterTable.sourceOrder to SortOrder.DESC)
.limit(opdsItemsPerPageBounded)
.offset(((pageNum - 1) * opdsItemsPerPageBounded).toLong())
.map {
OpdsLibraryUpdateAcqEntry(
chapter = it.toOpdsChapterListAcqEntry(),
mangaTitle = it[MangaTable.title],
mangaAuthor = it[MangaTable.author],
mangaId = it[MangaTable.id].value,
mangaSourceLang = it[SourceTable.lang],
mangaThumbnailUrl = it[MangaTable.thumbnail_url],
)
}
.toList()
val mangaIds = rawItems.map { it[MangaTable.id].value }.distinct()
val chapterCounts =
if (mangaIds.isNotEmpty()) {
ChapterTable
.select(ChapterTable.manga, ChapterTable.id.count())
.where { ChapterTable.manga inList mangaIds }
.groupBy(ChapterTable.manga)
.associate { it[ChapterTable.manga].value to it[ChapterTable.id.count()] }
} else {
emptyMap()
}
val items =
rawItems.map {
val mId = it[MangaTable.id].value
OpdsLibraryUpdateAcqEntry(
chapter = it.toOpdsChapterListAcqEntry(),
mangaTitle = it[MangaTable.title],
mangaAuthor = it[MangaTable.author],
mangaId = mId,
mangaSourceLang = it[SourceTable.lang],
mangaThumbnailUrl = it[MangaTable.thumbnail_url],
mangaTotalChapters = chapterCounts[mId] ?: 0L,
)
}
Pair(items, totalCount)
}
@@ -146,21 +272,38 @@ object ChapterRepository {
val totalCount = query.count()
val items =
val rawItems =
query
.orderBy(ChapterTable.lastReadAt to SortOrder.DESC)
.limit(opdsItemsPerPageBounded)
.offset(((pageNum - 1) * opdsItemsPerPageBounded).toLong())
.map {
OpdsHistoryAcqEntry(
chapter = it.toOpdsChapterListAcqEntry(),
mangaTitle = it[MangaTable.title],
mangaAuthor = it[MangaTable.author],
mangaId = it[MangaTable.id].value,
mangaSourceLang = it[SourceTable.lang],
mangaThumbnailUrl = it[MangaTable.thumbnail_url],
)
}
.toList()
val mangaIds = rawItems.map { it[MangaTable.id].value }.distinct()
val chapterCounts =
if (mangaIds.isNotEmpty()) {
ChapterTable
.select(ChapterTable.manga, ChapterTable.id.count())
.where { ChapterTable.manga inList mangaIds }
.groupBy(ChapterTable.manga)
.associate { it[ChapterTable.manga].value to it[ChapterTable.id.count()] }
} else {
emptyMap()
}
val items =
rawItems.map {
val mId = it[MangaTable.id].value
OpdsHistoryAcqEntry(
chapter = it.toOpdsChapterListAcqEntry(),
mangaTitle = it[MangaTable.title],
mangaAuthor = it[MangaTable.author],
mangaId = mId,
mangaSourceLang = it[SourceTable.lang],
mangaThumbnailUrl = it[MangaTable.thumbnail_url],
mangaTotalChapters = chapterCounts[mId] ?: 0L,
)
}
Pair(items, totalCount)
}

View File

@@ -11,6 +11,7 @@ import org.jetbrains.exposed.v1.core.and
import org.jetbrains.exposed.v1.core.eq
import org.jetbrains.exposed.v1.core.greater
import org.jetbrains.exposed.v1.core.inList
import org.jetbrains.exposed.v1.core.inSubQuery
import org.jetbrains.exposed.v1.core.intLiteral
import org.jetbrains.exposed.v1.core.like
import org.jetbrains.exposed.v1.core.lowerCase
@@ -36,8 +37,72 @@ import suwayomi.tachidesk.opds.dto.OpdsMangaDetails
import suwayomi.tachidesk.opds.dto.OpdsMangaFilter
import suwayomi.tachidesk.opds.dto.OpdsSearchCriteria
import suwayomi.tachidesk.opds.dto.PrimaryFilterType
import suwayomi.tachidesk.opds.util.OpdsStringUtil.formatSourceName
import suwayomi.tachidesk.server.serverConfig
/**
* Applies dynamic filters based on the current user configuration and cross-filters.
* Allows excluding a specific field to calculate mutual exclusion facet counts efficiently.
*
* @param criteria The filtering criteria.
* @param excludeField The field to exclude from filtering.
*/
fun Query.applyOpdsMangaFilter(
criteria: OpdsMangaFilter,
excludeField: String? = null,
) {
if (excludeField != "source_id") {
criteria.sourceId?.let { andWhere { MangaTable.sourceReference eq it } }
}
if (excludeField != "category_id") {
criteria.categoryId?.let { andWhere { CategoryMangaTable.category eq it } }
}
if (excludeField != "status_id") {
criteria.statusId?.let { andWhere { MangaTable.status eq it } }
}
if (excludeField != "lang_code") {
criteria.langCode?.let { andWhere { SourceTable.lang eq it } }
}
if (excludeField != "genre") {
criteria.genre?.let { genre ->
val genreTrimmed = genre.trim()
andWhere {
(MangaTable.genre like "%, $genreTrimmed, %") or
(MangaTable.genre like "$genreTrimmed, %") or
(MangaTable.genre like "%, $genreTrimmed") or
(MangaTable.genre eq genreTrimmed)
}
}
}
if (excludeField != "filter") {
criteria.filter?.let { filterVal ->
when (filterVal) {
"unread" -> {
andWhere {
MangaTable.id inSubQuery
ChapterTable.select(ChapterTable.manga).where { ChapterTable.isRead eq false }
}
}
"downloaded" -> {
andWhere {
MangaTable.id inSubQuery
ChapterTable.select(ChapterTable.manga).where { ChapterTable.isDownloaded eq true }
}
}
"ongoing" -> {
andWhere { MangaTable.status eq MangaStatus.ONGOING.value }
}
"completed" -> {
andWhere { MangaTable.status eq MangaStatus.COMPLETED.value }
}
}
}
}
}
/**
* Repository for fetching manga data tailored for OPDS feeds.
*/
@@ -60,53 +125,48 @@ object MangaRepository {
sourceLang = this[SourceTable.lang],
inLibrary = this[MangaTable.inLibrary],
status = this[MangaTable.status],
sourceName = this[SourceTable.name],
sourceName = formatSourceName(this[SourceTable.name], this[SourceTable.lang]),
lastFetchedAt = this[MangaTable.lastFetchedAt],
url = this[MangaTable.realUrl],
)
/**
* Centralized function to retrieve paginated, sorted, and filtered manga from the library.
* @param criteria Additional filtering criteria for categories, sources, etc.
* @param pageNum The page number for pagination.
* @param sort The sorting parameter.
* @param filter The filtering parameter.
* @param criteria Additional filtering criteria for categories, sources, etc.
* @return An [OpdsLibraryFeedResult] containing the list of manga, total count, and the specific filter name.
*/
fun getLibraryManga(
criteria: OpdsMangaFilter,
pageNum: Int,
sort: String?,
filter: String?,
criteria: OpdsMangaFilter,
): OpdsLibraryFeedResult =
transaction {
val unreadCountExpr = Case().When(ChapterTable.isRead eq false, intLiteral(1)).Else(intLiteral(0)).sum()
val unreadCount = unreadCountExpr.alias("unread_count")
// Base query with necessary joins for filtering and sorting
val query =
var baseJoin =
MangaTable
.join(SourceTable, JoinType.INNER, MangaTable.sourceReference, SourceTable.id)
.join(ChapterTable, JoinType.LEFT, MangaTable.id, ChapterTable.manga)
.join(CategoryMangaTable, JoinType.LEFT, MangaTable.id, CategoryMangaTable.manga)
if (criteria.categoryId != null) {
baseJoin = baseJoin.join(CategoryMangaTable, JoinType.LEFT, MangaTable.id, CategoryMangaTable.manga)
}
val query =
baseJoin
.select(MangaTable.columns + SourceTable.lang + SourceTable.name + unreadCount)
.where { MangaTable.inLibrary eq true }
.groupBy(MangaTable.id, SourceTable.lang, SourceTable.name)
// Apply specific filters from criteria
criteria.sourceId?.let { query.andWhere { MangaTable.sourceReference eq it } }
criteria.categoryId?.let { query.andWhere { CategoryMangaTable.category eq it } }
criteria.statusId?.let { query.andWhere { MangaTable.status eq it } }
criteria.langCode?.let { query.andWhere { SourceTable.lang eq it } }
criteria.genre?.let { genre ->
val genreTrimmed = genre.trim()
val genreCondition =
(MangaTable.genre like "%, $genreTrimmed, %") or
(MangaTable.genre like "$genreTrimmed, %") or
(MangaTable.genre like "%, $genreTrimmed") or
(MangaTable.genre eq genreTrimmed)
query.andWhere { genreCondition }
}
query.applyOpdsMangaFilter(criteria)
applyMangaLibrarySort(query, sort)
query.groupBy(MangaTable.id, SourceTable.lang, SourceTable.name)
// Efficiently get the name of the primary filter item
val specificFilterName =
@@ -114,10 +174,10 @@ object MangaRepository {
PrimaryFilterType.SOURCE -> {
criteria.sourceId?.let {
SourceTable
.select(SourceTable.name)
.select(SourceTable.name, SourceTable.lang)
.where { SourceTable.id eq it }
.firstOrNull()
?.get(SourceTable.name)
?.let { formatSourceName(it[SourceTable.name], it[SourceTable.lang]) }
}
}
@@ -150,8 +210,6 @@ object MangaRepository {
}
}
applyMangaLibrarySortAndFilter(query, sort, filter)
val totalCount = query.count()
val mangas =
query
@@ -245,6 +303,7 @@ object MangaRepository {
*/
fun getMangaDetails(mangaId: Int): OpdsMangaDetails? =
transaction {
val chapterCount = ChapterTable.select(ChapterTable.id).where { ChapterTable.manga eq mangaId }.count()
MangaTable
.select(MangaTable.id, MangaTable.title, MangaTable.thumbnail_url, MangaTable.author)
.where { MangaTable.id eq mangaId }
@@ -255,6 +314,7 @@ object MangaRepository {
title = it[MangaTable.title],
thumbnailUrl = it[MangaTable.thumbnail_url],
author = it[MangaTable.author],
totalChapters = chapterCount,
)
}
}
@@ -263,26 +323,15 @@ object MangaRepository {
* Applies sorting and filtering logic to a manga library query.
* @param query The Exposed SQL query to modify.
* @param sort The sorting parameter.
* @param filter The filtering parameter.
*/
private fun applyMangaLibrarySortAndFilter(
private fun applyMangaLibrarySort(
query: Query,
sort: String?,
filter: String?,
) {
val unreadCountExpr = Case().When(ChapterTable.isRead eq false, intLiteral(1)).Else(intLiteral(0)).sum()
val downloadedCountExpr = Case().When(ChapterTable.isDownloaded eq true, intLiteral(1)).Else(intLiteral(0)).sum()
val lastReadAtExpr = ChapterTable.lastReadAt.max()
val latestChapterDateExpr = ChapterTable.date_upload.max()
// Apply filtering using HAVING for aggregate functions or WHERE for direct columns
when (filter) {
"unread" -> query.having { unreadCountExpr greater 0 }
"downloaded" -> query.having { downloadedCountExpr greater 0 }
"ongoing" -> query.andWhere { MangaTable.status eq MangaStatus.ONGOING.value }
"completed" -> query.andWhere { MangaTable.status eq MangaStatus.COMPLETED.value }
}
// Apply sorting
when (sort) {
"alpha_asc" -> query.orderBy(MangaTable.title to SortOrder.ASC)
@@ -296,27 +345,44 @@ object MangaRepository {
}
/**
* Calculates the count of manga for various library filter facets.
* Calculates the count of manga for various library filter facets, respecting other active cross-filters.
* @param activeFilters The currently active filters to respect during count calculation.
* @return A map where keys are filter names and values are the counts.
*/
fun getLibraryFilterCounts(): Map<String, Long> =
fun getLibraryFilterCounts(activeFilters: OpdsMangaFilter): Map<String, Long> =
transaction {
val unreadCountExpr = Case().When(ChapterTable.isRead eq false, intLiteral(1)).Else(intLiteral(0)).sum()
val downloadedCountExpr = Case().When(ChapterTable.isDownloaded eq true, intLiteral(1)).Else(intLiteral(0)).sum()
var baseJoin =
MangaTable
.join(SourceTable, JoinType.INNER, MangaTable.sourceReference, SourceTable.id)
if (activeFilters.categoryId != null) {
baseJoin = baseJoin.join(CategoryMangaTable, JoinType.LEFT, MangaTable.id, CategoryMangaTable.manga)
}
val baseQuery =
MangaTable
.join(ChapterTable, JoinType.LEFT, MangaTable.id, ChapterTable.manga)
baseJoin
.select(MangaTable.id)
.where { MangaTable.inLibrary eq true }
.groupBy(MangaTable.id)
.withDistinct()
val unreadCount = baseQuery.copy().having { unreadCountExpr greater 0 }.count()
val downloadedCount = baseQuery.copy().having { downloadedCountExpr greater 0 }.count()
baseQuery.applyOpdsMangaFilter(activeFilters, excludeField = "filter")
val statusBaseQuery = MangaTable.select(MangaTable.id).where { MangaTable.inLibrary eq true }
val ongoingCount = statusBaseQuery.copy().andWhere { MangaTable.status eq MangaStatus.ONGOING.value }.count()
val completedCount = statusBaseQuery.copy().andWhere { MangaTable.status eq MangaStatus.COMPLETED.value }.count()
val unreadCount =
baseQuery
.copy()
.andWhere {
MangaTable.id inSubQuery
ChapterTable.select(ChapterTable.manga).where { ChapterTable.isRead eq false }
}.count()
val downloadedCount =
baseQuery
.copy()
.andWhere {
MangaTable.id inSubQuery
ChapterTable.select(ChapterTable.manga).where { ChapterTable.isDownloaded eq true }
}.count()
val ongoingCount = baseQuery.copy().andWhere { MangaTable.status eq MangaStatus.ONGOING.value }.count()
val completedCount = baseQuery.copy().andWhere { MangaTable.status eq MangaStatus.COMPLETED.value }.count()
mapOf(
"unread" to unreadCount,

View File

@@ -21,10 +21,12 @@ import suwayomi.tachidesk.opds.constants.OpdsConstants
import suwayomi.tachidesk.opds.dto.OpdsCategoryNavEntry
import suwayomi.tachidesk.opds.dto.OpdsGenreNavEntry
import suwayomi.tachidesk.opds.dto.OpdsLanguageNavEntry
import suwayomi.tachidesk.opds.dto.OpdsMangaFilter
import suwayomi.tachidesk.opds.dto.OpdsRootNavEntry
import suwayomi.tachidesk.opds.dto.OpdsSourceNavEntry
import suwayomi.tachidesk.opds.dto.OpdsStatusNavEntry
import suwayomi.tachidesk.opds.util.OpdsStringUtil.encodeForOpdsURL
import suwayomi.tachidesk.opds.util.OpdsStringUtil.formatSourceName
import suwayomi.tachidesk.server.serverConfig
import java.util.Locale
@@ -131,15 +133,14 @@ object NavigationRepository {
)
}
// ... (El resto del archivo permanece sin cambios)
fun getExploreSources(pageNum: Int): Pair<List<OpdsSourceNavEntry>, Long> =
transaction {
val query =
SourceTable
.join(ExtensionTable, JoinType.LEFT, onColumn = SourceTable.extension, otherColumn = ExtensionTable.id)
.select(SourceTable.id, SourceTable.name, ExtensionTable.apkName)
.select(SourceTable.id, SourceTable.name, SourceTable.lang, ExtensionTable.apkName)
.where { ExtensionTable.isInstalled eq true }
.groupBy(SourceTable.id, SourceTable.name, ExtensionTable.apkName)
.groupBy(SourceTable.id, SourceTable.name, SourceTable.lang, ExtensionTable.apkName)
.orderBy(SourceTable.name to SortOrder.ASC)
val totalCount = query.count()
@@ -150,7 +151,7 @@ object NavigationRepository {
.map {
OpdsSourceNavEntry(
id = it[SourceTable.id].value,
name = it[SourceTable.name],
name = formatSourceName(it[SourceTable.name], it[SourceTable.lang]),
iconUrl = it[ExtensionTable.apkName].let { apkName -> Extension.getExtensionIconUrl(apkName) },
mangaCount = null,
)
@@ -158,36 +159,71 @@ object NavigationRepository {
Pair(sources, totalCount)
}
fun getLibrarySources(pageNum: Int): Pair<List<OpdsSourceNavEntry>, Long> =
fun getLibrarySources(
pageNum: Int? = null,
activeFilters: OpdsMangaFilter = OpdsMangaFilter(),
): Pair<List<OpdsSourceNavEntry>, Long> =
transaction {
val mangaCount = MangaTable.id.countDistinct().alias("manga_count")
val query =
var baseJoin =
SourceTable
.join(MangaTable, JoinType.INNER, SourceTable.id, MangaTable.sourceReference)
.join(ExtensionTable, JoinType.LEFT, onColumn = SourceTable.extension, otherColumn = ExtensionTable.id)
.select(SourceTable.id, SourceTable.name, ExtensionTable.apkName, mangaCount)
if (activeFilters.categoryId != null) {
baseJoin = baseJoin.join(CategoryMangaTable, JoinType.LEFT, MangaTable.id, CategoryMangaTable.manga)
}
val query =
baseJoin
.select(SourceTable.id, SourceTable.name, SourceTable.lang, ExtensionTable.apkName, mangaCount)
.where { MangaTable.inLibrary eq true }
.groupBy(SourceTable.id, SourceTable.name, ExtensionTable.apkName)
.orderBy(SourceTable.name to SortOrder.ASC)
query.applyOpdsMangaFilter(activeFilters, excludeField = "source_id")
query
.groupBy(SourceTable.id, SourceTable.name, SourceTable.lang, ExtensionTable.apkName)
.orderBy(SourceTable.name to SortOrder.ASC)
val totalCount = query.count()
val sources =
if (pageNum != null) {
query
.limit(opdsItemsPerPageBounded)
.offset(((pageNum - 1) * opdsItemsPerPageBounded).toLong())
.map {
OpdsSourceNavEntry(
id = it[SourceTable.id].value,
name = it[SourceTable.name],
iconUrl = it[ExtensionTable.apkName].let { apkName -> Extension.getExtensionIconUrl(apkName) },
mangaCount = it[mangaCount],
)
}
}
val sources =
query.map {
OpdsSourceNavEntry(
id = it[SourceTable.id].value,
name = formatSourceName(it[SourceTable.name], it[SourceTable.lang]),
iconUrl = it[ExtensionTable.apkName].let { apkName -> Extension.getExtensionIconUrl(apkName) },
mangaCount = it[mangaCount],
)
}
Pair(sources, totalCount)
}
fun getCategories(pageNum: Int): Pair<List<OpdsCategoryNavEntry>, Long> =
fun getSourceDetails(sourceId: Long): Pair<String, String?>? =
transaction {
SourceTable
.join(ExtensionTable, JoinType.LEFT, onColumn = SourceTable.extension, otherColumn = ExtensionTable.id)
.select(SourceTable.name, SourceTable.lang, ExtensionTable.apkName)
.where { SourceTable.id eq sourceId }
.firstOrNull()
?.let {
val name = formatSourceName(it[SourceTable.name], it[SourceTable.lang])
val icon = Extension.getExtensionIconUrl(it[ExtensionTable.apkName])
Pair(name, icon)
}
}
fun getCategories(
pageNum: Int? = null,
activeFilters: OpdsMangaFilter = OpdsMangaFilter(),
): Pair<List<OpdsCategoryNavEntry>, Long> =
transaction {
val mangaCount = MangaTable.id.countDistinct().alias("manga_count")
@@ -195,35 +231,57 @@ object NavigationRepository {
CategoryTable
.join(CategoryMangaTable, JoinType.INNER, CategoryTable.id, CategoryMangaTable.category)
.join(MangaTable, JoinType.INNER, CategoryMangaTable.manga, MangaTable.id)
.join(SourceTable, JoinType.INNER, MangaTable.sourceReference, SourceTable.id)
.select(CategoryTable.id, CategoryTable.name, mangaCount)
.where { MangaTable.inLibrary eq true }
.groupBy(CategoryTable.id, CategoryTable.name)
.orderBy(CategoryTable.order to SortOrder.ASC)
query.applyOpdsMangaFilter(activeFilters, excludeField = "category_id")
query
.groupBy(CategoryTable.id, CategoryTable.name)
.orderBy(CategoryTable.order to SortOrder.ASC)
val totalCount = query.count()
val categories =
if (pageNum != null) {
query
.limit(opdsItemsPerPageBounded)
.offset(((pageNum - 1) * opdsItemsPerPageBounded).toLong())
.map {
OpdsCategoryNavEntry(
id = it[CategoryTable.id].value,
name = it[CategoryTable.name],
mangaCount = it[mangaCount],
)
}
}
val categories =
query.map {
OpdsCategoryNavEntry(
id = it[CategoryTable.id].value,
name = it[CategoryTable.name],
mangaCount = it[mangaCount],
)
}
Pair(categories, totalCount)
}
fun getGenres(
pageNum: Int,
locale: Locale,
pageNum: Int? = null,
activeFilters: OpdsMangaFilter = OpdsMangaFilter(),
): Pair<List<OpdsGenreNavEntry>, Long> =
transaction {
val allGenres =
var baseJoin =
MangaTable
.join(SourceTable, JoinType.INNER, MangaTable.sourceReference, SourceTable.id)
if (activeFilters.categoryId != null) {
baseJoin = baseJoin.join(CategoryMangaTable, JoinType.LEFT, MangaTable.id, CategoryMangaTable.manga)
}
val query =
baseJoin
.select(MangaTable.genre)
.where { MangaTable.inLibrary eq true }
query.applyOpdsMangaFilter(activeFilters, excludeField = "genre")
val allGenres =
query
.mapNotNull { it[MangaTable.genre] }
.flatMap { it.split(",").map(String::trim).filterNot(String::isBlank) }
@@ -231,21 +289,32 @@ object NavigationRepository {
val distinctGenres = genreCounts.keys.sorted()
val totalCount = distinctGenres.size.toLong()
val fromIndex = ((pageNum - 1) * opdsItemsPerPageBounded)
val toIndex = minOf(fromIndex + opdsItemsPerPageBounded, distinctGenres.size)
val finalGenres =
if (pageNum != null) {
val fromIndex = ((pageNum - 1) * opdsItemsPerPageBounded)
val toIndex = minOf(fromIndex + opdsItemsPerPageBounded, distinctGenres.size)
if (fromIndex < distinctGenres.size) distinctGenres.subList(fromIndex, toIndex) else emptyList()
} else {
distinctGenres
}
val paginatedGenres =
(if (fromIndex < distinctGenres.size) distinctGenres.subList(fromIndex, toIndex) else emptyList())
.map { genreName ->
OpdsGenreNavEntry(
id = genreName.encodeForOpdsURL(),
title = genreName,
mangaCount = genreCounts[genreName]?.toLong() ?: 0L,
)
}
finalGenres.map { genreName ->
OpdsGenreNavEntry(
id = genreName.encodeForOpdsURL(),
title = genreName,
mangaCount = genreCounts[genreName]?.toLong() ?: 0L,
)
}
Pair(paginatedGenres, totalCount)
}
fun getStatuses(locale: Locale): List<OpdsStatusNavEntry> {
fun getStatuses(
locale: Locale,
pageNum: Int? = null,
activeFilters: OpdsMangaFilter = OpdsMangaFilter(),
): Pair<List<OpdsStatusNavEntry>, Long> {
val statusStringResources: Map<MangaStatus, StringResource> =
mapOf(
MangaStatus.UNKNOWN to MR.strings.manga_status_unknown,
@@ -259,43 +328,98 @@ object NavigationRepository {
val statusCounts =
transaction {
MangaTable
.select(MangaTable.status, MangaTable.id.count())
.where { MangaTable.inLibrary eq true }
val countExpr = MangaTable.id.countDistinct().alias("manga_count")
var baseJoin =
MangaTable
.join(SourceTable, JoinType.INNER, MangaTable.sourceReference, SourceTable.id)
if (activeFilters.categoryId != null) {
baseJoin = baseJoin.join(CategoryMangaTable, JoinType.LEFT, MangaTable.id, CategoryMangaTable.manga)
}
val query =
baseJoin
.select(MangaTable.status, countExpr)
.where { MangaTable.inLibrary eq true }
query.applyOpdsMangaFilter(activeFilters, excludeField = "status_id")
query
.groupBy(MangaTable.status)
.associate { it[MangaTable.status] to it[MangaTable.id.count()] }
.associate { it[MangaTable.status] to it[countExpr] }
}
return MangaStatus.entries
.map { mangaStatus ->
val titleRes = statusStringResources[mangaStatus] ?: MR.strings.manga_status_unknown
OpdsStatusNavEntry(
id = mangaStatus.value,
title = titleRes.localized(locale),
mangaCount = statusCounts[mangaStatus.value] ?: 0L,
)
}.sortedBy { it.id }
val allStatuses =
MangaStatus.entries
.map { mangaStatus ->
val titleRes = statusStringResources[mangaStatus] ?: MR.strings.manga_status_unknown
OpdsStatusNavEntry(
id = mangaStatus.value,
title = titleRes.localized(locale),
mangaCount = statusCounts[mangaStatus.value] ?: 0L,
)
}.sortedBy { it.id }
val totalCount = allStatuses.size.toLong()
val paginatedStatuses =
if (pageNum != null) {
val fromIndex = ((pageNum - 1) * opdsItemsPerPageBounded)
val toIndex = minOf(fromIndex + opdsItemsPerPageBounded, allStatuses.size)
if (fromIndex < allStatuses.size) allStatuses.subList(fromIndex, toIndex) else emptyList()
} else {
allStatuses
}
return Pair(paginatedStatuses, totalCount)
}
fun getContentLanguages(uiLocale: Locale): List<OpdsLanguageNavEntry> =
fun getContentLanguages(
locale: Locale,
pageNum: Int? = null,
activeFilters: OpdsMangaFilter = OpdsMangaFilter(),
): Pair<List<OpdsLanguageNavEntry>, Long> =
transaction {
val mangaCount = MangaTable.id.countDistinct().alias("manga_count")
SourceTable
.join(MangaTable, JoinType.INNER, SourceTable.id, MangaTable.sourceReference)
.select(SourceTable.lang, mangaCount)
.where { MangaTable.inLibrary eq true }
var baseJoin =
SourceTable
.join(MangaTable, JoinType.INNER, SourceTable.id, MangaTable.sourceReference)
if (activeFilters.categoryId != null) {
baseJoin = baseJoin.join(CategoryMangaTable, JoinType.LEFT, MangaTable.id, CategoryMangaTable.manga)
}
val query =
baseJoin
.select(SourceTable.lang, mangaCount)
.where { MangaTable.inLibrary eq true }
query.applyOpdsMangaFilter(activeFilters, excludeField = "lang_code")
query
.groupBy(SourceTable.lang)
.orderBy(SourceTable.lang to SortOrder.ASC)
.map {
val totalCount = query.count()
if (pageNum != null) {
query
.limit(opdsItemsPerPageBounded)
.offset(((pageNum - 1) * opdsItemsPerPageBounded).toLong())
}
val languages =
query.map {
val langCode = it[SourceTable.lang]
OpdsLanguageNavEntry(
id = langCode,
title =
Locale.forLanguageTag(langCode).getDisplayName(uiLocale).replaceFirstChar { char ->
if (char.isLowerCase()) char.titlecase(uiLocale) else char.toString()
Locale.forLanguageTag(langCode).getDisplayName(locale).replaceFirstChar { char ->
if (char.isLowerCase()) char.titlecase(locale) else char.toString()
},
mangaCount = it[mangaCount],
)
}
Pair(languages, totalCount)
}
}

View File

@@ -33,6 +33,19 @@ object OpdsStringUtil {
return slug
}
/**
* Formats the source name appending the language code if applicable.
*/
fun formatSourceName(
name: String,
lang: String?,
): String =
if (lang.isNullOrBlank() || lang == "all") {
name
} else {
"$name (${lang.uppercase()})"
}
/**
* Formats a size in bytes to a human-readable representation.
* Uses binary (KiB, MiB, GiB, TiB) or decimal (KB, MB, GB, TB) units based on server configuration.

View File

@@ -36,6 +36,7 @@ import org.koin.core.context.startKoin
import org.koin.core.module.Module
import org.koin.dsl.module
import suwayomi.tachidesk.global.impl.KcefWebView.Companion.toCefCookie
import suwayomi.tachidesk.global.impl.sync.SyncManager
import suwayomi.tachidesk.graphql.types.DatabaseType
import suwayomi.tachidesk.i18n.LocalizationHelper
import suwayomi.tachidesk.manga.impl.backup.proto.ProtoBackupExport
@@ -70,11 +71,6 @@ import java.net.Authenticator
import java.net.PasswordAuthentication
import java.security.Security
import java.util.Locale
import kotlin.concurrent.thread
import kotlin.io.path.Path
import kotlin.io.path.createDirectories
import kotlin.io.path.div
import kotlin.math.roundToInt
private val logger = KotlinLogging.logger {}
@@ -517,6 +513,8 @@ fun applicationSetup() {
// start DownloadManager and restore + resume downloads
DownloadManager.restoreAndResumeDownloads()
SyncManager.scheduleSyncTask()
// asynchronously initialize CEF
GlobalScope.launch {
CEFManager.init()

View File

@@ -0,0 +1,224 @@
package suwayomi.tachidesk.server.database.migration
import de.neonew.exposed.migrations.helpers.SQLMigration
import suwayomi.tachidesk.graphql.types.DatabaseType
import suwayomi.tachidesk.server.serverConfig
@Suppress("ClassName", "unused")
class M0056_SyncYomi : SQLMigration() {
override val sql =
when (serverConfig.databaseType.value) {
DatabaseType.POSTGRESQL -> postgresQuery()
DatabaseType.H2 -> h2Query()
}
// language=postgresql
fun postgresQuery(): String =
"""
ALTER TABLE manga ADD COLUMN version BIGINT NOT NULL DEFAULT 0;
ALTER TABLE manga ADD COLUMN is_syncing BOOLEAN NOT NULL DEFAULT FALSE;
ALTER TABLE manga ADD COLUMN last_modified_at BIGINT NOT NULL DEFAULT 0;
ALTER TABLE chapter ADD COLUMN version BIGINT NOT NULL DEFAULT 0;
ALTER TABLE chapter ADD COLUMN is_syncing BOOLEAN NOT NULL DEFAULT FALSE;
ALTER TABLE chapter ADD COLUMN last_modified_at BIGINT NOT NULL DEFAULT 0;
ALTER TABLE category ADD COLUMN version BIGINT NOT NULL DEFAULT 0;
ALTER TABLE category ADD COLUMN uid BIGINT NOT NULL DEFAULT 0;
ALTER TABLE category ADD COLUMN is_syncing BOOLEAN NOT NULL DEFAULT FALSE;
ALTER TABLE category ADD COLUMN last_modified_at BIGINT NOT NULL DEFAULT 0;
CREATE OR REPLACE FUNCTION update_manga_version()
RETURNS trigger AS $$
BEGIN
IF NOT NEW.is_syncing
AND ROW(NEW.url, NEW.description, NEW.in_library)
IS DISTINCT FROM
ROW(OLD.url, OLD.description, OLD.in_library)
THEN
NEW.version := OLD.version + 1;
END IF;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER update_manga_version
AFTER UPDATE ON manga
FOR EACH ROW
EXECUTE FUNCTION update_manga_version();
CREATE OR REPLACE FUNCTION update_chapter_and_manga_version()
RETURNS trigger AS $$
BEGIN
IF NOT NEW.is_syncing
AND ROW(NEW.read, NEW.bookmark, NEW.last_page_read)
IS DISTINCT FROM
ROW(OLD.read, OLD.bookmark, OLD.last_page_read)
THEN
NEW.version := OLD.version + 1;
UPDATE manga SET version = version + 1 WHERE id = NEW.manga AND is_syncing = FALSE;
END IF;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER update_chapter_and_manga_version
AFTER UPDATE ON chapter
FOR EACH ROW
EXECUTE FUNCTION update_chapter_and_manga_version();
CREATE OR REPLACE FUNCTION update_manga_last_modified_at()
RETURNS trigger AS $$
BEGIN
NEW.last_modified_at := EXTRACT(EPOCH FROM NOW());
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER update_manga_last_modified_at
BEFORE UPDATE OR INSERT ON manga
FOR EACH ROW
EXECUTE FUNCTION update_manga_last_modified_at();
CREATE OR REPLACE FUNCTION update_chapter_last_modified_at()
RETURNS trigger AS $$
BEGIN
NEW.last_modified_at := EXTRACT(EPOCH FROM NOW());
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER update_chapter_last_modified_at
BEFORE UPDATE OR INSERT ON chapter
FOR EACH ROW
EXECUTE FUNCTION update_chapter_last_modified_at();
CREATE OR REPLACE FUNCTION insert_manga_category_update_version()
RETURNS trigger AS $$
BEGIN
UPDATE manga SET version = version + 1 WHERE id = NEW.manga AND is_syncing = FALSE;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER insert_manga_category_update_version
AFTER INSERT ON categorymanga
FOR EACH ROW
EXECUTE FUNCTION insert_manga_category_update_version();
CREATE OR REPLACE FUNCTION insert_category_uid()
RETURNS trigger AS $$
BEGIN
IF NEW.uid = 0 THEN
NEW.uid := RANDOM(1, 9223372036854775807);
END IF;
IF NEW.last_modified_at = 0 THEN
NEW.last_modified_at := EXTRACT(EPOCH FROM NOW());
END IF;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER insert_category_uid
BEFORE INSERT ON category
FOR EACH ROW
EXECUTE FUNCTION insert_category_uid();
CREATE OR REPLACE FUNCTION update_category_version()
RETURNS trigger AS $$
BEGIN
IF NOT NEW.is_syncing
AND ROW(NEW.name, NEW.sort_order)
IS DISTINCT FROM
ROW(OLD.name, OLD.sort_order)
THEN
NEW.version := NEW.version + 1;
NEW.last_modified_at := EXTRACT(EPOCH FROM NOW());
END IF;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER update_category_version
BEFORE UPDATE ON category
FOR EACH ROW
EXECUTE FUNCTION update_category_version();
""".trimIndent()
// language=h2
fun h2Query() =
"""
ALTER TABLE manga ADD COLUMN version BIGINT NOT NULL DEFAULT 0;
ALTER TABLE manga ADD COLUMN is_syncing BOOLEAN NOT NULL DEFAULT FALSE;
ALTER TABLE manga ADD COLUMN last_modified_at BIGINT NOT NULL DEFAULT 0;
ALTER TABLE chapter ADD COLUMN version BIGINT NOT NULL DEFAULT 0;
ALTER TABLE chapter ADD COLUMN is_syncing BOOLEAN NOT NULL DEFAULT FALSE;
ALTER TABLE chapter ADD COLUMN last_modified_at BIGINT NOT NULL DEFAULT 0;
ALTER TABLE category ADD COLUMN version BIGINT NOT NULL DEFAULT 0;
ALTER TABLE category ADD COLUMN uid BIGINT NOT NULL DEFAULT 0;
ALTER TABLE category ADD COLUMN is_syncing BOOLEAN NOT NULL DEFAULT FALSE;
ALTER TABLE category ADD COLUMN last_modified_at BIGINT NOT NULL DEFAULT 0;
CREATE TRIGGER update_manga_version
BEFORE UPDATE ON manga
FOR EACH ROW
CALL "suwayomi.tachidesk.server.database.trigger.UpdateMangaVersionTrigger";
CREATE TRIGGER update_chapter_and_manga_version
BEFORE UPDATE ON chapter
FOR EACH ROW
CALL "suwayomi.tachidesk.server.database.trigger.UpdateChapterAndMangaVersionTrigger";
CREATE TRIGGER update_manga_last_modified_at
BEFORE UPDATE ON manga
FOR EACH ROW
CALL "suwayomi.tachidesk.server.database.trigger.UpdateMangaLastModifiedAtTrigger";
CREATE TRIGGER insert_manga_last_modified_at
BEFORE INSERT ON manga
FOR EACH ROW
CALL "suwayomi.tachidesk.server.database.trigger.UpdateMangaLastModifiedAtTrigger";
CREATE TRIGGER update_chapter_last_modified_at
BEFORE UPDATE ON chapter
FOR EACH ROW
CALL "suwayomi.tachidesk.server.database.trigger.UpdateChapterLastModifiedAtTrigger";
CREATE TRIGGER insert_chapter_last_modified_at
BEFORE INSERT ON chapter
FOR EACH ROW
CALL "suwayomi.tachidesk.server.database.trigger.UpdateChapterLastModifiedAtTrigger";
CREATE TRIGGER insert_manga_category_update_version
AFTER INSERT ON categorymanga
FOR EACH ROW
CALL "suwayomi.tachidesk.server.database.trigger.InsertMangaCategoryUpdateVersionTrigger";
CREATE TRIGGER insert_category_uid
BEFORE INSERT ON category
FOR EACH ROW
CALL "suwayomi.tachidesk.server.database.trigger.InsertCategoryUidTrigger";
CREATE TRIGGER update_category_version
BEFORE UPDATE ON category
FOR EACH ROW
CALL "suwayomi.tachidesk.server.database.trigger.UpdateCategoryVersionTrigger";
""".trimIndent()
}

View File

@@ -0,0 +1,141 @@
package suwayomi.tachidesk.server.database.trigger
import org.h2.tools.TriggerAdapter
import java.sql.Connection
import java.sql.ResultSet
import kotlin.random.Random
import kotlin.time.Clock
@Suppress("unused")
class UpdateMangaVersionTrigger : TriggerAdapter() {
override fun fire(
conn: Connection,
oldRow: ResultSet,
newRow: ResultSet,
) {
val isSyncing = newRow.getBoolean("is_syncing")
val hasChanged =
oldRow.getString("url") != newRow.getString("url") ||
oldRow.getString("description") != newRow.getString("description") ||
oldRow.getBoolean("in_library") != newRow.getBoolean("in_library")
if (!isSyncing && hasChanged) {
val currentVersion = newRow.getLong("version")
newRow.updateLong("version", currentVersion + 1)
}
}
}
@Suppress("unused")
class UpdateChapterAndMangaVersionTrigger : TriggerAdapter() {
override fun fire(
conn: Connection,
oldRow: ResultSet,
newRow: ResultSet,
) {
val isSyncing = newRow.getBoolean("is_syncing")
val hasChanged =
oldRow.getBoolean("read") != newRow.getBoolean("read") ||
oldRow.getBoolean("bookmark") != newRow.getBoolean("bookmark") ||
oldRow.getInt("last_page_read") != newRow.getInt("last_page_read")
if (!isSyncing && hasChanged) {
val currentVersion = newRow.getLong("version")
newRow.updateLong("version", currentVersion + 1)
val mangaId = newRow.getInt("manga")
conn
.prepareStatement(
"UPDATE MANGA SET version = version + 1 WHERE id = ? AND NOT is_syncing",
).use {
it.setInt(1, mangaId)
it.executeUpdate()
}
}
}
}
@Suppress("unused")
class UpdateMangaLastModifiedAtTrigger : TriggerAdapter() {
override fun fire(
conn: Connection,
oldRow: ResultSet?,
newRow: ResultSet,
) {
newRow.updateLong("last_modified_at", Clock.System.now().epochSeconds)
}
}
@Suppress("unused")
class UpdateChapterLastModifiedAtTrigger : TriggerAdapter() {
override fun fire(
conn: Connection,
oldRow: ResultSet?,
newRow: ResultSet,
) {
newRow.updateLong("last_modified_at", Clock.System.now().epochSeconds)
}
}
@Suppress("unused")
class InsertMangaCategoryUpdateVersionTrigger : TriggerAdapter() {
override fun fire(
conn: Connection,
oldRow: ResultSet?,
newRow: ResultSet,
) {
val mangaId = newRow.getInt("manga")
conn
.prepareStatement(
"UPDATE MANGA SET version = version + 1 WHERE id = ? AND NOT is_syncing",
).use {
it.setInt(1, mangaId)
it.executeUpdate()
}
}
}
@Suppress("unused")
class InsertCategoryUidTrigger : TriggerAdapter() {
override fun fire(
conn: Connection,
oldRow: ResultSet?,
newRow: ResultSet,
) {
if (newRow.getLong("uid") == 0L) {
newRow.updateLong("uid", Random.nextLong(1, Long.MAX_VALUE))
}
if (newRow.getLong("last_modified_at") == 0L) {
newRow.updateLong(
"last_modified_at",
Clock.System.now().epochSeconds,
)
}
}
}
@Suppress("unused")
class UpdateCategoryVersionTrigger : TriggerAdapter() {
override fun fire(
conn: Connection,
oldRow: ResultSet,
newRow: ResultSet,
) {
val isSyncing = newRow.getBoolean("is_syncing")
val hasChanged =
oldRow.getString("name") != newRow.getString("name") ||
oldRow.getInt("sort_order") != newRow.getInt("sort_order")
if (!isSyncing && hasChanged) {
val currentVersion = newRow.getLong("version")
newRow.updateLong("version", currentVersion + 1)
newRow.updateLong(
"last_modified_at",
Clock.System.now().epochSeconds,
)
}
}
}

View File

@@ -11,9 +11,7 @@ import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.SupervisorJob
import kotlinx.coroutines.flow.first
import kotlinx.coroutines.flow.subscribe
import kotlinx.coroutines.launch
import kotlinx.coroutines.runBlocking
import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable
import kotlinx.serialization.json.Json

View File

@@ -136,7 +136,7 @@ object WebInterfaceManager {
fun getAboutInfo(): AboutWebUI {
val currentVersion = getLocalVersion()
val failedToGetVersion = currentVersion === "r-1"
val failedToGetVersion = currentVersion == "r-1"
if (failedToGetVersion) {
throw Exception("Failed to get current version")
}

View File

@@ -2,14 +2,12 @@ package suwayomi.tachidesk
import android.os.Handler
import android.os.Looper
import android.os.Message
import java.util.concurrent.CountDownLatch
import java.util.concurrent.TimeUnit
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertNotEquals
import kotlin.test.assertTrue
import kotlin.text.StringBuilder
class LooperThread : Thread() {
var mHandler: Handler? = null

View File

@@ -46,6 +46,7 @@ class TestUpdater : IUpdater {
TODO("Not yet implemented")
}
@Deprecated("Replaced with updates", replaceWith = ReplaceWith("updates"))
override val status: Flow<UpdateStatus>
get() = TODO("Not yet implemented")
override val updates: Flow<UpdateUpdates>

View File

@@ -28,7 +28,6 @@ import suwayomi.tachidesk.server.serverModule
import suwayomi.tachidesk.server.settings.SettingsRegistry
import suwayomi.tachidesk.server.util.AppMutex.handleAppMutex
import suwayomi.tachidesk.server.util.ConfigTypeRegistration
import suwayomi.tachidesk.server.util.SystemTray
import uy.kohesive.injekt.Injekt
import uy.kohesive.injekt.api.get
import xyz.nulldev.androidcompat.AndroidCompatInitializer