Commit Graph

347 Commits

Author SHA1 Message Date
schroda
3bac176bf6 Prevent UnsupportedOperationException in DownloadManager (#1521)
CopyOnWriteArraySet does not support the usage of "removeAll" with a predicate
2025-07-15 09:32:11 -04:00
Constantin Piber
df0078b725 [#1496] Image conversion (#1505)
* [#1496] First conversion attempt

* [#1496] Configurable conversion

* Fix: allow nested configs (map)

* [#1496] Support explicit `none` conversion

* Use MimeUtils for provided download

* [1496] Support image conversion on load for downloaded images

* Lint

* [#1496] Support conversion on fresh download as well

Previous commit was only for already downloaded images, now also for
fresh and cached

* [#1496] Refactor: Move where conversion for download happens

* Rewrite config handling, improve custom types

* Lint

* Add format to pages mutation

* Lint

* Standardize url encode

* Lint

* Config: Allow additional conversion parameters

* Implement conversion quality parameter

* Lint

* Implement a conversion util to allow fallback readers

* Add downloadConversions to api and backup, fix updateValue issues

* Lint

* Minor cleanup

* Update libs.versions.toml

---------

Co-authored-by: Syer10 <syer10@users.noreply.github.com>
2025-07-14 17:51:18 -04:00
schroda
09c950a890 Fix/gql download subscription errors spamming emits (#1518)
* Remove immediate download notification for latest gql subscription

There is a problem where too many immediate updates can cause the client to lag out (e.g., in case it has to update the queue in the cache based on the updates).
This happens in case e.g., a source is broken and all its downloads error out basically immediately.
With each errored out download, a new one starts, which causes an immediate notification to the clients.

* Determine downloader status from active state of downloader jobs

In case the downloader is active but all downloads are erroring out immediately, no download will have the DOWNLOADING status.
This then would result in the downloader status to constantly be STOPPED.

* Prevent multiple update for the same downloads

It was possible that multiple updates got added for the same download.
This caused issues with the graphql apollo client, because it wasn't able to correctly update the client cache.

* Set download error state only after reaching max retries

In case the max retries haven't been reached yet, the download will be retried and thus setting and emitting the error state will cause weird looking ui updates.
2025-07-14 17:50:03 -04:00
schroda
06c1eeb995 Add missing transaction context to manga category update (#1516)
fixes #1510
2025-07-14 17:49:03 -04:00
Constantin Piber
68a131dbeb [#1349] Basic Cookie Authentication (#1498)
* [#1349] Stub basic cookie authentication

* [#1349] Basic login page

Also adjusts WebView header color and shadow to match WebUI. WebUI uses
a background-image gradient to change the perceived color, which was not
noticed originally.

* [#1349] Handle login post

* [#1349] Redirect to previous URL

* [#1349] Return a basic 401 for api endpoints

Instead of redirecting to a visual login page, API should just indicate
the bad state

* Use more appropriate 303 redirect

* Update server/src/main/kotlin/suwayomi/tachidesk/server/JavalinSetup.kt

Co-authored-by: Mitchell Syer <Syer10@users.noreply.github.com>

* Update server/src/main/kotlin/suwayomi/tachidesk/server/JavalinSetup.kt

Co-authored-by: Mitchell Syer <Syer10@users.noreply.github.com>

* Lint

* Transition to AuthMode enum with migration path

* Make basicAuthEnabled auto property, Lint

* ConfigManager: Make sure to re-parse the config after migration

* basicAuth{Username,Password} -> auth{Username,Password}

* Lint

* Update server settings backup model

* Update comment

* Minor cleanup

* Improve backup legacy settings fix

* Lint

* Simplify config value migration

---------

Co-authored-by: Mitchell Syer <Syer10@users.noreply.github.com>
2025-07-06 12:08:29 -04:00
Mitchell Syer
8c4a2cb529 Add chapter lastReadAt to backups as BackupHistory (#1477)
* Add chapter lastReadAt to backups as BackupHistory

* MaxOrNull
2025-06-28 17:04:04 -04:00
Mitchell Syer
52201e2488 Add private to trackrecords filter (#1468)
* Add private to trackrecords filter

* Remove sourceMapping from base

* Format
2025-06-26 22:04:26 -04:00
Mitchell Syer
ee9de376a3 Fix new private parameter in tracking backup (#1463)
* Fix new private parameter in tracking backup

* Cleanup uneeded models
2025-06-22 20:17:31 -04:00
Mitchell Syer
b54dc6f967 Fix Tracking DisplayScore (#1461) 2025-06-22 15:53:25 -04:00
Mitchell Syer
abea85d831 Update Tracking Backend (#1457)
* Update Tracking Library

* Update Bangumi

* Update Anilist

* Update MangaUpdates

* Update MAL

* Add private to bind track

* Use null

* Remove old nullable

* Remove custom implementation of supportsTrackDeletion

* Add private to updateTrack

* Some descriptions

* Another description
2025-06-22 10:38:22 -04:00
schroda
149b549d8d Handle chapter marked as downloaded without downloaded files (#1448)
In case a chapter was marked as downloaded but did not have any downloaded files, an uncaught exception was thrown
2025-06-15 17:15:10 -04:00
schroda
4086a73727 Feature/backup suwayomi data (#1430)
* Export meta data

* Import meta data

* Add missing "opdsUseBinaryFileSize" setting to gql

* Export server settings

* Import server settings

* Streamline server config enum handling

* Use "restore amount" in backup import progress
2025-06-15 17:14:13 -04:00
schroda
ee4c852f1b Always update manga thumbnail on fetch (#1429)
It's possible that the cover changed, but the url is still the same.
In that case the cover never gets updated unless the downloaded/cached file gets deleted
2025-06-12 11:50:18 -04:00
schroda
1a5d334f6c Delete thumbnails during backup import (#1428)
Was accidentally removed with 633ea97848
2025-06-12 11:49:34 -04:00
schroda
7d72ff3514 Fix extracting "startDate" (#1427)
Value of json object was never properly accessed.
Start date can have different formats (yyyy-MM, yyyy-MM-dd) so it's not feasible to format it
2025-06-12 11:49:14 -04:00
schroda
e224e91100 Dequeue downloads of removed chapters (#1426)
Otherwise, graphql errors will be caused because the chapters won't be found and are unexpectedly null.

fixes #1358
2025-06-12 11:48:27 -04:00
schroda
ecea2ecdf5 Fix/initial scheduling of global update (#1416)
* Fix setting initial global update delay

In case no update has run yet, and the "last automated update" defaulted to 0, the calculation always resulted in a multiple of the interval.

This resulted for e.g., interval 24, to always be scheduled at 00:00.
For e.g., interval 6, it was always one of the following times: 00:00, 06:00, 12:00, 18:00; depending on the current system time.

* Delete the existing "last automated update time"

So that the update will be triggered based on the time the server got started again after the update.

* Extract migrations into separate functions

* Cleanup migration execution logic
2025-06-12 11:44:38 -04:00
schroda
1d7a60b630 Automatically truncate required varchar columns (#1423) 2025-06-12 11:39:19 -04:00
schroda
89421946af Properly deschedule active tasks (#1413) 2025-05-26 20:45:48 -04:00
Edge At Zero
d0f79ca473 Fix: Validate zipEntry directories during extension asset decompression (#1407) 2025-05-26 20:45:20 -04:00
Mitchell Syer
ec870759cf Add highest numbered chapter function in MangaType (#1397)
* Add highest numbered chapter function in MangaType

* Fix name
2025-05-22 19:58:09 -04:00
Shirish
0405a535c7 Feat: Adds OPDS Chapter Filtering/Ordering (#1392)
* Adds server level configs for OPDS

* PR comments

* Refactor server-reference.conf (itemsPerPage range)

* Coerce itemsPerPage (10, 5000) and default invalid sort orders to DESC

* Coerce itemsPerPage (10, 5000) and default invalid sort orders to DESC

* Change opdsChapterSortOrder type to Enum(SortOrder)

* Fix serialization of SortOrderEnum & Add `opdsShowOnlyDownloadedChapters` config
2025-05-22 19:57:55 -04:00
Mitchell Syer
538bd3f126 Improve Downloads Handling (#1387)
* Improve Downloads Handling

* Update known pagecount for downloaded chapters

* Get fresh data for downloadReady

* Format

* Assume downloaded if first page is found

* Filter out ComicInfoFile
2025-05-16 15:57:53 -04:00
Mitchell Syer
336f985894 Fix Downloaded pages with no cached pages from source (#1386) 2025-05-16 12:45:39 -04:00
schroda
59d2151c92 Prevent duplicated chapter pages (#1353)
In case "ChapterForDownload#asDownloadReady" was called in quick succession, the page list got inserted twice.

This caused problems with getting the images from the rest endpoint, because they are selected by sorting them by asc index and selecting the page by using the provided index as an offset.

This, however, only works as long as there are no duplicates, otherwise, page indexes 1, 2; 3, 4; 5, 6; ... will just return the same page.
2025-04-27 18:54:54 -04:00
KAAAsS
f8d73819ea [Feature] Support Bangumi Tracker (#1343)
* feat: Support Bangumi Tracker

Credits: Andreas, AntsyLich, Caleb Morris, Gauthier, MCAxiaz, MajorTanya, NarwhalHorns, arkon, fei long, jmir1, mutsumi, stevenyomi

* Use Suwayomi api keys

---------

Co-authored-by: Mitchell Syer <Syer10@users.noreply.github.com>
2025-04-12 19:34:04 -04:00
schroda
3167d8aa15 Fix/startup jvm error after installation update via msi (#1229)
* Remove existing installations with msi installer

* Remove unused x86 wxs file

* Uninstall old msi versions with different upgrade code

* Progress but error 2721 happens on install

* Remove added uninstall previous version wxs stuff

* Use revision as patch number

MSI only uninstalls previous versions in case the version number changed (it only checks the first three numbers (major, minor, patch)).
Thus, to prevent each preview install to result in it getting registered as a new "app" and for it to uninstall the old versions, we have to change the version on each release.

* Deprecate "BuildConfig.REVISION"

* Remove outdated env vars

---------

Co-authored-by: Syer10 <syer10@users.noreply.github.com>
2025-04-06 15:09:56 -04:00
schroda
226fad5594 Remove "default" category from backups (#1307)
Restoring a suwayomi backup in mihon created a category named "Default"
2025-03-22 19:34:57 -04:00
schroda
439e0c8284 Emit only updater job changes instead of full status (#1302)
The update subscription emitted the full update status, which, depending on how big the status was, took forever because the graphql subscription does not support data loader batching, causing it to run into the n+1 problem
2025-03-22 19:34:43 -04:00
Shirish
95d9293fe0 Add support for opds-pse for undownloaded chapters (#1278)
* Add OPDS page streaming for undownloaded chapters

* Add [D] in chapter title prefix when isDownloaded

* Removed Chapter.isDownloaded check in query for other opds endpoints

* Add chapter progression tracking for streaming and refactor code

* dd  Unicode for chapters with 0 pages [post pageRefresh]

* Add Library Updates feed and remove redundant metadata fetching for OPDS chapters and manga

* Address PR comments & add chapter markAsRead for cbzDownload

* Address PR comment/s

* Rem. markAsRead for chapter download

* Rem. markAsRead for chapter download

---------

Co-authored-by: ShowY <showypro@gmail.com>
2025-03-08 11:31:07 -05:00
Constantin Piber
3be165a551 Initial import of Kitsu tracker (#1297)
* Initial import of Kitsu tracker

Based on Mihon 6c6ea84509cc1bd859c880bebbc69067a241b358 because its
successor 9f99f03 relies on incompatible changes

* Kitsu: Avoid stupid long/int cast
2025-03-08 11:30:59 -05:00
schroda
633ea97848 Feature/optimize backup import (#1270)
* Optimize restoring manga chapters

* Streamline restoring manga data

* Optimize restoring manga trackers

* Simplify passing manga category restore data

* Properly prevent mangas from getting added to default category

76595233fc never actually worked...

* Extract logic to add manga to categories from gql mutation

* Optimize restoring manga categories

* Optimize restoring categories
2025-02-16 13:00:26 -05:00
schroda
36cb899b91 Prevent chapter lastReadPage coerceIn error (#1272)
coerceIn throws an error in case the max value is less than the min value ("Cannot coerce value to an empty range: maximum <max> is less than minimum <min>")

Regression from c8bd39b4bf
2025-02-14 23:05:04 -05:00
schroda
c8bd39b4bf Prevent negative lastPageRead values (#1267)
Co-authored-by: Mitchell Syer <Syer10@users.noreply.github.com>
2025-02-14 19:06:54 -05:00
schroda
733ba16af2 Fix/backup with duplicated chapters failure (#1269)
* Extract logic to restore manga chapters into function

* Extract logic to restore manga categories into function

* Extract logic to restore manga trackers into function

* Handle duplicated chapters in backup

In case a backup contained duplicated chapters for a manga, the manga failed to restore since the ChapterTable has a unique constraint to prevent multiple chapters with the same "url" and "mangaId"
2025-02-14 19:02:02 -05:00
schroda
37f57c0c55 Prevent marking chapter as downloaded without pages (#1268)
The Downloader marked chapters without any pages incorrectly as downloaded.
2025-02-14 19:01:46 -05:00
Zeedif
26aa684300 Add support for OPDS v1.2 to browse stored CBZ files (#1257)
* Añadiendo algunos cambios iniciales para probar OPDS

* Add suport to OPDS v1.2

* Added support for OPDS-PSE and reorganized controllers

* Rename chapterIndex to chapterId in the API and controller, and update descriptions in OPDS

* Refactor OPDS to use formatted timestamps and proxy thumbnail URLs

* Refactor OPDS to use formatted timestamps and proxy thumbnail URLs

* Update Manga API to download chapters cbz using only chapterId and improve chapter download query

* Optimize OPDS queries

* Update Manga API to download chapters cbz using only chapterId and improve chapter download query

* Optimize OPDS queries

* Use SourceDataClass to map sources and optimize thumbnail URL retrieval

* Kotlin lint errors in ChapterDownloadHelper and Opds

* Kotlin lint errors in ChapterDownloadHelper and Opds
2025-02-08 11:53:06 -05:00
Mitchell Syer
0b192cfa52 Normalize Paths (#1245)
* Normalize Paths

* Formatting

* Different format
2025-01-23 09:36:10 -05:00
schroda
de942440e3 Handle missing track search results on bind (#1196)
It's possible that a manga is bound to a tracker while there is no search result.
This happens when e.g. restoring a backup which includes track bindings for which there was never a tracker search.

In that case when trying to e.g. copy the binding to another manga, the mutation would fail due to not finding a search result.
These cases can be handled by additionally checking the TrackRecordTable to get the necessary track info.
2024-12-10 19:50:16 -05:00
Mitchell Syer
2e3af25dd4 Fix usage of deprecated functions (#1192)
* Fix usage of deprecated functions

* lint

* Lint

* Another
2024-12-07 23:56:42 -05:00
schroda
1d541a30ae Feature/update to exposed v0.57.0 (#1150)
* Update to exposed-migrations v3.5.0

* Update to kotlin-logging v7.0.0

* Update to exposed v0.46.0

* Update to exposed v0.47.0

* Update to exposed v0.55.0

* Update to exposed v0.56.0

* Update to exposed v0.57.0
2024-12-07 23:49:11 -05:00
Mitchell Syer
088552bf56 Fix Deprecation Warnings (#1187) 2024-12-07 14:13:06 -05:00
are-are-are
372b56bb1b add manga description (#1165) 2024-11-23 18:22:21 -05:00
Mitchell Syer
fb51834153 Fix RealUrl (#1162) 2024-11-20 22:57:42 -05:00
renovate[bot]
9cd8cb3d54 Update dependency io.javalin:javalin to v6 (#1152)
* Update dependency io.javalin:javalin to v6

* Simple compile fixes

* Simple compile fixes pass 2

* Add results to futures

* Setup jetty server and api routes

* Setup Cors

* Setup basic auth

* Documentation stubs

* Replace chapter mutex cache

* Fix compile

* Disable Jetty Logging

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Syer10 <syer10@users.noreply.github.com>
2024-11-17 15:00:53 -05:00
schroda
320a0971b4 Fix/gql download subscription (#1137)
* Properly set download update type on exceptions

* Always send FINISHED download update to client for deprecated subscription

By the time the status was sent to the client, the finished download item was already removed from the queue, causing the client to never get the latest status, thus, having an outdated cache

Regression introduced with 168b76cb0c
2024-11-15 21:30:09 -05:00
Mitchell Syer
9a51472726 Update Titles from the Source (#1115)
* Update Titles from the source

* Properly keep null fields
2024-11-14 18:08:31 -05:00
Mitchell Syer
0670f298cd Switch from Kodein to Koin (#1112)
* Switch from Kodein to Koin

* Ktlint
2024-11-14 18:08:19 -05:00
schroda
aa1e98544b Fix/invalid server settings gql mutation request (#1092)
* Validate setting values on mutation

* Handle invalid negative setting values

* Ensure at least one source is downloading at all times

* Prevent possible IllegalArgumentException

The "serverConfig.maxSourcesInParallel" value could have changed after the if-condition
2024-11-14 18:08:07 -05:00
schroda
168b76cb0c Feature/graphql download queue subscription send only updates (#1011)
* Emit only download changes instead of full status

The download subscription emitted the full download status, which, depending on how big the queue was, took forever because the graphql subscription does not support data loader batching, causing it to run into the n+1 problem

* Rename "DownloadManager#status" to "DownloadManager#updates"

* Add initial queue to download subscription type

Adds the current queue at the time of sending the initial message.
This field is null for all following messages after the initial one

* Optionally limit and omit download updates

To prevent the n+1 dataloader issue, the max number of updates included in the download subscription can be limited.
This way, the problem will be circumvented and instead, the latest download status should be (re-)fetched via the download status query, which does not run into this problem.

* Formatting
2024-11-14 18:07:14 -05:00