Commit Graph

380 Commits

Author SHA1 Message Date
schroda
ee4c852f1b Always update manga thumbnail on fetch (#1429)
It's possible that the cover changed, but the url is still the same.
In that case the cover never gets updated unless the downloaded/cached file gets deleted
2025-06-12 11:50:18 -04:00
schroda
1a5d334f6c Delete thumbnails during backup import (#1428)
Was accidentally removed with 633ea97848
2025-06-12 11:49:34 -04:00
schroda
7d72ff3514 Fix extracting "startDate" (#1427)
Value of json object was never properly accessed.
Start date can have different formats (yyyy-MM, yyyy-MM-dd) so it's not feasible to format it
2025-06-12 11:49:14 -04:00
schroda
e224e91100 Dequeue downloads of removed chapters (#1426)
Otherwise, graphql errors will be caused because the chapters won't be found and are unexpectedly null.

fixes #1358
2025-06-12 11:48:27 -04:00
schroda
ecea2ecdf5 Fix/initial scheduling of global update (#1416)
* Fix setting initial global update delay

In case no update has run yet, and the "last automated update" defaulted to 0, the calculation always resulted in a multiple of the interval.

This resulted for e.g., interval 24, to always be scheduled at 00:00.
For e.g., interval 6, it was always one of the following times: 00:00, 06:00, 12:00, 18:00; depending on the current system time.

* Delete the existing "last automated update time"

So that the update will be triggered based on the time the server got started again after the update.

* Extract migrations into separate functions

* Cleanup migration execution logic
2025-06-12 11:44:38 -04:00
schroda
1d7a60b630 Automatically truncate required varchar columns (#1423) 2025-06-12 11:39:19 -04:00
Zeedif
61f429896c feat(opds): implement full internationalization and refactor feed gen… (#1405)
* feat(opds): implement full internationalization and refactor feed generation

This commit introduces a comprehensive internationalization (i18n) framework
and significantly refactors the OPDS v1.2 implementation for improved
robustness, spec compliance, and localization.

Key changes:

Internationalization (`i18n`):
- Introduces `LocalizationService` to manage translations:
    - Loads localized strings from JSON files (e.g., `en.json`, `es.json`)
      stored in a new `i18n` data directory.
    - Default `en.json` and `es.json` files are bundled and copied from
      resources on first run if not present.
    - Supports template resolution with `$t()` cross-references, locale
      fallbacks (to "en" by default), and argument interpolation ({{placeholder}}).
- `ServerSetup` now initializes the `i18n` directory and `LocalizationService`.

OPDS Refactor & Enhancements:
- Replaces the previous `Opds.kt` and `OpdsDataClass.kt` with a new
  `OpdsFeedBuilder.kt` and a set of more granular, spec-aligned XML
  models (e.g., `OpdsFeedXml`, `OpdsEntryXml`, `OpdsLinkXml`).
- Integrates `LocalizationService` throughout all OPDS feeds:
    - All user-facing text (feed titles, entry titles, summaries,
      link titles, facet labels for sorting/filtering) is now localized.
    - Adds a `lang` query parameter to all OPDS endpoints to allow
      clients to request a specific UI language.
    - Uses the `Accept-Language` header as a fallback for language detection.
- The OpenSearch description (`/search` endpoint) is now localized and
  its template URL includes the determined language.
- Centralizes OPDS constants (namespaces, link relations, media types)
  in `OpdsConstants.kt`.
- Adds utility classes `OpdsDateUtil.kt`, `OpdsStringUtil.kt`, and
  `OpdsXmlUtil.kt` for common OPDS tasks.
- `MangaDataClass` now includes `sourceLang` to provide the content
  language of the manga in OPDS entries (`<dc:language>`).
- Updates OpenAPI documentation for OPDS endpoints with more detail
  and includes the new `lang` parameter.

Configuration:
- Adds `useBinaryFileSizes` server configuration option. File sizes in
  OPDS feeds now respect this setting (e.g., MiB vs MB), utilized via
  `OpdsStringUtil.formatFileSizeForOpds`.

This major refactor addresses the request for internationalization
originally mentioned in PR #1257 ("it would be great if messages were
adapted based on the user's language settings"). It builds upon the
foundational OPDS work in #1257 and subsequent enhancements in #1262,
#1263, #1278, and #1392, providing a more stable and extensible
OPDS implementation. Features like localized facet titles from #1392
are now fully integrated with the i18n system.

This resolves long-standing requests for better OPDS support (e.g., issue #769)
by making feeds more user-friendly, accessible, and standards-compliant,
also improving the robustness of features requested in #1390 (resolved by #1392)
and addressing underlying data needs for issues like #1265 (related to #1277, #1278).

* fix(opds): revert MIME type to application/xml for browser compatibility

* fix(opds): use chapter index for metadata feed and correct link relation

- Change `getChapterMetadataFeed` to use `chapterIndexFromPath` (sourceOrder)
  instead of `chapterIdFromPath` for fetching chapter data, ensuring
  consistency with how chapters are identified in manga feeds.
- Add error handling for cases where manga or chapter by index is not found.
- Correct OPDS link relation for chapter detail/fetch link in non-metadata
  chapter entries from `alternate` to `subsection` as per OPDS spec
  for navigation to more specific content or views.

* Use Moko-Resources

* Format

* Forgot the Languages.json

* refactor(opds)!: restructure OPDS feeds and introduce data repositories

This commit significantly refactors the OPDS v1.2 implementation by introducing dedicated repository classes for data fetching and by restructuring the feed generation logic for clarity and maintainability. The `chapterId` path parameter for chapter metadata feeds has been changed to `chapterIndex` (sourceOrder) to align with how chapters are identified in manga feeds.

BREAKING CHANGE: The OPDS endpoint for chapter metadata has changed from `/api/opds/v1.2/manga/{mangaId}/chapter/{chapterId}/fetch` to `/api/opds/v1.2/manga/{mangaId}/chapter/{chapterIndex}/fetch`. Clients will need to update to use the chapter's source order (index) instead of its database ID.

Key changes:
- Introduced `MangaRepository`, `ChapterRepository`, and `NavigationRepository` to encapsulate database queries and data transformation logic for OPDS feeds.
- Moved data fetching logic from `OpdsFeedBuilder` to these new repositories.
- `OpdsFeedBuilder` now primarily focuses on constructing the XML feed structure using DTOs provided by the repositories.
- Renamed `OpdsMangaAcqEntry.thumbnailUrl` to `rawThumbnailUrl` for clarity.
- Added various DTOs (e.g., `OpdsRootNavEntry`, `OpdsMangaDetails`, `OpdsChapterListAcqEntry`) to define clear data contracts between repositories and the feed builder.
- Simplified `OpdsV1Controller` by reorganizing feed endpoints into logical groups (Main Navigation, Filtered Acquisition, Item-Specific).
- Updated `OpdsAPI` to reflect the path parameter change for chapter metadata (`chapterIndex` instead of `chapterId`).
- Added `slugify()` utility to `OpdsStringUtil` for creating URL-friendly genre IDs.
- Standardized localization keys for root feed entry descriptions to use `*.entryContent` instead of `*.description`.
- Added `server.generated.BuildConfig` (likely from build process).

* style(opds): apply ktlint fixes

* Delete server/bin

* refactor(i18n): remove custom LocalizationService initialization

* refactor(i18n): remove unused imports from ServerSetup

* refactor(model): remove sourceLang from MangaDataClass

* refactor(opds): rename OPDS binary file size config property

- Rename `useBinaryFileSizes` to `opdsUseBinaryFileSizes` in code and config
- Update related condition check in formatFileSizeForOpds

BREAKING CHANGE: Existing server configurations using `server.useBinaryFileSizes` need to migrate to `server.opdsUseBinaryFileSizes`

* refactor(opds): improve OPDS endpoint structure and documentation

- Restructure endpoint paths for better resource hierarchy
- Add descriptive comments for each feed type and purpose
- Rename `/fetch` endpoint to `/metadata` for clarity
- Standardize feed naming conventions in route definitions

BREAKING CHANGE: Existing OPDS client integrations using old endpoint paths (`/manga/{mangaId}` and `/chapter/{chapterIndex}/fetch`) require updates to new paths (`/manga/{mangaId}/chapters` and `/chapter/{chapterIndex}/metadata`)

* fix(opds): Apply review suggestions for localization and comments

* Fix

* fix(opds): Update chapter links to include 'chapters' and 'metadata' in URLs

---------

Co-authored-by: Syer10 <syer10@users.noreply.github.com>
2025-05-26 20:46:14 -04:00
schroda
89421946af Properly deschedule active tasks (#1413) 2025-05-26 20:45:48 -04:00
Edge At Zero
d0f79ca473 Fix: Validate zipEntry directories during extension asset decompression (#1407) 2025-05-26 20:45:20 -04:00
Mitchell Syer
ec870759cf Add highest numbered chapter function in MangaType (#1397)
* Add highest numbered chapter function in MangaType

* Fix name
2025-05-22 19:58:09 -04:00
Shirish
0405a535c7 Feat: Adds OPDS Chapter Filtering/Ordering (#1392)
* Adds server level configs for OPDS

* PR comments

* Refactor server-reference.conf (itemsPerPage range)

* Coerce itemsPerPage (10, 5000) and default invalid sort orders to DESC

* Coerce itemsPerPage (10, 5000) and default invalid sort orders to DESC

* Change opdsChapterSortOrder type to Enum(SortOrder)

* Fix serialization of SortOrderEnum & Add `opdsShowOnlyDownloadedChapters` config
2025-05-22 19:57:55 -04:00
Mitchell Syer
538bd3f126 Improve Downloads Handling (#1387)
* Improve Downloads Handling

* Update known pagecount for downloaded chapters

* Get fresh data for downloadReady

* Format

* Assume downloaded if first page is found

* Filter out ComicInfoFile
2025-05-16 15:57:53 -04:00
Mitchell Syer
336f985894 Fix Downloaded pages with no cached pages from source (#1386) 2025-05-16 12:45:39 -04:00
schroda
785c0469ac Fix/m0045 prevent duplicated chapter pages migration (#1361)
* Fix "imageUrl" column name in migration

* Rename column "imageUrl" to "IMAGE_URL" of table "Page"
2025-04-27 19:33:15 -04:00
schroda
59d2151c92 Prevent duplicated chapter pages (#1353)
In case "ChapterForDownload#asDownloadReady" was called in quick succession, the page list got inserted twice.

This caused problems with getting the images from the rest endpoint, because they are selected by sorting them by asc index and selecting the page by using the provided index as an offset.

This, however, only works as long as there are no duplicates, otherwise, page indexes 1, 2; 3, 4; 5, 6; ... will just return the same page.
2025-04-27 18:54:54 -04:00
KAAAsS
f8d73819ea [Feature] Support Bangumi Tracker (#1343)
* feat: Support Bangumi Tracker

Credits: Andreas, AntsyLich, Caleb Morris, Gauthier, MCAxiaz, MajorTanya, NarwhalHorns, arkon, fei long, jmir1, mutsumi, stevenyomi

* Use Suwayomi api keys

---------

Co-authored-by: Mitchell Syer <Syer10@users.noreply.github.com>
2025-04-12 19:34:04 -04:00
schroda
3167d8aa15 Fix/startup jvm error after installation update via msi (#1229)
* Remove existing installations with msi installer

* Remove unused x86 wxs file

* Uninstall old msi versions with different upgrade code

* Progress but error 2721 happens on install

* Remove added uninstall previous version wxs stuff

* Use revision as patch number

MSI only uninstalls previous versions in case the version number changed (it only checks the first three numbers (major, minor, patch)).
Thus, to prevent each preview install to result in it getting registered as a new "app" and for it to uninstall the old versions, we have to change the version on each release.

* Deprecate "BuildConfig.REVISION"

* Remove outdated env vars

---------

Co-authored-by: Syer10 <syer10@users.noreply.github.com>
2025-04-06 15:09:56 -04:00
schroda
226fad5594 Remove "default" category from backups (#1307)
Restoring a suwayomi backup in mihon created a category named "Default"
2025-03-22 19:34:57 -04:00
schroda
439e0c8284 Emit only updater job changes instead of full status (#1302)
The update subscription emitted the full update status, which, depending on how big the status was, took forever because the graphql subscription does not support data loader batching, causing it to run into the n+1 problem
2025-03-22 19:34:43 -04:00
Shirish
95d9293fe0 Add support for opds-pse for undownloaded chapters (#1278)
* Add OPDS page streaming for undownloaded chapters

* Add [D] in chapter title prefix when isDownloaded

* Removed Chapter.isDownloaded check in query for other opds endpoints

* Add chapter progression tracking for streaming and refactor code

* dd  Unicode for chapters with 0 pages [post pageRefresh]

* Add Library Updates feed and remove redundant metadata fetching for OPDS chapters and manga

* Address PR comments & add chapter markAsRead for cbzDownload

* Address PR comment/s

* Rem. markAsRead for chapter download

* Rem. markAsRead for chapter download

---------

Co-authored-by: ShowY <showypro@gmail.com>
2025-03-08 11:31:07 -05:00
Constantin Piber
3be165a551 Initial import of Kitsu tracker (#1297)
* Initial import of Kitsu tracker

Based on Mihon 6c6ea84509cc1bd859c880bebbc69067a241b358 because its
successor 9f99f03 relies on incompatible changes

* Kitsu: Avoid stupid long/int cast
2025-03-08 11:30:59 -05:00
schroda
633ea97848 Feature/optimize backup import (#1270)
* Optimize restoring manga chapters

* Streamline restoring manga data

* Optimize restoring manga trackers

* Simplify passing manga category restore data

* Properly prevent mangas from getting added to default category

76595233fc never actually worked...

* Extract logic to add manga to categories from gql mutation

* Optimize restoring manga categories

* Optimize restoring categories
2025-02-16 13:00:26 -05:00
schroda
36cb899b91 Prevent chapter lastReadPage coerceIn error (#1272)
coerceIn throws an error in case the max value is less than the min value ("Cannot coerce value to an empty range: maximum <max> is less than minimum <min>")

Regression from c8bd39b4bf
2025-02-14 23:05:04 -05:00
schroda
c8bd39b4bf Prevent negative lastPageRead values (#1267)
Co-authored-by: Mitchell Syer <Syer10@users.noreply.github.com>
2025-02-14 19:06:54 -05:00
schroda
733ba16af2 Fix/backup with duplicated chapters failure (#1269)
* Extract logic to restore manga chapters into function

* Extract logic to restore manga categories into function

* Extract logic to restore manga trackers into function

* Handle duplicated chapters in backup

In case a backup contained duplicated chapters for a manga, the manga failed to restore since the ChapterTable has a unique constraint to prevent multiple chapters with the same "url" and "mangaId"
2025-02-14 19:02:02 -05:00
schroda
37f57c0c55 Prevent marking chapter as downloaded without pages (#1268)
The Downloader marked chapters without any pages incorrectly as downloaded.
2025-02-14 19:01:46 -05:00
Zeedif
26aa684300 Add support for OPDS v1.2 to browse stored CBZ files (#1257)
* Añadiendo algunos cambios iniciales para probar OPDS

* Add suport to OPDS v1.2

* Added support for OPDS-PSE and reorganized controllers

* Rename chapterIndex to chapterId in the API and controller, and update descriptions in OPDS

* Refactor OPDS to use formatted timestamps and proxy thumbnail URLs

* Refactor OPDS to use formatted timestamps and proxy thumbnail URLs

* Update Manga API to download chapters cbz using only chapterId and improve chapter download query

* Optimize OPDS queries

* Update Manga API to download chapters cbz using only chapterId and improve chapter download query

* Optimize OPDS queries

* Use SourceDataClass to map sources and optimize thumbnail URL retrieval

* Kotlin lint errors in ChapterDownloadHelper and Opds

* Kotlin lint errors in ChapterDownloadHelper and Opds
2025-02-08 11:53:06 -05:00
Mitchell Syer
0b192cfa52 Normalize Paths (#1245)
* Normalize Paths

* Formatting

* Different format
2025-01-23 09:36:10 -05:00
schroda
de942440e3 Handle missing track search results on bind (#1196)
It's possible that a manga is bound to a tracker while there is no search result.
This happens when e.g. restoring a backup which includes track bindings for which there was never a tracker search.

In that case when trying to e.g. copy the binding to another manga, the mutation would fail due to not finding a search result.
These cases can be handled by additionally checking the TrackRecordTable to get the necessary track info.
2024-12-10 19:50:16 -05:00
Mitchell Syer
2e3af25dd4 Fix usage of deprecated functions (#1192)
* Fix usage of deprecated functions

* lint

* Lint

* Another
2024-12-07 23:56:42 -05:00
schroda
1d541a30ae Feature/update to exposed v0.57.0 (#1150)
* Update to exposed-migrations v3.5.0

* Update to kotlin-logging v7.0.0

* Update to exposed v0.46.0

* Update to exposed v0.47.0

* Update to exposed v0.55.0

* Update to exposed v0.56.0

* Update to exposed v0.57.0
2024-12-07 23:49:11 -05:00
Mitchell Syer
088552bf56 Fix Deprecation Warnings (#1187) 2024-12-07 14:13:06 -05:00
are-are-are
372b56bb1b add manga description (#1165) 2024-11-23 18:22:21 -05:00
Mitchell Syer
fb51834153 Fix RealUrl (#1162) 2024-11-20 22:57:42 -05:00
renovate[bot]
9cd8cb3d54 Update dependency io.javalin:javalin to v6 (#1152)
* Update dependency io.javalin:javalin to v6

* Simple compile fixes

* Simple compile fixes pass 2

* Add results to futures

* Setup jetty server and api routes

* Setup Cors

* Setup basic auth

* Documentation stubs

* Replace chapter mutex cache

* Fix compile

* Disable Jetty Logging

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Syer10 <syer10@users.noreply.github.com>
2024-11-17 15:00:53 -05:00
schroda
320a0971b4 Fix/gql download subscription (#1137)
* Properly set download update type on exceptions

* Always send FINISHED download update to client for deprecated subscription

By the time the status was sent to the client, the finished download item was already removed from the queue, causing the client to never get the latest status, thus, having an outdated cache

Regression introduced with 168b76cb0c
2024-11-15 21:30:09 -05:00
Mitchell Syer
9a51472726 Update Titles from the Source (#1115)
* Update Titles from the source

* Properly keep null fields
2024-11-14 18:08:31 -05:00
Mitchell Syer
0670f298cd Switch from Kodein to Koin (#1112)
* Switch from Kodein to Koin

* Ktlint
2024-11-14 18:08:19 -05:00
schroda
aa1e98544b Fix/invalid server settings gql mutation request (#1092)
* Validate setting values on mutation

* Handle invalid negative setting values

* Ensure at least one source is downloading at all times

* Prevent possible IllegalArgumentException

The "serverConfig.maxSourcesInParallel" value could have changed after the if-condition
2024-11-14 18:08:07 -05:00
schroda
168b76cb0c Feature/graphql download queue subscription send only updates (#1011)
* Emit only download changes instead of full status

The download subscription emitted the full download status, which, depending on how big the queue was, took forever because the graphql subscription does not support data loader batching, causing it to run into the n+1 problem

* Rename "DownloadManager#status" to "DownloadManager#updates"

* Add initial queue to download subscription type

Adds the current queue at the time of sending the initial message.
This field is null for all following messages after the initial one

* Optionally limit and omit download updates

To prevent the n+1 dataloader issue, the max number of updates included in the download subscription can be limited.
This way, the problem will be circumvented and instead, the latest download status should be (re-)fetched via the download status query, which does not run into this problem.

* Formatting
2024-11-14 18:07:14 -05:00
Mitchell Syer
654a3cc7ed Use Backup.serializer() (#1088) 2024-09-16 21:18:01 -04:00
Mitchell Syer
841cdc474f Remove Broken Sources and Broken History (#1084) 2024-09-15 00:19:47 -04:00
schroda
0adbea3a43 Remove manga artist, author length limit (#1080) 2024-09-15 00:10:23 -04:00
Syer10
6c1fbfa63b [skip ci] Formatting 2024-09-03 21:37:18 -04:00
Antoine Aflalo
9a74ae5844 feat(comicinfo): add date fields to comic info (#1021)
* feat(comicinfo): add date fields to comic info

This will be parsed by Komga, Kavita etc ... and any other library management to also have the date of the chapter.

* refactor: improve code readability

Co-authored-by: Mitchell Syer <Syer10@users.noreply.github.com>

---------

Co-authored-by: Mitchell Syer <Syer10@users.noreply.github.com>
2024-08-31 18:55:13 -04:00
schroda
5dced82e5a Fix/missed automated task execution failure crashes server on startup (#1019)
* Catch automated backup task errors

* Catch automated udpate task errors

* Catch automated webui update task errors
2024-08-31 18:54:51 -04:00
schroda
9a1e4df408 Fix/server startup blocked by synchronous tasks (#1018)
* Launch missed auto backup task in background

* Launch missed auto global update task in background

* Launch missed auto webui update check task in background
2024-08-31 18:54:41 -04:00
schroda
5b08b81239 Initialize manga on add to library (#1016)
In case a manga gets added to the library which has not been initialized yet, it should be tried to initialize it.
Since it's not an error to have uninitialized manga in the library, this can be done in the background via the updater and the client receives the updated data via the update subscription.
2024-08-31 18:54:30 -04:00
schroda
7fac538ba3 Initialize uninitialized manga during global update (#1015)
They were only initialized in case the setting to refresh manga metadata during an update was enabled.
However, this should always be done for uninitialized manga, regardless of the setting.

06bfc33e72 prevents uninitialized manga from getting filtered out, however, it did not ensure to initialize the manga
2024-08-31 18:54:18 -04:00
schroda
ef6be74ec2 Fix/chapter downloaded check (#1012)
* Properly check for first page in cbz files

The download check for cbz files only checked if the archive existed but didn't check for the first page

* Streamline getImageImpl of ChapterDownloadProviders

* Exclude comic info file from page list

In case the download folder did not contain any page files, only the comic info file existed, which caused the download check to incorrectly detect the first page

* Add logging to ChapterForDownload#asDownloadReady
2024-08-31 18:54:06 -04:00