Commit Graph

62 Commits

Author SHA1 Message Date
Constantin Piber
81fe3f0108 Stop dumping cookies in the console (#1490) 2025-07-02 13:35:56 -04:00
Constantin Piber
a79dc580a5 Browser Webview (#1486)
* WebView: Add initial controller

Co-authored-by: Mitchell Syer <Syer10@users.noreply.github.com>

* WebView: Prepare page

* WebView: Basic HTML setup

* WebView: Improve navigation

* WebView: Refactor message class deserialization

* WebView: Refactor event message serialization

* WebView: Handle click events

* WebView: Fix events after refactor

* WebView: Fix normalizing of URLs

* WebView: HTML remove navigation buttons

* WebView: Handle more events

* WebView: Handle document change in events

* WebView: Refactor to send mutation events

* WebView: More mouse events

* WebView: Include bubbles, cancelable in event

Those seem to be important

* WebView: Attempt to support nested iframe

* WebView: Handle long titles

* WebView: Avoid setting invalid url

* WebView: Send mousemove

* WebView: Start switch to canvas-based render

* WebView: Send on every render

* WebView: Dynamic size

* WebView: Keyboard events

* WebView: Handle mouse events in CEF

This is important because JS can't click into iFrames, meaning the
previous solution doesn't work for captchas

* WebView: Cleanup

* WebView: Cleanup 2

* WebView: Document title

* WebView: Also send title on address change

* WebView: Load and flush cookies from store

* WebView: remove outdated TODOs

* Offline WebView: Load cookies from store

* Cleanup

* Add KcefCookieManager, need to figure out how to inject it

* ktLintFormat

* Fix a few cookie bugs

* Fix Webview on Windows

* Minor cleanup

* WebView: Remove /tmp image write, lint

* Remove custom cookie manager

* Multiple cookie fixes

* Minor fix

* Minor cleanup and add support for MacOS meta key

* Get enter working

* WebView HTML: Make responsive for mobile pages

* WebView: Translate touch events to mouse scroll

* WebView: Overlay an actual input to allow typing on mobile

Browsers will only show the keyboard if an input is focused. This also
removes the `tabstop` hack.

* WebView: Protect against occasional NullPointerException

* WebView: Use float for clientX/Y

* WebView: Fix ChromeAndroid being a pain

* Simplify enter fix

* NetworkHelper: Fix cache

* Improve CookieStore url matching, fix another cookie conversion issue

* Move distinctBy

* WebView: Mouse direction toggle

* Remove accidentally copied comment

---------

Co-authored-by: Mitchell Syer <Syer10@users.noreply.github.com>
2025-07-01 17:28:41 -04:00
Constantin Piber
a2fadbe513 Implement WebView via Playwright (#1434)
* Implement Android's Looper

Looper handles thread messaging. This is used by extensions when they
want to enqueue actions e.g. for sleeping while WebView does someting

* Stub WebView

* Continue stubbing ViewGroup for WebView

* Implement WebView via Playwright

* Lint

* Implement request interception

Supports Yidan

* Support WebChromeClient

For Bokugen

* Fix onPageStarted

* Make Playwright configurable

* Subscribe to config changes

* Fix exposing of functions

* Support data urls

* Looper: Fix infinite sleep

* Looper: Avoid killing the loop on exception

Just log it and continue

* Pump playwright's message queue periodically

https://playwright.dev/java/docs/multithreading#pagewaitfortimeout-vs-threadsleep

* Update server/src/main/kotlin/suwayomi/tachidesk/graphql/types/SettingsType.kt

Co-authored-by: Mitchell Syer <Syer10@users.noreply.github.com>

* Stub a KCef WebViewProvider

* Initial Kcef Webview implementation

Still buggy, on the second call it just seems to fall over

* Format, restructure to create browser on load

This is much more consistent, before we would sometimes see errors from
about:blank, which block the actual page

* Implement some small useful properties

* Move inline objects to class

* Handle requests in Kcef

* Move Playwright implementation

* Document Playwright settings, fix deprecated warnings

* Inject default user agent from NetworkHelper

* Move playwright to libs.versions.toml

* Lint

* Fix missing imports after lint

* Update server/src/main/kotlin/suwayomi/tachidesk/server/ServerSetup.kt

Co-authored-by: Mitchell Syer <Syer10@users.noreply.github.com>

* Fix default user agent set/get

Use System.getProperty instead of SystemProperties.get

* Configurable WebView provider implementation

* Simplify Playwright settings init

* Minor cleanup and improvements

* Remove playwright WebView impl

* Document WebView for Linux

---------

Co-authored-by: Mitchell Syer <Syer10@users.noreply.github.com>
2025-06-12 11:38:54 -04:00
schroda
1d541a30ae Feature/update to exposed v0.57.0 (#1150)
* Update to exposed-migrations v3.5.0

* Update to kotlin-logging v7.0.0

* Update to exposed v0.46.0

* Update to exposed v0.47.0

* Update to exposed v0.55.0

* Update to exposed v0.56.0

* Update to exposed v0.57.0
2024-12-07 23:49:11 -05:00
Mitchell Syer
088552bf56 Fix Deprecation Warnings (#1187) 2024-12-07 14:13:06 -05:00
Antoine Aflalo
df1cc2b8e9 fix(flaresolverr): fix cookie expiry for flaresolverr (#1070)
Cookie expiry is returned in seconds, the persistent cache expect it in miliseconds. Cookie is always considered as expired
2024-09-14 09:04:49 -04:00
Syer10
6c1fbfa63b [skip ci] Formatting 2024-09-03 21:37:18 -04:00
Antoine Aflalo
301980ab14 fix(flaresolverr): support possible rewrite flaresolverr (#1020)
This PR:
https://github.com/FlareSolverr/FlareSolverr/pull/1300

Solve a lot of issue with not solving challenge, however, the cookie don't have path, httpOnly, secure and sameSite.

By making them optional that should work for both version of flaresolverr.
2024-08-31 18:55:02 -04:00
AeonLucid
9e006166a8 Add setting to use the flaresolverr response (#990) 2024-07-28 15:57:40 -04:00
AeonLucid
25a62e33a1 Fix PersistentCookieStore for domains with an underscore (#989)
* Fix PersistentCookieStore for domains with an underscore

* Missed one uri
2024-07-28 15:57:30 -04:00
Chance Zibolski
dc124fb15c Make flaresolverr session options configurable (#854)
Signed-off-by: Chance Zibolski <chance.zibolski@gmail.com>
2024-01-24 21:11:53 -05:00
Chance Zibolski
9109d1ca3e Use a session with flaresolverr (#853)
Signed-off-by: Chance Zibolski <chance.zibolski@gmail.com>
2024-01-24 20:44:45 -05:00
Mitchell Syer
63e1082b97 Minor fixes for FlareSolverr (#851)
* Minor fixes for FlareSolverr

* Weird crash but ok
2024-01-24 17:49:51 -05:00
schroda
fc64f47589 Fix/excessive logging (#848)
* Remove log of mangas to update

This logged the full manga data objects in the list with information that is not needed (e.g. description of a manga).
Once a manga gets updated via the updater, it gets logged, which should be enough

* Include manga id in updater log

* Use "toString" to log mangas

* Change "HttpLoggingInterceptor" level to "BASIC"

Was unintentionally merged with d658e07583
2024-01-24 17:49:16 -05:00
Mitchell Syer
562b940d91 Remove dot before cookie (#845) 2024-01-23 19:21:33 -05:00
Mitchell Syer
d658e07583 Implement FlareSolverr (#844)
* Implement FlareSolverr

* Oops
2024-01-23 18:48:55 -05:00
Mitchell Syer
ce42e89e25 Add MangaUpdates (#834) 2024-01-21 11:45:34 -05:00
Mitchell Syer
7b290dc465 Update User Agent (#821) 2024-01-12 23:48:33 -05:00
Mitchell Syer
28e4ac8dcb Remove Playwright (#643) 2024-01-12 23:46:47 -05:00
Mitchell Syer
0d0e735d0e Fix brotli (#818) 2024-01-11 23:09:31 -05:00
Tachimanga
5a178ada74 add trackers support (#720)
* add trackers support

* Cleanup Tracker Code

* Add GraphQL support for Tracking

* Fix lint and deprecation errors

* remove password from logs

* Fixes after merge

* Disable tracking for now

* More disabled

---------

Co-authored-by: Syer10 <syer10@users.noreply.github.com>
2024-01-07 15:07:41 -05:00
Mitchell Syer
9b27d7ee23 Improve Http Client Configuration (#786)
* Improve Http Client Configuration

* Lint
2023-12-08 19:16:38 -05:00
Mitchell Syer
682c364647 Address Build Warnings and Cleanup (#707)
* Address build warnings and cleanup

* Actual name of who defined the protocol

* Remove uneeded detekt supression

* GraphQL Before-After cleanup

* Lint

* Cleanup unused functions

* Fix some discrepancies with the 1.5 source api and fix lang exception
2023-10-15 20:16:30 -04:00
Mitchell Syer
849acfca3d Switch to a new Ktlint Formatter (#705)
* Switch to new Ktlint plugin

* Add ktlintCheck to PR builds

* Run formatter

* Put ktlint version in libs toml

* Fix lint

* Use Zip4Java from libs.toml
2023-10-06 23:38:39 -04:00
Mitchell Syer
c8865ad185 Implement Non-Final 1.5 Extensions API (#699)
* Implement non-final 1.5 extensions API

* Bump lib version max

* Add visibility to preferences

* Add preference visibility
2023-10-04 22:01:45 -04:00
MangaCrushTeam
41fea1d2a0 remove @Synchronized in CloudflareInterceptor.kt for performance (#688) 2023-09-23 16:26:07 -04:00
schroda
321fbe22dd Feature/listen to server config value changes (#617)
* Make server config value changes subscribable

* Make server config value changes subscribable - Update usage

* Add util functions to listen to server config value changes

* Listen to server config value changes - Auto backups

* Listen to server config value changes - Auto global update

* Listen to server config value changes - WebUI auto updates

* Listen to server config value changes - Javalin update ip and port

* Listen to server config value changes - Update socks proxy

* Listen to server config value changes - Update debug log level

* Listen to server config value changes - Update system tray icon

* Update config values one at a time

In case settings are changed in quick succession it's possible that each setting update reverts the change of the previous changed setting because the internal config hasn't been updated yet.

E.g.
1. settingA changed
2. settingB changed
3. settingA updates config file
4. settingB updates config file (internal config hasn't been updated yet with change from settingA)
5. settingA updates internal config (settingA updated)
6. settingB updates internal config (settingB updated, settingA outdated)

now settingA is unchanged because settingB reverted its change while updating the config with its new value

* Always add log interceptor to OkHttpClient

In case debug logs are disabled then the KotlinLogging log level will be set to level > debug and thus, these logs won't get logged

* Rename "maxParallelUpdateRequests" to "maxSourcesInParallel"

* Use server setting "maxSourcesInParallel" for downloads

* Listen to server config value changes - downloads

* Always use latest server settings - Browser

* Always use latest server settings - folders

* [Test] Fix type error
2023-08-12 11:47:41 -04:00
Mitchell Syer
e093fe6a06 Add CookieManager implementation (#635)
* Add CookieManager implementation

* Remove Syncronized

* Rename CookieStore
2023-08-05 20:09:12 -04:00
Mitchell Syer
689847d864 Update dependencies (#611) 2023-08-04 22:48:24 -04:00
schroda
d4f9b0b1bc Feature/log to file (#607)
* Setup "logback" to write to file

To be able to dynamically set the log file save location, logback has to be setup via code instead of a config file

* Log OkHttp via logback

Otherwise, the logs would only get written to the console and thus, not be included in the log file

* Init logback

Has to be done after the config was loaded, otherwise, the root directory would be unknown.
Moved the log of the loaded config to the "applicationSetup" since otherwise, the log would not be included in the log file
2023-07-21 19:53:41 -04:00
Aria Moradi
a027d6df1b disable playwright for v0.6.7 2023-02-12 14:35:11 +03:30
Mitchell Syer
926a53a4b0 add support for Extensions Lib 1.4 (#496)
* Support extensions lib 1.4

* Fix build

* Support UpdateStrategy

* Update extension lib min/max to match Tachiyomi

* Use HttpSource.getMangaUrl and add Chapter.realUrl
2023-02-12 05:49:32 +03:30
Mitchell Syer
4cf7512ee0 Improve Playwright handling (#479)
* Improve playwright

* Move DriverJar.java and Chromium.kt
2023-01-08 01:17:53 +03:30
Aria Moradi
874aaf4e93 fix when playwright fails on providing a UA 2022-12-28 17:14:12 +03:30
Mitchell Syer
ab4d925a5a Get Playwright working (#462)
* Get Playwright working with ShadowJar

* Set system driver implementation

* Minor cleanup

* Fix run gradle task and re-add some removed code

* No need to get the FS if it already exists

* use java implementation

Co-authored-by: Aria Moradi <aria.moradi007@gmail.com>
2022-12-09 21:47:26 +03:30
Aria Moradi
07314ef018 get default User Agent from WebView (#457)
* get default User Agent from WebView

* make sure to close browser

Co-authored-by: Mitchell Syer <Mitchellptbo@gmail.com>

Co-authored-by: Mitchell Syer <Mitchellptbo@gmail.com>
2022-12-04 21:21:19 +03:30
Aria Moradi
f47dc6b9de WebView based cloudflare interceptor (#456)
* WebView based cloudflare interceptor

ported https://github.com/vvanglro/cf-clearance to kotlin

* code clean up

* Forgot to commit these

* Get ResolveWithWebView working
1. Make sure to .use all closeable resources
2. Use 10 seconds instead of 1 second for waiting for cloudflare(this was the most probable issue)
3. Use Extension UA when possible
4. Minor cleanup of logging

* rewrite and refactor

Co-authored-by: Syer10 <syer10@users.noreply.github.com>
2022-12-04 12:08:54 +03:30
Mitchell Syer
2195c3df76 Downloader Rewrite (#437)
* Downloader rewrite
- Rewrite downloader to use coroutines instead of a thread
- Remove unused Page functions
- Add page progress
- Add ProgressResponseBody
- Add support for canceling a download in the middle of downloading
- Fix clear download queue

* Minor fix

* Minor improvements
- notifyAllClients now launches in another thread and only sends new data every second
- Better handling of download queue checker in step()
- Minor improvements and fixes

* Reorder downloads

* Download in parallel by source

* Remove TODO
2022-11-08 04:39:26 +03:30
Mitchell Syer
a3c366c360 Lint (#423) 2022-10-22 15:38:14 +03:30
Aria Moradi
44ffed3f7c add support for tachiyomi extensions Lib 1.3 (#316)
* closes #315

* provide real values

* add support for tachiyomi extensions lib 1.3
2022-03-19 02:36:42 +03:30
Aria Moradi
4bbe51331f remove gson (#295)
* remove gson

* also remove kotson

* fix build
2022-02-19 05:12:20 +03:30
Aria Moradi
3d3939e808 better logs 2021-08-26 22:10:27 +04:30
Aria Moradi
65d9021c37 close response 2021-08-23 06:10:31 +04:30
Aria Moradi
66481a0391 NPE fix suggested by @syer10 2021-08-23 06:05:04 +04:30
Aria Moradi
a14a82bc9a fix oppsie, sync dependencies with tachiyomi 2021-08-23 05:27:39 +04:30
Aria Moradi
756c57a16e also intercept on 403 2021-08-23 04:56:27 +04:30
Aria Moradi
08443ceb3d remove comment 2021-08-23 04:20:04 +04:30
Aria Moradi
c215696f04 have a lighter log level 2021-08-23 04:17:40 +04:30
Aria Moradi
5ca42bf9b6 make it compile 2021-08-23 04:02:55 +04:30
Aria Moradi
3272b9dec5 add CloudflareInterceptor from TachiWeb-Server 2021-08-23 03:45:10 +04:30