146c91f2be
♻️ refactor: rename `RelayState` to `Relay` and privatize `load_templates`
2021-04-18 08:28:07 -05:00
c6f651387e
♻️ refactor: make `load_templates` private
2021-04-17 20:50:48 -05:00
ae4f102bdf
♻️ refactor: make git_version module private
2021-04-17 19:26:05 -05:00
fd238d8c2b
♻️ refactor: rename file_server::State to file_server::FileServer
2021-04-17 19:10:52 -05:00
ae33337156
♻️ refactor: clean up ptth_server
...
- Found I was passing the file server root twice
- Removed handlebars from the public API. The templates are fixed
when ptth_server ships, so I don't think users of the lib should
care what handlebars is.
- Making other stuff private where possible
2021-04-17 18:59:59 -05:00
86af3194e5
💥 breaking: remove password_is_bad from ptth_server
...
The password checking is not necessary with the new process of generating
high-entropy Base64 passwords and then approving them on the relay.
The password file takes a lot of space and this doesn't need to be part
of ptth_server's public API, if there is one, anyway.
2021-04-17 17:47:01 -05:00
d457feb35f
📝 docs: document ptth_core
2021-04-17 17:47:01 -05:00
a911e53e48
📝 docs: finish some basic docs for always_equal
2021-04-17 17:47:01 -05:00
e02dbf6e31
♻️ refactor: alphabetize
2021-04-17 17:47:01 -05:00
b975e463e8
📝 docs: start writing docs in `cargo doc` format
2021-04-17 14:57:30 -05:00
d16b0c4c61
🚨 `cargo check` pass
2021-04-10 09:41:21 -05:00
6b71b32cf5
♻️ refactor: extract routing module
2021-04-10 09:38:53 -05:00
6a84d2dbd5
🚧 start moving routing to its own module
2021-04-10 09:17:52 -05:00
29eafb3ce7
add placeholder for generating scraper keys
2021-04-09 20:00:08 -05:00
3408feb619
♻️ refactor: move gen_key up to ptth_core
2021-04-09 19:43:15 -05:00
240cd8dff1
♻️ refactor: extract gen_key pure function
2021-04-09 19:30:45 -05:00
d9669a7073
proof of concept for private browser cache based on etag and if-none-match
2021-04-03 17:26:53 +00:00
1df0f0f677
♻️ refactor: extract stream_file
2021-04-03 16:27:31 +00:00
ff73f501a4
🚧
2021-04-03 16:21:59 +00:00
3389292457
placeholder for caching - Always returns a random ETag header.
...
So the browser-side cache will always miss, but I tested that Firefox will at
least send us If-None-Match with the last ETag it saw.
2021-04-03 15:53:59 +00:00
6e24983ad7
🚨 refactor: cargo clippy
2021-04-03 15:30:32 +00:00
5a6d9314cc
🚨 refactor: appease clippy
2021-04-03 15:23:03 +00:00
0821ccaac6
♻️ refactor: move Markdown support in ptth_server out to a feature.
...
It may be removed in the future.
2021-04-03 15:17:28 +00:00
e5bf5f7d1f
♻️ refactor: flatten out indentation
2021-04-03 14:27:17 +00:00
832794f844
➕ in-memory audit logging
2021-04-02 23:19:33 -05:00
6927707501
♻️ refactor: use BoundedVec for unregistered_servers
2021-04-02 22:30:22 -05:00
2c4d3cf534
add BoundedVec generic struct
2021-04-02 22:14:23 -05:00
6ee9b6e7c9
🐛 bug: ptth_server: add 30-second timeout for _only_ http_listen requests
2021-04-02 20:58:19 -05:00
4dd08bdbb1
✨ bump ptth_server to 1.2.1 for the Windows patch
2021-03-21 19:10:40 +00:00
02d4362c3b
🏁 build: fix Windows build for ptth_server
2021-03-21 14:07:09 -05:00
2253ae3391
🐛 bug: fix reqwest feature needed for ptth_server
2021-03-21 18:28:08 +00:00
d15f8539cb
🎉 ptth_server to 1.2.0
2021-03-21 18:25:02 +00:00
3ff9c0f46c
add license
2021-03-21 18:22:32 +00:00
6f2c14670e
🎊 always_equal to 1.0.0
2021-03-21 18:21:48 +00:00
02070e3660
🐛 bug: add metadata to ptth_relay
2021-03-21 18:16:29 +00:00
fc4893f2e8
🎉 ptth_relay to 1.2.0
2021-03-21 18:11:39 +00:00
71a2b37fab
🎉 ptth_core to 1.2.0
2021-03-21 17:56:20 +00:00
f05257d833
🐛 bug fix requests timing out after 40 seconds.
...
This is such a stupid bug. I swear I checked this exact piece of code for
this exact bug and it wasn't there. But after I fixed it, I had no problem
running a download for 11 hours straight.
This fix won't affect a case where the firewall is actually closing long-
running connections (which is bad behavior, but it could happen) or a case
where the Internet is just flakey and the connection drops.
But it will fix the most common case where both client and server are on
robust connections and the download times out anyway.
2021-03-21 15:43:31 +00:00
c1118971b0
add --throttle-upload option
2021-03-21 15:43:15 +00:00
235e134cb2
♻️ refactor: more clippy lints, and make read_git_version simpler
2021-03-21 03:45:00 +00:00
c6b2418a49
♻️ refactor: split up large function at clippy's behest
2021-03-21 03:40:45 +00:00
50fc509c8b
🚨 fix a bunch of easy clippy warnings
2021-03-21 03:34:47 +00:00
f1f13d6e17
⬆️ deps: new version of dashmap
2021-03-21 03:15:01 +00:00
820099c2ca
⬆️ deps: replace ulid with rusty_ulid so I can drop rand 0.6.x
2021-03-21 03:09:44 +00:00
4dddfb1768
⬆️ update important crates to Tokio 1.4.0
2021-03-21 03:01:06 +00:00
ee1db02b08
♻️ refactor: use constructor for ConfigFile
2021-03-21 02:49:44 +00:00
453532e48e
add listen ID to http_listen
2021-03-16 15:15:16 +00:00
3302411daa
ptth_relay: update tokio, trying to find this bug
2021-03-16 14:50:52 +00:00
62821c546b
log error for 500 internal server error during 'listen'
2021-03-16 14:39:15 +00:00
456ab04f78
impl auto-gen-key
2021-03-15 20:34:32 +00:00
a356d981a8
add file_server_root CLI opt too
2021-03-15 20:02:37 +00:00
56496bf97b
add `--name` so caller program can provide name
2021-03-15 19:55:12 +00:00
13c1ddc01a
🐛 bug: fix tests not compiling
2021-03-15 19:29:54 +00:00
8a04481236
finish hooking up the unregistered servers page
2021-03-15 19:22:23 +00:00
699cb671ec
➕ add unregistered servers page
2021-03-15 19:11:57 +00:00
9648a9853c
wip: debug proxy now owns a filter which can drop or modify request bodies
2021-03-06 22:58:23 +00:00
a980d151fc
⬆️ deps: upgrade everything to Tokio 1.2.0
...
And pretend I didn't completely goober the Docker build somehow
2021-03-06 21:46:35 +00:00
4aa54c32d0
✅ test: add a test that uses debug_proxy
2021-03-06 18:55:05 +00:00
33746d9ace
♻️ refactor: debug_proxy into a lib
2021-03-05 04:03:40 +00:00
27336d8571
add debug_proxy which I can probably use to inject network problems during tests
2021-03-05 03:17:56 +00:00
6db94d3e4f
update: add endless_sink debug tool
2021-02-20 20:25:14 +00:00
aec822ba5f
⬆️ deps: Update to Rust 1.50.0
2021-02-20 19:12:38 +00:00
498d69eeb9
update: add throttled endless random garbage
2021-02-20 17:28:39 +00:00
09464c548a
update: add endless data source for debugging
2021-02-20 17:08:38 +00:00
f53973ddad
🐛 bug: make sure errors are sent to tracing
2021-02-20 16:36:45 +00:00
9a984f5a4e
♻️ refactor: ptth_relay: clean up Handlebars file names
2021-02-20 16:25:30 +00:00
0ffa129937
🚧 wip: add prefix for relay debugging tools
2021-02-20 16:23:12 +00:00
89213ebf8a
📦 build: preparing to publish on crates.io
2021-01-20 21:22:37 -06:00
031b03a13c
🎉 release: bump versions to 1.1.0 for everything in prod
...
I forgot to set the version in Cargo.toml files for 1.0.0.
I'm not gonna do 0.x versions because this is already live somewhere
and I don't like that 0.x adds complexity to versioning.
2021-01-20 20:49:32 -06:00
d7ed2ed931
✅ test: fix tests
2021-01-19 23:27:33 +00:00
c40abb0fe6
🚧 wip: I was working on some port-forwarding idea.
...
It was going to be generic over TCP and use 2 HTTP streams, one each way.
The plan's written down somewhere.
2021-01-19 23:25:24 +00:00
f94b40b6b8
🚧 wip: further POC
2021-01-19 23:25:24 +00:00
8d62b29319
update: both the downstream POCs work, with the bogus downstream.
2021-01-19 23:25:24 +00:00
0cb24695d0
🚧 wip: idea for tunneling TCP over HTTP
...
There's a lot of missing pieces, but the big picture is like this:
- Use 2 completely separate HTTP streams, and try to keep them alive as long
as possible, each in basically half-duplex mode
- Each stream has a long-running PUT and GET, sort of like station307
- Each end has to be terminated by a native app that either connects to a local
TCP server, or acts as a local TCP server
- No clue how it would work for multiple connections on the same port. Poorly,
I guess?
- It's probably gonna run like garbage because we're splitting TCP into
2 TCP streams, and although backpressure might work, the ACKs will be less
efficient. And the congestion control might get confused
My only goal is to tunnel Tracy over it, so that I can have that remotely.
2021-01-19 23:25:24 +00:00
574f660c1a
🚧 wip: can't remember what I was working on
2021-01-19 23:25:24 +00:00
29a6ad20ee
🚧 wip: adding http service to ptth_kv
2021-01-19 23:25:24 +00:00
eb927ef80d
✅ test: ptth_kv is slower with async, but still fast enough
2021-01-19 23:25:24 +00:00
27f31d0ef4
✅ test: measure performance for KV store idea
2021-01-19 23:25:24 +00:00
88e7839841
🚧 wip: outlining idea for Redis-like KV store in ptth_server
2021-01-03 18:09:00 +00:00
0a46fc05dc
🐛 bug: fix 30-day scraper keys having 7 days of validity.
...
This won't affect anything, because I had manually written the not_after for
the testing keys. Even the automated tests weren't using the new_30_day
function
2020-12-21 14:26:51 +00:00
cbf5583d06
update: tracing RSS MiB
2020-12-20 20:41:02 -06:00
137e8e1bf8
➕ update: add prototype for `tail -f` based on polling
2020-12-20 18:59:05 -06:00
e79925dc14
update: ptth_server metrics are at a stopping point for now
2020-12-20 18:09:24 -06:00
eec17b10a1
update: switch back to 60-second interval
2020-12-20 17:34:55 -06:00
bcc673b74b
♻️ refactor: extract metrics monitor function and add it to ptth_server
2020-12-20 17:32:13 -06:00
f335644b03
➕ update: measuring CPU usage every minute
2020-12-20 17:17:31 -06:00
1e5aa528c9
🚧 wip: collecting CPU time used.
2020-12-20 20:10:29 +00:00
96106e68fc
♻️ refactor: use file_server::State directly more
2020-12-20 19:55:20 +00:00
4cb0911b77
➕ update: add RSS MiB to server root page
2020-12-20 19:46:30 +00:00
e5103d48bd
➕ update: add ArcSwap to update gauges lock-free
2020-12-20 19:35:32 +00:00
b2b0bbc8fc
♻️ refactor: rename
2020-12-20 19:00:22 +00:00
64ac4baaa8
➕ update: print RSS in MiB at startup
2020-12-20 18:58:14 +00:00
bc361fa876
➕ update: add heim for measuring process RSS
2020-12-20 18:52:53 +00:00
47788f56e0
♻️ refactor: use `use X as Y`
2020-12-20 18:40:49 +00:00
066c95dc07
♻️ refactor: extract html.rs
2020-12-20 18:38:39 +00:00
4bd38180d0
♻️ refactor: remove lifetime on handlebars
2020-12-20 18:23:17 +00:00
88c3500fd0
♻️ refactor
2020-12-20 18:21:02 +00:00
1aff4389bd
♻️ refactor: move file server runtime state into file_server
2020-12-20 18:10:11 +00:00
31750d30fc
♻️ refactor: use new file_server::Config in ptth_server
2020-12-20 18:06:21 +00:00
009601e136
♻️ refactor: move file server config into file_server
2020-12-20 18:04:19 +00:00
e8d94da661
♻️ refactor: change PerInstance to Startup
2020-12-20 17:44:03 +00:00
d052f42507
🚧 wip: add placeholder for gauges
2020-12-20 17:41:00 +00:00
b54be58abc
🚨 refactor: rename InstanceMetrics to PerInstance
2020-12-18 23:45:30 +00:00
72b3b33206
🚨 refactor: fix some Clippy lints
2020-12-18 23:41:52 +00:00
d03c1a5476
➕ update: add InstanceMetrics and replace ServerInfo
2020-12-18 20:43:34 +00:00
626946b7b7
🔉 update: demote some boring logs from debug to trace
2020-12-16 19:06:15 -06:00
1e160ec55b
⭐ new: add subcommand in ptth_relay to hash API keys from the terminal
2020-12-16 14:57:47 +00:00
9ac44cfeb7
⭐ new: finish MVP for scraper auth.
...
Adding a SQLite DB to properly track the keys is going to take a while. For
now I'll just keep them in the config file and give them 30-day expirations.
2020-12-16 14:46:03 +00:00
cda627fa4b
⭐ new: add JSON API in server for dir listings
2020-12-15 05:15:17 +00:00
11f4b0e65b
✅ test: Fix tests broken by recent commits
2020-12-14 14:17:52 +00:00
fa5aa8b05a
🚧 wip: add server list API endpoint
2020-12-14 01:07:13 -06:00
e865ac56c7
🚨 refactor: fix some clippy / cargo check warnings
2020-12-13 20:05:52 -06:00
5d1b68dc90
📜 logging: don't log handlebars object
2020-12-13 04:47:47 +00:00
9c7b2b7a86
✅ test: add tests for scraper API test endpoint
2020-12-13 04:44:37 +00:00
0737edd8f8
👕 refactor: move handlebars out of RelayState
...
This will make it simpler to separate logic and presentation.
2020-12-13 04:03:30 +00:00
32e48697d5
👕 refactor: extract scraper_api module
2020-12-13 03:42:00 +00:00
5f947ed73c
👕 refactor: Extract relay_state module
2020-12-13 03:29:54 +00:00
532f99f770
💄 update: add dev mode and server Git version to server list
2020-12-13 03:00:45 +00:00
0e4839e146
🐳 build: add Tini to Docker image, simplify Git version injection
...
Building the Git version into the code meant that Cargo had to recompile
ptth_relay every time I built. This is annoying and it doesn't add anything.
I changed it to read the Git version from a text file which is absent by
default, and present in the Docker image.
2020-12-13 02:20:12 +00:00
670ce30667
✅ test: add end-to-end test for scraper API
2020-12-13 01:55:47 +00:00
1e81421444
👕 refactor: Extract functions for scraper API endpoints
2020-12-13 01:12:56 +00:00
6d68a77364
⭐ new (ptth_relay): add test endpoint for scrapers
...
Scrapers can auth using a shared (but hashed) API key.
The hash of the key is specified in ptth_relay.toml, and forces dev mode on.
2020-12-12 17:50:40 +00:00
004b98229a
🐛 bug: fix serde expecting phantom data in the config file
2020-12-12 17:12:38 +00:00
0eb1e7e38f
⭐ new: add code for scraper keys to expire and have limited durations
2020-12-12 17:11:22 +00:00
bf8e483d16
👕 refactor: merge the servers' config file and runtime representations
2020-12-12 15:57:22 +00:00
b43106393a
👕 refactor (ptth_relay): use the new blake3 deserializer
2020-12-12 15:50:38 +00:00
9bc4b57058
⭐ new (ptth_relay): add serde deserializer for blake3 hashes
2020-12-12 15:35:33 +00:00
0c5a37b441
🐳 build (ptth_relay): clean up Docker build process
...
The new method is much nicer and doesn't require the manual make-old-git
step. The top-level command is actually build_and_minimize.bash, which uses
`git archive` to unpack the last Git commit and build with _that_ Dockerfile
and Docker context. This is better for determinism. It's similar to our build
process for that one big project at work.
2020-12-12 05:08:58 +00:00
f6486b2c1a
🔧 config (ptth_relay): add feature flags
...
- dev mode
- scraper auth
These will gate features I'm adding soon.
2020-12-12 01:26:58 +00:00
c4b12eb806
🏁 Builds on Windows.
2020-12-10 06:24:56 +00:00
e8c020fbc6
Update --print-tripcode option
2020-11-30 16:15:27 +00:00
e59bb5b7bc
💥 Break relay config again
2020-11-30 15:52:15 +00:00
7925d9be95
♻️ Move server endpoints to their own file
2020-11-29 23:24:25 +00:00
8079b3f778
♻️ Remove "internal" from names in the internal module
2020-11-29 23:15:45 +00:00
bb4c4e803a
♻️ Extract file server internal parts to their own file
2020-11-29 23:12:56 +00:00
6e6e062c51
♻️ Move Markdown previewing to its own file
2020-11-29 22:58:38 +00:00
ab95485d92
♻️ Fix clippy issues
2020-11-29 22:41:48 +00:00
b94a3a1e17
Move byte range code into its own file
2020-11-29 22:31:54 +00:00
b43a6c2e4b
♻️ Move emoji icons into one place
2020-11-29 22:12:25 +00:00
028970cdf0
♻️ Remove the hyper dep from ptth_server.exe
2020-11-29 21:55:24 +00:00
f4b0c64e01
♻️ Move ptth_file_server into a bin crate so ptth_server can shed some
...
dependencies
2020-11-29 21:42:03 +00:00
d6430e39a9
♻️ Get rid of more unwraps and panics
2020-11-29 21:38:23 +00:00
7bd2450698
♻️ Fix pedantic clippy warnings
2020-11-29 20:22:40 +00:00
eada65d94b
♻️ Remove unwraps from file server module
2020-11-29 19:59:47 +00:00
f212931842
♻️ Remove more unwraps
2020-11-29 19:47:40 +00:00
720aae2201
♻️ Working on errors for file server and server
2020-11-29 19:19:59 +00:00
c3ff3deb8e
♻️ Fix some clippy warnings
2020-11-29 19:05:28 +00:00
47c59447f3
♻️ Splitting tests for file server into their own tests.rs
2020-11-29 18:50:51 +00:00
aad7f8e729
♻️ Remove all practically removable unwraps from ptth_relay
2020-11-29 18:39:51 +00:00
687cffdf90
♻️ Fixing clippy lints
2020-11-29 18:37:33 +00:00
5c0d7ea998
♻️ Removing unwraps
2020-11-29 17:09:25 +00:00
bf96d400b2
♻️ Removing unwraps
2020-11-29 17:09:19 +00:00
3e74f2b1ef
♻️ Fix unwrap()s in ptth_core.
...
Some were in an unused function, so I removed that function.
Some were in graceful_shutdown and can't really be handled. So I made
them into "expect"s
2020-11-29 16:36:59 +00:00
bbb88c01e8
♻️ Extract ptth_server crate. Docker still broken
2020-11-27 00:50:22 +00:00
4c9595ee2e
♻️ Extract crate ptth_relay
2020-11-27 00:20:18 +00:00
84bb326f37
♻️ Extract crate ptth_core
2020-11-27 00:03:11 +00:00
2b93aa8b83
🐛 Implement 416 Range Not Satisfiable
...
I had a lot of trouble getting AlwaysEqual to compile, so I tested it in a
completely separate crate and vendored it back into PTTH. It's also AGPLv3.
2020-11-09 00:55:45 +00:00
_