81 lines
3.3 KiB
Markdown
81 lines
3.3 KiB
Markdown
|
# Fire-and-forget logs / status key-values
|
||
|
|
|
||
|
|
(DMX6CO4G)
|
||
|
|
|
||
|
|
## Motivation
|
||
|
|
|
||
|
|
This feature allows ptth_server to act as a
|
||
|
|
[sidecar](https://docs.microsoft.com/en-us/azure/architecture/patterns/sidecar)
|
||
|
|
process to apps and scripts written in C, C++, Bash, or other languages
|
||
|
|
where networking is difficult.
|
||
|
|
|
||
|
|
ptth_server will hold volatile data such as high-detail logs or status data
|
||
|
|
that is not worth saving to files. This data can then be retrieved by human
|
||
|
|
users or scrapers through ptth_relay.
|
||
|
|
|
||
|
|
This feature is similar to Redis, in some ways.
|
||
|
|
|
||
|
|
## Interface with client apps
|
||
|
|
|
||
|
|
The lifecycle is this:
|
||
|
|
|
||
|
|
- A client app connects to ptth_server via TCP
|
||
|
|
- The client app sends data to ptth_server
|
||
|
|
- The client app MAY react to messages from ptth_server
|
||
|
|
- The TCP connection is closed by ptth_server OR the client app
|
||
|
|
|
||
|
|
The client app can send data with these semantics:
|
||
|
|
|
||
|
|
- "Status" i.e. "In this key directory, in this key, store this value"
|
||
|
|
- "Log" i.e. "In this log directory, in this key, store this log string"
|
||
|
|
|
||
|
|
Logs and status values do not share namespaces or quotas. ptth_server MUST be
|
||
|
|
configured with quotas in order to enable this feature. There is no "unlimited"
|
||
|
|
quota option, but the quotas MAY be set to any 64-bit value.
|
||
|
|
|
||
|
|
Keys, values, and log strings are binary-safe, but SHOULD be valid UTF-8.
|
||
|
|
If they are not valid UTF-8, they MAY not appear in client apps, or they may
|
||
|
|
appear as errors. ptth_server MUST behave correctly if it receives non-UTF-8
|
||
|
|
data.
|
||
|
|
|
||
|
|
The data should be considered volatile. ptth_server MUST NOT save the data
|
||
|
|
to disk. The client app SHOULD NOT buffer data internally if ptth_server is
|
||
|
|
refusing connections or unavailable. Client apps will see the data as
|
||
|
|
write-only - They cannot retrieve data from ptth_server.
|
||
|
|
|
||
|
|
ptth_server does not guarantee that data will be stored for any length of time,
|
||
|
|
or at all. All data is subject to quotas based on time, byte count, key count,
|
||
|
|
etc. Logs WILL be evicted in oldest-first order. Status values
|
||
|
|
replace older values written to the same key. If a key directory reaches
|
||
|
|
its quota, old values MAY be evicted in any order, including oldest-first,
|
||
|
|
alphabetical, or random.
|
||
|
|
|
||
|
|
Quotas: ptth_relay is the source of truth for quotas. ptth_server SHOULD NOT
|
||
|
|
store the quotas on disk, it SHOULD request them from ptth_relay when connecting,
|
||
|
|
keep them in memory, and periodically refresh them. All quotas are set
|
||
|
|
per-key-directory. ptth_relay is the source of truth for the list of key
|
||
|
|
directories.
|
||
|
|
|
||
|
|
The quotas are all maximums. The quotas (with recommended value in parens) are:
|
||
|
|
|
||
|
|
- Keys in the directory (1,024)
|
||
|
|
- Bytes for 1 key (1,024)
|
||
|
|
- Bytes for 1 value (1,024)
|
||
|
|
- Total payload (i.e. keys and values) bytes (262,144)
|
||
|
|
|
||
|
|
It's hard to set a perfect limit on RAM usage, so the quotas have many
|
||
|
|
different dimensions. There is some upper bound on RAM usage, but it will
|
||
|
|
somewhere above the number of bytes needed to store keys and values alone.
|
||
|
|
|
||
|
|
ptth_server MAY send these messages to a client app:
|
||
|
|
|
||
|
|
- "Go" - i.e. "Everything is good on my end. If you send data I will try to
|
||
|
|
store it."
|
||
|
|
- "Stop" - i.e. "Something is wrong. If you send data I will ignore you."
|
||
|
|
|
||
|
|
Unlike HTTP, there is no synchronization or dependency between the two
|
||
|
|
halves of the TCP connection - ptth_server can send a message at any time,
|
||
|
|
and the client app can send data at any time. "Go" does not mean that the
|
||
|
|
server will store the next data sent from the client, because a "Stop" message
|
||
|
|
could be in-flight.
|