Mirrors/cobalt
1
0
Fork 0
mirror of https://github.com/wukko/cobalt.git synced 2025-04-29 22:14:26 +02:00
Code Issues Packages Projects Releases Wiki Activity

docs: create a dedicated doc for api instance env variables

Browse Source 
& also move "api key file format" section to the actually relevant doc, aka `protect-an-instance`
This commit is contained in:
wukko 2025-04-01 00:38:04 +06:00
parent f70f88bc4c
commit 545971186a
No known key found for this signature in database
GPG Key ID: 3E30B3F26C7B4AA2
3 changed files with 283 additions and 91 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

228
docs/api-env-variables.md Normal file
View File

@ -0,0 +1,228 @@
# cobalt api instance environment variables
you can customize your processing instance's behavior using these environment variables. all of them but `API_URL` are optional.
this document is not final and will expand over time. feel free to improve it!
### general vars
| name | default | value example |
|:--------------------|:----------|:--------------------------------------|
| API_URL | | `https://api.url.example/` |
| API_PORT | `9000` | `1337` |
| COOKIE_PATH | | `/cookies.json` |
| PROCESSING_PRIORITY | | `10` |
| API_INSTANCE_COUNT | | `6` |
| API_REDIS_URL | | `redis://localhost:6379` |
| DISABLED_SERVICES | | `bilibili,youtube` |
[*view details*](#general)
### networking vars
| name | default | value example |
|:--------------------|:----------|:--------------------------------------|
| API_LISTEN_ADDRESS | `0.0.0.0` | `127.0.0.1` |
| API_EXTERNAL_PROXY | | `http://user:password@127.0.0.1:8080` |
| FREEBIND_CIDR | | `2001:db8::/32` |
[*view details*](#networking)
### limit vars
| name | default | value example |
|:-------------------------|:--------|:--------------|
| DURATION_LIMIT | `10800` | `18000` |
| TUNNEL_LIFESPAN | `90` | `120` |
| RATELIMIT_WINDOW | `60` | `120` |
| RATELIMIT_MAX | `20` | `30` |
| SESSION_RATELIMIT_WINDOW | `60` | `60` |
| SESSION_RATELIMIT | `10` | `10` |
[*view details*](#limits)
### security vars
| name | default | value example |
|:------------------|:--------|:--------------------------------------|
| CORS_WILDCARD | `1` | `0` |
| CORS_URL | | `https://web.url.example` |
| TURNSTILE_SITEKEY | | `1x00000000000000000000BB` |
| TURNSTILE_SECRET | | `1x0000000000000000000000000000000AA` |
| JWT_SECRET | | see [details](#security) |
| JWT_EXPIRY | `120` | `240` |
| API_KEY_URL | | `file://keys.json` |
| API_AUTH_REQUIRED | | `1` |
[*view details*](#security)
### service-specific vars
| name | value example |
|:---------------------------------|:-------------------------|
| CUSTOM_INNERTUBE_CLIENT | `IOS` |
| YOUTUBE_SESSION_SERVER | `http://localhost:8080/` |
| YOUTUBE_SESSION_INNERTUBE_CLIENT | `WEB_EMBEDDED` |
[*view details*](#service-specific)
## general
[*jump to the table*](#general-vars)
### API_URL
> [!NOTE]
> API_URL is required to run the API instance.
the URL from which your instance will be accessible. can be external or internal, but it must be a valid URL or else tunnels will not work.
the value is a URL.
### API_PORT
port from which the API server will be accessible.
the value is a number from 1024 to 65535.
### COOKIE_PATH
path to the `cookies.json` file relative to the current working directory of your cobalt instance (usually the main (src/api) folder).
### PROCESSING_PRIORITY
`nice` value for ffmpeg subprocesses. available only on unix systems.
note: the higher the nice value, the lower the priority. you can [read more about nice here](https://en.wikipedia.org/wiki/Nice_(Unix)).
the value is a number.
### API_INSTANCE_COUNT
supported only on linux and node.js `>=23.1.0`. when configured, cobalt will spawn multiple sub-instances amongst which requests will be balanced. `API_REDIS_URL` is required to use this option.
the value is a number.
### API_REDIS_URL
when configured, cobalt will use this redis instance for tunnel cache. required when `API_INSTANCE_COUNT` is more than 1, because else sub-instance wouldn't be able to share cache.
the value is a URL.
### DISABLED_SERVICES
comma-separated list which disables certain services from being used.
the value is a string of cobalt-supported services.
## networking
[*jump to the table*](#networking-vars)
### API_LISTEN_ADDRESS
defines the local address for the api instance. if you are using a docker container, you usually don't need to configure this.
the value is a local IP address.
### API_EXTERNAL_PROXY
URL of the proxy that will be passed to [`ProxyAgent`](https://undici.nodejs.org/#/docs/api/ProxyAgent) and used for all external requests. HTTP(S) only.
if some feature breaks when using a proxy, please make a new issue about it!
the value is a URL.
### FREEBIND_CIDR
IPv6 prefix used for randomly assigning addresses to cobalt requests. available only on linux systems.
setting a `FREEBIND_CIDR` allows cobalt to pick a random IP for every download and use it for all requests it makes for that particular download.
to use freebind in cobalt, you need to follow its [setup instructions](https://github.com/imputnet/freebind.js?tab=readme-ov-file#setup) first.
if you want to use this option and run cobalt in a docker container, you also need to set the `API_LISTEN_ADDRESS` env variable to `127.0.0.1` and set `network_mode` for the container to `host`.
the value is an IPv6 range.
## limits
[*jump to the table*](#limit-vars)
### DURATION_LIMIT
media duration limit, in **seconds**
the value is a number.
### TUNNEL_LIFESPAN
the duration for which tunnel info is stored in ram, **in seconds**.
it's recommended to keep this value either default or as low as possible to preserve efficiency and user privacy.
the value is a number.
### RATELIMIT_WINDOW
rate limit time window for api requests, but not session requests, in **seconds**.
the value is a number.
### RATELIMIT_MAX
amount of api requests to be allowed within the time window of `RATELIMIT_WINDOW`.
the value is a number.
### SESSION_RATELIMIT_WINDOW
rate limit time window for session creation requests, in **seconds**.
the value is a number.
### SESSION_RATELIMIT
amount of session requests to be allowed within the time window of `SESSION_RATELIMIT_WINDOW`.
the value is a number.
## security
[*jump to the table*](#security-vars)
> [!NOTE]
> in order to enable turnstile bot protection, `TURNSTILE_SITEKEY`, `TURNSTILE_SECRET`, and `JWT_SECRET` must be set. all three at once.
### CORS_WILDCARD
defines whether cross-origin resource sharing is enabled. when enabled, your instance will be accessible from foreign web pages.
the value is a number. 0: disabled. 1: enabled.
### CORS_URL
configures the [cross-origin resource sharing origin](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Access-Control-Allow-Origin). your instance will be available only from this URL if `CORS_WILDCARD` is set to `0`.
the value is a URL.
### TURNSTILE_SITEKEY
[cloudflare turnstile](https://www.cloudflare.com/products/turnstile/) sitekey used by the web client to request & solve a challenge to prove that the user is not a bot.
the value is a specific key.
### TURNSTILE_SECRET
[cloudflare turnstile](https://www.cloudflare.com/products/turnstile/) secret used by the processing instance to verify that the client solved the challenge successfully.
the value is a specific key.
### JWT_SECRET
the secret used for issuing JWT tokens for request authentication. the value must be a random, secure, and long string (over 16 characters).
the value is a specific key.
### JWT_EXPIRY
the duration of how long a cobalt-issued JWT token will remain valid, in seconds.
the value is a number.
### API_KEY_URL
the URL to the the external or local key database. for local files you have to specify a local path using the `file://` protocol.
see [the api key section](/docs/protect-an-instance.md#api-key-file-format) in the "how to protect your cobalt instance" document for more details.
the value is a URL.
### API_AUTH_REQUIRED
when set to `1`, the user always needs to be authenticated in some way before they can access the API (either via an api key or via turnstile, if enabled).
the value is a number.
## service-specific
[*jump to the table*](#service-specific-vars)
### CUSTOM_INNERTUBE_CLIENT
innertube client that will be used instead of the default one.
the value is a string.
### YOUTUBE_SESSION_SERVER
URL to an instance of [yt-session-generator](https://github.com/imputnet/yt-session-generator). used for automatically pulling `poToken` & `visitor_data` for youtube. can be local or remote.
the value is a URL.
### YOUTUBE_SESSION_INNERTUBE_CLIENT
innertube client that's compatible with botguard's (web) `poToken` and `visitor_data`.
the value is a string.

54
docs/protect-an-instance.md
View File

@ -114,7 +114,7 @@ if you want to use your instance outside of web interface, you'll need an api ke
>
> if api keys leak, you'll have to update/remove all UUIDs to revoke them.
1. create a `keys.json` file following [the schema and example here](/docs//run-an-instance.md#api-key-file-format).
1. create a `keys.json` file following [the schema and example down below](#api-key-file-format).
2. expose the `keys.json` to the docker container:
```yml
@ -148,3 +148,55 @@ environment:
### why not make keys exclusive by default?
keys may be useful for going around rate limiting,
while keeping the rest of api rate limited, with no turnstile in place.
## api key file format
the file is a JSON-serialized object with the following structure:
```typescript
type KeyFileContents = Record<
UUIDv4String,
{
name?: string,
limit?: number | "unlimited",
ips?: (CIDRString | IPString)[],
userAgents?: string[]
}
>;
```
where *`UUIDv4String`* is a stringified version of a UUIDv4 identifier.
- **name** is a field for your own reference, it is not used by cobalt anywhere.
- **`limit`** specifies how many requests the API key can make during the window specified in the `RATELIMIT_WINDOW` env.
- when omitted, the limit specified in `RATELIMIT_MAX` will be used.
- it can be also set to `"unlimited"`, in which case the API key bypasses all rate limits.
- **`ips`** contains an array of allowlisted IP ranges, which can be specified both as individual ips or CIDR ranges (e.g. *`["192.168.42.69", "2001:db8::48", "10.0.0.0/8", "fe80::/10"]`*).
- when specified, only requests from these ip ranges can use the specified api key.
- when omitted, any IP can be used to make requests with that API key.
- **`userAgents`** contains an array of allowed user agents, with support for wildcards (e.g. *`["cobaltbot/1.0", "Mozilla/5.0 * Chrome/*"]`*).
- when specified, requests with a `user-agent` that does not appear in this array will be rejected.
- when omitted, any user agent can be specified to make requests with that API key.
- if both `ips` and `userAgents` are set, the tokens will be limited by both parameters.
- if cobalt detects any problem with your key file, it will be ignored and a warning will be printed to the console.
an example key file could look like this:
```json
{
"b5c7160a-b655-4c7a-b500-de839f094550": {
"limit": 10,
"ips": ["10.0.0.0/8", "192.168.42.42"],
"userAgents": ["*Chrome*"]
},
"b00b1234-a3e5-99b1-c6d1-dba4512ae190": {
"limit": "unlimited",
"ips": ["192.168.1.2"],
"userAgents": ["cobaltbot/1.0"]
}
}
```
if you are configuring a key file, **do not use the UUID from the example** but instead generate your own. you can do this by running the following command if you have node.js installed:
`node -e "console.log(crypto.randomUUID())"`

92
docs/run-an-instance.md
View File

@ -54,93 +54,5 @@ sudo apt install nscd
sudo service nscd start
```
## list of environment variables for api
| variable name | default | example | description |
|:----------------------|:----------|:------------------------|:------------|
| `API_PORT` | `9000` | `9000` | changes port from which api server is accessible. |
| `API_LISTEN_ADDRESS` | `0.0.0.0` | `127.0.0.1` | changes address from which api server is accessible. **if you are using docker, you usually don't need to configure this.** |
| `API_URL` | ➖ | `https://api.cobalt.tools/` | changes url from which api server is accessible. <br> ***REQUIRED TO RUN THE API***. |
| `API_NAME` | `unknown` | `ams-1` | api server name that is shown in `/api/serverInfo`. |
| `API_EXTERNAL_PROXY` | ➖ | `http://user:password@127.0.0.1:8080`| url of the proxy that will be passed to [`ProxyAgent`](https://undici.nodejs.org/#/docs/api/ProxyAgent) and used for all external requests. HTTP(S) only. |
| `CORS_WILDCARD` | `1` | `0` | toggles cross-origin resource sharing. <br> `0`: disabled. `1`: enabled. |
| `CORS_URL` | not used | `https://cobalt.tools` | cross-origin resource sharing url. api will be available only from this url if `CORS_WILDCARD` is set to `0`. |
| `COOKIE_PATH` | not used | `/cookies.json` | path for cookie file relative to main folder. |
| `PROCESSING_PRIORITY` | not used | `10` | changes `nice` value* for ffmpeg subprocess. available only on unix systems. |
| `FREEBIND_CIDR` | ➖ | `2001:db8::/32` | IPv6 prefix used for randomly assigning addresses to cobalt requests. only supported on linux systems. see below for more info. |
| `RATELIMIT_WINDOW` | `60` | `120` | rate limit time window in **seconds**. |
| `RATELIMIT_MAX` | `20` | `30` | max requests per time window. requests above this amount will be blocked for the rate limit window duration. |
| `DURATION_LIMIT` | `10800` | `18000` | max allowed video duration in **seconds**. |
| `TUNNEL_LIFESPAN` | `90` | `120` | the duration for which tunnel info is stored in ram, **in seconds**. |
| `TURNSTILE_SITEKEY` | ➖ | `1x00000000000000000000BB` | [cloudflare turnstile](https://www.cloudflare.com/products/turnstile/) sitekey used by browser clients to request a challenge.\*\* |
| `TURNSTILE_SECRET` | ➖ | `1x0000000000000000000000000000000AA` | [cloudflare turnstile](https://www.cloudflare.com/products/turnstile/) secret used by cobalt to verify the client successfully solved the challenge.\*\* |
| `JWT_SECRET` | ➖ | ➖ | the secret used for issuing JWT tokens for request authentication. to choose a value, generate a random, secure, long string (ideally >=16 characters).\*\* |
| `JWT_EXPIRY` | `120` | `240` | the duration of how long a cobalt-issued JWT token will remain valid, in seconds. |
| `API_KEY_URL` | ➖ | `file://keys.json` | the location of the api key database. for loading API keys, cobalt supports HTTP(S) urls, or local files by specifying a local path using the `file://` protocol. see the "api key file format" below for more details. |
| `API_AUTH_REQUIRED` | ➖ | `1` | when set to `1`, the user always needs to be authenticated in some way before they can access the API (either via an api key or via turnstile, if enabled). |
| `API_REDIS_URL` | ➖ | `redis://localhost:6379` | when set, cobalt uses redis instead of internal memory for the tunnel cache. |
| `API_INSTANCE_COUNT` | ➖ | `2` | supported only on Linux and node.js `>=23.1.0`. when configured, cobalt will spawn multiple sub-instances amongst which requests will be balanced. |
| `DISABLED_SERVICES` | ➖ | `bilibili,youtube` | comma-separated list which disables certain services from being used. |
| `CUSTOM_INNERTUBE_CLIENT` | ➖ | `IOS` | innertube client that will be used instead of the default one. |
| `YOUTUBE_SESSION_SERVER` | ➖ | `http://localhost:8080/` | url to an instance of [invidious' youtube-trusted-session-generator](https://github.com/iv-org/youtube-trusted-session-generator) or its fork/counterpart. used for automatically pulling poToken & visitor_data for youtube. can be local or remote. |
\* the higher the nice value, the lower the priority. [read more here](https://en.wikipedia.org/wiki/Nice_(Unix)).
\*\* in order to enable turnstile bot protection, all three **`TURNSTILE_SITEKEY`, `TURNSTILE_SECRET` and `JWT_SECRET`** need to be set.
#### FREEBIND_CIDR
setting a `FREEBIND_CIDR` allows cobalt to pick a random IP for every download and use it for all
requests it makes for that particular download. to use freebind in cobalt, you need to follow its [setup instructions](https://github.com/imputnet/freebind.js?tab=readme-ov-file#setup) first. if you configure this option while running cobalt
in a docker container, you also need to set the `API_LISTEN_ADDRESS` env to `127.0.0.1`, and set
`network_mode` for the container to `host`.
## api key file format
the file is a JSON-serialized object with the following structure:
```typescript
type KeyFileContents = Record<
UUIDv4String,
{
name?: string,
limit?: number | "unlimited",
ips?: (CIDRString | IPString)[],
userAgents?: string[]
}
>;
```
where *`UUIDv4String`* is a stringified version of a UUIDv4 identifier.
- **name** is a field for your own reference, it is not used by cobalt anywhere.
- **`limit`** specifies how many requests the API key can make during the window specified in the `RATELIMIT_WINDOW` env.
- when omitted, the limit specified in `RATELIMIT_MAX` will be used.
- it can be also set to `"unlimited"`, in which case the API key bypasses all rate limits.
- **`ips`** contains an array of allowlisted IP ranges, which can be specified both as individual ips or CIDR ranges (e.g. *`["192.168.42.69", "2001:db8::48", "10.0.0.0/8", "fe80::/10"]`*).
- when specified, only requests from these ip ranges can use the specified api key.
- when omitted, any IP can be used to make requests with that API key.
- **`userAgents`** contains an array of allowed user agents, with support for wildcards (e.g. *`["cobaltbot/1.0", "Mozilla/5.0 * Chrome/*"]`*).
- when specified, requests with a `user-agent` that does not appear in this array will be rejected.
- when omitted, any user agent can be specified to make requests with that API key.
- if both `ips` and `userAgents` are set, the tokens will be limited by both parameters.
- if cobalt detects any problem with your key file, it will be ignored and a warning will be printed to the console.
an example key file could look like this:
```json
{
"b5c7160a-b655-4c7a-b500-de839f094550": {
"limit": 10,
"ips": ["10.0.0.0/8", "192.168.42.42"],
"userAgents": ["*Chrome*"]
},
"b00b1234-a3e5-99b1-c6d1-dba4512ae190": {
"limit": "unlimited",
"ips": ["192.168.1.2"],
"userAgents": ["cobaltbot/1.0"]
}
}
```
if you are configuring a key file, **do not use the UUID from the example** but instead generate your own. you can do this by running the following command if you have node.js installed:
`node -e "console.log(crypto.randomUUID())"`
## list of environment variables
[this section has moved](/docs/api-env-variables.md) to a dedicated document that is way easier to understand and maintain. go check it out!