Mirrors/cobalt
1
0
Fork 0
mirror of https://github.com/wukko/cobalt.git synced 2025-04-29 22:14:26 +02:00
Code Issues Packages Projects Releases Wiki Activity

docs: create a dedicated doc for api instance env variables

Browse Source 
& also move "api key file format" section to the actually relevant doc, aka `protect-an-instance`
This commit is contained in:
wukko 2025-04-01 00:38:04 +06:00
parent f70f88bc4c
commit 545971186a
No known key found for this signature in database
GPG Key ID: 3E30B3F26C7B4AA2
3 changed files with 283 additions and 91 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

228
docs/api-env-variables.md Normal file
View File

@ -0,0 +1,228 @@
# cobalt api instance environment variables
you can customize your processing instance's behavior using these environment variables. all of them but `API_URL` are optional.
this document is not final and will expand over time. feel free to improve it!
### general vars
| name | default | value example |
|:--------------------|:----------|:--------------------------------------|
| API_URL | | `https://api.url.example/` |
| API_PORT | `9000` | `1337` |
| COOKIE_PATH | | `/cookies.json` |
| PROCESSING_PRIORITY | | `10` |
| API_INSTANCE_COUNT | | `6` |
| API_REDIS_URL | | `redis://localhost:6379` |
| DISABLED_SERVICES | | `bilibili,youtube` |
[*view details*](#general)
### networking vars
| name | default | value example |
|:--------------------|:----------|:--------------------------------------|
| API_LISTEN_ADDRESS | `0.0.0.0` | `127.0.0.1` |
| API_EXTERNAL_PROXY | | `http://user:password@127.0.0.1:8080` |
| FREEBIND_CIDR | | `2001:db8::/32` |
[*view details*](#networking)
### limit vars
| name | default | value example |
|:-------------------------|:--------|:--------------|
| DURATION_LIMIT | `10800` | `18000` |
| TUNNEL_LIFESPAN | `90` | `120` |
| RATELIMIT_WINDOW | `60` | `120` |
| RATELIMIT_MAX | `20` | `30` |
| SESSION_RATELIMIT_WINDOW | `60` | `60` |
| SESSION_RATELIMIT | `10` | `10` |
[*view details*](#limits)
### security vars
| name | default | value example |
|:------------------|:--------|:--------------------------------------|
| CORS_WILDCARD | `1` | `0` |
| CORS_URL | | `https://web.url.example` |
| TURNSTILE_SITEKEY | | `1x00000000000000000000BB` |
| TURNSTILE_SECRET | | `1x0000000000000000000000000000000AA` |
| JWT_SECRET | | see [details](#security) |
| JWT_EXPIRY | `120` | `240` |
| API_KEY_URL | | `file://keys.json` |
| API_AUTH_REQUIRED | | `1` |
[*view details*](#security)
### service-specific vars
| name | value example |
|:---------------------------------|:-------------------------|
| CUSTOM_INNERTUBE_CLIENT | `IOS` |
| YOUTUBE_SESSION_SERVER | `http://localhost:8080/` |
| YOUTUBE_SESSION_INNERTUBE_CLIENT | `WEB_EMBEDDED` |
[*view details*](#service-specific)
## general
[*jump to the table*](#general-vars)
### API_URL
> [!NOTE]
> API_URL is required to run the API instance.
the URL from which your instance will be accessible. can be external or internal, but it must be a valid URL or else tunnels will not work.
the value is a URL.
### API_PORT
port from which the API server will be accessible.
the value is a number from 1024 to 65535.
### COOKIE_PATH
path to the `cookies.json` file relative to the current working directory of your cobalt instance (usually the main (src/api) folder).
### PROCESSING_PRIORITY
`nice` value for ffmpeg subprocesses. available only on unix systems.
note: the higher the nice value, the lower the priority. you can [read more about nice here](https://en.wikipedia.org/wiki/Nice_(Unix)).
the value is a number.
### API_INSTANCE_COUNT
supported only on linux and node.js `>=23.1.0`. when configured, cobalt will spawn multiple sub-instances amongst which requests will be balanced. `API_REDIS_URL` is required to use this option.
the value is a number.
### API_REDIS_URL
when configured, cobalt will use this redis instance for tunnel cache. required when `API_INSTANCE_COUNT` is more than 1, because else sub-instance wouldn't be able to share cache.
the value is a URL.
### DISABLED_SERVICES
comma-separated list which disables certain services from being used.
the value is a string of cobalt-supported services.
## networking
[*jump to the table*](#networking-vars)
### API_LISTEN_ADDRESS
defines the local address for the api instance. if you are using a docker container, you usually don't need to configure this.
the value is a local IP address.
### API_EXTERNAL_PROXY
URL of the proxy that will be passed to [`ProxyAgent`](https://undici.nodejs.org/#/docs/api/ProxyAgent) and used for all external requests. HTTP(S) only.
if some feature breaks when using a proxy, please make a new issue about it!
the value is a URL.
### FREEBIND_CIDR
IPv6 prefix used for randomly assigning addresses to cobalt requests. available only on linux systems.
setting a `FREEBIND_CIDR` allows cobalt to pick a random IP for every download and use it for all requests it makes for that particular download.
to use freebind in cobalt, you need to follow its [setup instructions](https://github.com/imputnet/freebind.js?tab=readme-ov-file#setup) first.
if you want to use this option and run cobalt in a docker container, you also need to set the `API_LISTEN_ADDRESS` env variable to `127.0.0.1` and set `network_mode` for the container to `host`.
the value is an IPv6 range.
## limits
[*jump to the table*](#limit-vars)
### DURATION_LIMIT
media duration limit, in **seconds**
the value is a number.
### TUNNEL_LIFESPAN
the duration for which tunnel info is stored in ram, **in seconds**.
it's recommended to keep this value either default or as low as possible to preserve efficiency and user privacy.
the value is a number.
### RATELIMIT_WINDOW
rate limit time window for api requests, but not session requests, in **seconds**.
the value is a number.
### RATELIMIT_MAX
amount of api requests to be allowed within the time window of `RATELIMIT_WINDOW`.
the value is a number.
### SESSION_RATELIMIT_WINDOW
rate limit time window for session creation requests, in **seconds**.
the value is a number.
### SESSION_RATELIMIT
amount of session requests to be allowed within the time window of `SESSION_RATELIMIT_WINDOW`.
the value is a number.
## security
[*jump to the table*](#security-vars)
> [!NOTE]
> in order to enable turnstile bot protection, `TURNSTILE_SITEKEY`, `TURNSTILE_SECRET`, and `JWT_SECRET` must be set. all three at once.
### CORS_WILDCARD
defines whether cross-origin resource sharing is enabled. when enabled, your instance will be accessible from foreign web pages.
the value is a number. 0: disabled. 1: enabled.
### CORS_URL
configures the [cross-origin resource sharing origin](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Access-Control-Allow-Origin). your instance will be available only from this URL if `CORS_WILDCARD` is set to `0`.
the value is a URL.
### TURNSTILE_SITEKEY
[cloudflare turnstile](https://www.cloudflare.com/products/turnstile/) sitekey used by the web client to request & solve a challenge to prove that the user is not a bot.
the value is a specific key.
### TURNSTILE_SECRET
[cloudflare turnstile](https://www.cloudflare.com/products/turnstile/) secret used by the processing instance to verify that the client solved the challenge successfully.
the value is a specific key.
### JWT_SECRET
the secret used for issuing JWT tokens for request authentication. the value must be a random, secure, and long string (over 16 characters).
the value is a specific key.
### JWT_EXPIRY
the duration of how long a cobalt-issued JWT token will remain valid, in seconds.
the value is a number.
### API_KEY_URL
the URL to the the external or local key database. for local files you have to specify a local path using the `file://` protocol.
see [the api key section](/docs/protect-an-instance.md#api-key-file-format) in the "how to protect your cobalt instance" document for more details.
the value is a URL.
### API_AUTH_REQUIRED
when set to `1`, the user always needs to be authenticated in some way before they can access the API (either via an api key or via turnstile, if enabled).
the value is a number.
## service-specific
[*jump to the table*](#service-specific-vars)
### CUSTOM_INNERTUBE_CLIENT
innertube client that will be used instead of the default one.
the value is a string.
### YOUTUBE_SESSION_SERVER
URL to an instance of [yt-session-generator](https://github.com/imputnet/yt-session-generator). used for automatically pulling `poToken` & `visitor_data` for youtube. can be local or remote.
the value is a URL.
### YOUTUBE_SESSION_INNERTUBE_CLIENT
innertube client that's compatible with botguard's (web) `poToken` and `visitor_data`.
the value is a string.

54
docs/protect-an-instance.md
View File

@ -114,7 +114,7 @@ if you want to use your instance outside of web interface, you'll need an api ke
> >
> if api keys leak, you'll have to update/remove all UUIDs to revoke them. > if api keys leak, you'll have to update/remove all UUIDs to revoke them.
1. create a `keys.json` file following [the schema and example here](/docs//run-an-instance.md#api-key-file-format). 1. create a `keys.json` file following [the schema and example down below](#api-key-file-format).
2. expose the `keys.json` to the docker container: 2. expose the `keys.json` to the docker container:
```yml ```yml
@ -148,3 +148,55 @@ environment:
### why not make keys exclusive by default? ### why not make keys exclusive by default?
keys may be useful for going around rate limiting, keys may be useful for going around rate limiting,
while keeping the rest of api rate limited, with no turnstile in place. while keeping the rest of api rate limited, with no turnstile in place.
## api key file format
the file is a JSON-serialized object with the following structure:
```typescript
type KeyFileContents = Record<
UUIDv4String,
{
name?: string,
limit?: number | "unlimited",
ips?: (CIDRString | IPString)[],
userAgents?: string[]
}
>;
```
where *`UUIDv4String`* is a stringified version of a UUIDv4 identifier.
- **name** is a field for your own reference, it is not used by cobalt anywhere.
- **`limit`** specifies how many requests the API key can make during the window specified in the `RATELIMIT_WINDOW` env.
- when omitted, the limit specified in `RATELIMIT_MAX` will be used.
- it can be also set to `"unlimited"`, in which case the API key bypasses all rate limits.
- **`ips`** contains an array of allowlisted IP ranges, which can be specified both as individual ips or CIDR ranges (e.g. *`["192.168.42.69", "2001:db8::48", "10.0.0.0/8", "fe80::/10"]`*).
- when specified, only requests from these ip ranges can use the specified api key.
- when omitted, any IP can be used to make requests with that API key.
- **`userAgents`** contains an array of allowed user agents, with support for wildcards (e.g. *`["cobaltbot/1.0", "Mozilla/5.0 * Chrome/*"]`*).
- when specified, requests with a `user-agent` that does not appear in this array will be rejected.
- when omitted, any user agent can be specified to make requests with that API key.
- if both `ips` and `userAgents` are set, the tokens will be limited by both parameters.
- if cobalt detects any problem with your key file, it will be ignored and a warning will be printed to the console.
an example key file could look like this:
```json
{
"b5c7160a-b655-4c7a-b500-de839f094550": {
"limit": 10,
"ips": ["10.0.0.0/8", "192.168.42.42"],
"userAgents": ["*Chrome*"]
},
"b00b1234-a3e5-99b1-c6d1-dba4512ae190": {
"limit": "unlimited",
"ips": ["192.168.1.2"],
"userAgents": ["cobaltbot/1.0"]
}
}
```
if you are configuring a key file, **do not use the UUID from the example** but instead generate your own. you can do this by running the following command if you have node.js installed:
`node -e "console.log(crypto.randomUUID())"`

92
docs/run-an-instance.md
View File

@ -54,93 +54,5 @@ sudo apt install nscd
sudo service nscd start sudo service nscd start
``` ```
## list of environment variables for api ## list of environment variables
| variable name | default | example | description | [this section has moved](/docs/api-env-variables.md) to a dedicated document that is way easier to understand and maintain. go check it out!
|:----------------------|:----------|:------------------------|:------------|
| `API_PORT` | `9000` | `9000` | changes port from which api server is accessible. |
| `API_LISTEN_ADDRESS` | `0.0.0.0` | `127.0.0.1` | changes address from which api server is accessible. **if you are using docker, you usually don't need to configure this.** |
| `API_URL` | ➖ | `https://api.cobalt.tools/` | changes url from which api server is accessible. <br> ***REQUIRED TO RUN THE API***. |
| `API_NAME` | `unknown` | `ams-1` | api server name that is shown in `/api/serverInfo`. |
| `API_EXTERNAL_PROXY` | ➖ | `http://user:password@127.0.0.1:8080`| url of the proxy that will be passed to [`ProxyAgent`](https://undici.nodejs.org/#/docs/api/ProxyAgent) and used for all external requests. HTTP(S) only. |
| `CORS_WILDCARD` | `1` | `0` | toggles cross-origin resource sharing. <br> `0`: disabled. `1`: enabled. |
| `CORS_URL` | not used | `https://cobalt.tools` | cross-origin resource sharing url. api will be available only from this url if `CORS_WILDCARD` is set to `0`. |
| `COOKIE_PATH` | not used | `/cookies.json` | path for cookie file relative to main folder. |
| `PROCESSING_PRIORITY` | not used | `10` | changes `nice` value* for ffmpeg subprocess. available only on unix systems. |
| `FREEBIND_CIDR` | ➖ | `2001:db8::/32` | IPv6 prefix used for randomly assigning addresses to cobalt requests. only supported on linux systems. see below for more info. |
| `RATELIMIT_WINDOW` | `60` | `120` | rate limit time window in **seconds**. |
| `RATELIMIT_MAX` | `20` | `30` | max requests per time window. requests above this amount will be blocked for the rate limit window duration. |
| `DURATION_LIMIT` | `10800` | `18000` | max allowed video duration in **seconds**. |
| `TUNNEL_LIFESPAN` | `90` | `120` | the duration for which tunnel info is stored in ram, **in seconds**. |
| `TURNSTILE_SITEKEY` | ➖ | `1x00000000000000000000BB` | [cloudflare turnstile](https://www.cloudflare.com/products/turnstile/) sitekey used by browser clients to request a challenge.\*\* |
| `TURNSTILE_SECRET` | ➖ | `1x0000000000000000000000000000000AA` | [cloudflare turnstile](https://www.cloudflare.com/products/turnstile/) secret used by cobalt to verify the client successfully solved the challenge.\*\* |
| `JWT_SECRET` | ➖ | ➖ | the secret used for issuing JWT tokens for request authentication. to choose a value, generate a random, secure, long string (ideally >=16 characters).\*\* |
| `JWT_EXPIRY` | `120` | `240` | the duration of how long a cobalt-issued JWT token will remain valid, in seconds. |
| `API_KEY_URL` | ➖ | `file://keys.json` | the location of the api key database. for loading API keys, cobalt supports HTTP(S) urls, or local files by specifying a local path using the `file://` protocol. see the "api key file format" below for more details. |
| `API_AUTH_REQUIRED` | ➖ | `1` | when set to `1`, the user always needs to be authenticated in some way before they can access the API (either via an api key or via turnstile, if enabled). |
| `API_REDIS_URL` | ➖ | `redis://localhost:6379` | when set, cobalt uses redis instead of internal memory for the tunnel cache. |
| `API_INSTANCE_COUNT` | ➖ | `2` | supported only on Linux and node.js `>=23.1.0`. when configured, cobalt will spawn multiple sub-instances amongst which requests will be balanced. |
| `DISABLED_SERVICES` | ➖ | `bilibili,youtube` | comma-separated list which disables certain services from being used. |
| `CUSTOM_INNERTUBE_CLIENT` | ➖ | `IOS` | innertube client that will be used instead of the default one. |
| `YOUTUBE_SESSION_SERVER` | ➖ | `http://localhost:8080/` | url to an instance of [invidious' youtube-trusted-session-generator](https://github.com/iv-org/youtube-trusted-session-generator) or its fork/counterpart. used for automatically pulling poToken & visitor_data for youtube. can be local or remote. |
\* the higher the nice value, the lower the priority. [read more here](https://en.wikipedia.org/wiki/Nice_(Unix)).
\*\* in order to enable turnstile bot protection, all three **`TURNSTILE_SITEKEY`, `TURNSTILE_SECRET` and `JWT_SECRET`** need to be set.
#### FREEBIND_CIDR
setting a `FREEBIND_CIDR` allows cobalt to pick a random IP for every download and use it for all
requests it makes for that particular download. to use freebind in cobalt, you need to follow its [setup instructions](https://github.com/imputnet/freebind.js?tab=readme-ov-file#setup) first. if you configure this option while running cobalt
in a docker container, you also need to set the `API_LISTEN_ADDRESS` env to `127.0.0.1`, and set
`network_mode` for the container to `host`.
## api key file format
the file is a JSON-serialized object with the following structure:
```typescript
type KeyFileContents = Record<
UUIDv4String,
{
name?: string,
limit?: number | "unlimited",
ips?: (CIDRString | IPString)[],
userAgents?: string[]
}
>;
```
where *`UUIDv4String`* is a stringified version of a UUIDv4 identifier.
- **name** is a field for your own reference, it is not used by cobalt anywhere.
- **`limit`** specifies how many requests the API key can make during the window specified in the `RATELIMIT_WINDOW` env.
- when omitted, the limit specified in `RATELIMIT_MAX` will be used.
- it can be also set to `"unlimited"`, in which case the API key bypasses all rate limits.
- **`ips`** contains an array of allowlisted IP ranges, which can be specified both as individual ips or CIDR ranges (e.g. *`["192.168.42.69", "2001:db8::48", "10.0.0.0/8", "fe80::/10"]`*).
- when specified, only requests from these ip ranges can use the specified api key.
- when omitted, any IP can be used to make requests with that API key.
- **`userAgents`** contains an array of allowed user agents, with support for wildcards (e.g. *`["cobaltbot/1.0", "Mozilla/5.0 * Chrome/*"]`*).
- when specified, requests with a `user-agent` that does not appear in this array will be rejected.
- when omitted, any user agent can be specified to make requests with that API key.
- if both `ips` and `userAgents` are set, the tokens will be limited by both parameters.
- if cobalt detects any problem with your key file, it will be ignored and a warning will be printed to the console.
an example key file could look like this:
```json
{
"b5c7160a-b655-4c7a-b500-de839f094550": {
"limit": 10,
"ips": ["10.0.0.0/8", "192.168.42.42"],
"userAgents": ["*Chrome*"]
},
"b00b1234-a3e5-99b1-c6d1-dba4512ae190": {
"limit": "unlimited",
"ips": ["192.168.1.2"],
"userAgents": ["cobaltbot/1.0"]
}
}
```
if you are configuring a key file, **do not use the UUID from the example** but instead generate your own. you can do this by running the following command if you have node.js installed:
`node -e "console.log(crypto.randomUUID())"`