# Storage block

|     |     |
| --- | --- |
|  | This documentation is for<br>a previous<br>version of Cerbos. Choose 0.53.0 from the version picker at the top right or navigate to [https://docs.cerbos.dev](https://docs.cerbos.dev/) for the latest version. |

Cerbos supports multiple backends for storing policies. Which storage driver to use is defined by the `driver` setting.

## Blob driver

Cerbos policies can be stored in AWS S3, Google Cloud Storage, or any other S3-compatible storage systems such as [Minio](https://www.minio.io/).

### Configuration keys

- `bucket`: Required. A URL specifying the service (e.g. S3, GCS), the storage bucket and any other configuration parameters required by the provider.

- AWS S3: `s3://my-bucket?region=us-west-1`. Must specify region in the URL.

- Google Cloud Storage: `gs://my-bucket`

- S3-compatible (e.g. Minio): `s3://my-bucket?endpoint=http://my.minio.local:8080&hostname_immutable=true&region=local`. Must specify region in the URL.

- `prefix`: Optional. Look for policies only under this key prefix.

- `workDir`: Optional. Path to the local directory to download the policies to. Defaults to the system cache directory if not specified.

- `updatePollInterval`: Optional. How frequently the blob store should be checked to discover new or updated policies. Defaults to 0 — which disables polling.

- `requestTimeout`: Optional. HTTP request timeout. It takes an HTTP request to download a policy file. Defaults to 5s.

- `downloadTimeout`: Optional. Timeout to download all policies from the the storage provider. Must be greater than the `requestTimeout`. Defaults to 60s.

|     |     |
| --- | --- |
|  | Setting the `updatePollInterval` to a low value could increase resource consumption in both the client and the server systems. Some managed service providers may even impose rate limits or temporary suspensions on your account if the number of requests is too high. |

Credentials for accessing the storage buckets are retrieved from the environment. The method of specifying credentials in the environment vary by cloud provider and security configuration. Usually, it involves defining environment variables such as `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` for S3 and `GOOGLE_APPLICATION_CREDENTIALS` for GCS. Refer to the relevant cloud provider documentation for more details.

- AWS: [https://docs.aws.amazon.com/sdk-for-go/api/aws/session/](https://docs.aws.amazon.com/sdk-for-go/api/aws/session/)

- Google: [https://cloud.google.com/docs/authentication/provide-credentials-adc](https://cloud.google.com/docs/authentication/provide-credentials-adc)

### AWS S3

```yaml
storage:
  driver: "blob"
  blob:
    bucket: "s3://my-bucket-name?region=us-east-2"
    prefix: policies
    workDir: ${HOME}/tmp/cerbos/work
    updatePollInterval: 15s
    downloadTimeout: 30s
    requestTimeout: 10s
```

### Google Cloud Storage

```yaml
storage:
  driver: "blob"
  blob:
    bucket: "gs://my-bucket-name"
    workDir: ${HOME}/tmp/cerbos/work
    updatePollInterval: 10s
```

### Minio local container

```yaml
storage:
  driver: "blob"
  blob:
    bucket: "s3://my-bucket-name?endpoint=http://localhost:9000&hostname_immutable=true&region=local"
    workDir: ${HOME}/tmp/cerbos/work
    updatePollInterval: 10s
```

## Disk driver

The disk driver is a way to serve the policies from a directory on the filesystem. Any `.yaml`, `.yml` or `.json` files in the directory tree rooted at the given path will be read and parsed as policies.

### Static fileset with no change detection

```yaml
storage:
  driver: disk
  disk:
    directory: /etc/cerbos/policies
```

### Dynamic fileset with change detection

```yaml
storage:
  driver: disk
  disk:
    directory: /etc/cerbos/policies
    watchForChanges: true
```

### Archive files

Alternatively, you can opt to archive and/or compress your policies directory into a Zip (`.zip`), Tar (`.tar`) or Gzip file (`.tgz` or `.tar.gz`). The archive is assumed to be laid out like a standard policy directory. It must contain no non-policy YAML files.

You specify the file in your config like so:

### Archived fileset using a Zip file

```yaml
storage:
  driver: disk
  disk:
    directory: /etc/cerbos/policies.zip
```

## Git driver

Git is the preferred method of storing Cerbos policies. The server is smart enough to detect when new commits are made to the git repository and refresh its state based on the changes.

- Git repositories can be local (`file` protocol) or remote (`ssh` or `https`). Please note that the local `file` protocol requires `git` to be available and cannot be used with the Cerbos container.

- If no `branch` is specified, the default branch would be the `master` branch.

- If no `subDir` is specified, the entire repository would be scanned for policies (`.yaml`, `.yml` or `.json`).

- The `checkoutDir` is the working directory of the server and must be writable by the server process.

- If `updatePollInterval` is set to 0, the source repository will not be polled to pick up any new commits.

### Local git repository

```yaml
storage:
  driver: "git"
  git:
    protocol: file
    url: file://${HOME}/tmp/cerbos/policies
    checkoutDir: ${HOME}/tmp/cerbos/work
    updatePollInterval: 10s
```

### Remote git repository accessed over HTTPS

```yaml
storage:
  driver: "git"
  git:
    protocol: https
    url: https://github.com/cerbos/policy-test.git
    branch: main
    subDir: policies
    checkoutDir: ${HOME}/tmp/work/policies
    updatePollInterval: 60s
    operationTimeout: 30s
    https:
      username: cerbos
      password: ${GITHUB_TOKEN}
```

### Remote git repository accessed over SSH

```yaml
storage:
  driver: "git"
  git:
    protocol: ssh
    url: github.com:cerbos/policy-test.git
    branch: main
    subDir: policies
    checkoutDir: ${HOME}/tmp/cerbos/work
    updatePollInterval: 60s
    ssh:
      user: git
      privateKeyFile: ${HOME}/.ssh/id_rsa
```

## Hub driver

Connects the PDP to a Cerbos Hub [Deployment](https://docs.cerbos.dev/cerbos-hub/deployments). A Cerbos Hub Deployment tracks a connected Git repository or a [managed Cerbos Hub store](https://docs.cerbos.dev/cerbos-hub/policy-stores) and automatically compiles, tests and pushes an optimized policy bundle to the PDP whenever a policy change is committed.

## MySQL driver

The MySQL storage backend is one of the dynamic stores that supports adding or updating policies at runtime through the [Admin API](https://docs.cerbos.dev/cerbos/0.50.0/configuration/server#admin-api).

The driver configuration expects the connection details to be provided as a DSN in the following form:

```
[username[:password]@][protocol[(address)]]/dbname[?param1=value1&...&paramN=valueN]
```

You can use environment variable references in the URL to avoid storing credentials as part of the Cerbos configuration file.

### Using MySQL as a storage backend for Cerbos

```yaml
storage:
  driver: "mysql"
  mysql:
    dsn: "${MYSQL_USER}:${MYSQL_PASSWORD}@tcp(localhost:3306)/cerbos"
```

### Secure connections

If your MySQL server requires TLS or if you want to use RSA key pair-based password exchange, you can configure those settings as follows:

### TLS certificates

```yaml
storage:
  driver: "mysql"
  mysql:
    dsn: "${MYSQL_USER}:${MYSQL_PASSWORD}@tcp(localhost:3306)/cerbos?tls=mysecuretls"
    tls:
      mysecuretls:
        caCert: /path/to/ca_certificate.crt
        cert: /path/to/certificate.crt
        key: /path/to/private.key
```

### Server public key

```yaml
storage:
  driver: "mysql"
  mysql:
    dsn: "${MYSQL_USER}:${MYSQL_PASSWORD}@tcp(localhost:3306)/cerbos?serverPubKey=mypubkey"
    serverPubKey:
      mypubkey: /path/to/server_public_key.pem
```

### Connection pool

Cerbos uses a connection pool when connecting to a database. You can configure the connection pool settings by adding a `connPool` section to the driver configuration.

### Connection retries

Cerbos attempts to connect to the database on startup and exits if connection cannot be established after three attempts. You can configure the connection retry settings using the `connRetry` options.

### Database object definitions

You can customise the script below to suit your environment. Make sure to specify a strong password for the `cerbos_user` user.

```sql
CREATE DATABASE IF NOT EXISTS cerbos CHARACTER SET utf8mb4;
CREATE TABLE IF NOT EXISTS policy (...);
CREATE TABLE IF NOT EXISTS policy_dependency (...);
CREATE TABLE IF NOT EXISTS policy_ancestor (...);
CREATE TABLE IF NOT EXISTS policy_revision (...);
CREATE TABLE IF NOT EXISTS attr_schema_defs (...);
CREATE USER IF NOT EXISTS cerbos_user IDENTIFIED BY 'changeme';
GRANT SELECT,INSERT,UPDATE,DELETE ON cerbos.policy TO cerbos_user;
```

## Overlay driver

You can provide redundancy by configuring an `overlay` driver, which wraps a `base` and a `fallback` driver. Under normal operation, the base driver will be targeted as usual. However, if the driver consistently errors, the PDP will start targeting the fallback driver instead. The fallback is determined by a configurable [circuit breaker pattern](https://learn.microsoft.com/en-us/previous-versions/msp-n-p/dn589784(v=pandp.10)).

```yaml
storage:
  driver: "overlay"
  overlay:
    baseDriver: postgres
    fallbackDriver: disk
    fallbackErrorThreshold: 5 # number of errors that occur within the fallbackErrorWindow to trigger failover
    fallbackErrorWindow: 5s # the rolling window in which errors are aggregated
  disk:
    directory: policies
    watchForChanges: true
  postgres:
    url: "postgres://${PG_USER}:${PG_PASSWORD}@localhost:5432/postgres?sslmode=disable&search_path=cerbos"
```

## Postgres driver

The Postgres storage backend is one of the dynamic stores that supports adding or updating policies at runtime through the [Admin API](https://docs.cerbos.dev/cerbos/0.50.0/configuration/server#admin-api).

The driver configuration expects the connection details to be provided as connection URL. See [Postgres connstring documentation](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING) for more information. Use the `search_path` parameter to point to the schema containing the Cerbos tables.

### Using Postgres as a storage backend for Cerbos

```yaml
storage:
  driver: "postgres"
  postgres:
    url: "postgres://${PG_USER}:${PG_PASSWORD}@localhost:5432/postgres?sslmode=disable&search_path=cerbos"
```

### Database object definitions

You can customise the script below to suit your environment. Make sure to specify a strong password for the `cerbos_user` user.

```sql
CREATE SCHEMA IF NOT EXISTS cerbos;
CREATE TABLE IF NOT EXISTS policy (...);
CREATE TABLE IF NOT EXISTS policy_dependency (...);
CREATE TABLE IF NOT EXISTS policy_ancestor (...);
CREATE TABLE IF NOT EXISTS policy_revision (...);
CREATE USER cerbos_user WITH PASSWORD 'changeme';
GRANT CONNECT ON DATABASE postgres TO cerbos_user;
```

## SQLite3 driver

The SQLite3 storage backend is one of the dynamic stores that supports adding or updating policies at runtime through the [Admin API](https://docs.cerbos.dev/cerbos/0.50.0/configuration/server#admin-api).

### In-memory ephemeral database

```yaml
storage:
  driver: "sqlite3"
  sqlite3:
    dsn: "file::memory:?cache=shared"
```

### On-disk persistent database

```yaml
storage:
  driver: "sqlite3"
  sqlite3:
    dsn: "file:/tmp/cerbos.sqlite?mode=rwc&cache=shared&_fk=true"
```