Storage block :: Cerbos Authorization Management Platform // Documentation

Storage block

This documentation is for an as-yet unreleased version of Cerbos PDP. Choose 0.53.0 from the version picker at the top right or navigate to https://docs.cerbos.dev for the latest version.

Cerbos supports multiple backends for storing policies. Which storage driver to use is defined by the driver setting.

Blob driver

Cerbos policies can be stored in AWS S3, Google Cloud Storage, or any other S3-compatible storage systems such as Minio.

Configuration keys

Credentials for accessing the storage buckets are retrieved from the environment. The method of specifying credentials in the environment vary by cloud provider and security configuration. Usually, it involves defining environment variables such as AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY for S3 and GOOGLE_APPLICATION_CREDENTIALS for GCS. Refer to the relevant cloud provider documentation for more details.

AWS S3

storage:
  driver: "blob"
  blob:
    bucket: "s3://my-bucket-name?region=us-east-2"
    prefix: policies
    workDir: ${HOME}/tmp/cerbos/work
    updatePollInterval: 15s
    downloadTimeout: 30s
    requestTimeout: 10s

Google Cloud Storage

storage:
  driver: "blob"
  blob:
    bucket: "gs://my-bucket-name"
    workDir: ${HOME}/tmp/cerbos/work
    updatePollInterval: 10s

Minio local container

storage:
  driver: "blob"
  blob:
    bucket: "s3://my-bucket-name?endpoint=http://localhost:9000&hostname_immutable=true&region=local"
    workDir: ${HOME}/tmp/cerbos/work
    updatePollInterval: 10s

Disk driver

The disk driver is a way to serve the policies from a directory on the filesystem. Any .yaml, .yml or .json files in the directory tree rooted at the given path will be read and parsed as policies.

Static fileset with no change detection

storage:
  driver: disk
  disk:
    directory: /etc/cerbos/policies

Dynamic fileset with change detection

storage:
  driver: disk
  disk:
    directory: /etc/cerbos/policies
    watchForChanges: true

Archive files

Alternatively, you can opt to archive and/or compress your policies directory into a Zip (.zip), Tar (.tar) or Gzip file (.tgz or .tar.gz). The archive is assumed to be laid out like a standard policy directory. It must contain no non-policy YAML files.

Archived fileset using a Zip file

storage:
  driver: disk
  disk:
    directory: /etc/cerbos/policies.zip

Git driver

Git is the preferred method of storing Cerbos policies. The server is smart enough to detect when new commits are made to the git repository and refresh its state based on the changes.

Local git repository

storage:
  driver: "git"
  git:
    protocol: file
    url: file://${HOME}/tmp/cerbos/policies
    checkoutDir: ${HOME}/tmp/cerbos/work
    updatePollInterval: 10s

Remote git repository accessed over HTTPS

storage:
  driver: "git"
  git:
    protocol: https
    url: https://github.com/cerbos/policy-test.git
    branch: main
    subDir: policies
    checkoutDir: ${HOME}/tmp/work/policies
    updatePollInterval: 60s
    operationTimeout: 30s
    https:
      username: cerbos
      password: ${GITHUB_TOKEN}

Remote git repository accessed over SSH

storage:
  driver: "git"
  git:
    protocol: ssh
    url: github.com:cerbos/policy-test.git
    branch: main
    subDir: policies
    checkoutDir: ${HOME}/tmp/cerbos/work
    updatePollInterval: 60s
    ssh:
      user: git
      privateKeyFile: ${HOME}/.ssh/id_rsa

Hub driver

Connects the PDP to a Cerbos Hub Deployment. A Cerbos Hub Deployment tracks a connected Git repository or a managed Cerbos Hub store and automatically compiles, tests and pushes an optimized policy bundle to the PDP whenever a policy change is committed.

MySQL driver

The MySQL storage backend is one of the dynamic stores that supports adding or updating policies at runtime through the Admin API.

The driver configuration expects the connection details to be provided as a DSN in the following form:

[username[:password]@][protocol[(address)]]/dbname[?param1=value1&...&paramN=valueN]

Using MySQL as a storage backend for Cerbos

storage:
  driver: "mysql"
  mysql:
    dsn: "${MYSQL_USER}:${MYSQL_PASSWORD}@tcp(localhost:3306)/cerbos"

Secure connections

If your MySQL server requires TLS or if you want to use RSA key pair-based password exchange, you can configure those settings as follows:

TLS certificates

storage:
  driver: "mysql"
  mysql:
    dsn: "${MYSQL_USER}:${MYSQL_PASSWORD}@tcp(localhost:3306)/cerbos?tls=mysecuretls"
    tls:
      mysecuretls:
        caCert: /path/to/ca_certificate.crt
        cert: /path/to/certificate.crt
        key: /path/to/private.key

Server public key

storage:
  driver: "mysql"
  mysql:
    dsn: "${MYSQL_USER}:${MYSQL_PASSWORD}@tcp(localhost:3306)/cerbos?serverPubKey=mypubkey"
    serverPubKey:
      mypubkey: /path/to/server_public_key.pem

Connection pool

Cerbos uses a connection pool when connecting to a database. You can configure the connection pool settings by adding a connPool section to the driver configuration.

storage:
  driver: "mysql"
  mysql:
    dsn: "${MYSQL_USER}:${MYSQL_PASSWORD}@tcp(localhost:3306)/cerbos"
    connPool:
      maxLifeTime: 5m
      maxIdleTime: 3m
      maxOpen: 10
      maxIdle: 5

Connection retries

Cerbos attempts to connect to the database on startup and exits if connection cannot be established after three attempts. You can configure the connection retry settings using the connRetry options.

Database object definitions

You can customise the script below to suit your environment. Make sure to specify a strong password for the cerbos_user user.

CREATE DATABASE IF NOT EXISTS cerbos CHARACTER SET utf8mb4;

USE cerbos;

CREATE TABLE IF NOT EXISTS policy (
    id BIGINT PRIMARY KEY,
    kind VARCHAR(128) NOT NULL,
    name VARCHAR(1024) NOT NULL,
    version VARCHAR(128) NOT NULL,
    scope VARCHAR(512),
    description TEXT,
    disabled BOOLEAN default false,
    definition BLOB);

CREATE TABLE IF NOT EXISTS policy_dependency (
    policy_id BIGINT NOT NULL,
    dependency_id BIGINT NOT NULL,
    PRIMARY KEY (policy_id, dependency_id),
    FOREIGN KEY (policy_id) REFERENCES policy(id) ON DELETE CASCADE);

CREATE TABLE IF NOT EXISTS policy_ancestor (
    policy_id BIGINT NOT NULL,
    ancestor_id BIGINT NOT NULL,
    PRIMARY KEY (policy_id, ancestor_id),
    FOREIGN KEY (policy_id) REFERENCES policy(id) ON DELETE CASCADE);

CREATE TABLE IF NOT EXISTS policy_revision (
    revision_id INTEGER AUTO_INCREMENT PRIMARY KEY,
    action ENUM('INSERT', 'UPDATE', 'DELETE'),
    id BIGINT NOT NULL,
    kind VARCHAR(128),
    name VARCHAR(1024),
    version VARCHAR(128),
    scope VARCHAR(512),
    description TEXT,
    disabled BOOLEAN,
    definition BLOB);

CREATE TABLE IF NOT EXISTS attr_schema_defs (
    id VARCHAR(255) PRIMARY KEY,
    definition JSON);

CREATE USER IF NOT EXISTS cerbos_user IDENTIFIED BY 'changeme';
GRANT SELECT,INSERT,UPDATE,DELETE ON cerbos.policy TO cerbos_user;
GRANT SELECT,INSERT,UPDATE,DELETE ON cerbos.attr_schema_defs TO cerbos_user;
GRANT SELECT,INSERT,UPDATE,DELETE ON cerbos.policy_dependency TO cerbos_user;
GRANT SELECT,INSERT,UPDATE,DELETE ON cerbos.policy_ancestor TO cerbos_user;
GRANT SELECT,INSERT,DELETE ON cerbos.policy_revision TO cerbos_user;

Overlay driver

You can provide redundancy by configuring an overlay driver, which wraps a base and a fallback driver. Under normal operation, the base driver will be targeted as usual. However, if the driver consistently errors, the PDP will start targeting the fallback driver instead.

storage:
  driver: "overlay"
  overlay:
    baseDriver: postgres
    fallbackDriver: disk
    fallbackErrorThreshold: 5 # number of errors that occur within the fallbackErrorWindow to trigger failover
    fallbackErrorWindow: 5s # the rolling window in which errors are aggregated
  disk:
    directory: policies
    watchForChanges: true
  postgres:
    url: "postgres://${PG_USER}:${PG_PASSWORD}@localhost:5432/postgres?sslmode=disable&search_path=cerbos"

Postgres driver

The Postgres storage backend is one of the dynamic stores that supports adding or updating policies at runtime through the Admin API.

Using Postgres as a storage backend for Cerbos

storage:
  driver: "postgres"
  postgres:
    url: "postgres://${PG_USER}:${PG_PASSWORD}@localhost:5432/postgres?sslmode=disable&search_path=cerbos"

Connection pool

storage:
  driver: "postgres"
  postgres:
    url: "postgres://${PG_USER}:${PG_PASSWORD}@localhost:5432/postgres?sslmode=disable&search_path=cerbos"
    connPool:
      maxLifeTime: 5m
      maxIdleTime: 3m
      maxOpen: 10
      maxIdle: 5

Connection retries

Database object definitions

You can customise the script below to suit your environment. Make sure to specify a strong password for the cerbos_user user.

CREATE SCHEMA IF NOT EXISTS cerbos;

SET search_path TO cerbos;

CREATE TABLE IF NOT EXISTS policy (
    id bigint NOT NULL PRIMARY KEY,
    kind VARCHAR(128) NOT NULL,
    name VARCHAR(1024) NOT NULL,
    version VARCHAR(128) NOT NULL,
    scope VARCHAR(512),
    description TEXT,
    disabled BOOLEAN default false,
    definition BYTEA
);

CREATE TABLE IF NOT EXISTS policy_dependency (
    policy_id BIGINT,
    dependency_id BIGINT,
    PRIMARY KEY (policy_id, dependency_id),
    FOREIGN KEY (policy_id) REFERENCES cerbos.policy(id) ON DELETE CASCADE
);

CREATE TABLE IF NOT EXISTS policy_ancestor (
    policy_id BIGINT,
    ancestor_id BIGINT,
    PRIMARY KEY (policy_id, ancestor_id),
    FOREIGN KEY (policy_id) REFERENCES cerbos.policy(id) ON DELETE CASCADE
);

CREATE TABLE IF NOT EXISTS policy_revision (
    revision_id SERIAL PRIMARY KEY,
    action VARCHAR(64),
    id BIGINT,
    kind VARCHAR(128),
    name VARCHAR(1024),
    version VARCHAR(128),
    scope VARCHAR(512),
    description TEXT,
    disabled BOOLEAN,
    definition BYTEA,
    update_timestamp TIMESTAMPTZ DEFAULT CURRENT_TIMESTAMP
);

CREATE TABLE IF NOT EXISTS attr_schema_defs (
    id VARCHAR(255) PRIMARY KEY,
    definition JSON
);

CREATE OR REPLACE FUNCTION process_policy_audit() RETURNS TRIGGER AS $policy_audit$
    BEGIN
        IF (TG_OP = 'DELETE') THEN
            INSERT INTO policy_revision(action, id, kind, name, version, scope, description, disabled, definition)
            VALUES('DELETE', OLD.id, OLD.kind, OLD.name, OLD.version, OLD.scope, OLD.description, OLD.disabled, OLD.definition);
        ELSIF (TG_OP = 'UPDATE') THEN
            INSERT INTO policy_revision(action, id, kind, name, version, scope, description, disabled, definition)
            VALUES('UPDATE', NEW.id, NEW.kind, NEW.name, NEW.version, NEW.scope, NEW.description, NEW.disabled, NEW.definition);
        ELSIF (TG_OP = 'INSERT') THEN
            INSERT INTO policy_revision(action, id, kind, name, version, scope, description, disabled, definition)
            VALUES('INSERT', NEW.id, NEW.kind, NEW.name, NEW.version, NEW.scope, NEW.description, NEW.disabled, NEW.definition);
        END IF;
        RETURN NULL;
    END;
$policy_audit$ LANGUAGE plpgsql;

CREATE TRIGGER policy_audit
AFTER INSERT OR UPDATE OR DELETE ON policy
FOR EACH ROW EXECUTE PROCEDURE process_policy_audit();

CREATE USER cerbos_user WITH PASSWORD 'changeme';
GRANT CONNECT ON DATABASE postgres TO cerbos_user;
GRANT USAGE ON SCHEMA cerbos TO cerbos_user;
GRANT SELECT,INSERT,UPDATE,DELETE ON cerbos.policy, cerbos.policy_dependency, cerbos.policy_ancestor, cerbos.attr_schema_defs TO cerbos_user;
GRANT SELECT,INSERT,DELETE ON cerbos.policy_revision TO cerbos_user;
GRANT USAGE,SELECT ON cerbos.policy_revision_revision_id_seq TO cerbos_user;

SQLite3 driver

The SQLite3 storage backend is one of the dynamic stores that supports adding or updating policies at runtime through the Admin API.

In-memory ephemeral database

storage:
  driver: "sqlite3"
  sqlite3:
    dsn: "file::memory:?cache=shared"

On-disk persistent database

storage:
  driver: "sqlite3"
  sqlite3:
    dsn: "file:/tmp/cerbos.sqlite?mode=rwc&cache=shared&_fk=true"