Skip to main content

Data Sources

Connect to external databases, data warehouses, and cloud storage services. Once configured, these connections are available in your workflows via the STRONGLY_SERVICES environment variable.

Supported Data Source Types​

Choose from over 60 supported data source types across 9 categories:

CategoryProvidersUse Cases
RelationalMySQL, PostgreSQL, MSSQL, Oracle, Redshift, Snowflake, BigQuery, CockroachDB, CrateDB, TimescaleDB, QuestDB, ClickHouse, SingleStore, GreenplumTransactional data, structured queries, analytics
Document/NoSQLMongoDB, Elasticsearch, DynamoDB, Firestore, Supabase, CouchDB, CouchbaseDocument storage, search, flexible schemas
Key-Value/CacheRedis, MemcachedCaching, session storage, real-time data
GraphNeo4j, Amazon Neptune, ArangoDB, TigerGraphGraph relationships, knowledge graphs
VectorMilvus, Pinecone, Weaviate, Qdrant, Chroma, pgvector, LanceDB, Vespa, MarqoAI embeddings, semantic search, RAG systems
Multi-ModelSurrealDB, FaunaDBMulti-paradigm data modeling
SpreadsheetAirtable, Google Sheets, Baserow, NocoDB, SeaTable, GristLow-code databases, spreadsheet data
Cloud StorageAmazon S3, Google Cloud Storage, MinIO, Azure Blob StorageFile storage, data lakes, backups
Message QueueRabbitMQ, Apache Kafka, Amazon SQS, Apache PulsarEvent streaming, message brokering
GenericJDBC, ODBCAny JDBC/ODBC-compatible database
Additional Documented Types

Dedicated configuration pages exist for the most commonly used types listed above. All other types (MSSQL, Oracle, Redshift, Elasticsearch, DynamoDB, Neo4j, Pinecone, Kafka, etc.) follow the same general creation workflow described in How to Connect Data Sources. The creation form dynamically renders the correct credential fields for every supported type.

Using Data Sources in Workflows​

When you attach a data source to your workflow during deployment, its credentials are automatically injected via the STRONGLY_SERVICES environment variable. Credentials are decrypted server-side and included as top-level fields (not nested under a credentials key). A connectionString is also included for supported types (MySQL, PostgreSQL, MongoDB, Redis, MSSQL, Oracle, Redshift, Snowflake, Neo4j, ClickHouse).

STRONGLY_SERVICES Structure​

{
  "datasources": {
    "prod_postgres": {
      "type": "postgres",
      "name": "prod-postgres",
      "host": "postgres.example.com",
      "port": 5432,
      "database": "myapp",
      "username": "app_user",
      "password": "actual_password",
      "ssl": true,
      "connectionString": "postgresql://app_user:actual_password@postgres.example.com:5432/myapp"
    }
  }
}

The data source key is the data source name sanitized to lowercase with non-alphanumeric characters replaced by underscores (e.g., prod-postgres becomes prod_postgres).

Each data source entry contains:

  • type -- the data source type string (e.g., postgres, mysql, s3)
  • name -- the display label of the data source
  • All credential fields spread at the top level (e.g., host, port, username, password, accessKeyId, etc.)
  • connectionString -- auto-generated connection string (only for supported types)

Python Example -- PostgreSQL​

import os, json
import psycopg2

# Parse STRONGLY_SERVICES environment variable
services = json.loads(os.getenv('STRONGLY_SERVICES', '{}'))
datasources = services.get('datasources', {})

# Get your data source by its sanitized name
pg = datasources['prod_postgres']

# Connect using top-level credential fields
conn = psycopg2.connect(
    host=pg['host'],
    port=pg['port'],
    database=pg['database'],
    user=pg['username'],
    password=pg['password'],
    sslmode='require' if pg.get('ssl') else 'prefer'
)

# Or use the connection string directly
conn = psycopg2.connect(pg['connectionString'])

# Execute queries
cursor = conn.cursor()
cursor.execute("SELECT * FROM users WHERE active = true")
users = cursor.fetchall()

Node.js Example -- MongoDB​

const { MongoClient } = require('mongodb');

// Parse STRONGLY_SERVICES environment variable
const services = JSON.parse(process.env.STRONGLY_SERVICES || '{}');
const datasources = services.datasources || {};

// Get MongoDB data source (fields are at the top level)
const mongo = datasources['prod_mongodb'];

// Build connection string from top-level fields
const authPart = mongo.username ? `${mongo.username}:${mongo.password}@` : '';
const url = `mongodb://${authPart}${mongo.host}:${mongo.port}/${mongo.database}`;

// Connect
const client = new MongoClient(url);
await client.connect();
const db = client.db(mongo.database);

// Query
const users = await db.collection('users').find({ active: true }).toArray();

Security​

Credential Encryption​

All credentials are encrypted at rest. The encryption key is configured via the API_KEY_ENCRYPTION_SECRET environment variable (minimum 32 characters required). Credentials are never sent to the client -- they are excluded from all publications and only decrypted server-side when needed for connection testing, metadata fetching, or building STRONGLY_SERVICES.

Permission Model​

Data sources support three levels of access control:

  • Private (default): Only the owner can access the data source
  • All Users (allowAllUsers): All users in the organization can use the connection
  • Specific Users (allowedUsers): Only selected users can access the data source

Permissions are managed through the datasources.updatePermissions, datasources.share, and datasources.unshare methods. In multi-tenant (SaaS) mode, sharing is restricted to users within the same organization.

Managing Data Sources​

View Connection Details​

  • Click data source name to view details page
  • See connection string (passwords masked)
  • View metadata: tables, schemas, databases, collections, buckets
  • Check last tested time and status

Test Connection​

The platform automatically tests the connection when creating or updating a data source. You can also manually trigger a test using the Test Connection button on the data source details page. The test result updates the data source status to connected or error.

For types with native driver support (MySQL, PostgreSQL, MongoDB, Redis, S3/MinIO, Snowflake, BigQuery, Oracle, MSSQL, Neo4j, Elasticsearch, Redshift, ClickHouse, QuestDB, DynamoDB, SQS, GCS), the platform performs an actual connection test (e.g., SELECT 1, ping, ListBuckets). For other types, credentials are saved with a message indicating verification will happen at runtime.

Schema Discovery​

After connecting, use Refresh Metadata to discover the structure of your data source. Depending on the type, this returns:

  • Tables/Collections: Available tables or collections
  • Schemas: Database schemas (PostgreSQL, Snowflake, Redshift, Oracle, etc.)
  • Databases: Available databases
  • Buckets: S3/cloud storage buckets
  • Size and row counts: Approximate storage size and row counts

Full native metadata connectors are available for: MySQL, PostgreSQL, MongoDB, Redis, S3, MinIO, Snowflake, BigQuery, Oracle, Redshift, Neo4j, and PostgreSQL-compatible databases (CockroachDB, CrateDB, TimescaleDB, Greenplum, pgvector, QuestDB) and MySQL-compatible databases (SingleStore). Other types use a generic HTTP-based connector that returns empty metadata.

Additionally, you can fetch column-level metadata for individual tables using the table columns feature (supported for MySQL, PostgreSQL, Oracle, Redshift, Snowflake, BigQuery, and MongoDB).

Update Credentials​

  • Click Edit button
  • Update label, description, credentials, or tags
  • Connection is re-tested automatically when credentials change
  • Click Save to apply changes

Usage Tracking​

  • View usage count and last used timestamp
  • Usage count increments when data sources are used in workflows

Delete Data Source​

  • Click Delete button
  • Confirm deletion (irreversible)
  • Associated permissions are also removed
  • Workflows using this source will lose connection
Security

All credentials are encrypted at rest and never transmitted to the client. Passwords are never displayed in plaintext after initial entry.