marivo.datasource#

Project-level datasource authoring and management API.

Registration & lifecycle#

`connect`	Open a context-manageable live ibis backend for a datasource.
`DatasourceConnection`	Context-manageable datasource backend connection.
`register`	Create or replace a project datasource file from a DatasourceSpec.
`DatasourceSpec`	Represent a PEP 604 union type
`DuckDBSpec`	DuckDB datasource specification.
`SQLiteSpec`	SQLite datasource specification.
`TrinoSpec`	Trino datasource specification.
`MySQLSpec`	MySQL datasource specification.
`PostgresSpec`	Postgres datasource specification.
`ClickHouseSpec`	ClickHouse datasource specification.
`load`	Load the project datasource catalog.
`list`	List configured project datasources as a displayable DatasourceList.
`remove`	Delete the named project datasource file.
`test`	Round-trip the backend and best-effort cache validated env secrets.

Connection ownership#

connect() returns a caller-owned connection; prefer with md.connect(...). Inspection, sampling, semantic preview, source health, and parity release their operation-owned connections before returning. Analysis caches connections until Session.close(). Connections retain validated environment-secret provenance; host-injected secrets are never cached.

Internal handshakes have an independent 30-second deadline. raw_sql and sampling/preview scopes retain their execution timeout, rather than an end-to-end budget. test() retains its whole-roundtrip deadline. SQLite opens locally on its caller’s thread and cannot be forcibly interrupted during synchronous open.

Credential injection#

`credential_scope`	Select a host credential resolver for new operations and runtimes.
`CredentialRequest`	Read-only reference and operation facts passed to a host resolver.
`CredentialResolver`	Synchronous host adapter; owns authorization and service timeouts.
`SecretValue`	An explicitly readable, non-serializable secret string.
`DatasourceCredentialError`	Host credential resolution failed, with reference-only repair facts.
`DatasourceCredentialScopeError`	The selected resolver differs from the runtime's captured source.

Source constructors#

`csv`	Build a typed DuckDB file source descriptor for CSV files.
`source_param`	Declare one required runtime request parameter for a physical source.
`json`	Build a typed DuckDB JSON physical-source descriptor.
`parquet`	Build a DuckDB file source descriptor for Parquet files.
`duckdb`	Declare a DuckDB datasource.
`sqlite`	Declare a SQLite datasource.
`postgres`	Declare a Postgres datasource.
`mysql`	Declare a MySQL datasource.
`clickhouse`	Declare a ClickHouse datasource.
`trino`	Declare a Trino datasource.
`table`	Build a physical table source descriptor.
`source_column`	Declare one typed physical identifier for a table column binding.

Inspection & snapshots#

`inspect`	Inspect a physical source through metadata and system-catalog hooks only.
`SourceInspection`
`DiscoverySnapshot`	Immutable bounded rows and profiles from one explicit source acquisition.
`PartitionInspection`
`PhysicalExtent`
`Partitioning`
`ExecutionCapabilities`
`raw_sql`	Run terminal SQL exploration and load the complete query result.

DiscoverySnapshot.contract().show() displays the query-free read contract for a snapshot, including its selected columns, retained-row shape, and value evidence state. When persist_values=True, snapshot.retained_values is a tuple of dictionaries keyed by the selected column names; with the default persist_values=False it remains unavailable and no follow-up query occurs.

Scope helpers#

Datasource specs and explicit scopes expose bounded render() / show() cards for agent-relevant state. Specs show only their core connection target, credential field names, and hidden-configuration count; scopes show guards and a bounded predicate preview. Read .fields / .env_refs or .values for exact data, and .contract() for mechanical continuation.

`partition`	Build an explicitly partitioned authoring acquisition scope.
`time_range`	Build a bounded half-open temporal acquisition scope.
`unpruned`	Build an explicitly unpruned authoring acquisition scope.

Description#

python -m marivo help is an environment bootstrap only. Use the sole public coordinator, marivo.help("datasource.<target>"), for bounded focused help rendered from the datasource registry. The md namespace executes datasource operations and intentionally has no md.help() alias. A SourceInspection card points to marivo.help("datasource.SourceInspection.sample") for the complete single-process acquisition and query-free projection chain.

describe

Show literal fields and env refs for one datasource.

Catalog & refs#

Datasource identities use marivo.semantic.ref.datasource(...). The datasource module accepts that exact ref at inspection and raw-SQL boundaries, but does not define a second ref type.

`DatasourceCatalog`	Read-only catalog over configured project datasources.
`DatasourceList`	Displayable collection of configured project datasource summaries.
`DatasourceSummary`	Summary row for one configured project datasource.
`DatasourceDescription`	Literal fields and env refs for one datasource.

Datasource vs source#

md.duckdb(...) and md.sqlite(...) declare datasources. md.table(...) is the source descriptor for internal tables/views inside either datasource. md.parquet(...) and md.csv(...) remain DuckDB file sources. md.json(...) covers DuckDB JSON files and URLs, including a parameterized JSON-body POST request. These sources are used with a datasource ref in inspection and snapshot calls; they are not datasource declarations.

Metadata & sources#

`TableSource`	Represent a PEP 604 union type
`TableColumnBindingIR`	One typed physical identifier in a projected table source.
`PartitionScope`	Explicit pruned selection and positive acquisition guards.
`UnprunedScope`	Explicit unpruned acquisition with positive guards.

Results#

md.raw_sql(...) returns a bounded terminal RawSqlResult. Its shape and row_count describe returned bounded rows: row_count == shape[0] == returned_row_count. Read requested_limit and is_truncated alongside that count; it is not full-source cardinality. Ordered columns and isolated to_pandas() are available, but RawSqlResult has no contract(), typed affordances, or typed-analysis re-entry.

`DatasourceTestResult`	Result of a datasource connectivity round-trip.
`DatasourceFailure`	Bounded structured cause for one failed datasource connection test.