DubUrl.Provider.Text 0.65.0

DubUrl.Providers

Provider implementation packages, database-free unit tests, and real database QA for DubUrl.

The projects under Providers use released DubUrl Core packages and DubUrl.Schema 0.104.0. Use the following commands to build and test.

dotnet build DubUrl.Providers.slnx
dotnet test QA/DubUrl.Providers.Testing
dotnet test DubUrl.BulkCopy.Testing

The provider unit suite compiles the tests alongside their owning provider and runs without a database. Schema source-compatibility QA can opt into a local package with ./build/Initialize-SchemaQa.ps1 -SourceCommit <sha> before building in the same PowerShell session.

The shared live fixtures are in QA/DubUrl.Provider.QA. Their four namespaces are Schema, Connection, BulkCopy, and Query under DubUrl.Providers.QA.Contracts. Concrete fixtures supply their real driver, SQL, and expected behavior. Engine regressions live in Specific subfolders.

Run the embedded schema suites with:

./QA/Schema/run.ps1 -Database Sqlite
./QA/Schema/run.ps1 -Database DuckDb

The schema runner needs Python with defusedxml for TRX verification. PostgreSQL, SQL Server, Oracle, SingleStore, and MySQL also need Docker. Existing provider workflows under .github/workflows show the matching provisioning scripts and supported platforms.

Schema QA verifies each required shared scenario passed exactly once. See QA/Schema/README.md for capability expectations and runner details.

Every active provider QA job requires a passing live connection lifecycle, customer query, schema discovery capability check, and bulk-copy capability check. Native bulk-copy adapters also inherit the live row and null round trip. Schema providers continue to run the seven full schema suites with the mandatory scenario manifest.

CI live QA is disabled for Snowflake, Teradata, and SSAS Tabular and Multidimensional. SSAS keeps its named checks for its package and manifest checks only. The required Apache Drill QA check builds the provider and validates its manifest. Live ODBC QA requires an authenticated driver distribution and is excluded on public runners.

The published DubUrl.ProviderTesting 0.60.0 package remains available for compatibility. The internal DubUrl.Provider.QA project is not published.

Structured schema columns

Schema rendering uses the column's explicit StructuredStorage. The following table describes SQL representation, database enforcement, and catalog recovery independently.

An optional JSON object shape describes the caller's data. Native JSON columns validate syntax within the engine's supported JSON values. The caller supplies any value codec. DuckDB native collections support reversible scalar leaves and unshaped nested JSON. Their DDL rejects required nested values and duplicate struct names under DuckDB's case-insensitive comparison. SingleStore qualifies signed integer and floating-point components up to 16,384 dimensions. SQL Server qualifies up to 1,998 single-precision vector dimensions.

DuckDB vectors support signed and unsigned 8-, 16-, 32-, and 64-bit integers and float32 or float64 components. The ARRAY type fixes the dimensions. The column check counts non-null components, so a nullable vector column can still store SQL NULL. Bare fixed arrays and nested vectors are outside the supported discovery shape.

PostgreSQL and TimescaleDB arrays support Boolean, signed 16-, 32-, and 64-bit integers, float32 and float64, text, binary, UUID, date, and unshaped JSONB elements. Bare rank-one declarations produce incomplete discovery because PostgreSQL accepts higher-rank values in those columns. Multidimensional declarations, non-default collations, domains, and unsupported element types or modifiers also produce incomplete discovery. TimescaleDB discovery reads the hypertable's column definitions and excludes the extension's default internal schemas. Hypertable partitioning is outside schema rendering.

For the opt-in PostgreSQL and TimescaleDB live tests in PostgreSqlArrayLiveTests, set DUBURL_POSTGRES_ARRAY_QA_URL or DUBURL_TIMESCALE_ARRAY_QA_URL to the corresponding disposable database and select the individual test by its fully qualified name.

Bind each PostgreSQL struct shape to a schema-qualified name with PgsqlDialect.WithCompositeType(name, shape). Register nested composites first. Use RenderCompositeTypeDefinitions() to provision new types before executing the table plan. An existing matching type needs only the binding. The table planner reuses that name across columns. Discovery recovers the portable shape, and the raw metadata exposes the qualified name on MetadataColumn.NativeType for rebinding. Supported attributes use the same scalar leaves as PostgreSQL arrays, unshaped JSONB, or another bound composite. Required fields, domains, table row types, and unsupported type modifiers are outside this mapping. The opt-in tests in PostgreSqlCompositeLiveTests use DUBURL_POSTGRES_COMPOSITE_QA_CONNECTION_STRING.

For Oracle, use WithServerVersion(serverVersion, compatibilityVersion) when the database's COMPATIBLE setting differs from the server release. The one-argument overload assumes they match. Oracle qualifies fixed dense vectors up to 65,535 dimensions. Flexible dimensions or formats, sparse storage, and binary vectors produce incomplete discovery diagnostics.

Bind each Oracle shape to one schema-qualified type name. Use WithVarrayType(name, shape, capacity) for ordered arrays and WithObjectType(name, shape) for structs. Register dependencies first and execute RenderNamedTypeDefinitions() before the table plan when provisioning new types. Existing matching types need only the binding. Supported members include signed 16-, 32-, and 64-bit integers using scale-zero NUMBER, float32, float64, VARCHAR2(4000 BYTE) text, Oracle DATE as DbType.DateTime, and other bound named types. Raw column metadata carries the qualified name and a top-level VARRAY's capacity in Length for rebinding. Nested tables are unordered, and PL/SQL associative arrays are outside SQL column mapping. The discovery reader reports unsupported named types and attributes as incomplete, including VARRAY declarations with required elements.

SingleStore discovers native JSON from the catalog's column type. SingleStore's procedural ARRAY and RECORD types are unavailable as table column types. JSON can carry arrays and objects with application-defined structure.

MariaDB discovery treats LONGTEXT COLLATE utf8mb4_bin as JSON text when the catalog also shows the matching column-level JSON_VALID check. The JSON alias recreates that check during rendering. MariaDB has no native array, map, or struct column mapping. Set DUBURL_MARIADB_STRUCTURED_QA_CONNECTION_STRING to a disposable MariaDB 11.7.1+ database to run the focused round-trip test:

dotnet test QA/DubUrl.Providers.Testing -f net10.0 --filter "FullyQualifiedName=DubUrl.Provider.MySql.Tests.Schema.MariaDbStructuredTypeLiveTests.JsonTextAndVectorRoundTrip"

Use each engine’s WithServerVersion to qualify CockroachDB vectors (24.2+, up to 16,000 dimensions), CrateDB vectors (5.5+, up to 2,048 dimensions), and QuestDB arrays (9.0+). Catalog discovery covers tables and columns on these routes. CrateDB supports nullable object fields and nullable array elements. CrateDB text arrays are outside discovery because the catalog also reports bounded character arrays as text_array. QuestDB higher-rank arrays produce incomplete discovery because the column model has no rectangular tensor type.

Use a SQLite build with JSON functions enabled. Callers can write JSONB using jsonb(?1) and read it as JSON text using json(column). Discovery distinguishes the renderer’s validated forms from ordinary TEXT and BLOB, and reports bare JSON and JSONB declarations as incomplete. Native array, map, struct, and vector column shapes are outside SQLite’s structured mappings.

For the three opt-in round-trip tests, set DUBURL_COCKROACH_STRUCTURED_QA_CONNECTION_STRING, DUBURL_CRATE_STRUCTURED_QA_CONNECTION_STRING, or DUBURL_QUEST_STRUCTURED_QA_CONNECTION_STRING for the corresponding disposable engine. Use QuestDB’s Npgsql Server Compatibility Mode=NoTypeLoading. Select the individual test by its fully qualified name in PgCompatibleStructuredLiveTests.

Snowflake collection leaves support Boolean, text, date, double, and signed 16-, 32-, and 64-bit integers. Integral leaves use scale-zero NUMBER with declared precision. Vectors are supported as top-level columns up to 4,096 dimensions. Snowflake discovery covers standard tables, columns, and comments in the current database or the databases selected by the caller. Bare semi-structured arrays and objects, unsupported leaf declarations, and unavailable type details produce incomplete discovery diagnostics.

Set DUBURL_SNOWFLAKE_QA_CONNECTION_STRING to a disposable Snowflake schema to run the opt-in round-trip test:

dotnet test QA/DubUrl.Providers.Testing -f net10.0 --filter "FullyQualifiedName=DubUrl.Provider.Snowflake.Tests.Schema.SnowflakeStructuredTypeLiveTests.AllFiveShapesRoundTrip"

Use SchemaInspector with the registered provider reader to recover structured descriptors. Recognized types outside the model, such as DuckDB fixed arrays without a recognized component check or SingleStore vectors with unrepresentable component types, produce an incomplete discovery diagnostic. Legacy scalar columns and native type hints retain their rendering path.