sqlite: add NewConnector, a driver.Connector for sql.OpenDB (2c7e3eb3) · Commits · cznic / sqlite

CHANGELOG.md

+3 −0

Original line number	Diff line number	Diff line
		# Changelog

		- 2026-07-30 v1.56.0:
		- Add `NewConnector`, returning a `database/sql/driver.Connector` for use with `sql.OpenDB`. It opens the same connections `sql.Open("sqlite", dsn)` does, from the same registered driver, so every function, collation, connection hook and virtual table module registered through this package applies to them. It exists for callers that need to interpose on the physical connections `database/sql` opens — tracing, metrics, connection-scoped setup — which `sql.Open` gives no access to: such a caller can embed the returned `Connector`, override `Connect`, and pass its own wrapper to `sql.OpenDB`. Previously the only way to reach the registered driver was the `db, _ := sql.Open("sqlite", ""); drv := db.Driver(); db.Close()` idiom, which works only because `sql.Open` does not connect and this driver does not implement `driver.DriverContext`; and the only way to get a wrapper into a `sql.DB` was `sql.Register`, which is process-global, panics on a name it has already seen, and cannot be undone, so a library had to invent a unique driver name per configuration. `sql.OpenDB` registers nothing. Constructing a `&sqlite.Driver{}` is not an alternative — its fields are unexported, so it carries none of the registrations. Only the syntax of the DSN query string is checked by `NewConnector`; parameter values continue to be validated when the connection is opened, so an unknown parameter or an out-of-range value is reported by `Connect` rather than at construction. Nothing about the existing `sql.Open` path changes: `Driver` deliberately still does not implement `driver.DriverContext`, so `sql.Open` remains lazy and DSN errors continue to surface where they always have. A runnable sample is in `examples/connector`. Resolves [GitLab issue #253](https://gitlab.com/cznic/sqlite/-/issues/253), thanks Alessandro Segala (@ItalyPaleAle)!

		- 2026-07-20 v1.55.0:
		- Add `github.com/mattn/go-sqlite3`-compatible shorthand DSN query parameters to ease migration from that driver: `_busy_timeout`/`_timeout`, `_foreign_keys`/`_fk`, `_journal_mode`/`_journal`, `_synchronous`/`_sync`, `_auto_vacuum`/`_vacuum`, and `_query_only`, each setting the correspondingly named PRAGMA. Values are validated against the same set `mattn/go-sqlite3` accepts (case-insensitive) and an unrecognized value fails the connection with an error, so a typo such as `_synchronous=fu1l` or `_foreign_keys=yes_please` is reported rather than silently downgrading durability or dropping foreign-key enforcement. The keys are applied in a fixed order independent of their order in the DSN — `_busy_timeout` and `_auto_vacuum` before any `_pragma` values (`auto_vacuum` must be set before the database is first written), the rest after, and `_query_only` last — and where a key and its alias are both supplied the alias wins, matching `mattn/go-sqlite3`; selection is by presence rather than by value, so supplying the alias empty (`_foreign_keys=on&_fk=`) suppresses the PRAGMA rather than deferring to the primary key, again matching that driver. Behavior change to note: prior releases ignored these keys entirely, so a DSN carried over from a `mattn/go-sqlite3` setup changes in two ways. A recognized key that previously did nothing now takes effect — `_foreign_keys=on` begins enforcing constraints against data that may already violate them, `_journal_mode=wal` persistently converts the database file, and `_query_only=1` makes the connection read-only. And a value outside the accepted set now fails the connection with an error where the same DSN previously opened successfully — for example a duration-style `_busy_timeout=5s` or `_timeout=5000ms`, neither of which is the integer that key requires. Review such DSNs before upgrading. `_pragma` is unchanged and no pre-existing parameter changes meaning, though see the following entry for a change in when all of them are validated.
		- See [GitLab merge request #134](https://gitlab.com/cznic/sqlite/-/merge_requests/134), thanks Toni Spets (@beeper-hifi) and Ian Chechin!

CLAUDE.md

+1 −1

Original line number	Diff line number	Diff line
		@@ -17,7 +17,7 @@ The hand-written Go on top of that transpiled core implements the `database/sql/
		- `vfs/` — exposes a Go `fs.FS` as a read-only SQLite VFS. `vfs.New(fsys)` returns a registered VFS name; open with `?vfs=<name>`. C side is transpiled per platform from `vfs/c/vfs.c` via the `vfs/Makefile`.
		- `vtab/` — Go-facing virtual-table API (no dependency on the transpiled C). `vtab.RegisterModule(db, name, module)` registers modules on new connections only; the bridge to C lives in the top-level `vtab.go`. See `vtab/doc.go` for the contract (Updater/Renamer/Transactional optional interfaces, re-entrancy rules, ArgIndex/Omit semantics).
		- `vendor_libs/main.go` (build tag `none`) — regeneration tool. Reads transpiled `ccgo_<goos>_<goarch>.go` from sibling repos `../libsqlite3` and `../libsqlite_vec`, rewrites package names and imports, and writes `lib/sqlite_.go` / `vec/vec_.go`. Invoked by `make vendor`.
		- `examples/` — runnable samples: `example1`, `vtab_basic`, `vtab_csv`, `vtab_match`, `vtab_regexp`.
		- `examples/` — runnable samples: `example1`, `connector`, `vtab_basic`, `vtab_csv`, `vtab_match`, `vtab_regexp`.
		- `addport.go`, `issue198/`, `issue120.diff` — porting/regression scaffolding kept around for reference; not built.

		## Commands

connector.go

0 → 100644

+108 −0

Original line number	Diff line number	Diff line
		// Copyright 2026 The Sqlite Authors. All rights reserved.
		// Use of this source code is governed by a BSD-style
		// license that can be found in the LICENSE file.

		package sqlite // import "modernc.org/sqlite"

		import (
		"context"
		"database/sql/driver"
		"net/url"
		"strings"
		)

		// NewConnector returns a driver.Connector that opens connections to dsn using
		// the driver this package registers as "sqlite" -- the one carrying every
		// function, collation, connection hook and virtual table module registered
		// through RegisterFunction, RegisterScalarFunction,
		// RegisterDeterministicScalarFunction, RegisterCollationUtf8,
		// RegisterConnectionHook and vtab.RegisterModule. The dsn syntax and the
		// supported query parameters are documented on Driver.Open.
		//
		// The returned value is intended for sql.OpenDB:
		//
		// c, err := sqlite.NewConnector("file:app.db?_pragma=foreign_keys(1)")
		// if err != nil {
		// return err
		// }
		// db := sql.OpenDB(c)
		// defer db.Close()
		//
		// For opening a database this is equivalent to sql.Open("sqlite", dsn). It
		// exists for callers that need to interpose on the physical connections
		// database/sql opens -- tracing, metrics, or connection-scoped setup. Such a
		// caller can embed the returned Connector, override Connect, and pass its own
		// wrapper to sql.OpenDB:
		//
		// type tracer struct{ driver.Connector }
		//
		// func (t tracer) Connect(ctx context.Context) (driver.Conn, error) {
		// conn, err := t.Connector.Connect(ctx)
		// // ... wrap conn ...
		// return conn, err
		// }
		//
		// base, err := sqlite.NewConnector(dsn)
		// if err != nil {
		// return err
		// }
		// db := sql.OpenDB(tracer{base})
		//
		// Reaching the same driver through sql.Open requires sql.Register, which is
		// process-global, rejects a name it has already seen with a panic, and offers
		// no way to undo a registration; a library doing the above would have to
		// invent a unique name per configuration. sql.OpenDB registers nothing.
		//
		// Only the syntax of the dsn query string is checked here -- a malformed
		// query, such as one carrying an invalid percent-escape, is reported
		// immediately. Parameter values are validated when a connection is opened, so
		// an unknown parameter or an out-of-range value is reported by Connect, and
		// hence by the first use of the sql.DB, rather than by NewConnector.
		//
		// The returned Connector is safe for concurrent use; database/sql calls
		// Connect from multiple goroutines as it grows the pool. As with sql.Open, it
		// does not itself open a connection.
		func NewConnector(dsn string) (driver.Connector, error) {
		if _, err := url.ParseQuery(dsnQuery(dsn)); err != nil {
		return nil, err
		}

		return &connector{dsn: dsn}, nil
		}

		// connector implements driver.Connector on top of the package-level driver. It
		// holds no mutable state, so the concurrent Connect calls database/sql makes
		// need no synchronization here.
		type connector struct {
		dsn string
		}

		// Connect implements driver.Connector.
		//
		// ctx is honored only up to the point the open begins: sqlite3_open_v2 is a
		// blocking call with no cancellation hook, so a cancellation arriving once it
		// is under way takes effect no earlier than the first statement run on the
		// returned connection.
		func (c *connector) Connect(ctx context.Context) (driver.Conn, error) {
		if err := ctx.Err(); err != nil {
		return nil, err
		}

		return d.Open(c.dsn)
		}

		// Driver implements driver.Connector. It returns the driver registered as
		// "sqlite", so (*sql.DB).Driver() reports the same value for a database opened
		// through this Connector as for one opened with sql.Open("sqlite", dsn).
		func (c *connector) Driver() driver.Driver { return d }

		// dsnQuery returns the query-parameter portion of dsn, or "" when it has none.
		// It must agree with the split newConn performs, which likewise ignores a '?'
		// in the first position; TestConnectorDSNSplitMatchesOpen guards that.
		func dsnQuery(dsn string) string {
		if pos := strings.IndexRune(dsn, '?'); pos >= 1 {
		return dsn[pos+1:]
		}

		return ""
		}

connector_test.go

0 → 100644

+285 −0

Original line number	Diff line number	Diff line
		// Copyright 2026 The Sqlite Authors. All rights reserved.
		// Use of this source code is governed by a BSD-style
		// license that can be found in the LICENSE file.

		package sqlite // import "modernc.org/sqlite"

		import (
		"context"
		"database/sql"
		"database/sql/driver"
		"path/filepath"
		"strings"
		"sync/atomic"
		"testing"
		)

		// The registrations below go on the package-level driver and therefore affect
		// every connection the test binary opens afterwards. They are named uniquely
		// and the connection hook is inert unless the DSN carries connHookMarker, so
		// they cannot perturb the rest of the suite.
		const connHookMarker = "connector_test_hook_marker"

		var connHookCalls int64

		func init() {
		MustRegisterDeterministicScalarFunction("connector_test_answer", 0,
		func(*FunctionContext, []driver.Value) (driver.Value, error) {
		return int64(42), nil
		})
		MustRegisterCollationUtf8("connector_test_collation",
		func(left, right string) int { return strings.Compare(left, right) })
		RegisterConnectionHook(func(_ ExecQuerierContext, dsn string) error {
		if strings.Contains(dsn, connHookMarker) {
		atomic.AddInt64(&connHookCalls, 1)
		}
		return nil
		})
		}

		// TestConnectorOpenDB exercises the sql.OpenDB path end to end.
		func TestConnectorOpenDB(t *testing.T) {
		c, err := NewConnector("file::memory:")
		if err != nil {
		t.Fatalf("NewConnector: %v", err)
		}

		db := sql.OpenDB(c)
		defer db.Close()

		if _, err := db.Exec(`CREATE TABLE t(i INT); INSERT INTO t VALUES(1), (2)`); err != nil {
		t.Fatalf("Exec: %v", err)
		}

		var n int
		if err := db.QueryRow(`SELECT sum(i) FROM t`).Scan(&n); err != nil {
		t.Fatalf("QueryRow: %v", err)
		}
		if g, e := n, 3; g != e {
		t.Fatalf("got %v, expected %v", g, e)
		}
		}

		// TestConnectorAppliesGlobalRegistrations is the point of
		// https://gitlab.com/cznic/sqlite/-/issues/253: connections handed out by the
		// Connector must carry the functions, collations and hooks registered on the
		// package-level driver, which a caller-constructed &Driver{} does not.
		func TestConnectorAppliesGlobalRegistrations(t *testing.T) {
		before := atomic.LoadInt64(&connHookCalls)

		c, err := NewConnector("file::memory:?_pragma=application_id(1)&x=" + connHookMarker)
		if err != nil {
		t.Fatalf("NewConnector: %v", err)
		}

		db := sql.OpenDB(c)
		defer db.Close()

		var n int
		if err := db.QueryRow(`SELECT connector_test_answer()`).Scan(&n); err != nil {
		t.Fatalf("registered function not available: %v", err)
		}
		if g, e := n, 42; g != e {
		t.Fatalf("got %v, expected %v", g, e)
		}

		var b bool
		if err := db.QueryRow(`SELECT 'a' < 'b' COLLATE connector_test_collation`).Scan(&b); err != nil {
		t.Fatalf("registered collation not available: %v", err)
		}
		if !b {
		t.Fatal("collation returned an unexpected ordering")
		}

		if g := atomic.LoadInt64(&connHookCalls); g <= before {
		t.Fatalf("connection hook not called: %v, was %v", g, before)
		}
		}

		// TestConnectorDriverIsRegisteredDriver checks that the Connector reports the
		// same driver value database/sql hands out for sql.Open("sqlite", ...), which
		// is what makes it a drop-in for code reaching the driver through db.Driver().
		func TestConnectorDriverIsRegisteredDriver(t *testing.T) {
		viaOpen, err := sql.Open("sqlite", "file::memory:")
		if err != nil {
		t.Fatalf("sql.Open: %v", err)
		}
		defer viaOpen.Close()

		c, err := NewConnector("file::memory:")
		if err != nil {
		t.Fatalf("NewConnector: %v", err)
		}

		viaConnector := sql.OpenDB(c)
		defer viaConnector.Close()

		if g, e := viaConnector.Driver(), viaOpen.Driver(); g != e {
		t.Fatalf("got %p, expected %p", g, e)
		}
		if _, ok := viaConnector.Driver().(*Driver); !ok {
		t.Fatalf("got %T, expected *sqlite.Driver", viaConnector.Driver())
		}
		}

		// countingConnector is the wrapper an instrumentation library writes. Note
		// that it needs no sql.Register and therefore no globally unique driver name.
		type countingConnector struct {
		driver.Connector

		connects int64
		}

		func (c *countingConnector) Connect(ctx context.Context) (driver.Conn, error) {
		atomic.AddInt64(&c.connects, 1)
		return c.Connector.Connect(ctx)
		}

		// TestConnectorWrapping covers the use case the Connector exists for:
		// interposing on the physical connections database/sql opens.
		func TestConnectorWrapping(t *testing.T) {
		base, err := NewConnector("file:" + filepath.Join(t.TempDir(), "wrap.db"))
		if err != nil {
		t.Fatalf("NewConnector: %v", err)
		}

		wrapped := &countingConnector{Connector: base}
		db := sql.OpenDB(wrapped)
		defer db.Close()
		db.SetMaxOpenConns(2)

		if g := atomic.LoadInt64(&wrapped.connects); g != 0 {
		t.Fatalf("sql.OpenDB connected eagerly: %v", g)
		}

		// Holding two sql.Conn at once forces exactly two physical connections.
		ctx := context.Background()
		c1, err := db.Conn(ctx)
		if err != nil {
		t.Fatalf("Conn: %v", err)
		}
		defer c1.Close()

		c2, err := db.Conn(ctx)
		if err != nil {
		t.Fatalf("Conn: %v", err)
		}
		defer c2.Close()

		if g, e := atomic.LoadInt64(&wrapped.connects), int64(2); g != e {
		t.Fatalf("got %v physical connects, expected %v", g, e)
		}

		// The wrapper must not have cost us the global registrations.
		var n int
		if err := c1.QueryRowContext(ctx, `SELECT connector_test_answer()`).Scan(&n); err != nil {
		t.Fatalf("registered function not available through wrapper: %v", err)
		}
		if g, e := n, 42; g != e {
		t.Fatalf("got %v, expected %v", g, e)
		}
		}

		// TestConnectorRejectsMalformedQuery covers the eager half of the validation
		// contract: a dsn whose query string is not parseable fails at construction.
		func TestConnectorRejectsMalformedQuery(t *testing.T) {
		for _, dsn := range []string{
		"file::memory:?_pragma=%zz",
		"file::memory:?%",
		"file::memory:?a=%2",
		} {
		c, err := NewConnector(dsn)
		if err == nil {
		t.Errorf("%q: got a Connector, expected an error", dsn)
		continue
		}
		if c != nil {
		t.Errorf("%q: got a non-nil Connector alongside %v", dsn, err)
		}
		}
		}

		// TestConnectorDefersValueValidation covers the lazy half: parameter values
		// are checked when the connection is opened, not by NewConnector.
		func TestConnectorDefersValueValidation(t *testing.T) {
		const dsn = "file::memory:?_txlock=bogus"

		c, err := NewConnector(dsn)
		if err != nil {
		t.Fatalf("NewConnector rejected %q eagerly: %v", dsn, err)
		}

		db := sql.OpenDB(c)
		defer db.Close()

		if err := db.Ping(); err == nil {
		t.Fatal("Ping succeeded, expected the bad _txlock to be reported")
		} else if !strings.Contains(err.Error(), "_txlock") {
		t.Fatalf("got %v, expected it to mention _txlock", err)
		}
		}

		// TestConnectorContextCanceled checks Connect reports an already-cancelled
		// context rather than opening a connection.
		func TestConnectorContextCanceled(t *testing.T) {
		c, err := NewConnector("file::memory:")
		if err != nil {
		t.Fatalf("NewConnector: %v", err)
		}

		ctx, cancel := context.WithCancel(context.Background())
		cancel()

		conn, err := c.Connect(ctx)
		if err == nil {
		conn.Close()
		t.Fatal("Connect succeeded on a cancelled context")
		}
		if g, e := err, context.Canceled; g != e {
		t.Fatalf("got %v, expected %v", g, e)
		}
		}

		// TestConnectorDSNSplitMatchesOpen guards the invariant dsnQuery's comment
		// relies on: NewConnector must never reject a dsn newConn would have accepted.
		func TestConnectorDSNSplitMatchesOpen(t *testing.T) {
		for _, v := range []struct {
		dsn string
		query string
		}{
		{"", ""},
		{":memory:", ""},
		{"file::memory:", ""},
		{"file::memory:?", ""},
		{"file::memory:?a=b", "a=b"},
		{"file::memory:?a=b&c=d", "a=b&c=d"},
		{"?a=b", ""}, // A '?' in the first position is part of the filename.
		{"x?a=b?c=d", "a=b?c=d"},
		} {
		if g, e := dsnQuery(v.dsn), v.query; g != e {
		t.Errorf("dsnQuery(%q): got %q, expected %q", v.dsn, g, e)
		}
		}

		// Whatever NewConnector rejects, opening must reject too. Every dsn here
		// is memory-backed, so a successful open touches no file.
		for _, dsn := range []string{
		"file::memory:",
		"file::memory:?",
		"file::memory:?_pragma=%zz",
		"file::memory:?%",
		"file::memory:?vfs=a&vfs=b",
		"file::memory:?_error_rc=maybe",
		} {
		_, cErr := NewConnector(dsn)
		if cErr == nil {
		continue
		}

		conn, oErr := d.Open(dsn)
		if oErr == nil {
		conn.Close()
		t.Errorf("%q: NewConnector rejected it (%v) but Open accepted it", dsn, cErr)
		}
		}
		}

doc.go

+6 −0

Original line number	Diff line number	Diff line
		@@ -81,6 +81,12 @@
		//
		// ...
		//
		// [NewConnector] is an alternative entry point returning a
		// [driver.Connector] for use with [sql.OpenDB]. It opens the same
		// connections sql.Open does, from the same driver, and exists for callers that
		// need to interpose on them -- tracing, metrics, or connection-scoped setup --
		// which sql.Open gives no access to. See its docstring for an example.
		//
		// # Debug and development versions
		//
		// A comma separated list of options can be passed to `go generate` via the