Fix handling of delimited object identifiers

Merge Request Checklist

  • Link to an issue
  • Include the proposed fix or feature
  • Include and update tests for the modified code
  • Include a documentation change
  • Add an entry in the Unreleased section of the CHANGELOG

This MR addresses Permifrost's handling of double-quoted object identifiers, resolving bugs that would generate invalid statements.

The root cause, broadly speaking, is the immediate conversions of all identifiers to lowercase, which does not work if the identifier is case sensitive, as delimited identifiers are. There's some challenge to resolve this, as we lowercase everything very far upstream, right within the SnowflakeConnector methods themselves.

The strategy I took was to expand the use of the SnowflakeConnecter.snowflaky method, which converts identifier names to comply with Snowflake's requirements. I made some updates to the method itself to account for a few cases that weren't previously included, like leading integers and mixed case going in quotes and leaving '$' ok as unquoted, to name a couple. The net result is that instead of immediately lowercasing all identifiers via .lower() when we return query results, we run the identifiers through snowflaky(). Most of the identifiers will just come out lowercased, but when that's not appropriate, we apply double-quotes or leave double-quotes as-is. The method is also designed to be idempotent, so even if we run identifiers through snowflaky() again downstream, it shouldn't make an impact.

What this does not do: I omitted support for object identifiers that include a period and left a note indicating as such. Including this was going to make the regex way more complex, and I wanted to try to avoid that in what felt like an already significant change. I can create a follow-up issue for this and we can add support for this in the future if there's a need. Note that this applies only to object identifiers. Permifrost still supports usernames with periods, like first.last@company.com.

We also omit support for objects named with reserved keywords. These are not currently supported and this MR does not change that.

This is a bug fix so no documentation update required.

Once your merge request is complete, please assign it the Permifrost maintainers using the /assign_reviewer @gitlab-data/permifrost_maintainers placed in a comment for the review cycle. Once the review cycle is finished, the reviewer shall approve the merge request so it can be merged by any Permifrost developer, maintainer, or owner.

Edited by William Herrmann

Merge request reports

Loading