Document the semantic layer

### Important Schemas

The Production Database in the EDW is used for reporting and analysis by Data Consumers at GitLab. It is composed of 4 major schemas which are `COMMON_`, `SPECIFIC`, `LEGACY_` and `WORKSPACE_` schemas. Below are descriptions of each Schema:

The Production Database in the EDW is used for reporting and analysis by Data Consumers at GitLab. It is composed of 5 major schemas which are `COMMON_`, `SPECIFIC`, `LEGACY_`, `WORKSPACE_`, and `SEMANTIC` schemas. Below are descriptions of each Schema:

Our warehouse architecture is organized into four distinct schemas:

Our warehouse architecture is organized into five distinct schemas:

1. **COMMON Schema:** Houses our Enterprise Dimensional Model (EDM), serving as the heart of our integrated application data. This schema implements the Kimball methodology to ensure the highest data quality standards.

1. **SPECIFIC Schema:** Maintains independent application data that doesn't require integration with other systems, following our Trusted Data Development process while avoiding unnecessary complexity.

1. **WORKSPACE Schema:** Provides a flexible environment for experimentation and prototyping, serving as a staging area for future EDM solutions.

1. **LEGACY Schema:** Maintains historical modeling approaches as we strategically deprecate and migrate critical systems to our modern architecture.

1. **SEMANTIC Schema:** Acts as the primary entry point to the semantic layer for agentic and AI tool access, providing curated, documented views with embedded Agent Skills that contain all context needed to correctly query and interpret the data.

## Dimensional Modeling Fundamentals

## Schemas

Our schema design reflects the natural progression of data through our warehouse, from initial ingestion to final consumption. Raw data enters through dedicated schemas, undergoes transformation in preparation schemas, and ultimately resides in consumption-ready schemas that support our dimensional model. This layered approach allows us to maintain clean handoffs between different stages of data processing while ensuring data quality and consistency throughout the pipeline. Each schema serves as a checkpoint in our data's journey, with clear responsibilities and governance rules that help maintain the integrity of our data warehouse.

Our schema design reflects the natural progression of data through our warehouse, from initial ingestion to final consumption. Raw data enters through dedicated schemas, undergoes transformation in preparation schemas, and ultimately resides in consumption-ready schemas that support our dimensional model for human reporting and analysis, or in AI Semantic schemas that provide a curated, agent-accessible interface to that same data. This layered approach allows us to maintain clean handoffs between different stages of data processing while ensuring data quality and consistency throughout the pipeline. Each schema serves as a checkpoint in our data's journey, with clear responsibilities and governance rules that help maintain the integrity of our data warehouse.

## Common Prep Schema

In our targets vs actuals reporting, we use a scaffold table to ensure data completeness and consistency. This table maintains a comprehensive structure that covers all time periods and attribute combinations, whether or not sales activity occurred.

## AI Semantic Schemas

The AI Semantic schemas provide a curated, controlled interface to the data warehouse designed specifically for agentic and AI tools. Rather than relying on warehouse introspection or metadata crawling, these schemas offer a stable, intentionally designed set of views that agents can discover and use reliably.

Two schemas serve this purpose:

- **`prod.SEMANTIC`** — publicly accessible views for agentic tools

- **`prod.RESTRICTED_SAFE_SEMANTIC`** — access-controlled equivalent for views that expose sensitive data

### Key Design Principles

All objects in these schemas are non-materialized (views, functions, and semantic views). This ensures that changes can be deployed immediately without requiring a full model rebuild. Views act as entry points to existing report tables, avoiding duplication of business logic. All objects are managed via dbt for maintainability, testability, and scalable development.

### View Types

| Type | Description |

|------|-------------|

| Verified Query | Curated by subject matter experts |

| Discovered Query | Built from evaluating user query patterns |

| Semantic View | Constructed from data relationships |

| User Defined Table Function | Requires strict parameters |

### Documentation Pattern

Each view is documented following a pattern designed for agent consumption:

- **Schema description** — a description of the schema itself that participates in agent search alongside individual view descriptions

- **Short description** — a concise summary of what the view provides, used during progressive disclosure search to identify candidate views without overwhelming the agent's context window

- **Full usage description** — a comprehensive description that is loaded once a view is selected. This is structured as and functions as an Agent Skill: it contains all the context needed to correctly query and interpret the view, including usage guidance, parameter expectations, and result interpretation

- **Category and priority** — metadata used to rank and filter search results

### How Agents Use These Schemas

Agents interact with these schemas using a progressive disclosure pattern:

1. The agent searches the schema using the schema description and view short descriptions to identify relevant views, returning ranked matches without loading full details for every result

2. The agent selects the most relevant view and loads its full usage description, which functions as a use-specific Agent Skill providing complete querying context

3. The agent queries the view directly using the embedded guidance, or reads the underlying SQL to make informed ad-hoc modifications

## Specific Schema

The `SPECIFIC` schema is to be used for tables that perform a reporting function and act as a source of truth but do not conform to the dimensional modeling structure of the Enterprise Dimensional Model.

### Tableau Integration Best Practices

Since Tableau is our primary visualization tool, all EDW models should be designed with Tableau compatibility in mind. This means following consistent naming conventions and planning migrations carefully.

Since Tableau is our primary visualization tool, all EDW models intended for human reporting should be designed with Tableau compatibility in mind, following consistent naming conventions and planning migrations carefully. This guidance applies to models in the COMMON, COMMON_MART, SPECIFIC, and related schemas. Objects in the AI Semantic schemas (`SEMANTIC` and `RESTRICTED_SAFE_SEMANTIC`) are designed for agentic tool consumption and follow a different design pattern described in the [AI Semantic Schemas](#ai-semantic-schemas) section.

Models should follow our [naming standards](/handbook/enterprise-data/platform/edw/#naming-standards) and use the correct prefixes (`mart_`, `rpt_`, `fct_`, `dim_`, etc) to reflect their purpose and EDW layer. Because Tableau workbooks break when table names change, it’s important to choose stable names from the start. Migrations should be coordinated with stakeholders, scheduled outside of high-usage periods such as month or quarter end, and followed by testing of all dependent reports to ensure continued functionality.

1. Use prefixes in table and column names to group like data. Data will remain logically grouped when sorted alphabetically, e.g. dim_geo_location, dim_geo_region, dim_geo_sub_region.

1. Use dimension table names in primary and foreign key naming. This makes it clear to the user what table will need to be joined to pull in additional attributes. For example, the primary key for dim_crm_account is dim_crm_account_id. If this field appears in fct_subscription, it will be named dim_crm_account_id to make it clear the user will need to join to dim_crm_account to get additional account details.

1. Dimension, fact, and mart tables are not to contain references to operational systems. We abstract the name away from the source system the data is produced into a name that describes the business entity or semantic significance of the data. For example, data from Salesforce is described as `crm` in the dimensional model and not `sfdc` or `salesforce`.

1. **AI SEMANTIC VIEWS:** Objects in the `SEMANTIC` and `RESTRICTED_SAFE_SEMANTIC` schemas use a prefix based on the type of object, followed by a plain-language name that reflects the business question the object is designed to answer:

   | Type | Prefix | Example |

   |------|--------|---------|

   | Verified Query | `vq_` | `vq_monthly_arr_by_sales_segment` |

   | Discovered Query | `dq_` | `dq_open_opportunities_by_region` |

   | Semantic View | `sv_` | `sv_customer_health_score` |

   | User Defined Table Function | `udtf_` | `udtf_arr_for_account_and_period` |

### File-Based Data Sources