Skip to content
Snippets Groups Projects
Commit 169d3d63 authored by Kevin Dietz's avatar Kevin Dietz
Browse files

update auth example

parent 4ef4c68a
No related branches found
No related tags found
1 merge request!50Kdietz update snowflake connection
Pipeline #1103067273 passed with warnings
%% Cell type:markdown id: tags:
### Main functions provided for connecting to SnowFlake
#### Required imports
%% Cell type:code id: tags:
``` python
from gitlabdata.orchestration_utils import (
data_science_engine_factory,
query_dataframe,
query_executor,
ds_query_dataframe,
)
from os import environ as env
import pandas as pd
from sqlalchemy.engine.base import Engine
import logging
```
%% Cell type:markdown id: tags:
#### data_science_engine_factory
Returns a SQL alchemy engine, based off of the data available in profiles.yml.
This function accepts two parameters:
1. profile_target - The profile target denotes the `output` field in the ./.dbt/profiles.yml file and defaults to 'dev'.
2. schema - schema to select data from. Optional.
3. in_docker - boolean, defaults to false if you are running this notebook from within docker (i.e. with `make jupyter-docker`)
%% Cell type:markdown id: tags:
#### Example usage when running `make jupyter-local`
%% Cell type:code id: tags:
``` python
engine = data_science_engine_factory()
engine = data_science_engine_factory(env, profile_target='dev', run_target='local')
```
%% Cell type:markdown id: tags:
#### query_dataframe
After instantiating an engine object you should now be able to query data as normal using SQL. To reduce the amount of repeated code and enable easier data-science tasks we have written the function query_dataframe, which queries the DB and returns the dataset as a dataframe.
%% Cell type:code id: tags:
``` python
schema_check = f"SELECT * FROM PROD.INFORMATION_SCHEMA.TABLES LIMIT 10 "
d = query_dataframe(engine, schema_check)
d = ds_query_dataframe(engine, schema_check)
```
%% Cell type:code id: tags:
``` python
d
```
%% Cell type:code id: tags:
``` python
```
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment