Setting project via `gcloud` CLI does not work
I was trying out ELI5 for the first time today, and setting project via gcloud CLI does not work.
Steps:
- Trying to run
poetry run eli5 code-suggestions evaluate \
--dataset="code_generation_mbpp_all_sanitized" \
--source=gitlab \
--limit=1 \
--experiment-prefix=mmj-2511-experiment-mistral \
--intent=generationbut ELI5 failed with
╭───────────────────────────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────────────────────────╮
│ /Users/gitlab/eli5/eli5/cli/codesuggestions/commands.py:104 in evaluate │
│ │
│ 101 ): │
│ 102 │ client: Client = ctx.obj.langsmith_client │
│ 103 │ │
│ ❱ 104 │ codesuggestions.evaluate( │
│ 105 │ │ client, │
│ 106 │ │ dataset, │
│ 107 │ │ limit=limit, │
│ │
│ /Users/gitlab/eli5/eli5/codesuggestions/evaluate.py:173 in evaluate │
│ │
│ 170 │ if evaluator_names is None: │
│ 171 │ │ evaluator_names = ["exact_match", "similarity_score"] │
│ 172 │ │
│ ❱ 173 │ evaluators = [EVALUATORS[e]() for e in evaluator_names] │
│ 174 │ │
│ 175 │ if evaluate_with_llm and "llm" not in evaluator_names: │
│ 176 │ │ evaluators.append(anthropic_llm_correctness_evaluator()) │
│ │
│ /Users/gitlab/eli5/eli5/codesuggestions/evaluators/similarity_score.py:39 in similarity_score_evaluator │
│ │
│ 36 │ │
│ 37 │ return LangChainStringEvaluator( │
│ 38 │ │ RateLimitEmbeddingDistanceEvalChain( │
│ ❱ 39 │ │ │ embeddings=VertexAIEmbeddings(model_name="textembedding-gecko@003"), distanc │
│ 40 │ │ ), │
│ 41 │ │ prepare_data=prepare_data, │
│ 42 │ ) │
│ │
│ /Users/gitlab/eli5/.venv/lib/python3.12/site-packages/langchain_google_vertexai/embeddings.py:150 in __init__ │
│ │
│ 147 │ │ """Initialize the sentence_transformer.""" │
│ 148 │ │ if model_name: │
│ 149 │ │ │ kwargs["model_name"] = model_name │
│ ❱ 150 │ │ super().__init__( │
│ 151 │ │ │ project=project, │
│ 152 │ │ │ location=location, │
│ 153 │ │ │ credentials=credentials, │
│ │
│ /Users/gitlab/eli5/.venv/lib/python3.12/site-packages/pydantic/main.py:212 in __init__ │
│ │
│ 209 │ │ """ │
│ 210 │ │ # `__tracebackhide__` tells pytest and some other tools to omit this function fr │
│ 211 │ │ __tracebackhide__ = True │
│ ❱ 212 │ │ validated_self = self.__pydantic_validator__.validate_python(data, self_instance │
│ 213 │ │ if self is not validated_self: │
│ 214 │ │ │ warnings.warn( │
│ 215 │ │ │ │ 'A custom validator is returning a value other than `self`.\n' │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
ValidationError: 1 validation error for VertexAIEmbeddings
Value error, Unable to find your project. Please provide a project ID by:
- Passing a constructor argument
- Using vertexai.init()
- Setting project using 'gcloud config set project my-project'
- Setting a GCP environment variable
- To create a Google Cloud project, please follow guidance at https://developers.google.com/workspace/guides/create-project [type=value_error,
input_value={'project': None, 'locati... 'default_metadata': ()}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.9/v/value_errorI tried setting the google project via
gcloud config set project my-project-idbut it continued to fail.
Workaround: Setting the same project-id as ENV variable worked, so like GOOGLE_CLOUD_PROJECT=my-project-id