--debug-level=True help function didn't help a new Meltano user

Was just debugging meltano with a new Meltano user. They hit an exception with tap-csv, had the standard "failed while in discovery" which pointing to logs (which they found), and then they tried to run with --log-level=DEBUG but it wasn't clear where to put --log-level=DEBUG so they did something like this

meltano elt tap-csv target-jsonl --job-id=abc --log-level=DEBUG

Easy fix is to probably just extend the message a bit to give an example of where to put --log-level? Easy MR to make but not sure if you'll agree!

@vischous yep - anything to make it easier / clearer!

I think the line is here https://gitlab.com/meltano/meltano/-/blob/master/src/meltano/cli/elt.py#L255

Want to make an MR?

@kgpayne any other place we recommend this? I know you touched a lot of this code recently.

Done!

changed milestone to %February 2022

added Accepting Merge Requests kindBug valuestreamMeltano labels

added to epic &123

set weight to 1

@vischous - How about adding a link to a docs page with example? I would hesitate to give a sample with the full cli context unless it's something they can copy-paste.

This is probably a lot more work, but if we have access to the original CLI command we could try to construct/inject it for them - in which case then I think it become easy to read and copy-pastable.

Given that it would probably be a lot more work to inject the --log-level to the right place on their behalf, I think I'm inclined to just link to our docs with more examples/explanation of the --log-level option.

What do you think, @vischous, @tayloramurphy, @edgarrmondragon ?

This link looks about right: https://docs.meltano.com/reference/command-line-interface#debugging

@aaronsteers I agree about leaving CLI behavior as is and ref'ing to docs for usage. I wonder if CLI auto-completion would help

Thanks, @edgarrmondragon.

I agree about leaving CLI behavior as is and ref'ing to docs for usage.

Yeah, probably that's our best bet at this stage.

I wonder if CLI auto-completion would help

@edgarrmondragon - Good idea. I really like the idea of CLI autocompletion, and certainly willing to invest in it, but my past experience is that only power users actually know how to install the auto-complete symbols. So, for new users, I think we have to assume even if we have auto-complete available, many less technical users likely still wouldn't have access to it at the terminal.

Ways around this:

• A web-based terminal could have auto-complete symbols already enabled.
• A devcontainer-based approach could similarly already have auto-complete enabled out of box.
• We can publish docs on how to install the symbols on Mac and/or other OSes.

@aaronsteers linking to the docs makes sense too.

Could we also update the guidance to just say or re-run the command adding the '--log-level=debug' CLI flag after 'meltano' and before your command or something like it? Not critical, but could be helpful.

@tayloramurphy - Good idea.

Maybe just giving the first snippet with an ellipses (meltano --log-level=debug ...) is a nice balance, such as:

...or re-run the command using `meltano --log-level=debug ...`

For more information on debugging and logging: 
  https://docs.meltano.com/reference/command-line-interface#debugging

Love it

Thanks for all the back and forth!

Took a quick stab at this and as I was going through I noticed that the message of

ELT could not be completed: Extractor failed
For more detailed log messages, check the generated log file '/home/visch/git/tap-clickup/.meltano/logs/elt/2022-03-02T204844--tap-clickup--target-jsonl/5507fc55-1197-4d2b-b907-c4fff397e50e/elt.log'.
or re-run the command using 'meltano --log-level=debug ...' CLI flag.
For more information on debugging and logging: https://docs.meltano.com/reference/command-line-interface#debugging

Seems nice, but /home/visch/git/tap-clickup/.meltano/logs/elt/2022-03-02T204844--tap-clickup--target-jsonl/5507fc55-1197-4d2b-b907-c4fff397e50e/elt.log doesn't have anymore information than what is printed to console.

Ended up going with

ELT could not be completed: Extractor failed.
For more detailed log messages re-run the command using 'meltano --log-level=debug ...' CLI flag.
Note that you can also check the generated log file at '/home/visch/git/tap-clickup/.meltano/logs/elt/2022-03-02T205550--tap-clickup--target-jsonl/28254941-ff5c-48c3-ae4c-668875fedc1f/elt.log'.
For more information on debugging and logging: https://docs.meltano.com/reference/command-line-interface#debugging

changed milestone to %April 2022

mentioned in merge request !2554 (merged)

assigned to @vischous

added Community-Contributed MR label

changed milestone to %March 2022

closed with merge request !2554 (merged)

mentioned in commit b9d47bd6