User should be able to install taps and targets in the UI
Problem to solve
Users need to manage their taps and targets in the UI in order to minimize the amount of CLI interaction.
Target audience
Users who prefer using the webapp over the command line.
Further details
- Enable a non-technical user to get data from a source and load it to a target: a user should be able to see all available taps and targets. Focus of this issue is mostly on the UI, rather back-end functionality.
Proposal
@bencodezen could we list the engineering tasks required to have the ability to view, add, manage and remove extractors and loaders as well as any prerequisites and blockers? Let's discuss feasibility and any considerations with details first thing on Monday? (cc: @dmor)
- Prerequisite:
-
Meltano start
should open a browser - we will be building assuming this is done
- Design
-
Have an ETL page where potentially we can have an accordion design for Taps and Targets (ex: https://www.smashingmagazine.com/2017/06/designing-perfect-accordion-checklist/) -
Once we open the accordion we can see the list of available taps/targets (ex: https://datastudio.google.com/data). -
Once a user clicks on one, the thumbnail ?flips? and requires all credentials required. We should have whatever one needs to configure currently in the .env file as editable. Let's assume we will separate the configs for the tap from the configs of the targetThis task accounted for in #520 (closed)
export FLASK_ENV=development
export SQLITE_DATABASE=meltano
export PG_PASSWORD=warehouse
export PG_USERNAME=warehouse
export PG_ADDRESS=localhost
export PG_SCHEMA=meltano
export PG_PORT=5502
export PG_DATABASE=warehouse
export GITLAB_API_PROJECTS=''
export GITLAB_API_START_DATE=''
- Engineering tasks required:
-
Build out a new page layout focused on ETL - we should not need a separate settings page anymore to set db variables as these would be on the target thumbnail -
Build an accordion component to toggle between taps and targets -
Build a search pad and autocomplete filtering for available taps -
Build an API to manage (incl. add, remove, edit target or tap)This task accounted for in #520 (closed) -
Build a card for taps/targets - plan is to take the logo of say postgres and put into the square. The card should expand, so that someone can make edits - not only fill in but also add additional fields. -
Build an API to fetch all current taps/targets, i.e. taking discovery.yaml in order to expose the data in the UI - Build remove plugin functionality with API integration
- Configure plugin variables through UI
- Design Tasks
-
@bencodezen Apply styles based on design inspiration discussion with @valexieva -
@bencodezen Interaction model with the installation process (to let users know something is installing) -
@bencodezen check out @derek-knox basic wire-frame to make sure all consistent and documented -
@dmor - should we have the user be able to get back to the CLI and perform the next steps required to proceed to etl and get data in the dashboard or we should have this feature with a flag and not show until fully functional e2e user experience exists?
What does success look like, and how can we measure that?
(Define both the success metrics and acceptance criteria. Note that success metrics indicate the desired business outcomes, while acceptance criteria indicate when the solution is working correctly. If there is no way to measure success, link to an issue that will implement a way to measure this)
A user should be able to add, view, manage and remove extractors and loaders from a page in the UI.
Regression test
(Ensure the feature doesn't cause any regressions)
-
Write adequate test cases and submit test results -
Test results should be reviewed by a person from the team
Links / references
Please note that this was taken from GitLab, to be changed accordingly