Improve logging of remote development module of agentk
Currently, all logs are debug. So we have no visibility into the system by default. We cannot convert them to Info logs as it is because they contain a lot of information.
We should clean up the logging such that there are some minimal info logs about
- When an error has occurred
- When partial reconciliation has started
- When full reconciliation has started
For debugging,
- Whenever reconciliation happens, we log the
namespace/nameof the workspace being sent to agentk - We SHOULD NOT log the entire YAML received from rails that we apply
- Currently, whenever there are any state changes in the deployment, we see a lot of debug logs. Reduce the noise of them significantly.
- Should we have a log when we receive an information from informer?
- Other improvements.
In general the goal should be that by default we have some information if things are going good by default(at Info level) and some additional useful information at debug level.
Introducing the debug logs should not result in a plethora of logs.
We are doing a similar effort on Rails - Switch all remote_dev branch log statements to ... (#408783 - closed)
Availability and Testing
Ensure updated test coverage in unit/integration/feature tests.
Relevant MR: gitlab-org/cluster-integration/gitlab-agent!945 (merged)
Activity
added to epic &10356 (closed)
You can refer to the Features by Group handbook page for guidance.
If you are unsure about the correct group, please do not leave the issue without a group label, and refer to GitLab's shared responsibility functionality guidelines for more information on how to triage this kind of issue.
This message was generated automatically. You're welcome to improve it.
added devopscreate sectiondev labels
I'm adding a few things related to agentk in the Beta since it looks like most of our changes are almost done there. These issues won't be anything significant that would require a lot of code changes.
Ok, then I'm adding this to the next iteration, adding to %16.0 and marking Deliverable
assigned to @hkhanna2
changed iteration to Workspaces May 1, 2023 - May 7, 2023
changed milestone to %16.0
added Deliverable label
mentioned in issue gitlab-org/quality/triage-reports#12229 (closed)
added quad-planningcomplete-action label
Notes from IPM:
- @shekharpatnaik has converted most/all of the logs to use debug already
- This is the only visibility we have into agentk errors for 16.0
Just to add more context here
- The default info logs are very less (if any?).
- If debug logs are enabled, they are way too much even for when you have just 4-5 workspaces. For any real traffic, it is not at all meaningful to enable debug logs.
If everyone is okay with that, I'm fine pushing this out of %16.0
Edited by Vishal Tak
If we're pushing this out of %16.0, perhaps I'll un-assign myself from this issue?
If we're pushing this out of %16.0, perhaps I'll un-assign myself from this issue?
That should be fine, we can reassign when we reprioritize it.
Also, lets remember to remove the milestones, iteration, and Deliverable label when we depriortize things (I'll do that now).
removed milestone %16.0
removed Deliverable label
removed iteration Workspaces May 1, 2023 - May 7, 2023
unassigned @hkhanna2
changed milestone to %16.0
added Deliverable workflowin dev labels and removed workflowready for development label
changed iteration to Workspaces May 1, 2023 - May 7, 2023
mentioned in merge request gitlab-org/cluster-integration/gitlab-agent!945 (merged)
added workflowin review label and removed workflowin dev label
assigned to @hkhanna2
removed iteration Workspaces May 1, 2023 - May 7, 2023
changed iteration to Workspaces May 8, 2023 - May 14, 2023
added workflowcomplete label and removed workflowin review label
added rd-workflowdone label
