Refactor Runner logging
Overview
Currently, we have two types of loggers. System loggers and Job trace
logger. But we don't distinguish well between them and we don't pass this
around in our codebase very well which results in some logs missing some
keys or context. For example you can't say "Get me all the logs for
this specific job". Something we even just do directly logrus.Info
with 0 contexts and you end up just looking through the code base and
trying to find out what we are logging for debugging.
Recently we have been structuring our application in an ideal way where we pass in the logger around in structs, this provides a good way for testability and logging keys in general. However, we are creating interfaces specific for the package and end up creating adapters for them. This turns out a bit messy because of code duplication, and not every package implements the same logging interface.
Example
Show example of a Job run and the logs it produces.
Proposal
Create two types of loggers that are created at the start of the program/job and then have these loggers passed around and only use this 1 interface.
This will not fix everything we still have to update packages and structs to accept this logger so it will take a long time and should be done gradually, starting with new packages we create/structs, and then going back and refactoring existing implementations.
Benefits
We will be able to start to question answers like:
- "What are the logs for all the job"
- "Will this log go in the system logger, or Job trace"
- "Darn, I can't add a debug logline here because this logger
interface only has
Warn
I have to update the interface"
We define around Domain models meaning we should only have two types of loggers:
- System loggers
- Trace loggers
With this, we also have 1 central place where we can define common keys such as JobID called (where the logline was called), we don't have a mess of different IDs for the same thing.
Define an Interface
To make testing and dependency management easy we should define an interface which will be passed around. We should have 1 interface for both the Job and System logger.
For example the logger interface can look something like:
type Logger interface {
Debug(msg, args)
Info(msg, args)
Warn(msg, args)
Err(msg, args)
}
This interface should probably be defined inside of golang-org/labkit
which can reused by other projects.
Definitions
System logger
The system logger is the logs we print inside of STDOUT or inside of the syslog (if running as daemon). This should be initialized at the start of the program
Trace logger
This is the log lines that the user sees inside of the job page.