Implement artifacts operations as subprocesses
Jacob Vosmaer [10:06]
@kamilt: what does "spawning yourself" mean?
Kamil Trzciński [10:14]
@jacobvosmaer: basically, this: `exec.Command(os.Args[0], …)`
Jacob Vosmaer [10:14]
@kamilt: os.Args[0] is not reliable
[10:16]
exec(3) lets the caller set arg0 to whatever they want. it does not have to be a valid path on disk
Kamil Trzciński [10:18]
@jacobvosmaer: I’m using this in Runner: https://github.com/kardianos/osext. It uses syscalls to the the hard work.
GitHub
kardianos/osext
osext - Extensions to the standard "os" package. Executable and ExecutableFolder.
(edited)
Jacob Vosmaer [10:20]
why do crazy stuff like that to spawn a one-off process?
[10:20]
I don't think it is an appropriate solution
Kamil Trzciński [10:20]
to make dependencies simpler, it’s easier to distribute one binary than a bunch
Jacob Vosmaer [10:21]
I don't like it. I think it is the opposite of simple
[10:22]
brb
Kamil Trzciński [10:22]
We are talking about emotions now, not a real arguments :simple_smile:
Grzegorz Bizon [10:22]
I think we need to think about to extending GitLab with support for small Go binaries
[10:23]
It will be easier in the future to load small binary into memory than to spawn workhorse, than can be quite big application within next few months, what do you think?
Kamil Trzciński [10:24]
@grzegorz: 8MB vs 9MB? The app is already loaded. It doesn’t cost you more than a work memory.(edited)
Grzegorz Bizon [10:25]
@kamilt: Well, I just wonder if extending workhorse, every time we need some Go implementation will scale good enough(edited)
Kamil Trzciński [10:26]
Second: `gitlab-workhorse` is good name for it to, because: the zip metadata generation is „slow“ and memory consuming, extracting zip file can be the same.
Grzegorz Bizon [10:26]
@kamilt: I'm not talking about artifacts/metadata now, but it is more general issue I think(edited)
[10:27]
I think that our current implementation is really good (big thanks to @kamilt) and will get better after moving implementation to separate process. I'm just thinking about future usage of Go.(edited)
Kamil Trzciński [10:28]
@grzegorz: I’m talking about resolving the issue, not adding a lot of work doing that „maybe“ right, right now. I’m not sure that this is good idea to have separate apps for everything in Go. It’s more common in Go world than any else to have a single binary :simple_smile:(edited)
[10:29]
The one of the reasons is that each binary cost you ~8MB of memory (the Go runtime).(edited)
Grzegorz Bizon [10:29]
@kamilt: I like it how it is done right now (already reviewed this MR today)
Kamil Trzciński [10:30]
Does it affect „scalability“? I don’t see it.
[10:30]
@grzegorz: It’s ok-ish.
Grzegorz Bizon [10:31]
@kamilt: I'm not saying that we should change something now, or do something different now, following YAGNI now is really reasonable. I'm just saying that we do not know what we will encounter in the future, but we will need more Go eventually, probably.(edited)
Kamil Trzciński [10:32]
@grzegorz: for the same reason I don’t see benefit adding it as separate binary :simple_smile:
Grzegorz Bizon [10:33]
@kamilt: There is none, now. How will it look like in there future? I'm just skeptical about growing workhorse too much. Best solutions are simple solutions, making workhorse that solves simple problems more complex, may not scale good in the future.