make.org 16.8 KB
Newer Older
doshitan's avatar
doshitan committed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
---
title: Make Tips
toc: true
---

I tend to use make as a task runner, a purpose it is not expressly suited to,
but works well enough.

This own site's [[https://gitlab.com/doshitan/doshitan.com/blob/master/makefile][makefile]].

Note I almost always use and build for [[https://www.gnu.org/software/make/][GNU Make]], which is [[#gnu-vs-bsd-make][different in some ways
than other makes]].

* Basics
See the [[https://www.gnu.org/software/make/manual/make.html#Introduction][intro section]] of the GNU Make manual.

Basic format:

#+BEGIN_SRC makefile
# v-- filename of the thing the recipe builds or an action/task name
target: prerequisites ...
#         ^-- other targets to build before this one
# v-- must start with a tab character, no spaces here
	recipe # <-- lines to execute
	...
#+END_SRC

Each line execute in a separate subshell, so you can not set variables inside a
target.

If you have a long line, use backslashes to break it up across lines.
#+BEGIN_SRC make
do-thing:
	command thing $(SOME_VAR) \
	--a-flag \
	--another-flag \
	--output=$(SOME_DIR)
#+END_SRC

~@~ at the beginning of a line suppresses make echoing the command.

#+BEGIN_SRC makefile
echo:
	echo "Hello, world"
#+END_SRC

#+BEGIN_SRC bash
$ make echo
echo "Hello, world"
"Hello, world"
#+END_SRC

vs.

#+BEGIN_SRC makefile
echo:
	@echo "Hello, world"
#+END_SRC

#+BEGIN_SRC bash
$ make echo
"Hello, world"
#+END_SRC

~-~ at the beginning of a line suppresses errors on that line from killing
target build.

#+BEGIN_SRC makefile
clean:
	-rm -f *.o
#+END_SRC

Finishes successfully even if the ~rm~ errors (like if there are no ~.o~ files
to clean).

You can combine them:

#+BEGIN_SRC makefile
clean:
	@-rm -f *.o
#+END_SRC

Doesn't say what it's doing or care if it errors.

If make is called without a target, it will run the first target listed in the
makefile, unless the ~.DEFAULT_GOAL~ variable is set, then whatever that says
gets run.
* Capitalize ~makefile~ or not
It [[https://www.gnu.org/software/make/manual/html_node/Makefile-Names.html][doesn't really matter]]. ~Makefile~ is conventional.

Note that GNU make searches for ~makefile~ before ~Makefile~, so if both are
present (why?), the lowercase one wins.

I prefer lowercase ~makefile~ as it's ever so slightly easier to type and it's
usually surrounded by other lowercased names so it looks better to me to match.
* Make as a task runner
Often in projects you have a number of commands that you may wish to run on
occasion. There are many existing task runner solutions out there, the
JavaScript world in particular seems to love to create them. I shy away from
these as I think often they are overcomplicated for the task at hand.

If a project has a JavaScript dependency, often it will use the ~scripts~
section in it's ~package.json~ file, which can easily be run with ~npm run
<script name>~. I dislike this for a few reasons:
1. I avoid JavaScript dependencies as much as possible
2. I generally want a package manager to be responsible for one thing, getting packages
3. I prefer to have npm scripts disabled globally as they are a [[https://blog.npmjs.org/post/141702881055/package-install-scripts-vulnerability][security hazard]]

Advantages of make:
- Been around forever
- Available everywhere and probably already installed
- Just let's you write shell scripts
- Easy/automatic [[#shell-tab-complete][tab-completion support]] for targets in most shells

Disadvantages of make:
- Old and thus not purposely designed for today's environment
- It's ways and appearances can be odd to the uninitiated, or to the initiated
  that have been away for a time

For me, a simple makefile strikes a good balance for many projects.

My suggested approach is this:
1. At the beginning, use a simple makefile for common tasks
2. For bigger tasks, write a standalone script, store it in a ~scripts/~
   directory and call the script in the makefile
3. When you get to the point where you have lots of scripts or some complicated
   behavior, write a custom application for managing your project. This might
   literally just be a CLI library wrapping some scripts, but it's very nice
   having a management interface specifically tuned to your project (and having
   a proper programming language to build it in). Use a build system library
   like [[https://shakebuild.com/][Shake]] if needed.
* Shell tab-complete
Many shells support autocomplete for make targets, zsh ships with support by
default, there is often a ~bash-completion~ package for your system which will
enable it in bash.

This is pretty handy, to be able to just type ~make~ then hit
@@html:<kbd>Tab</kbd>@@ to see a list of targets and have tab-completion for
finishing target names.
* Variables
There are [[https://www.gnu.org/software/make/manual/make.html#Flavors][two flavors of variables]] in make. In short, variables defined with ~=~
are /recursively expanded/ and their value is evaluated every time they are
referenced, which has performance consequences if it's defined in terms of functions.

The other kind of variables are defined with ~:=~ which are /simply expanded
variables/ that get evaluated only once, when make first encounters them. These
behave much more understandably and should be preferred by default.

There's also another form, ~?=~ which sets a variable only if it hasn't been
defined, which can be useful to define defaults for variables you may want to
inherit from the environment. More on that in [[#passing-arguments-to-make-targets][passing arguments]].
* Passing arguments to make targets
Okay, so there are basically two ways to get values to a target.

** From environment
All environment variables[fn::Except ~MAKE~ and ~SHELL~.] [[https://www.gnu.org/software/make/manual/make.html#Environment][become make variables]]
inside the makefile. This is probably the most common way to modify makes
behavior. If you were to do:

#+BEGIN_SRC bash
THING=val OTHER_THING=val2 make thing
#+END_SRC

~$(THING)~ and ~$(OTHER_THING)~ would be available in the makefile.

The important thing about inheriting environment variables is that they are only
inherited if they are not set in the makefile[fn::Unless you set ~-e~ or use the
~override~ directive on the variable.]. For instance, if our makefile was:

#+BEGIN_SRC makefile
msg = hello

echo:
	@echo $(msg)
#+END_SRC

And then try:

#+BEGIN_SRC bash
$ msg=world make echo
hello
#+END_SRC

The output is ~hello~, *not* ~world~.

This is where the ~?=~ variable definition comes in.

If we had instead written our makefile like:

#+BEGIN_SRC makefile
msg ?= hello

echo:
	@echo $(msg)
#+END_SRC

Then:

#+BEGIN_SRC bash
$ msg=world make echo
world
#+END_SRC

So in short, define variables with ~?=~ if you want to enable the variables to
be overridden by an environment variable (which is usually what you want).
** As arguments to make
You can also set make variables by passing them as arguments to make. They can
come before or after the target name.

#+BEGIN_SRC bash
make THING=val OTHER_THING=val2 thing
#+END_SRC

#+BEGIN_SRC bash
make thing THING=val OTHER_THING=val2
#+END_SRC

Unlike environment variables, it doesn't matter how these variables are defined
in the makefile, their value is what is set in the argument, any definitions
for them are ignored in the makefile.
** Arbitrary arguments
Okay so setting arguments is cool, but what if you want to pass arbitrary
arguments to a target? Well, just use a general variable name, like ~args~.

Write your target like:

#+BEGIN_SRC makefile
ls:
	ls $(args)
#+END_SRC

Which you could then call like:

#+BEGIN_SRC bash
$ make ls args='-la ~'
# ls -la ~
#+END_SRC

You can even set default arguments on a per-target basis, say:

#+BEGIN_SRC makefile
ls: args=~/
ls:
	ls $(args)

run: args=--flag -t file
run:
	command $(args)
#+END_SRC

But if you *really* don't want to have to type ~args=""~, you can bend and contort
to support passing the other arguments to make directly to a target. Reproduced
from [[https://stackoverflow.com/a/14061796][this StackOverflow answer]].

Write your makefile like:

#+BEGIN_SRC makefile
# If the first argument is "run"...
ifeq (run,$(firstword $(MAKECMDGOALS)))
  # use the rest as arguments for "run"
  RUN_ARGS := $(wordlist 2,$(words $(MAKECMDGOALS)),$(MAKECMDGOALS))
  # ...and turn them into do-nothing targets
  $(eval $(RUN_ARGS):;@:)
endif

prog: # ...
    # ...

.PHONY: run
run: prog
    @echo prog $(RUN_ARGS)
#+END_SRC

You could then do:

#+BEGIN_SRC bash
$ make run foo bar baz
prog foo bar baz
#+END_SRC

If you want to pass options to the target, you do need to separate them from
make with a ~--~ (like other commands), so something like:

#+BEGIN_SRC makefile
make run -- --from here --to there
#+END_SRC

Generally, if you do need to pass arbitrary arguments to a target all the time,
I would suggest writing a script and running it directly, maybe with make
targets for the common cases. But you do you.
* Auto-documented makefiles
[[https://marmelab.com/blog/2016/02/29/auto-documented-makefile.html][This post]] describes the approach in detail and is quite handy. In sort, annotate
your targets like:

#+BEGIN_SRC make
deps: ## Install project dependencies
#+END_SRC

Add the magic incantation

#+BEGIN_SRC make
help:
	@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-10s\033[0m %s\n", $$1, $$2}'
#+END_SRC

And then when you run ~make help~ you get a nicely formatted help page, with the
simple example above:

#+BEGIN_SRC bash
deps    Install project dependencies
#+END_SRC
* Set SHELL
By default, regardless of what your personal shell is, make targets run under
~/bin/sh~. You can [[https://www.gnu.org/software/make/manual/html_node/Choosing-the-Shell.html][change this]] by explicitly setting the ~SHELL~ variable either
globally in the makefile or for a specific target like:
#+BEGIN_SRC make
clean: SHELL:=bash
	rm $(SOME_DIR)/{one,two,three}/*.junk
#+END_SRC
* GNU vs BSD make
There are other makes out there, but generally I only think about GNU and BSD
variants. As mentioned at the top, I almost always write for GNU make
specifically.

If you want to more portable makefiles, avoid using anything listed on the
[[https://www.gnu.org/software/make/manual/html_node/Features.html][features]] and [[https://www.gnu.org/software/make/manual/html_node/Missing.html][missing]] pages for GNU make.

I don't recommend it, but you can also maintain separate ~GNUmakefile~ and
~BSDmakefile~ files and the different makes will pick the right one
automatically[fn::GNU make for example looks for ~GNUmakefile~, ~makefile~, and
~Makefile~, in that order, unless told to read a specific file with
~-f/--file~.]. For stuff that isn't version specific, you can put it in a
~makefile.common~ (or whatever), then ~include makefile.common~ in each of the
others to easy the maintenance burden.
* Actually building things
Make is first and foremost, a build system. Designed to take source files, do
something with them to generate other files (like an executable). So if you
actually do have file dependencies, make has some nice features.

Take a super simple example.

#+BEGIN_SRC makefile
test.html: test.md
	pandoc test.md --from markdown --to html --output test.html
#+END_SRC

This is a rule that knows how to build the ~test.html~ file from its markdown
source file, ~test.md~. We can run ~make test.html~ to build it.

The nice thing is make only does the actual work if it needs to by comparing the
timestamps for the prerequisites and targets, if the prerequisite has been
modified after the currently generated targets timestamp, then it knows it
should rerun the recipe, otherwise it doesn't. Which you can see if you run
~make test.html~ again, make will say there is no work to do.

But we are repeating ourselves a bit, make knows the target we intend to build
and the source inputs before it goes to run the recipe and has special variables
we can use for them.

#+BEGIN_SRC makefile
test.html: test.md
	pandoc $< --from markdown --to html --output $@
#+END_SRC

~$<~ is the expanded prerequisite name (~test.md~ in this case).
~$@~ is the expanded target name (~test.html~ in this case).

Better. But we can generalize a little more, say if we have a bunch of markdown
files, maybe documentation stuff, we can use a [[https://www.gnu.org/software/make/manual/html_node/Pattern-Rules.html][pattern rule]]:

#+BEGIN_SRC makefile
%.html: %.md
	pandoc $< --from markdown --to html --output $@
#+END_SRC

This is a rule that knows how to build *any* html file from its corresponding
markdown source file. So we can just say ~make test.html~ or ~make other.html~
and it will generate them if needed.

Okay, but we don't want to have to run a bunch of ~make <file>.html~ commands in
order to get all our html files generated. Let's write a rule that has as prerequisites
all the html files, so when we run that target it will trigger the build of all
the html files.

#+BEGIN_SRC makefile
html_files := test.html \
              other.html

all: $(html_files)

%.html: %.md
	pandoc $< --from markdown --to html --output $@
#+END_SRC

So we've added a list of the html files in the variable ~html_files~ and added
it as prerequisites to the target ~all~, so when we run ~make all~, all the html
files will be generated.

But it could be tedious to maintain the list of html files by hand, adding a new
line every time we create a new markdown file. Let's automate that.

#+BEGIN_SRC makefile
md_files  := $(wildcard *.md)
html_files := $(patsubst %.md,%.html,$(md_files))

all: $(html_files)

%.html: %.md
	pandoc $< --from markdown --to html --output $@
#+END_SRC

So we've used the [[https://www.gnu.org/software/make/manual/html_node/Wildcard-Function.html][~wildcard~ function]] to get a list of all the markdown files
in the directory. We then swap their file extention with the [[https://www.gnu.org/software/make/manual/html_node/Text-Functions.html][~patsubst~
function]]. Everything else is the same as before.

We can now just add markdown files as we wish, ~make all~ will build or rebuild
them as necessary.

Another wonderful feature of make, is it's parallel building, through the
~-j/--jobs~ option. You sometimes need to write your rules with parallel
building in mind to get the full benefit of it, but as we've written our rules
here, we can run say ~make all -j 5~ to build five files at once, a potentially
big time saver if you have the compute capacity.

This could make our directory a little messy, let's add a ~clean~ target to
delete the generated files.

#+BEGIN_SRC makefile
md_files  := $(wildcard *.md)
html_files := $(patsubst %.md,%.html,$(md_files))

all: $(html_files)

clean:
	-rm $(html_files)

%.html: %.md
	pandoc $< --from markdown --to html --output $@
#+END_SRC

Now we can run ~make clean~ to remove all those html files, if we want to force
regenerating all of them from scratch, or we just don't need them anymore.

This is just a simple example, but hopefully illustrative of the how we can take
advantage of some of make's features to build a simple and powerful tool.
* Bigger example
Extracted from a old, but real project.

#+BEGIN_SRC makefile
.PHONY: build check-elm clean clean-elm deps distclean install
.SECONDEXPANSION:
.DEFAULT_GOAL := help

BUILDDIR ?= '_build'
DESTDIR ?= '../server/priv/static'


build: $(addprefix $(BUILDDIR)/, elm.js index.html)
build: ## Builds application into $(BUILDDIR)
	cp -r assets $(BUILDDIR)/
	cp -r node_modules/uswds/dist/* $(BUILDDIR)/assets/


build-prod: build
build-prod: ## Builds application in $(BUILDDIR), w/ optimizations
	mv $(BUILDDIR)/elm.js $(BUILDDIR)/elm-unoptimized.js
	closure-compiler --js $(BUILDDIR)/elm-unoptimized.js --js_output_file $(BUILDDIR)/elm.js --compilation_level SIMPLE
	rm $(BUILDDIR)/elm-unoptimized.js


install: ## Puts build files into $(DESTDIR)
	mkdir -p $(DESTDIR)
	cp -r $(BUILDDIR)/* $(DESTDIR)


deps: ## Installs project dependencies
	npm install
	elm package install --yes
	cd tests && elm package install --yes


clean: clean-elm
clean: ## Deletes build artifacts
	rm -r $(BUILDDIR)


clean-elm: ## Deletes Elm build artifacts
	rm -r elm-stuff/build-artifacts


distclean: clean
distclean: ## Deletes all non-source code stuff
	rm -r elm-stuff node_modules


check-elm: clean-elm $(BUILDDIR)/elm.js
check-elm: ## Rebuilds application for warnings


test:
	node_modules/.bin/elm-test


ELM_FILES := $(shell find src/ -type f -name '*.elm')
$(BUILDDIR)/elm.js: $(ELM_FILES) | $$(@D)/.
	elm make --warn src/Main.elm --output $@


$(BUILDDIR)/index.html: index.html | $$(@D)/.
	cp index.html $@


# Trick to make it easy to ensure a target's parent directories are created
# before we try to use them, just add a `| $$(@D)/.` to the dependencies of a
# target to ensure the directories in it's path are created
%/.:
	mkdir -p $@


# Self-documenting make file
# (http://marmelab.com/blog/2016/02/29/auto-documented-makefile.html)
help: ## Displays this help screen
	@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-10s\033[0m %s\n", $$1, $$2}'
	@echo ""
	@echo "BUILDDIR: $(BUILDDIR)"
	@echo "DESTDIR: $(DESTDIR)"
#+END_SRC