make.org 27.9 KB
Newer Older
doshitan's avatar
doshitan committed
1
2
3
---
title: Make Tips
toc: true
doshitan's avatar
doshitan committed
4
toc-margin: true
5
published: 2019-04-16T11:23:06-05:00
6
modified: 2020-09-24T22:19:41-04:00
doshitan's avatar
doshitan committed
7
8
9
10
11
---

I tend to use make as a task runner, a purpose it is not expressly suited to,
but works well enough.

doshitan's avatar
doshitan committed
12
This site's own [[https://gitlab.com/doshitan/doshitan.com/blob/master/makefile][makefile]].
doshitan's avatar
doshitan committed
13
14
15
16
17

Note I almost always use and build for [[https://www.gnu.org/software/make/][GNU Make]], which is [[#gnu-vs-bsd-make][different in some ways
than other makes]].

* Basics
doshitan's avatar
doshitan committed
18
19
See the [[https://www.gnu.org/software/make/manual/make.html#Introduction][intro section]] of the GNU Make manual. And [[https://www.jfranken.de/homepages/johannes/vortraege/make.en.html][this page]] is a good intro and
reference.
doshitan's avatar
doshitan committed
20
21
22
23
24
25
26
27
28
29
30
31

Basic format:

#+BEGIN_SRC makefile
# v-- filename of the thing the recipe builds or an action/task name
target: prerequisites ...
#         ^-- other targets to build before this one
# v-- must start with a tab character, no spaces here
	recipe # <-- lines to execute
	...
#+END_SRC

doshitan's avatar
doshitan committed
32
33
34
35
36
37
38
Each line execute in a separate shell, so you can not set variables inside a
target. Chain commands together with ~&&~ or ~;~, e.g.,
#+BEGIN_SRC makefile
thing:
	a_command && b_command # execute in same shell
	a_command; b_command # also executes in same shell
#+END_SRC
doshitan's avatar
doshitan committed
39
40

If you have a long line, use backslashes to break it up across lines.
doshitan's avatar
doshitan committed
41
#+BEGIN_SRC makefile
doshitan's avatar
doshitan committed
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
do-thing:
	command thing $(SOME_VAR) \
	--a-flag \
	--another-flag \
	--output=$(SOME_DIR)
#+END_SRC

~@~ at the beginning of a line suppresses make echoing the command.

#+BEGIN_SRC makefile
echo:
	echo "Hello, world"
#+END_SRC

#+BEGIN_SRC bash
$ make echo
echo "Hello, world"
"Hello, world"
#+END_SRC

vs.

#+BEGIN_SRC makefile
echo:
	@echo "Hello, world"
#+END_SRC

#+BEGIN_SRC bash
$ make echo
"Hello, world"
#+END_SRC

~-~ at the beginning of a line suppresses errors on that line from killing
target build.

#+BEGIN_SRC makefile
clean:
doshitan's avatar
doshitan committed
79
	-rm *.o
doshitan's avatar
doshitan committed
80
81
82
#+END_SRC

Finishes successfully even if the ~rm~ errors (like if there are no ~.o~ files
doshitan's avatar
doshitan committed
83
84
to clean, for this specific case you'd probably just use the ~-f~ on ~rm~
itself, but this is just an example).
doshitan's avatar
doshitan committed
85
86
87
88
89

You can combine them:

#+BEGIN_SRC makefile
clean:
doshitan's avatar
doshitan committed
90
	@-rm *.o
doshitan's avatar
doshitan committed
91
92
93
94
95
96
97
98
99
100
101
#+END_SRC

Doesn't say what it's doing or care if it errors.

If make is called without a target, it will run the first target listed in the
makefile, unless the ~.DEFAULT_GOAL~ variable is set, then whatever that says
gets run.
* Capitalize ~makefile~ or not
It [[https://www.gnu.org/software/make/manual/html_node/Makefile-Names.html][doesn't really matter]]. ~Makefile~ is conventional.

Note that GNU make searches for ~makefile~ before ~Makefile~, so if both are
doshitan's avatar
doshitan committed
102
103
104
105
106
107
108
109
110
111
112
113
present, the lowercase one wins.

This can be an interesting feature in a shared project. For instance, if you
have a version tracked/shared ~Makefile~, then a local ~makefile~ that does an
~include Makefile~ to pull in the shared one, you can then add personal targets
to your ~makefile~, things that are only useful for you or that you are giving a
trial run for a while.

This is also potentially dangerous, mixing local modifications in the same
interface, safer to have a differently named, like a ~make.local~ that you have
to ~make -f make.local <target>~, but sometimes it's nice to have your tweaks
presented in the same interface.
doshitan's avatar
doshitan committed
114
115
116
117
118

I prefer lowercase ~makefile~ as it's ever so slightly easier to type and it's
usually surrounded by other lowercased names so it looks better to me to match.
* Make as a task runner
Often in projects you have a number of commands that you may wish to run on
doshitan's avatar
doshitan committed
119
120
occasion. There are many existing task runner solutions out there[fn::The
JavaScript world in particular seems to love to create them.]. I shy away from
doshitan's avatar
doshitan committed
121
122
123
124
125
126
these as I think often they are overcomplicated for the task at hand.

If a project has a JavaScript dependency, often it will use the ~scripts~
section in it's ~package.json~ file, which can easily be run with ~npm run
<script name>~. I dislike this for a few reasons:
1. I avoid JavaScript dependencies as much as possible
doshitan's avatar
doshitan committed
127
128
2. I generally want a package manager to be responsible for one thing, getting
   packages
doshitan's avatar
doshitan committed
129
3. I prefer to have npm scripts disabled globally as they are a [[https://blog.npmjs.org/post/141702881055/package-install-scripts-vulnerability][security hazard]]
doshitan's avatar
doshitan committed
130
131
132
133
134
135
136
4. Can't have comments (because JSON)

Similar reasons apply to pipenv scripts or whatever, especially point two.

If you already have a build system, e.g., rake, Ant/Maven/Gradle, SCons, etc.,
maybe it's easier to stuff tasks in there, but even then I think a simple
makefile might have room for high level project management stuff.
doshitan's avatar
doshitan committed
137
138
139
140
141

Advantages of make:
- Been around forever
- Available everywhere and probably already installed
- Just let's you write shell scripts
doshitan's avatar
doshitan committed
142
143
144
145
146
147
148
149
150
- Independent of your project specific language environment, a couple advantages to this:
  - A more stable tool, language specific tooling usually grows and changes a
    lot more than make will, so it can paper over those changes. It's also
    divorced from you package management, so less churn, easier to track
    history, etc.
  - A common interface between different projects (which are all potentially
    written in a variety of different languages). You can build up common
    targets and practices around make that can be shared between all your
    projects, providing a consistent interface when bouncing between projects.
doshitan's avatar
doshitan committed
151
152
153
154
155
156
157
158
159

Disadvantages of make:
- Old and thus not purposely designed for today's environment
- It's ways and appearances can be odd to the uninitiated, or to the initiated
  that have been away for a time

For me, a simple makefile strikes a good balance for many projects.

My suggested approach is this:
doshitan's avatar
doshitan committed
160
161
162
1. At the beginning, use a simple makefile for common tasks. Whenever you find
   yourself needing to run some command (or sequence of commands) more than
   once, stick it in your makefile.
doshitan's avatar
doshitan committed
163
2. For bigger tasks, write a standalone script, store it in a ~scripts/~
doshitan's avatar
doshitan committed
164
   directory and call the script in the makefile.
doshitan's avatar
doshitan committed
165
3. When you get to the point where you have lots of scripts or some complicated
doshitan's avatar
doshitan committed
166
167
168
169
170
   behavior, write a custom application for managing your project. This might be
   just a CLI library wrapping some scripts, but it's very nice having a
   management interface specifically tuned to your project (and having a proper
   programming language to build it in). Use a build system library like [[https://shakebuild.com/][Shake]]
   if needed.
doshitan's avatar
doshitan committed
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
* Shell tab-complete
Many shells support autocomplete for make targets, zsh ships with support by
default, there is often a ~bash-completion~ package for your system which will
enable it in bash.

This is pretty handy, to be able to just type ~make~ then hit
@@html:<kbd>Tab</kbd>@@ to see a list of targets and have tab-completion for
finishing target names.
* Variables
There are [[https://www.gnu.org/software/make/manual/make.html#Flavors][two flavors of variables]] in make. In short, variables defined with ~=~
are /recursively expanded/ and their value is evaluated every time they are
referenced, which has performance consequences if it's defined in terms of functions.

The other kind of variables are defined with ~:=~ which are /simply expanded
variables/ that get evaluated only once, when make first encounters them. These
behave much more understandably and should be preferred by default.

There's also another form, ~?=~ which sets a variable only if it hasn't been
defined, which can be useful to define defaults for variables you may want to
inherit from the environment. More on that in [[#passing-arguments-to-make-targets][passing arguments]].
* Passing arguments to make targets
Okay, so there are basically two ways to get values to a target.

** From environment
All environment variables[fn::Except ~MAKE~ and ~SHELL~.] [[https://www.gnu.org/software/make/manual/make.html#Environment][become make variables]]
inside the makefile. This is probably the most common way to modify makes
behavior. If you were to do:

#+BEGIN_SRC bash
THING=val OTHER_THING=val2 make thing
#+END_SRC

~$(THING)~ and ~$(OTHER_THING)~ would be available in the makefile.

The important thing about inheriting environment variables is that they are only
inherited if they are not set in the makefile[fn::Unless you set ~-e~ or use the
~override~ directive on the variable.]. For instance, if our makefile was:

#+BEGIN_SRC makefile
msg = hello

echo:
	@echo $(msg)
#+END_SRC

And then try:

#+BEGIN_SRC bash
$ msg=world make echo
hello
#+END_SRC

The output is ~hello~, *not* ~world~.

This is where the ~?=~ variable definition comes in.

If we had instead written our makefile like:

#+BEGIN_SRC makefile
msg ?= hello

echo:
	@echo $(msg)
#+END_SRC

Then:

#+BEGIN_SRC bash
$ msg=world make echo
world
#+END_SRC

So in short, define variables with ~?=~ if you want to enable the variables to
be overridden by an environment variable (which is usually what you want).
** As arguments to make
You can also set make variables by passing them as arguments to make. They can
come before or after the target name.

#+BEGIN_SRC bash
make THING=val OTHER_THING=val2 thing
#+END_SRC

#+BEGIN_SRC bash
make thing THING=val OTHER_THING=val2
#+END_SRC

Unlike environment variables, it doesn't matter how these variables are defined
in the makefile, their value is what is set in the argument, any definitions
for them are ignored in the makefile.
** Arbitrary arguments
Okay so setting arguments is cool, but what if you want to pass arbitrary
arguments to a target? Well, just use a general variable name, like ~args~.

Write your target like:

#+BEGIN_SRC makefile
ls:
	ls $(args)
#+END_SRC

Which you could then call like:

#+BEGIN_SRC bash
$ make ls args='-la ~'
# ls -la ~
#+END_SRC

You can even set default arguments on a per-target basis, say:

#+BEGIN_SRC makefile
ls: args=~/
ls:
	ls $(args)

run: args=--flag -t file
run:
	command $(args)
#+END_SRC

But if you *really* don't want to have to type ~args=""~, you can bend and contort
to support passing the other arguments to make directly to a target. Reproduced
from [[https://stackoverflow.com/a/14061796][this StackOverflow answer]].

Write your makefile like:

#+BEGIN_SRC makefile
# If the first argument is "run"...
ifeq (run,$(firstword $(MAKECMDGOALS)))
  # use the rest as arguments for "run"
  RUN_ARGS := $(wordlist 2,$(words $(MAKECMDGOALS)),$(MAKECMDGOALS))
  # ...and turn them into do-nothing targets
  $(eval $(RUN_ARGS):;@:)
endif

prog: # ...
    # ...

.PHONY: run
run: prog
    @echo prog $(RUN_ARGS)
#+END_SRC

You could then do:

#+BEGIN_SRC bash
$ make run foo bar baz
prog foo bar baz
#+END_SRC

If you want to pass options to the target, you do need to separate them from
make with a ~--~ (like other commands), so something like:

#+BEGIN_SRC makefile
make run -- --from here --to there
#+END_SRC

Generally, if you do need to pass arbitrary arguments to a target all the time,
I would suggest writing a script and running it directly, maybe with make
targets for the common cases. But you do you.
doshitan's avatar
doshitan committed
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
* User defined functions

There are a variety of built-in functions, but it's often useful to define your
own reusable block of code.

You can define "functions" with the [[https://www.gnu.org/software/make/manual/make.html#index-define][~define~ directive]], like so:
#+BEGIN_SRC makefile
define run_thing
	do_thing_with_first_arg $(1)
	do_another_thing_with_other_arg $(2)
endef
#+END_SRC

The ~define~ directive is just a way to build up a multiline variable, so what
we are really doing is creating a variable. Using a variable to store commands
is also called a [[https://www.gnu.org/software/make/manual/make.html#Canned-Recipes][canned recipe]].

Then to use it, pass it to the [[https://www.gnu.org/software/make/manual/make.html#Call-Function][~call~ function]] in the form of ~$(call
<name_of_function>[,param][,param][,])~, like so:

#+BEGIN_SRC makefile
$(call run_thing,foo,bar)
#+END_SRC

Which would execute:
#+BEGIN_SRC makefile
do_thing_with_first_arg foo
do_another_thing_with_other_arg bar
#+END_SRC

More concretely, say you wanted to print a test coverage report after every run
of the test suite, as for a Python project using ~pytest~ and ~coverage.py~:
#+BEGIN_SRC makefile
define run_tests
	poetry run coverage run --source src -m pytest $(1)
	poetry run coverage report
endef

test: ## Run all tests
	$(call run_tests,.)

## say this project's convetion is to mark tests that are integration tests, so
## therefore, to run the unit tests, we want to run everything not marked as an
## integration test
test-unit: ## Run just unit tests
	$(call run_tests,-m "not integration")
#+END_SRC

Notably the ~call~ function allows us to parameterize the definition (the ~$(1)~
and ~$(2)~ above), which is usually what I need, but if you just want to collect
a few lines that need run as-is in a few places, can just use the function as a
regular variable, e.g., ~$(run_thing)~.
doshitan's avatar
doshitan committed
382
383
384
385
* Auto-documented makefiles
[[https://marmelab.com/blog/2016/02/29/auto-documented-makefile.html][This post]] describes the approach in detail and is quite handy. In sort, annotate
your targets like:

doshitan's avatar
doshitan committed
386
#+BEGIN_SRC makefile
doshitan's avatar
doshitan committed
387
388
389
deps: ## Install project dependencies
#+END_SRC

doshitan's avatar
doshitan committed
390
391
392
393
394
395
396
Add the magic incantation:

#+BEGIN_SRC makefile
help: ## Displays this help screen
	@grep -Eh '^[[:print:]]+:.*?##' $(MAKEFILE_LIST) | \
	sort -d | \
	awk -F':.*?## ' '{printf "\033[36m%s\033[0m\t%s\n", $$1, $$2}' | \
doshitan's avatar
doshitan committed
397
	column -ts "$$(printf '\t')"
doshitan's avatar
doshitan committed
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
#+END_SRC

And then when you run ~make help~ you get a nicely formatted help page.

Quick line-by-line breakdown, first grep the makefile(s) for our special comments:

#+BEGIN_SRC
┌ tell make not to print the line, we are only interested in the output of the command
|                  ┌ the regex to match your comment pattern
|                  |                    ┌ automatic variable, holds all filenames that have been parsed for make rules on this invocation
|         ┌--------+----------┐ ┌-------+------┐
@grep -Eh '^[[:print:]]+:.*?##' $(MAKEFILE_LIST) | \
       ||
       |└ don't print filenames, for when multiple makefiles are parsed (say by including another)
       └ extended regex support
#+END_SRC

Then we sort the lines grep returns:
#+BEGIN_SRC
sort -d | \
#+END_SRC

Then add some color to the target name, which means: split the line into it's
parts, color the name, then recombine:
#+BEGIN_SRC
         ┌ split the input based on our comment pattern
         |         set color ┐          ┌ reset color
    ┌----+----┐           ┌--+---┐  ┌--+--┐
awk -F':.*?## ' '{printf "\033[36m%s\033[0m\t%s\n", $$1, $$2}' | \
                                  └┘       └┘└┘
                      target name ┘        |  └ comment/help text
                                           └ separator character (TAB in this case)
#+END_SRC

Then nicely align everything:
#+BEGIN_SRC
        ┌ table mode, so columns and rows fill correctly
        |┌ set separator character between columns
        ||
doshitan's avatar
doshitan committed
437
438
439
440
column -ts "$$(printf '\t')"
            |└-----+------┘
            |      └ get a literal tab character
            └ escape the $ for the tab character since we are in make
doshitan's avatar
doshitan committed
441
442
443
444
#+END_SRC

I use a tab character to separate the target and help text as it's unlikely to
be used in the help text, feel free to chose a different one, just keep it in
445
sync between the ~awk~ and ~column~.
doshitan's avatar
doshitan committed
446

doshitan's avatar
doshitan committed
447
448
You can of course go simpler, something like:
#+BEGIN_SRC makefile
doshitan's avatar
doshitan committed
449
help:
doshitan's avatar
doshitan committed
450
	grep -E "^[[:print:]]+: ##" [Mm]akefile
doshitan's avatar
doshitan committed
451
452
#+END_SRC

doshitan's avatar
doshitan committed
453
454
Which will show the targets in the order they are in the makefile, even
highlighting the target name (and other comment pattern junk).
doshitan's avatar
doshitan committed
455

doshitan's avatar
doshitan committed
456
457
458
459
460
Or with straight ~awk~[fn::Sourced from https://github.com/moby/moby/blob/master/Makefile]
#+BEGIN_SRC makefile
	@awk 'BEGIN {FS = ":.*?## "} /^[a-zA-Z0-9_-]+:.*?## / {gsub("\\\\n",sprintf("\n%22c",""), $$2);printf "\033[36m%-20s\033[0m %s\n", $$1, $$2}' $(MAKEFILE_LIST)
#+END_SRC

doshitan's avatar
doshitan committed
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
Or use a different pattern:
#+BEGIN_SRC makefile
# target: help - Display callable targets.
help:
	@grep -E "^# target:" [Mm]akefile

# target: list - List source files
list:
    # Won't work. Each command is in separate shell
    cd src
    ls

    # Correct, continuation of the same shell
    cd src; \
    ls

# target: dist - Make a release.
dist:
    tar -cf  $(RELEASE_DIR)/$(RELEASE_FILE) && \
    gzip -9  $(RELEASE_DIR)/$(RELEASE_FILE).tar
#+END_SRC

Ultimately, you're just grepping some files, do whatever works for you.
* When order matters
The prerequisites of a target are not guaranteed to be run in any particular
order. For instance:

#+BEGIN_SRC makefile
thing: do-one do-two do-three
	./scripts/thing.sh
doshitan's avatar
doshitan committed
491
#+END_SRC
doshitan's avatar
doshitan committed
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611

So the ~do-{one,two,three}~ targets will get run before ~thing~ does, but the
order they get run in not deterministic, ~do-one~ is not guaranteed to run
before ~do-two~ and so on[fn::This is ignoring what prerequisites the ~do-~
targets might have for the moment].

Now they often *do* get run in the order you list them and for simple, small
things, you can usually get by, you just have to be careful the targets are not
a part of any other targets you might want to run with ~-j~.

But sometimes this matters, sometimes you really want to run a series of other
make targets in a guaranteed order. For this you have to reach for [[https://www.gnu.org/software/make/manual/html_node/Recursion.html][recursive
make]].

WARNING: Recursive make can be a bit of a gotcha, and should be avoided if
possible in simple cases. For instance, variables that are not marked for export
are not automatically inherited to the sub-make. Read up before you start down
the recursive path.

#+BEGIN_SRC makefile
thing:
	$(MAKE) do-one
	$(MAKE) do-two
	$(MAKE) do-three
	./scripts/thing.sh
#+END_SRC

~$(MAKE)~ is just a variable that is set to the same make command that is
currently running.

Calling ~make~ inside of a make target is traditionally used to call other
makefiles in sub-directories to build smaller components of a system and so make
will print some extra information about what directory it's running in when
called recursively.

For task-runner purposes this can be a little annoying. GNU make has a easy flag
to turn this off though, so I'd recommend setting a special variable with the
flag and use it, like:

#+BEGIN_SRC makefile
# (q)uiet (make), use whatever you want
QMAKE := $(MAKE) --no-print-directory

thing:
	$(QMAKE) do-one
	$(QMAKE) do-two
	$(QMAKE) do-three
	./scripts/thing.sh
#+END_SRC

If you want to quiet if for every recursive make call, then you could just add
it to ~MAKEFLAGS~ like:

#+BEGIN_SRC makefile
MAKEFLAGS += --no-print-directory

thing:
	$(MAKE) do-one
	$(MAKE) do-two
	$(MAKE) do-three
	./scripts/thing.sh
#+END_SRC

And achieve the same thing.

For non-GNU make you can set the ~-s~ flag instead, but that silences *all*
output, which is usually less desirable.

*If* you can manage to structure your targets just so, then you can enforce some
ordering with just prerequisites, like:

#+BEGIN_SRC makefile
do-one:
do-two: do-one
do-three: do-two

# technically would only need to say do-three
thing: do-one do-two do-three
	./scripts/thing.sh
#+END_SRC

But often you can't structure your targets like this.

Also, there are things called [[https://www.gnu.org/software/make/manual/html_node/Prerequisite-Types.html][order-only prerequisites]], but don't let that name
fool you, they do not get run in a specified order, they are just prerequisites
that are always considered out of date and so are always run (which sometimes
matters).
* Running other targets
Our old friend recursive make.

#+BEGIN_SRC makefile
build:
	gcc -Wall $(args)

build-dev:
	$(MAKE) build args='-O0'

build-prod:
	$(MAKE) build args='-O9'
#+END_SRC

Though sometimes it might be better to pull out the common bits into a variable,
and not call make, like:

#+BEGIN_SRC makefile
BUILD_CMD := gcc -Wall

build:
	$(BUILD_CMD) $(args)

build-dev:
	$(BUILD_CMD) -Og

build-prod:
	$(BUILD_CMD) -O2
#+END_SRC

But sometimes your other targets you want to call, have certain prerequisites
setup that you don't want to have to duplicate and other stuff. So sometimes it
is easier to use recursive make.
doshitan's avatar
doshitan committed
612
613
614
615
* Set SHELL
By default, regardless of what your personal shell is, make targets run under
~/bin/sh~. You can [[https://www.gnu.org/software/make/manual/html_node/Choosing-the-Shell.html][change this]] by explicitly setting the ~SHELL~ variable either
globally in the makefile or for a specific target like:
doshitan's avatar
doshitan committed
616
617

#+BEGIN_SRC makefile
doshitan's avatar
doshitan committed
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
clean: SHELL:=bash
	rm $(SOME_DIR)/{one,two,three}/*.junk
#+END_SRC
* GNU vs BSD make
There are other makes out there, but generally I only think about GNU and BSD
variants. As mentioned at the top, I almost always write for GNU make
specifically.

If you want to more portable makefiles, avoid using anything listed on the
[[https://www.gnu.org/software/make/manual/html_node/Features.html][features]] and [[https://www.gnu.org/software/make/manual/html_node/Missing.html][missing]] pages for GNU make.

I don't recommend it, but you can also maintain separate ~GNUmakefile~ and
~BSDmakefile~ files and the different makes will pick the right one
automatically[fn::GNU make for example looks for ~GNUmakefile~, ~makefile~, and
~Makefile~, in that order, unless told to read a specific file with
~-f/--file~.]. For stuff that isn't version specific, you can put it in a
~makefile.common~ (or whatever), then ~include makefile.common~ in each of the
others to easy the maintenance burden.
* Actually building things
Make is first and foremost, a build system. Designed to take source files, do
something with them to generate other files (like an executable). So if you
actually do have file dependencies, make has some nice features.

Take a super simple example.

#+BEGIN_SRC makefile
test.html: test.md
	pandoc test.md --from markdown --to html --output test.html
#+END_SRC

This is a rule that knows how to build the ~test.html~ file from its markdown
source file, ~test.md~. We can run ~make test.html~ to build it.

The nice thing is make only does the actual work if it needs to by comparing the
timestamps for the prerequisites and targets, if the prerequisite has been
modified after the currently generated targets timestamp, then it knows it
should rerun the recipe, otherwise it doesn't. Which you can see if you run
~make test.html~ again, make will say there is no work to do.

But we are repeating ourselves a bit, make knows the target we intend to build
and the source inputs before it goes to run the recipe and has special variables
we can use for them.

#+BEGIN_SRC makefile
test.html: test.md
	pandoc $< --from markdown --to html --output $@
#+END_SRC

~$<~ is the expanded prerequisite name (~test.md~ in this case).
~$@~ is the expanded target name (~test.html~ in this case).

doshitan's avatar
doshitan committed
669
670
Find more [[https://www.gnu.org/software/make/manual/html_node/Automatic-Variables.html][automatic variables]] in the docs.

doshitan's avatar
doshitan committed
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
Better. But we can generalize a little more, say if we have a bunch of markdown
files, maybe documentation stuff, we can use a [[https://www.gnu.org/software/make/manual/html_node/Pattern-Rules.html][pattern rule]]:

#+BEGIN_SRC makefile
%.html: %.md
	pandoc $< --from markdown --to html --output $@
#+END_SRC

This is a rule that knows how to build *any* html file from its corresponding
markdown source file. So we can just say ~make test.html~ or ~make other.html~
and it will generate them if needed.

Okay, but we don't want to have to run a bunch of ~make <file>.html~ commands in
order to get all our html files generated. Let's write a rule that has as prerequisites
all the html files, so when we run that target it will trigger the build of all
the html files.

#+BEGIN_SRC makefile
html_files := test.html \
              other.html

all: $(html_files)

%.html: %.md
	pandoc $< --from markdown --to html --output $@
#+END_SRC

So we've added a list of the html files in the variable ~html_files~ and added
it as prerequisites to the target ~all~, so when we run ~make all~, all the html
files will be generated.

But it could be tedious to maintain the list of html files by hand, adding a new
line every time we create a new markdown file. Let's automate that.

#+BEGIN_SRC makefile
md_files  := $(wildcard *.md)
html_files := $(patsubst %.md,%.html,$(md_files))

all: $(html_files)

%.html: %.md
	pandoc $< --from markdown --to html --output $@
#+END_SRC

So we've used the [[https://www.gnu.org/software/make/manual/html_node/Wildcard-Function.html][~wildcard~ function]] to get a list of all the markdown files
in the directory. We then swap their file extention with the [[https://www.gnu.org/software/make/manual/html_node/Text-Functions.html][~patsubst~
function]]. Everything else is the same as before.

We can now just add markdown files as we wish, ~make all~ will build or rebuild
them as necessary.

Another wonderful feature of make, is it's parallel building, through the
~-j/--jobs~ option. You sometimes need to write your rules with parallel
building in mind to get the full benefit of it, but as we've written our rules
here, we can run say ~make all -j 5~ to build five files at once, a potentially
big time saver if you have the compute capacity.

This could make our directory a little messy, let's add a ~clean~ target to
delete the generated files.

#+BEGIN_SRC makefile
md_files  := $(wildcard *.md)
html_files := $(patsubst %.md,%.html,$(md_files))

all: $(html_files)

clean:
	-rm $(html_files)

%.html: %.md
	pandoc $< --from markdown --to html --output $@
#+END_SRC

Now we can run ~make clean~ to remove all those html files, if we want to force
regenerating all of them from scratch, or we just don't need them anymore.

This is just a simple example, but hopefully illustrative of the how we can take
advantage of some of make's features to build a simple and powerful tool.
doshitan's avatar
doshitan committed
749
750
751
752
753
754
755
756
* Libraries
You can import other makefiles with the ~include <file>~ feature, which means it
is possible to develop "libraries" of useful things you can copy
around/submodule/fetch from wherever.

Some existing ones to use or borrow from:
- GNU Make Standard Library: https://gmsl.sourceforge.io/
- https://github.com/cloudposse/build-harness
doshitan's avatar
doshitan committed
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
* Bigger example
Extracted from a old, but real project.

#+BEGIN_SRC makefile
.PHONY: build check-elm clean clean-elm deps distclean install
.SECONDEXPANSION:
.DEFAULT_GOAL := help

BUILDDIR ?= '_build'
DESTDIR ?= '../server/priv/static'


build: $(addprefix $(BUILDDIR)/, elm.js index.html)
build: ## Builds application into $(BUILDDIR)
	cp -r assets $(BUILDDIR)/
	cp -r node_modules/uswds/dist/* $(BUILDDIR)/assets/


build-prod: build
build-prod: ## Builds application in $(BUILDDIR), w/ optimizations
	mv $(BUILDDIR)/elm.js $(BUILDDIR)/elm-unoptimized.js
	closure-compiler --js $(BUILDDIR)/elm-unoptimized.js --js_output_file $(BUILDDIR)/elm.js --compilation_level SIMPLE
	rm $(BUILDDIR)/elm-unoptimized.js


install: ## Puts build files into $(DESTDIR)
	mkdir -p $(DESTDIR)
	cp -r $(BUILDDIR)/* $(DESTDIR)


deps: ## Installs project dependencies
	npm install
	elm package install --yes
	cd tests && elm package install --yes


clean: clean-elm
clean: ## Deletes build artifacts
	rm -r $(BUILDDIR)


clean-elm: ## Deletes Elm build artifacts
	rm -r elm-stuff/build-artifacts


distclean: clean
distclean: ## Deletes all non-source code stuff
	rm -r elm-stuff node_modules


check-elm: clean-elm $(BUILDDIR)/elm.js
check-elm: ## Rebuilds application for warnings


test:
	node_modules/.bin/elm-test


ELM_FILES := $(shell find src/ -type f -name '*.elm')
$(BUILDDIR)/elm.js: $(ELM_FILES) | $$(@D)/.
	elm make --warn src/Main.elm --output $@


$(BUILDDIR)/index.html: index.html | $$(@D)/.
	cp index.html $@


# Trick to make it easy to ensure a target's parent directories are created
# before we try to use them, just add a `| $$(@D)/.` to the dependencies of a
# target to ensure the directories in it's path are created
%/.:
	mkdir -p $@


# Self-documenting make file
# (http://marmelab.com/blog/2016/02/29/auto-documented-makefile.html)
help: ## Displays this help screen
	@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-10s\033[0m %s\n", $$1, $$2}'
	@echo ""
	@echo "BUILDDIR: $(BUILDDIR)"
	@echo "DESTDIR: $(DESTDIR)"
#+END_SRC
doshitan's avatar
doshitan committed
839
840
841
842

* Misc.
- http://clarkgrubb.com/makefile-style-guide
- http://make.mad-scientist.net/papers/rules-of-makefiles/
doshitan's avatar
doshitan committed
843
844
- https://blog.mindlessness.life/2019/11/17/the-language-agnostic-all-purpose-incredible-makefile.html
- http://gromnitsky.users.sourceforge.net/articles/notes-for-new-make-users/