INSTALL.md 119 KB
Newer Older
1
# EmulationStation Desktop Edition (ES-DE) v1.2 - Building and advanced configuration
Leon Styhre's avatar
Leon Styhre committed
2

Leon Styhre's avatar
Leon Styhre committed
3
4
**Note:** This is a quite technical document intended for those that are interested in compiling ES-DE from source code, or would like to customize the configuration. If you just want to start using the software, check out [USERGUIDE.md](USERGUIDE.md) instead.

Leon Styhre's avatar
Leon Styhre committed
5
Table of contents:
Leon Styhre's avatar
Leon Styhre committed
6

Leon Styhre's avatar
Leon Styhre committed
7
8
9
[[_TOC_]]

## Development Environment
10

Leon Styhre's avatar
Leon Styhre committed
11
12
ES-DE is developed and compiled using Clang/LLVM and GCC on Unix, Clang/LLVM on macOS and MSVC and GCC (MinGW) on Windows.

Leon Styhre's avatar
Leon Styhre committed
13
CMake is the build system for all the supported operating systems, used in conjunction with `make` on Unix and macOS and `nmake` and `make` on Windows. Xcode on macOS or Visual Studio on Windows are not required for building ES-DE and they have not been used during the development. The only exception is notarization of codesigned macOS packages which require the `altool` and `stapler` binaries that come bundled with Xcode.
14

15
16
For automatic code formatting [clang-format](https://clang.llvm.org/docs/ClangFormat.html) is used.

17
Any code editor can be used of course, but I recommend [VSCode](https://code.visualstudio.com).
18

19
## Building on Unix
20

21
There are some dependencies that need to be fulfilled in order to build ES-DE. These are detailed per operating system below.
22

23
**Debian/Ubuntu**
Leon Styhre's avatar
Leon Styhre committed
24

25
All of the required packages can be installed with apt-get:
26
```
27
sudo apt-get install build-essential clang-format git cmake libsdl2-dev libavcodec-dev libavfilter-dev libavformat-dev libavutil-dev libfreeimage-dev libfreetype6-dev libcurl4-openssl-dev libpugixml-dev libasound2-dev libgl1-mesa-dev
Leon Styhre's avatar
Leon Styhre committed
28
29
```

30
31
32
33
If building with the optional VLC video player, the following packages are also needed:
```
sudo apt-get install vlc libvlc-dev
```
Leon Styhre's avatar
Leon Styhre committed
34

35
**Fedora**
Leon Styhre's avatar
Leon Styhre committed
36

37
38
On Fedora you first need to install the RPM Fusion repository:

Leon Styhre's avatar
Leon Styhre committed
39
```
40
41
42
sudo dnf install \
https://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm \
https://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm
43
```
44

45
46
47
48
49
Then you can use dnf to install all the required packages:
```
sudo dnf install gcc-c++ clang-tools-extra cmake rpm-build SDL2-devel ffmpeg-devel freeimage-devel freetype-devel curl-devel pugixml-devel alsa-lib-devel mesa-libGL-devel
```
If building with the optional VLC video player, the following packages are also needed:
50
51


Leon Styhre's avatar
Leon Styhre committed
52
```
53
sudo dnf install vlc vlc-devel
Leon Styhre's avatar
Leon Styhre committed
54
55
```

56
**Manjaro**
Leon Styhre's avatar
Leon Styhre committed
57
58
59

Use pacman to install all the required packages:

60
```
61
sudo pacman -S gcc clang make cmake pkgconf sdl2 ffmpeg freeimage freetype2 pugixml
62
63
64
65
66
```

If building with the optional VLC video player, the following package is also needed:
```
sudo pacman -S vlc
67
68
```

69
70
71
72
**Raspberry Pi OS (Raspian)**

All of the required packages can be installed with apt-get:
```
73
sudo apt-get install clang-format cmake libsdl2-dev libavcodec-dev libavfilter-dev libavformat-dev libavutil-dev libfreeimage-dev libcurl4-gnutls-dev libpugixml-dev
74
75
76
77
78
79
80
81
82
83
84
85
```

If building with the optional VLC video player, the following packages are also needed:
```
sudo apt-get install vlc libvlc-dev
```

To build with CEC support you also need to install these packages:
```
sudo apt-get install libcec-dev libp8-platform-dev
```

86
87
88
89
The Raspberry Pi 4/400 is the minimum recommended version and earlier boards have not been tested. The GPU memory should be set to at least 256 MB using `raspi-config` and the GL driver must be set to `GL (Fake KMS)` or the performance will be horrible.

Note that low-level ALSA sound support has been removed from ES-DE which means that a sound server like PulseAudio or PipeWire is required.

90
**FreeBSD**
Leon Styhre's avatar
Leon Styhre committed
91

92
93
Use pkg to install the dependencies:
```
94
pkg install llvm-devel git pkgconf cmake sdl2 ffmpeg freeimage pugixml
95
96
```

97
98
99
100
101
If building with the optional VLC video player, the following package is also needed:
```
pkg install vlc
```

Leon Styhre's avatar
Leon Styhre committed
102
Clang/LLVM and curl should already be included in the base OS installation.
103

104
**NetBSD**
Leon Styhre's avatar
Leon Styhre committed
105

106
107
Use pkgin to install the dependencies:
```
108
pkgin install clang git cmake pkgconf SDL2 ffmpeg4 freeimage pugixml
109
110
111
112
113
```

If building with the optional VLC video player, the following package is also needed:
```
pkgin vlc
114
115
```

116
NetBSD ships with GCC by default, and although you should be able to use Clang/LLVM, it's probably easier to just stick to the default compiler environment. The reason why the clang package needs to be installed is to get clang-format onto the system.
117

118
**OpenBSD**
Leon Styhre's avatar
Leon Styhre committed
119

120
121
Use pkg_add to install the dependencies:
```
122
123
124
125
126
127
pkg_add clang-tools-extra cmake pkgconf sdl2 ffmpeg freeimage
```

If building with the optional VLC video player, the following package is also needed:
```
pkg_add vlc
128
129
```

Leon Styhre's avatar
Leon Styhre committed
130
In the same manner as for FreeBSD, Clang/LLVM and curl should already be installed by default.
131
132
133
134

Pugixml does exist in the package collection but somehow this version is not properly detected by CMake, so you need to compile this manually as well:

```
Leon Styhre's avatar
Leon Styhre committed
135
git clone https://github.com/zeux/pugixml.git
136
137
138
139
140
141
142
cd pugixml
git checkout v1.10
cmake .
make
make install
```

Leon Styhre's avatar
Leon Styhre committed
143
**Cloning and compiling ES-DE**
144

145
To clone the source repository, run the following:
146

147
```
148
git clone https://gitlab.com/es-de/emulationstation-de.git
149
150
```

151
Then generate the Makefile and build the software:
152

153
```
154
cd emulationstation-de
155
cmake .
156
157
158
make
```

159
By default the master branch will be used, which is where development takes place. To instead build a stable release, switch to the `stable-x.x` branch for the version, for example:
Leon Styhre's avatar
Leon Styhre committed
160
161
162

```
cd emulationstation-de
163
git checkout stable-1.2
Leon Styhre's avatar
Leon Styhre committed
164
165
166
167
cmake .
make
```

168
169
170
171
172
173
174
To build ES-DE with the VLC video player in addition to the default FFmpeg player, enable the VLC_PLAYER option, for example:
```
cmake -DVLC_PLAYER=on .
make
```

To create a debug build, run this:
175
```
176
cmake -DCMAKE_BUILD_TYPE=Debug .
177
178
make
```
Leon Styhre's avatar
Leon Styhre committed
179
Keep in mind that a debug version will be much slower due to all compiler optimizations being disabled.
180

181
To create a profiling build (optimized with debug symbols), run this:
182
183
184
185
186
```
cmake -DCMAKE_BUILD_TYPE=Profiling .
make
```

187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
To enable AddressSanitizer which helps with identifying memory issues like corruption bugs and buffer overflows, build with the ASAN option:
```
cmake -DCMAKE_BUILD_TYPE=Debug -DASAN=on .
make
```

To enable ThreadSanitizer which helps with identifying data races for multi-threaded code, build with the TSAN option:
```
cmake -DCMAKE_BUILD_TYPE=Debug -DTSAN=on .
make
```

To enable UndefinedBehaviorSanitizer which helps with identifying bugs that may otherwise be hard to find, build with the UBSAN option:
```
cmake -DCMAKE_BUILD_TYPE=Debug -UBSAN=on .
make
```

Leon Styhre's avatar
Leon Styhre committed
205
To get stack traces printed as well, set this environment variable:
206
207
208
209
210
211
212
```
export UBSAN_OPTIONS=print_stacktrace=1
```

These tools aren't very useful without debug symbols so only use them for a Debug or Profiling build. Clang and GCC support all three tools. Note that ASAN and TSAN can't be combined.

As for advanced debugging, Valgrind is a very powerful and useful tool which can analyze many aspects of the application. Be aware that some of the Valgrind tools should be run with an optimized build, and some with optimizations turned off. Refer to the Valgrind documentation for more information.
213
214
215
216
217
218
219

The most common tool is Memcheck to check for memory leaks, which you run like this:

```
valgrind --tool=memcheck --leak-check=full --log-file=../valgrind_run_01 ./emulationstation
```

Leon Styhre's avatar
Leon Styhre committed
220
221
222
223
224
There are numerous flags that can be used, for example this will also track reachable memory which could indicate further leaks:
```
valgrind --tool=memcheck --leak-check=full --track-origins=yes --show-reachable=yes --log-file=../valgrind_run_01 ./emulationstation
```

225
Another helpful tool is the Callgrind call-graph analyzer:
226
```
Leon Styhre's avatar
Leon Styhre committed
227
valgrind --tool=callgrind --log-file=../valgrind_run_01 ./emulationstation
228
229
```

230
The output file can be loaded into an application such as KCachegrind for data analysis.
231
232


233
234

Yet another very useful Valgrind tool is the Massif heap profiler, which can be run like this:
235
```
236
valgrind --tool=massif --massif-out-file=../massif.out.01 ./emulationstation
237
238
```

239
The output file can be loaded into an application such as Massif-Visualizer for analysis.
240

241
Another useful tool is `scan-build`, assuming you use Clang/LLVM. This is a static analyzer that runs during compilation to provide a very helpful HTML report of potential bugs (well it should be actual bugs but some false positives could be included). You need to run it for both the cmake and make steps, here's an example:
242
243
244
245
246
247

```
scan-build cmake -DCMAKE_BUILD_TYPE=Debug .
scan-build make -j6
```

248
You open the report with the `scan-view` command which lets you read it using your web browser. Note that the compilation time is much longer when using the static analyzer compared to a normal build. As well this tool generates a lot of extra files and folders in the build tree, so it may make sense to run it in a separate copy of the source folder to avoid having to clean up all this extra data when the analysis has been completed.
249

250
251
252
253
254
255
256
An even more advanced static analyzer is `clang-tidy`, to use it first make sure it's installed on your system and then run the following:
```
cmake -DCLANG_TIDY=on .
```

Even though many irrelevant checks are filtered out via the configuration, this will still likely produce a quite huge report (at least until most of the recommendations have been implemented). In the same manner as for scan-view, the compilation time is much longer when using this static analyzer compared to a normal build.

257
258
259
260
261
262
263
To build ES-DE with experimental FFmpeg video hardware decoding support, enable the following option:

```
cmake -DVIDEO_HW_DECODING=on .
make
```

264
To build ES-DE with CEC support, enable the corresponding option, for example:
265
266

```
267
cmake -DCEC=on .
268
269
make
```
270
You will most likely need to install additional packages to get this to build. On Debian-based systems these are _libcec-dev_ and _libp8-platform-dev_. Note that the CEC support is currently untested.
271

Leon Styhre's avatar
Leon Styhre committed
272
To build with the GLES renderer, run the following:
273
```
274
cmake -DGLES=on .
275
276
make
```
277
The GLES renderer is quite limited as there is no shader support for it, so ES-DE will definitely not look as pretty as when using the default OpenGL renderer. This option is basically deprecated as the releases for all supported platforms are built using the desktop OpenGL renderer (including the Raspberry Pi).
278

279
Running multiple compile jobs in parallel is a good thing as it speeds up the build time a lot (scaling almost linearly). Here's an example telling make to run 6 parallel jobs:
Leon Styhre's avatar
Leon Styhre committed
280
281
282
283
284

```
make -j6
```

Leon Styhre's avatar
Leon Styhre committed
285
286
By default ES-DE will install under /usr on Linux, /usr/pkg on NetBSD and /usr/local on FreeBSD and OpenBSD although this can be changed by setting the `CMAKE_INSTALL_PREFIX` variable.

Leon Styhre's avatar
Leon Styhre committed
287
The following example will build the application for installtion under /opt:
288
289
290
291
292

```
cmake -DCMAKE_INSTALL_PREFIX=/opt .
```

293
It's important to understand that this is not only the directory used by the install script, the CMAKE_INSTALL_PREFIX variable also modifies code inside ES-DE used to locate the required program resources. So it's necessary that the install prefix corresponds to the location where the application will actually be installed.
294

Leon Styhre's avatar
Leon Styhre committed
295
296
On Linux, if you're not building a package and instead intend to install using `make install` it's recommended to set the installation prefix to /usr/local instead of /usr.

Leon Styhre's avatar
Leon Styhre committed
297
**Compilers**
298

Leon Styhre's avatar
Leon Styhre committed
299
Both Clang/LLVM and GCC work fine for building ES-DE.
300

301
I did some small benchmarks comparing Clang 10.0 to GCC 9.3.0 with the ES-DE v1.1 codebase on an Intel Xeon W-2245 @ 3.90GHz running Kubuntu 20.04.2 LTS and it's pretty interesting.
302
303

Advantages with Clang (vs GCC):
304
305
306
307
308
* 8% smaller binary size for a release build
* 31% smaller binary size for a debug build
* 16% faster compile time for a release build
* 25% faster compile time for a debug build
* 13% faster application startup time for a release build
309
310
311
312
313
* 4% faster application startup time for a debug build

*Release build: Optimizations enabled, debug info disabled, binary stripped.* \
*Debug build: Optimizations disabled, debug info enabled, binary not stripped.*

Leon Styhre's avatar
Leon Styhre committed
314
This Clang debug build is LLVM "native", i.e. intended to be debugged using the LLVM project debugger LLDB. The problem is that this is still not well integrated with VSCode that I use for development so I need to keep using GDB. But this is problematic as the libstd++ data required by GDB is missing in the binary, making it impossible to see the values of for instance std::string variables.
315

316
It's possible to activate the additional debug info needed by GDB by using the flag `-D_GLIBCXX_DEBUG`. I've added this to CMakeLists.txt when using Clang, but this bloats the binary and makes the code much slower. Actually, instead of a 4% faster application startup, it's now 25% slower. The same goes for the binary size, instead of 31% smaller it's now 5% larger. The compilation time is still less than GCC but only by 10% instead of 25%.
317

318
But I'm expecting this issue to be resolved in the future so the workaround can be removed.
319

320
It's by the way very easy to switch between LLVM and GCC using Ubuntu, just use the `update-alternatives` command:
321
322

```
323
myusername@computer:~$ sudo update-alternatives --config c++
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
[sudo] password for user:
There are 2 choices for the alternative c++ (providing /usr/bin/c++).

  Selection    Path              Priority   Status
------------------------------------------------------------
* 0            /usr/bin/g++       20        auto mode
  1            /usr/bin/clang++   10        manual mode
  2            /usr/bin/g++       20        manual mode

Press <enter> to keep the current choice[*], or type selection number: 1
update-alternatives: using /usr/bin/clang++ to provide /usr/bin/c++ (c++) in manual mode
```

Following this, just re-run cmake and make and the binary should be built by Clang instead.

339
**Installing**
340
341
342
343
344
345
346

Installing the software requires root permissions, the following command will install all the required application files:

```
sudo make install
```

Leon Styhre's avatar
Leon Styhre committed
347
Assuming the default installation prefix /usr has been used, this is the directory structure for the installation:
348
349

```
Leon Styhre's avatar
Leon Styhre committed
350
351
352
353
354
355
356
357
/usr/bin/emulationstation
/usr/share/man/man6/emulationstation.6.gz
/usr/share/applications/emulationstation.desktop
/usr/share/emulationstation/LICENSE
/usr/share/emulationstation/licenses/*
/usr/share/emulationstation/resources/*
/usr/share/emulationstation/themes/*
/usr/share/pixmaps/emulationstation.svg
358
359
```

Leon Styhre's avatar
Leon Styhre committed
360
361
362
However, when installing manually instead of building a package, it's recommended to change the install prefix to /usr/local instead of /usr.

Be aware that if using the GNOME desktop environment, /usr/share/pixmaps/emulationstation.svg must exist in order for the ES-DE icon to be shown in the Dash and task switcher.
363

364
ES-DE will look in the following locations for application resources, in the listed order:
365

Leon Styhre's avatar
Leon Styhre committed
366
367
368
* \<home\>/.emulationstation/resources/
* \<install prefix\>/share/emulationstation/resources/
* \<ES-DE executable directory\>/resources/
369

Leon Styhre's avatar
Leon Styhre committed
370
The resources directory is critical, without it the application won't start.
371

372
As well the following locations will be searched for themes, also in the listed order:
Leon Styhre's avatar
Leon Styhre committed
373

Leon Styhre's avatar
Leon Styhre committed
374
375
376
* \<home\>/.emulationstation/themes/
* \<install prefix\>/share/emulationstation/themes/
* \<ES-DE executable directory\>/themes/
377

Leon Styhre's avatar
Leon Styhre committed
378
A theme is not mandatory to start the application, but ES-DE will be basically useless without it.
379

380
As indicated above, the home directory will always take precedence and any resources or themes located there will override the ones in the installation path, or in the path of the ES-DE executable.
381

Leon Styhre's avatar
Leon Styhre committed
382
**Creating .deb and .rpm packages**
383

384
Creation of Debian .deb packages is enabled by default, simply run `cpack` to generate the package:
385
386

```
387
myusername@computer:~/emulationstation-de$ cpack
388
389
390
391
392
393
CPack: Create package using DEB
CPack: Install projects
CPack: - Run preinstall target for: emulationstation-de
CPack: - Install project: emulationstation-de []
CPack: Create package
CPackDeb: - Generating dependency list
394
CPack: - package: /home/myusername/emulationstation-de/emulationstation-de-1.2.0-x64.deb generated.
395
396
```

397
You may want to check that the dependencies look fine, as they're (mostly) automatically generated by CMake:
Leon Styhre's avatar
Leon Styhre committed
398
399

```
400
dpkg -I ./emulationstation-de-1.2.0-x64.deb
Leon Styhre's avatar
Leon Styhre committed
401
402
403
404
405
```

The package can now be installed using a package manager, for example apt:

```
406
sudo apt install ./emulationstation-de-1.2.0-x64.deb
Leon Styhre's avatar
Leon Styhre committed
407
```
Leon Styhre's avatar
Leon Styhre committed
408

409
To build an RPM package instead, set the flag LINUX_CPACK_GENERATOR to RPM when running cmake, for example:
410
411

```
412
cmake -DLINUX_CPACK_GENERATOR=RPM .
413
```
414
415
416

Then simply run `cpack`:

417
```
418
419
420
421
422
423
424
myusername@computer:~/emulationstation-de$ cpack
CPack: Create package using RPM
CPack: Install projects
CPack: - Run preinstall target for: emulationstation-de
CPack: - Install project: emulationstation-de []
CPack: Create package
CPackRPM: Will use GENERATED spec file: /home/myusername/emulationstation-de/_CPack_Packages/Linux/RPM/SPECS/emulationstation-de.spec
425
CPack: - package: /home/myusername/emulationstation-de/emulationstation-de-1.2.0-x64.rpm generated.
426
427
```

428
On Fedora, you need to install rpmbuild before this command can be run:
Leon Styhre's avatar
Leon Styhre committed
429
430
431
```
sudo dnf install rpm-build
```
432

433
After the package generation you can check that the metadata looks fine using the `rpm` command:
434
```
435
rpm -qi ./emulationstation-de-1.2.0-x64.rpm
436
437
```

438
To see the automatically generated dependencies, run this:
Leon Styhre's avatar
Leon Styhre committed
439
```
440
rpm -q --requires ./emulationstation-de-1.2.0-x64.rpm
Leon Styhre's avatar
Leon Styhre committed
441
442
443
444
```

And of course, you can also install the package:
```
445
sudo dnf install ./emulationstation-de-1.2.0-x64.rpm
Leon Styhre's avatar
Leon Styhre committed
446
```
Leon Styhre's avatar
Leon Styhre committed
447

448
449
450
**Creating an AppImage**

The process to create a Linux AppImage is completely automated. You simply run the AppImage creation script, which has to be executed from the root of the repository:
451

452
453
454
455
```
tools/create_AppImage.sh
```

Leon Styhre's avatar
Leon Styhre committed
456
This script has only been tested on Ubuntu (20.04 LTS and 22.04 LTS) and it's recommended to go for an older operating system when building the AppImage to achieve compatibility with a large number of distributions. This does come with some sacrifices though, such as the use of an older SDL version which may not support the latest game controllers.
457

Leon Styhre's avatar
Leon Styhre committed
458
The script will delete CMakeCache.txt and run cmake with the BUNDLED_CERTS option, as otherwise scraping wouldn't work on Fedora (and probably not on openSUSE and a few other distributions as well).
459
460

After creating the AppImage it's recommended to delete CMakeCache.txt manually so the BUNDLED_CERTS option is not accidentally enabled when building the other packages.
461

Leon Styhre's avatar
Leon Styhre committed
462
463
464
465
466
467
468
To build the Steam Deck-specific AppImage, run the following:
```
tools/create_AppImage_SteamDeck.sh
```

This is similar to the regular AppImage but does not build with the BUNDLED_CERTS option and changes some settings like the VRAM limit.

Leon Styhre's avatar
Leon Styhre committed
469
Both _appimagetool_ and _linuxdeploy_ are required for the build process but they will be downloaded automatically by the script if they don't exist. So to force an update to the latest build tools, delete these two AppImages prior to running the build script.
Leon Styhre's avatar
Leon Styhre committed
470

471
472
473
## Building on macOS

ES-DE for macOS is built using Clang/LLVM which is the default compiler for this operating system. It's pretty straightforward to build software on this OS. The main problem is that there is no native package manager, but as there are several third party package managers available, this can be partly compensated for. The use of one of them, [Homebrew](https://brew.sh), is detailed below.
474

475
**Setting up the build tools**
476

477
Install the Command Line Tools which include Clang/LLVM, Git, make etc. Simply open a terminal and enter the command `clang`. This will open a dialog that will let you download and install the tools.
478

479
Following this, install the Homebrew package manager which will simplify the installation of some additional required packages. Run the following in a terminal window:
480
```
Leon Styhre's avatar
Leon Styhre committed
481
482
483
484
485
486
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
```

If running on an M1 Mac, you also need to add the following to your `~/.zshrc` shell profile file:
```
export PATH=/opt/homebrew/bin:$PATH
487
488
```

489
**Package installation with Homebrew**
490

491
Install the required tools:
492
493

```
494
brew install clang-format cmake pkg-config nasm yasm
495
496
```

497
If building with the optional VLC video player, then run this as well:
498
499

```
500
brew install --cask vlc
501
502
```

Leon Styhre's avatar
Leon Styhre committed
503
**Developer mode**
504
505
506
507
508
509

Enable developer mode to avoid annoying password requests when attaching the debugger to a process:
```
sudo /usr/sbin/DevToolsSecurity --enable
```

510
**Cloning and compiling**
511
512
513
514

To clone the source repository, run the following:

```
515
git clone https://gitlab.com/es-de/emulationstation-de.git
516
517
```

518
For macOS all dependencies are built in-tree in the `external` directory tree. There are two scripts in the tools directory that automate this entirely and they are executed such as this:
519
520
521

```
cd emulationstation-de
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
tools/macOS_dependencies_setup.sh
tools/macOS_dependencies_build.sh
```
This can take quite a while as multiple packages are downloaded and compiled. It's important to not have any of the dependency libraries installed using Homebrew as they will interfere with the in-tree build.

Re-running macOS_dependencies_setup.sh will delete and download all dependencies again, and re-running macOS_dependencies_build.sh will clean and rebuild all packages. If you want to recompile a single package, make sure to first set the MACOSX_DEPLOYMENT_TARGET variable:
```
export MACOSX_DEPLOYMENT_TARGET=10.14
```

Then manually recompile the package, for example:
```
cd external/pugixml
rm CMakeCache.txt
cmake .
make clean
make -j4
cp libpugixml.a ../..
```

After all dependencies have been built, generate the Makefile and build ES-DE:

```
545
546
547
548
cmake .
make
```

549
By default the master branch will be used, which is where development takes place. To instead build a stable release, switch to the `stable-x.x` branch for the version, for example:
Leon Styhre's avatar
Leon Styhre committed
550
551
552

```
cd emulationstation-de
553
git checkout stable-1.2
Leon Styhre's avatar
Leon Styhre committed
554
555
556
557
cmake .
make
```

558
559
560
561
562
563
564
To build ES-DE with the VLC video player in addition to the default FFmpeg player, enable the VLC_PLAYER option:
```
cmake -DVLC_PLAYER=on .
make
```

To generate a debug build, run this:
565
566
567
568
```
cmake -DCMAKE_BUILD_TYPE=Debug .
make
```
569
Keep in mind that the debug version will be much slower due to all compiler optimizations being disabled.
570

571
To enable AddressSanitizer which helps with identifying memory issues like corruption bugs and buffer overflows, build with the ASAN option:
Leon Styhre's avatar
Leon Styhre committed
572
```
573
574
cmake -DCMAKE_BUILD_TYPE=Debug -DASAN=on .
make
Leon Styhre's avatar
Leon Styhre committed
575
```
576

577
578
579
580
581
To enable ThreadSanitizer which helps with identifying data races for multi-threaded code, build with the TSAN option:
```
cmake -DCMAKE_BUILD_TYPE=Debug -DTSAN=on .
make
```
582

583
584
585
586
587
To enable UndefinedBehaviorSanitizer which helps with identifying bugs that may otherwise be hard to find, build with the UBSAN option:
```
cmake -DCMAKE_BUILD_TYPE=Debug -UBSAN=on .
make
```
588

Leon Styhre's avatar
Leon Styhre committed
589
To get stack traces printed as well, set this environment variable:
590
591
592
```
export UBSAN_OPTIONS=print_stacktrace=1
```
593

594
These tools aren't very useful without debug symbols so only use them for a Debug or Profiling build. Note that ASAN and TSAN can't be combined.
595

596
Specifically on macOS it seems as if AddressSanitizer generates a lot of false positives regarding container overflows, so it may be necessary to ignore these:
597
```
598
export ASAN_OPTIONS=detect_container_overflow=0
599
600
```

601
Running `make -j6` (or whatever number of parallel jobs you prefer) speeds up the compilation time if you have cores to spare.
602

Leon Styhre's avatar
Leon Styhre committed
603
After building ES-DE and trying to execute the application, there could be issues with finding the dynamic link libraries for VLC (assuming VLC was enabled for the build) as these are not installed into a standard location but rather into the /Applications folder. As such, you may need to set the DYLD_LIBRARY_PATH environment variable to find the VLC libraries. Note that this is not intended or required for the release build that will be shipped in a DMG installer or if you manually install ES-DE using _make install_. It's only needed to be able to run the binary from the build directory. You should add this to your shell profile file to avoid having to set it each time you open a new terminal window:
604
605
606
```
export DYLD_LIBRARY_PATH=/Applications/VLC.app/Contents/MacOS/lib
```
607

608
Running ES-DE from the build directory may be a bit flaky as there is no Info.plist file available which is required for setting the proper window mode and such. It's therefore recommended to run the application from the installation directory for any more in-depth testing. But normal debugging can of course be done from the build directory.
609

610
**Building for the M1 (ARM) processor**
611

612
The build steps detailed above should in theory work identically on an M1 processor but possibly some of the dependencies will not build correctly and may need manual patching. Cross-compiling using an Intel processor has been attempted but failed due to multiple issues with dependencies refusing to build.
613

614
**Code signing**
615

616
Due to the Apple notarization requirement implemented as of macOS 10.14.5 a build with simple code signing is needed for versions up to 10.13 and another build with both code signing and notarization is required for version 10.14 and higher.
617

618
macOS code signing is beyond the scope of this document, but the CMake option MACOS_CODESIGN_IDENTITY is used to specify the code signing certificate identity, for example:
619
```
620
cmake -DMACOS_CODESIGN_IDENTITY="My Name" .
621
622
```

623
Assuming the code signing ceritificate is properly setup in Keychain Access, the process will be automatic and the resulting DMG package can be notarized as-is. For some reason code signing fails if run via an SSH session, so in order for the cpack command to succeed it needs to run from a terminal window started via the GUI.
624

625
**Legacy build**
626

627
Normally ES-DE is meant to be built for macOS 10.14 and higher, but a legacy build for earlier operating system versions can be enabled. This has been tested with a minimum version of 10.11. It's unclear if it works with even older macOS releases.
628

629
To enable a legacy build, change the CMAKE_OSX_DEPLOYMENT_TARGET variable in CMakeLists.txt from 10.14 to whatever version you would like to build for. This will disable Hardened Runtime if signing is enabled and it will add "legacy" to the DMG installer filename when running CPack. It will also enable the bundled TLS/SSL certificates. As these older macOS releases are no longer receiving patches from Apple, certificates have likely expired meaning the scraper would not work if the bundled certificates were not used.
Leon Styhre's avatar
Leon Styhre committed
630

631
You also need to modify es-app/assets/EmulationStation-DE_Info.plist and set the key SMinimumSystemVersion to the version you're building for. And finally CMAKE_OSX_DEPLOYMENT_TARGET needs to be updated in tools/macOS_dependencies_build.sh. This script then needs to be executed to rebuild all dependencies for the configured macOS version.
632

633
**Installing**
Leon Styhre's avatar
Leon Styhre committed
634

635
As macOS does not have any package manager which would have handled the library dependencies, we need to bundle the required shared libraries with the application. This is almost completely automated by the build scripts.
Leon Styhre's avatar
Leon Styhre committed
636

637
The only exceptions are these libraries for the optional VLC video player:
Leon Styhre's avatar
Leon Styhre committed
638
```
639
640
libvlc.dylib
libvlccore.dylib
Leon Styhre's avatar
Leon Styhre committed
641
642
```

643
If building the VLC video player, copy these files from the /Applications/VLC.app/Contents/MacOS/lib/ directory to the emulationstation-de build folder.
644

645
In addition to these you need to create a `plugins` directory and copy over the following libraries, which are located in /Applications/VLC.app/Contents/MacOS/plugins/:
646
647

```
648
649
libadummy_plugin.dylib
libamem_plugin.dylib
650
libaudio_format_plugin.dylib
651
libauhal_plugin.dylib
652
653
654
655
656
657
658
659
660
661
662
663
libavcodec_plugin.dylib
libconsole_logger_plugin.dylib
libfilesystem_plugin.dylib
libfreetype_plugin.dylib
libswscale_plugin.dylib
libtrivial_channel_mixer_plugin.dylib
libvmem_plugin.dylib
libwave_plugin.dylib
libx264_plugin.dylib
libx265_plugin.dylib
```

Leon Styhre's avatar
Leon Styhre committed
664
On macOS you can install the application as a normal user, i.e. no root privileges are required. Simply run the following:
665
666
667
668
669

```
make install
```

670
This will be the directory structure for the installation (the VLC-related files are optional):
671
672
673
674

```
/Applications/EmulationStation Desktop Edition.app/Contents/Info.plist
/Applications/EmulationStation Desktop Edition.app/Contents/MacOS/EmulationStation
675
/Applications/EmulationStation Desktop Edition.app/Contents/MacOS/libSDL2-2.0.dylib
676
677
678
679
680
/Applications/EmulationStation Desktop Edition.app/Contents/MacOS/libavcodec.58.dylib
/Applications/EmulationStation Desktop Edition.app/Contents/MacOS/libavfilter.7.dylib
/Applications/EmulationStation Desktop Edition.app/Contents/MacOS/libavformat.58.dylib
/Applications/EmulationStation Desktop Edition.app/Contents/MacOS/libavutil.56.dylib
/Applications/EmulationStation Desktop Edition.app/Contents/MacOS/libfdk-aac.2.dylib
681
/Applications/EmulationStation Desktop Edition.app/Contents/MacOS/libfreetype.6.dylib
682
683
684
/Applications/EmulationStation Desktop Edition.app/Contents/MacOS/libpostproc.55.dylib
/Applications/EmulationStation Desktop Edition.app/Contents/MacOS/libswresample.3.dylib
/Applications/EmulationStation Desktop Edition.app/Contents/MacOS/libswscale.5.dylib
685
686
/Applications/EmulationStation Desktop Edition.app/Contents/MacOS/libvlc.dylib
/Applications/EmulationStation Desktop Edition.app/Contents/MacOS/libvlccore.dylib
687
688
/Applications/EmulationStation Desktop Edition.app/Contents/MacOS/libvorbis.0.4.9.dylib
/Applications/EmulationStation Desktop Edition.app/Contents/MacOS/libvorbisenc.2.0.12.dylib
689
690
691
692
693
694
695
696
/Applications/EmulationStation Desktop Edition.app/Contents/MacOS/plugins/*
/Applications/EmulationStation Desktop Edition.app/Contents/Resources/EmulationStation-DE.icns
/Applications/EmulationStation Desktop Edition.app/Contents/Resources/LICENSE
/Applications/EmulationStation Desktop Edition.app/Contents/Resources/licenses/*
/Applications/EmulationStation Desktop Edition.app/Contents/Resources/resources/*
/Applications/EmulationStation Desktop Edition.app/Contents/Resources/themes/*
```

697
ES-DE will look in the following locations for application resources, in the listed order:
698

Leon Styhre's avatar
Leon Styhre committed
699
700
701
* \<home\>/.emulationstation/resources/
* \<ES-DE executable directory\>/../Resources/resources/
* \<ES-DE executable directory\>/resources/
702

703
The resources directory is critical, without it the application won't start.
704

705
As well the following locations will be searched for themes, also in the listed order:
706

Leon Styhre's avatar
Leon Styhre committed
707
708
709
* \<HOME\>/.emulationstation/themes/
* \<ES-DE executable directory\>/../Resources/themes/
* \<ES-DE executable directory\>/themes/
710

711
A theme is not mandatory to start the application, but ES-DE will be basically useless without it.
712

713
As indicated above, the home directory will always take precedence and any resources or themes located there will override the ones in the path of the ES-DE executable.
714

715
**Creating a .dmg installer**
716

717
Simply run `cpack` to build a .dmg disk image/installer:
718
719

```
720
myusername@computer:~/emulationstation-de$ cpack
721
CPack: Create package using Bundle
722
723
724
725
CPack: Install projects
CPack: - Run preinstall target for: emulationstation-de
CPack: - Install project: emulationstation-de []
CPack: Create package
726
CPack: - package: /Users/myusername/emulationstation-de/EmulationStation-DE-1.2.0-x64.dmg generated.
727
728
```

729
## Building on Windows
730

731
Both MSVC and MinGW (GCC) work fine for building ES-DE on Windows.
732

733
Although I would prefer to exclude support for MSVC, this compiler simply works much better when developing as it's much faster than MinGW when linking debug builds and when actually debugging. But for release builds MinGW is very fast and ES-DE starts around 18% faster when built with MinGW meaning this compiler probably generates more efficient code overall. As well MSVC requires a lot more junk DLL files to be distributed with the application so it's not a good candidate for the final release build.
734

735
For this reason I think MSVC makes sense for development and MinGW for the releases.
736

737
**MSVC setup**
738

739
740
741
742
743
744
745
746
Install Git for Windows: \
[https://gitforwindows.org](https://gitforwindows.org)

Download the Visual Studio Build Tools (choose Visual Studio Community edition): \
[https://visualstudio.microsoft.com/downloads](https://visualstudio.microsoft.com/downloads)

It seems as if Microsoft has dropped support for installing the Build Tools without the Visual Studio IDE, at least I've been unable to find a way to exclude it. But I just pretend it's not installed and use VSCode instead which works perfectly fine.

747
During installation, choose the Desktop development with C++ workload with the following options (version details may differ):
748
749

```
750
MSVC v143 - VS 2022 C++ x64/x86 build tools (Latest)
751
752
Windows 10 SDK
Just-In-Time debugger
753
754
755
756
757
C++ AddressSanitizer
```

If you will only use MSVC and not MinGW, then also add this option:
```
758
759
760
C++ CMake tools for Windows
```

761
If not installing the CMake version supplied by Microsoft, you need to make sure that you have a recent version on your machine or CMake will not be able to detect MSVC correctly.
762

763
764
As well you may need to install the latest version of Microsoft Visual C++ Redistributable which can be downloaded here:\
https://docs.microsoft.com/en-us/cpp/windows/latest-supported-vc-redis
765
766


767
768
769
770
771
The way the MSVC environment works is that a specific developer shell is provided where the build environment is properly configured. You open this from the Start menu via `Visual Studio 2022` -> `Visual Studio tools` -> `VC` -> `x64 Native Tools Command Prompt for VS 2022 Current`.

It's important to choose the x64-specific shell and not the x86 variant, as ES-DE will only compile as a 64-bit application.

**MinGW (GCC) setup**
772
773

Download the following packages and install them:
774

775
[https://gitforwindows.org](https://gitforwindows.org)
776

777
[https://cmake.org/download](https://cmake.org/download)
778

779
Download the _MinGW-w64 based_ version of GCC: \
Leon Styhre's avatar
Leon Styhre committed
780
[https://jmeubank.github.io/tdm-gcc](https://jmeubank.github.io/tdm-gcc)
781

782
After installation, make a copy of `TDM-GCC-64\bin\mingw32-make` to `make` just for convenience.
783

784
785
Version 9.2.0 of MinGW has been confirmed to work fine, but 10.3.0 appears broken as it causes huge performance problems for the FFmpeg function avfilter_graph_free() with execution times going from milliseconds to hundreds of milliseconds or even seconds.

786
Note that most GDB builds for Windows have broken Python support so that pretty printing won't work. The recommended MinGW distribution should work fine though.
787

788
**Other preparations**
789

790
791
792
793
794
In order to get clang-format onto the system you need to download and install Clang: \
[https://llvm.org/builds](https://llvm.org/builds)

Just run the installer and make sure to select the option _Add LLVM to the system PATH for current user_.

795
Install your editor of choice, I use [VSCode](https://code.visualstudio.com).
796

797
Configure Git. I won't get into the details on how this is done, but there are many resources available online to support with this. The `Git Bash` shell shipped with Git for Windows is very useful though as it's somewhat reproducing a Unix environment using MinGW/MSYS2.
798

799
It's strongly recommended to set line breaks to Unix-style (line feed only) directly in the editor. But if not done, lines breaks will anyway be converted when running clang-format on the code, as explained [here](INSTALL.md#using-clang-format-for-automatic-code-formatting).
800

801
In the descriptions below it's assumed that all build steps for MinGW/GCC will be done in the Git Bash shell, and all the build steps for MSVC will be done in the MSVC developer console (x64 Native Tools Command Prompt for VS).
802

803

804
**Download the dependency packages**
805

806
807
808
FFmpeg (choose the n4.4 package with win64-gpl-shared in the filename, the snapshot version will not work) \
[https://github.com/BtbN/FFmpeg-Builds/releases](https://github.com/BtbN/FFmpeg-Builds/releases)

809
FreeImage (binary distribution) \
810
811
[https://sourceforge.net/projects/freeimage](https://sourceforge.net/projects/freeimage)

Leon Styhre's avatar
Leon Styhre committed
812
curl (Windows 64 bit binary, select the MinGW version even if using MSVC) \
813
[https://curl.haxx.se/download.html](https://curl.haxx.se/download.html)
814

815
SDL2 (development libraries, MinGW or VC/MSVC) \
816
[https://www.libsdl.org/download-2.0.php](https://www.libsdl.org/download-2.0.php)
817

818
libVLC (both win64 binary and source distributions)  - optional, if building with the VLC video player\
819
[https://ftp.lysator.liu.se/pub/videolan/vlc](https://ftp.lysator.liu.se/pub/videolan/vlc)
820

821
Uncompress the files from the above packages to a suitable directory, for example `C:\Programming\Dependencies`
Leon Styhre's avatar
Leon Styhre committed
822

823
Append `_src` or something appropriate to the VLC source distribution directory as it has the same name as the binary distribution.
824

825
826
GLEW\
[http://glew.sourceforge.net](http://glew.sourceforge.net)
827

828
If using MinGW, this library needs to be compiled from source as the pre-built libraries don't seem to work with GCC. The GitHub repo seems to be somewhat broken as well, therefore the manual download is required. It's recommended to get the source in zip format and uncompress it to the same directory as the other libraries listed above.
829

Leon Styhre's avatar
Leon Styhre committed
830
Then simply build the required glew32.dll library:
831
832
833
834
835
836

```
unzip glew-2.1.0.zip
cd glew-2.1.0
make
```
Leon Styhre's avatar
Leon Styhre committed
837
You will probably see a huge amount of compiler warnings, and the glewinfo.exe tool may fail to build, but we don't need it so it's not an issue.
838

839
840
841
If using MSVC, simply download the binary distribution of GLEW.

The following packages are not readily available for Windows, so clone the repos and build them yourself:
842
843
844
845

[FreeType](https://www.freetype.org)
```
git clone git://git.savannah.gnu.org/freetype/freetype2.git
846
cd freetype2
Leon Styhre's avatar
Leon Styhre committed
847
git checkout VER-2-10-4
848
mkdir build
849
cd build
850
851
852
853
854
855
856
857
858
859
```

MSVC:
```
cmake -G "NMake Makefiles" -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=ON ..
nmake
```

MinGW:
```
860
861
862
863
864
865
cmake -G "MinGW Makefiles" -DBUILD_SHARED_LIBS=ON ..
make
```

[pugixml](https://pugixml.org)
```
Leon Styhre's avatar
Leon Styhre committed
866
git clone https://github.com/zeux/pugixml.git
867
cd pugixml
868
git checkout v1.10
869
870
871
872
873
874
875
876
877
878
879
```

MSVC:

```
cmake -G "NMake Makefiles" -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=ON .
nmake
```

MinGW:
```
880
881
882
883
cmake -G "MinGW Makefiles" -DBUILD_SHARED_LIBS=ON .
make
```

884
**Clone the ES-DE repository**
885

Leon Styhre's avatar
Leon Styhre committed
886
This works the same as on Unix or macOS, just run the following:
887
888

```
889
git clone https://gitlab.com/es-de/emulationstation-de.git
890
891
```

892
By default the master branch will be used, which is where development takes place. To instead build a stable release, switch to the `stable-x.x` branch for the version, for example:
Leon Styhre's avatar
Leon Styhre committed
893
894
895

```
cd emulationstation-de
896
git checkout stable-1.2
Leon Styhre's avatar
Leon Styhre committed
897
898
```

899
**Setup the include directories**
900

901
As there is no standardized include directory structure in Windows, you need to provide the include files manually.
902

903
Make a directory in your build environment tree, for instance under `C:\Programming\include`
904

Leon Styhre's avatar
Leon Styhre committed
905
Copy the include files for curl, FFmpeg, FreeImage, FreeType, GLEW, pugixml, SDL2 and optionally VLC to this directory.
906
907
908

You may need to create the SDL2 directory manually and copy the header files there.

909
910
911
For FFmpeg, copy the directories libavcodec, libavfilter, libavformat and libavutil.

It should look something like this:
912
913
914
915
916
917
918

```
$ ls -1 include/
curl/
FreeImage.h
freetype/
ft2build.h
919
GL/
920
921
922
923
libavcodec/
libavfilter/
libavformat/
libavutil/
924
925
926
pugiconfig.hpp
pugixml.hpp
SDL2/
927
vlc/            (only if building with the VLC video player)
928
929
```

930
**Copy the required library files to the ES-DE build directory**
931
932
933

As there's no package manager in Windows and no way to handle dependencies, we need to ship all the required shared libraries with the application.

934
Copy the files to the `emulationstation-de` build directory. Most of them will come from the packages that were provided in the previous steps of this guide.
935

Leon Styhre's avatar
Leon Styhre committed
936
**Required files for MSVC:**