diff --git a/articles/c.org b/articles/c.org index 1471bb448d74c3d643d76f0c61fe92fbeb191f87..07c18d290cb60889b49c3947528e5423df4b3cec 100644 --- a/articles/c.org +++ b/articles/c.org @@ -1,4 +1,4 @@ -% C: The Dark Corners +#+TITLE: C: The Dark Corners C was designed with simplicity in mind. Despite this, C has a lot of dark corners that are not necessarily well known. Here follows an incomplete @@ -7,418 +7,522 @@ collection of them. For historical reasons, we also include some C++ intricacies that may be source of confusion when writing C programs. +* Constancy + :PROPERTIES: + :CUSTOM_ID: constancy + :END: -# Constancy +** Constant definition + :PROPERTIES: + :CUSTOM_ID: constant-definition + :END: -## Constant definition - -The const key word always applies to _the identifier to the left_, when any, +The =const= key word always applies to /the identifier to the left/, when any, or to the right otherwise. Both following lines declare a pointer over a constant integer. - const int * pi; - int const * pi; +#+BEGIN_EXAMPLE + const int * pi; + int const * pi; +#+END_EXAMPLE The following, however, declares a constant pointer over an integer. - int * const pi; +#+BEGIN_EXAMPLE + int * const pi; +#+END_EXAMPLE Pay special attention when declaring pointers to arrays because of the operator precedence. Here we have an array of 12 pointers to constant integers. - const int *pi[12]; +#+BEGIN_EXAMPLE + const int *pi[12]; +#+END_EXAMPLE The next one is a pointer to an array of 12 constant integers. - const int (*pi)[12]; +#+BEGIN_EXAMPLE + const int (*pi)[12]; +#+END_EXAMPLE It is always possible to make something constant, but the opposite is not true. -In C++, it is possible to add the const key word next to a method prototype to +In C++, it is possible to add the =const= key word next to a method prototype to specify that it will not modify the attributes. - -## Constant pointers +** Constant pointers + :PROPERTIES: + :CUSTOM_ID: constant-pointers + :END: The following is forbidden: - char *pc; - const char **ppc; +#+BEGIN_EXAMPLE + char *pc; + const char **ppc; - ppc = &pc; // Forbidden! + ppc = &pc; // Forbidden! +#+END_EXAMPLE -This would break the constancy rule, since it would be possible to change **ppc -value through *pc. +This would break the constancy rule, since it would be possible to change +=**ppc= value through =*pc=. Suppose it would not be forbidden: - const char c = 'a'; // Constant variable. - char *pc; // Pointer through which we will change c. - const char **ppc = &pc; // Forbidden, but assume it is not. +#+BEGIN_EXAMPLE + const char c = 'a'; // Constant variable. + char *pc; // Pointer through which we will change c. + const char **ppc = &pc; // Forbidden, but assume it is not. - *ppc = &c; // Legal. - *pc = 'b'; // Change c. + *ppc = &c; // Legal. + *pc = 'b'; // Change c. +#+END_EXAMPLE -So ppc goes through pc to c. Since pc is not a pointer to a constant, we -can change the value, thus ppc constancy is broken. +So =ppc= goes through =pc= to =c=. Since =pc= is not a pointer to a constant, we +can change the value, thus =ppc= constancy is broken. - -## C/C++ difference for const +** C/C++ difference for =const= + :PROPERTIES: + :CUSTOM_ID: cc-difference-for-const + :END: In C, the following - const int a = 10; - int *p = &a; - *p = 30; +#+BEGIN_EXAMPLE + const int a = 10; + int *p = &a; + *p = 30; - printf("&a: %u, a: %d\n", &a, a); - printf("&p: %u, p: %d\n", p, *p); - return 0; + printf("&a: %u, a: %d\n", &a, a); + printf("&p: %u, p: %d\n", p, *p); + return 0; +#+END_EXAMPLE outputs as expected - &a: 1021510500, a: 30 - &p: 1021510500, p: 30 +#+BEGIN_EXAMPLE + &a: 1021510500, a: 30 + &p: 1021510500, p: 30 +#+END_EXAMPLE -But in C++, the previous code won't be allowed since the const keyword is more +But in C++, the previous code won't be allowed since the =const= keyword is more restrictive. There is a workaround though: - const int a = 10; - int *p = (int*)(&a); - *p = 30; +#+BEGIN_EXAMPLE + const int a = 10; + int *p = (int*)(&a); + *p = 30; - printf("&a: %u, a: %d\n", &a, a); - printf("&p: %u, p: %d\n", p, *p); + printf("&a: %u, a: %d\n", &a, a); + printf("&p: %u, p: %d\n", p, *p); +#+END_EXAMPLE but the output will be: - &a: 1021510500, a: 10 - &p: 1021510500, p: 30 +#+BEGIN_EXAMPLE + &a: 1021510500, a: 10 + &p: 1021510500, p: 30 +#+END_EXAMPLE Yes, that is the same address and two different values! -This is because C++ handles const as an immediate value, not a variable. It -behaves similarly to #define. The address of a const, albeit grammatically +This is because C++ handles =const= as an immediate value, not a variable. It +behaves similarly to =#define=. The address of a =const=, albeit grammatically defined, is rather meaningless. +** Constants as static array initializers + :PROPERTIES: + :CUSTOM_ID: constants-as-static-array-initializers + :END: -## Constants as static array initializers - -Semantically speaking, the const keyword refers to _immutable variables_ and -not _constant variables_, which is an interesting oxymoron. +Semantically speaking, the =const= keyword refers to /immutable variables/ and +not /constant variables/, which is an interesting oxymoron. -As such, const variables should not be used to initialize static arrays of -some size, since the standard requires a semantic constant here, i.e. an integer +As such, =const= variables should not be used to initialize static arrays of +some size, since the standard requires a semantic constant here, i.e. an integer or a preprocessor expression that expands to an integer. - int array1[17]; - const unsigned int sz = sizeof array1; - int array2[sizeof array1]; // OK - int array3[sz]; // Wrong +#+BEGIN_EXAMPLE + int array1[17]; + const unsigned int sz = sizeof array1; + int array2[sizeof array1]; // OK + int array3[sz]; // Wrong +#+END_EXAMPLE -In practice, most compilers accept const variables in that case. +In practice, most compilers accept =const= variables in that case. +* Function argument evaluation order + :PROPERTIES: + :CUSTOM_ID: function-argument-evaluation-order + :END: -# Function argument evaluation order +From /The C Programming Language/: -From _The C Programming Language_: - -> The order in which function arguments are evaluated is unspecified, so the -> statement -> printf("%d %d\n", ++n, power(2, n)); -> can produce different results with different compilers, depending on whether n -> is incremented before power is called. +#+BEGIN_QUOTE + The order in which function arguments are evaluated is unspecified, so the + statement printf("%d %d\n", ++n, power(2, n)); can produce different results + with different compilers, depending on whether n is incremented before power + is called. +#+END_QUOTE Thus it is good practice to avoid expressions in function calls. +* Arrays + :PROPERTIES: + :CUSTOM_ID: arrays + :END: -# Arrays - -Arrays are _not_ pointers! There is a small number of cases when they behave +Arrays are /not/ pointers! There is a small number of cases when they behave differently. The following test is true: - array[0] == *array +#+BEGIN_EXAMPLE + array[0] == *array +#+END_EXAMPLE -From the [C standard][]: +From the [[http://www.open-std.org/JTC1/SC22/WG14/www/docs/n1570.pdf][C +standard]]: -> Except when it is the operand of the sizeof operator, the _Alignof operator, -> or the unary & operator, or is a string literal used to initialize an array, -> an expression that has type "array of type" is converted to an expression with -> type "pointer to type" that points to the initial element of the array object -> and is not an lvalue. If the array object has register storage class, the -> behavior is undefined. +#+BEGIN_QUOTE + Except when it is the operand of the =sizeof= operator, the \_Alignof + operator, or the unary & operator, or is a string literal used to initialize + an array, an expression that has type "array of type" is converted to an + expression with type "pointer to type" that points to the initial element of + the array object and is not an lvalue. If the array object has register + storage class, the behavior is undefined. +#+END_QUOTE +** Using =sizeof= + :PROPERTIES: + :CUSTOM_ID: using-sizeof + :END: -## Using sizeof +The =sizeof= operator is dynamic and follows its own set of rules as described +by the standard. When the argument is an array, it will return the total number +of bytes. -The sizeof operator is dynamic and follows its own set of rules as -described by the standard. When the argument is an array, it will return the -total number of bytes. +#+BEGIN_EXAMPLE + long array[3]; + long *p = array; + printf("%zu\n", sizeof(array)); + printf("%zu\n", sizeof(p)); +#+END_EXAMPLE - long array[3]; - long *p = array; - printf("%zu\n", sizeof(array)); - printf("%zu\n", sizeof(p)); +On machines where =long= is 8 bytes and pointers are 4 bytes, this will output: -On machines where long is 8 bytes and pointers are 4 bytes, this will output: +#+BEGIN_EXAMPLE + 24 + 4 +#+END_EXAMPLE - 24 - 4 +Arrays are automatically converted to pointers in function arguments. Thus the +behavior of =sizeof= is special only within the scope of an array declaration. -Arrays are automatically converted to pointers in function arguments. -Thus the behavior of sizeof is special only within the scope of an array -declaration. +#+BEGIN_EXAMPLE + void foo(int array[]) { + printf("foo: sizeof array == %zu\n", sizeof array); + } - void foo(int array[]) { - printf("foo: sizeof array == %zu\n", sizeof array); - } + void bar(int array[12]) { + printf("bar: sizeof array == %zu\n", sizeof array); + } - void bar(int array[12]) { - printf("bar: sizeof array == %zu\n", sizeof array); - } + int main() { + int array[10]; - int main() { - int array[10]; + printf("main: sizeof array == %zu\n", sizeof array); + foo(array); + bar(array); - printf("main: sizeof array == %zu\n", sizeof array); - foo(array); - bar(array); - - return 0; - } + return 0; + } +#+END_EXAMPLE For multidimensional arrays, only the outermost dimension is converted to a -pointers. For instance, int array[M][N] will be cast to int (*)[N]. -The following will output the size of a pointer. - - void foo(int *array[3]) { - printf("foo: sizeof array == %zu\n", sizeof array); - } - - int main() { - int arr[2][3] = {{10, 20, 30}, {40, 50, 60}}; - foo(arr); - return 0; - } - - -## Addressing arrays +pointers. For instance, =int array[M][N]= will be cast to =int (*)[N]=. The +following will output the size of a pointer. + +#+BEGIN_EXAMPLE + void foo(int *array[3]) { + printf("foo: sizeof array == %zu\n", sizeof array); + } + + int main() { + int arr[2][3] = {{10, 20, 30}, {40, 50, 60}}; + foo(arr); + return 0; + } +#+END_EXAMPLE + +** Addressing arrays + :PROPERTIES: + :CUSTOM_ID: addressing-arrays + :END: Arrays have a type signature that differs from pointers. The signature of a -pointer to an n-array of T is T (*)[n]. +pointer to an n-array of T is =T (*)[n]=. - long array[3]; - long *p; - long **pp; - long (*ap)[3]; +#+BEGIN_EXAMPLE + long array[3]; + long *p; + long **pp; + long (*ap)[3]; - p = &array; // Wrong - pp = &array; // Wrong - ap = &array; // OK + p = &array; // Wrong + pp = &array; // Wrong + ap = &array; // OK +#+END_EXAMPLE -Note that the warning about type comes from the dereferences (&), since the +Note that the warning about type comes from the dereferences (=&=), since the following code does not prompt any warning: - long array[3]; - long *p; - long (*ap)[3]; - - p = array; // OK this time - ap = &array; // OK +#+BEGIN_EXAMPLE + long array[3]; + long *p; + long (*ap)[3]; + p = array; // OK this time + ap = &array; // OK +#+END_EXAMPLE Conversely, an pointer cannot be assigned to an array: - long array[3]; - long *p; - array = p; // Wrong +#+BEGIN_EXAMPLE + long array[3]; + long *p; + array = p; // Wrong +#+END_EXAMPLE - -## Arrays as strings +** Arrays as strings + :PROPERTIES: + :CUSTOM_ID: arrays-as-strings + :END: Arrays can only be initialized with semantic constants. - char *p = "hello"; - char t0[] = "world"; - char t1[] = {'f', 'o', 'o'}; - char t2[] = p; // Error. - char t3[] = (char*) "foo"; // Error. - -There is another _major_ difference in the initialization of pointers -against arrays. The pointer will only set its value to the address of hello -stored in the static memory segment of the program, whereas the array will copy -world from this same segment to its allocated memory. The array can be -modified afterwards, unlike the underlying value of the pointer. - - -# Implicit cast - -Numbers are automatically upcast in function calls. -Compare - - unsigned char a = 255; - a++; - printf("%d\n", a); +#+BEGIN_EXAMPLE + char *p = "hello"; + char t0[] = "world"; + char t1[] = {'f', 'o', 'o'}; + char t2[] = p; // Error. + char t3[] = (char*) "foo"; // Error. +#+END_EXAMPLE + +There is another /major/ difference in the initialization of pointers against +arrays. The pointer will only set its value to the address of =hello= stored in +the static memory segment of the program, whereas the array will copy =world= +from this same segment to its allocated memory. The array can be modified +afterwards, unlike the underlying value of the pointer. + +* Implicit cast + :PROPERTIES: + :CUSTOM_ID: implicit-cast + :END: + +Numbers are automatically upcast in function calls. Compare + +#+BEGIN_EXAMPLE + unsigned char a = 255; + a++; + printf("%d\n", a); +#+END_EXAMPLE and - unsigned char a = 255; - printf("%d\n", a+1); +#+BEGIN_EXAMPLE + unsigned char a = 255; + printf("%d\n", a+1); +#+END_EXAMPLE -There is no loss of information during an upcast, except for the char type. C -does not specify whether a char should be signed. Thus signed or unsigned +There is no loss of information during an upcast, except for the =char= type. C +does not specify whether a =char= should be signed. Thus =signed= or =unsigned= should be used to ensure portability. -From _The C Programming Language_, section 2.7: +From /The C Programming Language/, section 2.7: -> Conversion rules are more complicated when unsigned operands are involved. The -> problem is that comparisons between signed and unsigned values are -> machine-dependent, because they depend on the sizes of the various integer -> types. For example, suppose that int is 16 bits and long is 32 bits. Then -1L -> < 1U, because 1U, which is an int, is promoted to a signed long. But -1L > -> 1UL, because -1L is promoted to unsigned long and thus appears to be a large -> positive number. +#+BEGIN_QUOTE + Conversion rules are more complicated when unsigned operands are involved. The + problem is that comparisons between signed and unsigned values are + machine-dependent, because they depend on the sizes of the various integer + types. For example, suppose that int is 16 bits and long is 32 bits. Then -1L + < 1U, because 1U, which is an int, is promoted to a signed long. But -1L > + 1UL, because -1L is promoted to unsigned long and thus appears to be a large + positive number. +#+END_QUOTE See appendix A6 in the book for more implicit conversion rules. +* Bit shifting + :PROPERTIES: + :CUSTOM_ID: bit-shifting + :END: -# Bit shifting - -Be wary of the difference between a _logical shift_ and an _arithmetic shift_. -See [this Wikipedia article][w arithshift] for more details. -Note that it only matters for right shifting. +Be wary of the difference between a /logical shift/ and an /arithmetic shift/. +See [[https://en.wikipedia.org/wiki/Arithmetic_shift][this Wikipedia article]] +for more details. Note that it only matters for right shifting. The C behaviour is architecture-dependent for signed numbers. - -# Modulo operation +* Modulo operation + :PROPERTIES: + :CUSTOM_ID: modulo-operation + :END: In C99, the result of a modulo operation has the sign of the dividend: - printf("-5 % 2 = %d\n", -5 % 2); - printf("5 % -2 = %d\n", 5 % -2); +#+BEGIN_EXAMPLE + printf("-5 % 2 = %d\n", -5 % 2); + printf("5 % -2 = %d\n", 5 % -2); +#+END_EXAMPLE To test whether an integer is odd, you must compare to 0, not 1. Otherwise, the result will be incorrect when the dividend is negative. - if (n % 2 == 1) // WRONG! - if (n % 2 != 0) // Correct. +#+BEGIN_EXAMPLE + if (n % 2 == 1) // WRONG! + if (n % 2 != 0) // Correct. +#+END_EXAMPLE -# Operator precedence +* Operator precedence + :PROPERTIES: + :CUSTOM_ID: operator-precedence + :END: The choice for operator precedence in C can be counter-intuitive at times. The -expression a & b == 7 is parsed as a & (b == 7). - -See [this Wikipedia article][w opprec] for more details. +expression =a & b == 7= is parsed as =a & (b == 7)=. +See +[[https://en.wikipedia.org/wiki/Operators_in_C_and_C%2B%2B#Operator_precedence][this +Wikipedia article]] for more details. -# File reading +* File reading + :PROPERTIES: + :CUSTOM_ID: file-reading + :END: -When a text file is open in text-mode, (e.g. using the "r" option), POSIX -specifies that the "b" option is ignored. Some non-POSIX operating systems, +When a text file is open in text-mode, (e.g. using the ="r"= option), POSIX +specifies that the ="b"= option is ignored. Some non-POSIX operating systems, however, may try to be too smart. They will expect a "standard" end-of-line, -such as \r\n. Which will obviously produce unexpected results on files with -"\n" line breaks. The "b" option does not harm and helps for portability. +such as =\r\n=. Which will obviously produce unexpected results on files with +="\n"= line breaks. The ="b"= option does not harm and helps for portability. - -# Globals +* Globals + :PROPERTIES: + :CUSTOM_ID: globals + :END: Pre-declarations can appear any number of times in C. They can appear only once in C++, or the compiler will complain about double definitions of globals: - #include +#+BEGIN_EXAMPLE + #include - int global; - int global; - int global = 3; + int global; + int global; + int global = 3; - void change() { - global = 17; - } + void change() { + global = 17; + } - int main() { - printf("%d\n", global); - change(); - printf("%d\n", global); - return 0; - } + int main() { + printf("%d\n", global); + change(); + printf("%d\n", global); + return 0; + } +#+END_EXAMPLE In C, it will display the following: - 3 - 17 - +#+BEGIN_EXAMPLE + 3 + 17 +#+END_EXAMPLE -# Pointer arithmetic +* Pointer arithmetic + :PROPERTIES: + :CUSTOM_ID: pointer-arithmetic + :END: It is not safe to assume that pointer arithmetic results in any integral type. Some architectures may have memory addresses indexed over 64-bit values, while -using data over 32 bits. This behavior can be controlled from stdlib.h. For -example, a pointer difference is stored as a type ptrdiff_t. +using data over 32 bits. This behavior can be controlled from =stdlib.h=. For +example, a pointer difference is stored as a type =ptrdiff_t=. +* Size of void + :PROPERTIES: + :CUSTOM_ID: size-of-void + :END: -# Size of void +With GCC, =sizeof(void) == 1= is true. This is non standard, but the behaviour +is not clearly specified either. Using =-pedantic= will output a warning. -With GCC, sizeof(void) == 1 is true. This is non standard, but the behaviour -is not clearly specified either. Using -pedantic will output a -warning. - - -# Alignment +* Alignment + :PROPERTIES: + :CUSTOM_ID: alignment + :END: Do not expect the memory layout in structures to be as the code describes it: the compiler is free to pad some memory for optimization purposes. -This proves dangerous when serializing data. Use the offsetof macro to get the +This proves dangerous when serializing data. Use the =offsetof= macro to get the real offset of each structure member. - struct {char a; int b;} foo; - struct {char a; char b;} bar; +#+BEGIN_EXAMPLE + struct {char a; int b;} foo; + struct {char a; char b;} bar; - printf("sizeof foo == %zu\n", sizeof foo); - printf("&foo == %p\n", &foo); - printf("&foo.a == %p\n", &foo.a); - printf("&foo.b == %p\n", &foo.b); + printf("sizeof foo == %zu\n", sizeof foo); + printf("&foo == %p\n", &foo); + printf("&foo.a == %p\n", &foo.a); + printf("&foo.b == %p\n", &foo.b); - printf("sizeof bar == %zu\n", sizeof bar); - printf("&bar == %p\n", &bar); - printf("&bar.a == %p\n", &bar.a); - printf("&bar.b == %p\n", &bar.b); + printf("sizeof bar == %zu\n", sizeof bar); + printf("&bar == %p\n", &bar); + printf("&bar.a == %p\n", &bar.a); + printf("&bar.b == %p\n", &bar.b); +#+END_EXAMPLE - -# Precompiled headers +* Precompiled headers + :PROPERTIES: + :CUSTOM_ID: precompiled-headers + :END: Compiling a header file may yield an unexpected result: some compilers such as GCC will recognize the extension and act accordingly. In that case, building a -header will not result in an executable, but in a _precompiled header_, that is, +header will not result in an executable, but in a /precompiled header/, that is, an optimization for large headers. If you want to force or prevent the build of precompiled headers, GCC allows for specifying the input language: - # The .xml file will be seen as a C header file. - gcc -x c-header myfile.xml - # The .h file will be compiled into an executable. - gcc -x c myfile.h - +#+BEGIN_EXAMPLE + # The .xml file will be seen as a C header file. + gcc -x c-header myfile.xml + # The .h file will be compiled into an executable. + gcc -x c myfile.h +#+END_EXAMPLE -# Final note +* Final note + :PROPERTIES: + :CUSTOM_ID: final-note + :END: The numerous dark corners of C require some getting used to. It is helpful and good practice to make heavy use of your compiler's warning flags, together with some fine 'lint' tools. - -# References - -* BSD/GNU man pages -* The C Programming Language, D. Ritchie & B. Kernighan -* [Draft of the C standard][C standard] -* [Wikipedia/Arithmetic shift][w arithshift] -* [Wikipedia/Operator precedence][w opprec] -* [Wikipedia/Type conversion](https://en.wikipedia.org/wiki/Type_conversion) - -[w arithshift]: https://en.wikipedia.org/wiki/Arithmetic_shift -[w opprec]: https://en.wikipedia.org/wiki/Operators_in_C_and_C%2B%2B#Operator_precedence -[C standard]: http://www.open-std.org/JTC1/SC22/WG14/www/docs/n1570.pdf +* References + :PROPERTIES: + :CUSTOM_ID: references + :END: + +- BSD/GNU man pages +- The C Programming Language, D. Ritchie & B. Kernighan +- [[http://www.open-std.org/JTC1/SC22/WG14/www/docs/n1570.pdf][Draft of the C + standard]] +- [[https://en.wikipedia.org/wiki/Arithmetic_shift][Wikipedia/Arithmetic shift]] +- [[https://en.wikipedia.org/wiki/Operators_in_C_and_C%2B%2B#Operator_precedence][Wikipedia/Operator + precedence]] +- [[https://en.wikipedia.org/wiki/Type_conversion][Wikipedia/Type conversion]] diff --git a/articles/cinema.org b/articles/cinema.org index fdf91011ecb064712185bf10a536fd1f9091d91e..ff4ad30edef3f26668a8294d7532f279560eb07e 100644 --- a/articles/cinema.org +++ b/articles/cinema.org @@ -1,128 +1,146 @@ -% Cinema Frivolities - -An insight of Hollywood: *Writing movies for fun and profit* (Thomas Lennon & Robert Ben Garant, 2012) - -# Some uncanny movies - -* 12 Angry Men -* 3 Idiots -* 50/50 -* A Clockwork Orange -* A History of Violence -* A.I. Artificial Intelligence -* Abre los ojos (Open Your Eyes) -* American Beauty -* Analyze This -* Atonement -* Before Midnight -* Before Sunrise -* Before Sunset -* Boyhood -* Bronson -* Buried -* Captain Fantastic -* Carnage -* Casablanca -* Catch Me If You Can -* Children of Men -* Chinjeolhan geumjassi (Lady Vengeance) -* Chugyeogja (The Chaser) -* Clerks -* Clerks II -* Cloud Atlas -* Confessions of a Dangerous Mind -* Das Experiment (The Experiment) -* Dev D -* District 9 -* Don Jon -* Donnie Darko -* Eastern Promises -* Ed Wood -* El laberinto del fauno (Pan's Labyrinth) -* Election -* Eternal Sunshine of the Spotless Mind -* Everything Is Illuminated -* Eyes wide shut -* Falling Down -* Filth -* Gangs of Wasseypur -* Goksung (The Wailing) -* Gongdong gyeongbi guyeok JSA (J.S.A.: Joint Security Area) -* Groundhog Day -* Hunt for the Wilderpeople -* Il buono, il brutto, il cattivo (The Good, the Bad and the Ugly) -* In Bruges -* Jisatsu sâkuru (Suicide Club) and one interesting [decryption](suicideclub.html) -* Joheunnom nabbeunnom isanghannom (The Good, the Bad, and the Weird) -* L.A. Confidential -* Little Miss Sunshine -* Lord of War -* Madeo (Mother) -* Magnolia -* Minority Report -* Mr. Nobody -* Mullholland Dr. -* My Cousin Vinny -* Nightcrawler -* O Brother, Where Art Thou? -* Oldeuboi -* Only God Forgives -* Perfume: The Story of a Murderer -* Revolutionary Road -* Safety Not Guaranteed -* Shallow Grave -* Smokin' Aces -* Stoker -* Thank You for Smoking -* The Act of Killing -* The Ambassador -* The Breakfast Club -* The Imaginarium of Doctor Parnassus -* The Killer Inside Me -* The Man from Earth -* Three Kings -* True Romance -* Twelve Monkeys -* Up in the Air -* We Need to Talk About Kevin -* Wo hu cang long (Crouching Tiger, Hidden Dragon) -* World's Greatest Dad -* Youth - -# French age certifications +#+TITLE: Cinema Frivolities + +An insight of Hollywood: /Writing movies for fun and profit/ (Thomas Lennon & +Robert Ben Garant, 2012) + +* Some uncanny movies + :PROPERTIES: + :CUSTOM_ID: some-uncanny-movies + :END: + +- 12 Angry Men +- 3 Idiots +- 50/50 +- A Clockwork Orange +- A History of Violence +- A.I. Artificial Intelligence +- Abre los ojos (Open Your Eyes) +- American Beauty +- Analyze This +- Atonement +- Before Midnight +- Before Sunrise +- Before Sunset +- Boyhood +- Bronson +- Buried +- Captain Fantastic +- Carnage +- Casablanca +- Catch Me If You Can +- Children of Men +- Chinjeolhan geumjassi (Lady Vengeance) +- Chugyeogja (The Chaser) +- Clerks +- Clerks II +- Cloud Atlas +- Confessions of a Dangerous Mind +- Das Experiment (The Experiment) +- Dev D +- District 9 +- Don Jon +- Donnie Darko +- Eastern Promises +- Ed Wood +- El laberinto del fauno (Pan's Labyrinth) +- Election +- Eternal Sunshine of the Spotless Mind +- Everything Is Illuminated +- Eyes wide shut +- Falling Down +- Filth +- Gangs of Wasseypur +- Goksung (The Wailing) +- Gongdong gyeongbi guyeok JSA (J.S.A.: Joint Security Area) +- Groundhog Day +- Hunt for the Wilderpeople +- Il buono, il brutto, il cattivo (The Good, the Bad and the Ugly) +- In Bruges +- Jisatsu sâkuru (Suicide Club) and one interesting + [[file:suicideclub.html][decryption]] +- Joheunnom nabbeunnom isanghannom (The Good, the Bad, and the Weird) +- L.A. Confidential +- Little Miss Sunshine +- Lord of War +- Madeo (Mother) +- Magnolia +- Minority Report +- Mr. Nobody +- Mullholland Dr. +- My Cousin Vinny +- Nightcrawler +- O Brother, Where Art Thou? +- Oldeuboi +- Only God Forgives +- Perfume: The Story of a Murderer +- Revolutionary Road +- Safety Not Guaranteed +- Shallow Grave +- Smokin' Aces +- Stoker +- Thank You for Smoking +- The Act of Killing +- The Ambassador +- The Breakfast Club +- The Imaginarium of Doctor Parnassus +- The Killer Inside Me +- The Man from Earth +- Three Kings +- True Romance +- Twelve Monkeys +- Up in the Air +- We Need to Talk About Kevin +- Wo hu cang long (Crouching Tiger, Hidden Dragon) +- World's Greatest Dad +- Youth + +* French age certifications + :PROPERTIES: + :CUSTOM_ID: french-age-certifications + :END: (U = no restriction) -* 127 Hours: U -* 300: 12 -* A History of Violence: 12 -* Chugyeogja (The Chaser): 12 (with warning) -* Drive: 12 -* Eastern Promises: 12 -* El laberinto del fauno (Pan's Labyrinth): 12 -* Eyes Wide Shut: U -* Kick-Ass: U -* Killer Joe: 12 -* Waltz with Bashir: U -* Watchmen: 12 - -# Computers in movies - -## Jurassic Park +- 127 Hours: U +- 300: 12 +- A History of Violence: 12 +- Chugyeogja (The Chaser): 12 (with warning) +- Drive: 12 +- Eastern Promises: 12 +- El laberinto del fauno (Pan's Labyrinth): 12 +- Eyes Wide Shut: U +- Kick-Ass: U +- Killer Joe: 12 +- Waltz with Bashir: U +- Watchmen: 12 + +* Computers in movies + :PROPERTIES: + :CUSTOM_ID: computers-in-movies + :END: + +** Jurassic Park + :PROPERTIES: + :CUSTOM_ID: jurassic-park + :END: When Lex tries to take control of one of the computers in the command centre, we get to contemplate how she skillfully navigates the system using a (laggy) 3D file browser. Surprisingly, this is not just an eccentricity of some movie -writer, the program, called fsn, actually existed by the time the movie was -written. See [Wikipedia/Fsn](https://en.wikipedia.org/wiki/Fsn). +writer, the program, called =fsn=, actually existed by the time the movie was +written. See [[https://en.wikipedia.org/wiki/Fsn][Wikipedia/Fsn]]. -An open source and "modern" clone called [fsv](http://fsv.sourceforge.net/) has -been developed, if you'd feel like playing Lex with your computer. +An open source and "modern" clone called [[http://fsv.sourceforge.net/][fsv]] +has been developed, if you'd feel like playing Lex with your computer. -> "It's a UNIX system! I know this!" -> -- Lex Murphy +#+BEGIN_QUOTE + "It's a UNIX system! I know this!" -- Lex Murphy +#+END_QUOTE -## The Social Network +** The Social Network + :PROPERTIES: + :CUSTOM_ID: the-social-network + :END: There is a famous urban legend around M. Zuckerberg who purportedly "hacked" its campus network while being intoxicated. The myth of the genius struck us one @@ -130,17 +148,20 @@ more time. While some dark arcane of hackerism spread through the collective mind, the movie shows a much more realistic version of the petty offense: - His computer runs KDE3, a Unix desktop environment popular around the events -depicted in the movie. -- He uses some "wget magic". wget is a popular command-line network -downloader. It is ideal to write scripts for batch downloads. -- He subsequently writes some scripts to automate the queries (e.g. when limited -to 20 pictures per page). + depicted in the movie. +- He uses some "wget magic". =wget= is a popular command-line network + downloader. It is ideal to write scripts for batch downloads. +- He subsequently writes some scripts to automate the queries (e.g. when limited + to 20 pictures per page). - He fires up Emacs, and modifies some Perl script. -Despite the lack of details, all the steps happen to be mundane tasks -related to automated network queries. +Despite the lack of details, all the steps happen to be mundane tasks related to +automated network queries. -## Tron: Legacy +** Tron: Legacy + :PROPERTIES: + :CUSTOM_ID: tron-legacy + :END: The movie shows a (surprisingly) witty use of computers. @@ -149,42 +170,50 @@ we get a short glimpse at one terminal running an Eshell session, the Elisp shell of Emacs. Most certainly the top of the world in Geekland... One subliminal frame in the same scene shows a session of Tetris run from Emacs, -while the command hanoi-unix was just entered... +while the command =hanoi-unix= was just entered... When the main character first discovers the secret room with the terminal, his -first reflex is to type the following _meaningful_ commands: +first reflex is to type the following /meaningful/ commands: - $whoami - flynn +#+BEGIN_EXAMPLE +$ whoami + flynn +#+END_EXAMPLE -If you'd be given a computer that has kept running for years, you'd like to -know who was using it last. This is what he is checking. +If you'd be given a computer that has kept running for years, you'd like to know +who was using it last. This is what he is checking. - $uname -a - SolarOS 4.0.1 Generic_50203-02 sun4m i386 - Unknown.Unknown +#+BEGIN_EXAMPLE +$ uname -a + SolarOS 4.0.1 Generic_50203-02 sun4m i386 + Unknown.Unknown +#+END_EXAMPLE -uname is a universal command for showing the type of system the machine is +=uname= is a universal command for showing the type of system the machine is running. It appropriately shows the OS name (a cross-over between SunOS and Solaris?), the version, the kernel (Generic...), the machine hardware name (i386, a popular architecture by Intel), and possibly the "Unknown" hostname, which unveils the latent mystery around the machine! - $login -n root - Login incorrect - login: backdoor - No home directory specified in password file! - Logging in with home=/ +#+BEGIN_EXAMPLE +$ login -n root + Login incorrect + login: backdoor + No home directory specified in password file! + Logging in with home=/ +#+END_EXAMPLE -Now this gets a little more creative: login with the backdoor name is a very naive -method to hack your way toward full-privileged access to a computer! +Now this gets a little more creative: login with the =backdoor= name is a very +naive method to hack your way toward full-privileged access to a computer! - # bin/history - ... +#+BEGIN_EXAMPLE + # bin/history + ... +#+END_EXAMPLE -history is usually a command embedded in the shell, but nevermind... It would -have been much more sensible to run the history command when logged as -flynn, and not as backdoor. +=history= is usually a command embedded in the shell, but nevermind... It would +have been much more sensible to run the =history= command when logged as +=flynn=, and not as =backdoor=. It shows the list of the last commands that were typed. Again, it totally makes sense to do so if you want to investigate how a computer was used the last time. @@ -194,8 +223,10 @@ program, the editing of the last will, some system status checks and finally the execution of the Grid simulator (last command), which projects our hero to a virtual world. +** Nmap vs. Hollywood + :PROPERTIES: + :CUSTOM_ID: nmap-vs.hollywood + :END: -## Nmap vs. Hollywood - -See [official website](https://nmap.org/movies/) for an impressive list of +See [[https://nmap.org/movies/][official website]] for an impressive list of movies in which Nmap gets to be a rock star! diff --git a/articles/cinema/suicideclub.org b/articles/cinema/suicideclub.org index 873294b0384600544fff0a3de78fe6225d33fe7c..deb746c9229ac57b4ad703a91cb6b45c090af2d1 100644 --- a/articles/cinema/suicideclub.org +++ b/articles/cinema/suicideclub.org @@ -1,4 +1,4 @@ -% Cinema: Suicide Club +#+TITLE: Cinema: Suicide Club By Manuel-Hoerth. @@ -8,25 +8,90 @@ Post Edited: Fri Oct 14 2011 09:00:58. Originally from http://www.imdb.com/title/tt0312843/board/nest/189698427. +* For those who after watching it couldn't figure it out + :PROPERTIES: + :CUSTOM_ID: for-those-who-after-watching-it-couldnt-figure-it-out + :END: -# For those who after watching it couldn't figure it out +So you watched the movie and where like me and thought "What the heck"? I have +to admit i haven't figured out everything either, as i am too lazy to watch the +movie multiple times myself - if i where to do that i am sure i could see even +more things. But anyway i think i figured the main thing out that everyone was +wondering about, namely the answer to the "what the heck is going on" -question. -So you watched the movie and where like me and thought "What the heck"? -I have to admit i haven't figured out everything either, as i am too lazy to watch the movie multiple times myself - if i where to do that i am sure i could see even more things. -But anyway i think i figured the main thing out that everyone was wondering about, namely the answer to the "what the heck is going on" -question. +Strangely nobody has actually posted it so far - instead the posts i read on +here so far are actually pretty misleading like saying "it's a Japanese thing +you can't understand it from a western perspective" or "it's a Buddhist thing" +or whatever... that's BS. And actually that's precisely the reason why nobody +has figured it out cause we are all expecting some esoteric eastern +explanation... and so we start to think much too complicated and we go offtrack. +Sure it's a Japanese movie, but have you noticed that the Japanese are actually +even bigger fans of western culture than we are of eastern culture? Why else +would they try to speak English all the time or have references to the Rocky +Horror Picture Show? -Strangely nobody has actually posted it so far - instead the posts i read on here so far are actually pretty misleading like saying "it's a Japanese thing you can't understand it from a western perspective" or "it's a Buddhist thing" or whatever... that's BS. -And actually that's precisely the reason why nobody has figured it out cause we are all expecting some esoteric eastern explanation... and so we start to think much too complicated and we go offtrack. Sure it's a Japanese movie, but have you noticed that the Japanese are actually even bigger fans of western culture than we are of eastern culture? Why else would they try to speak English all the time or have references to the Rocky Horror Picture Show? +So here goes the spoiler: Yes, it's a cultural thing... and yes it's true that +only people with a certain cultural background can understand it. But the +cultural background required is not Japanese, it's German! To put it blunt, this +movie is nothing more than a gory, modernized, Japanese version of the Pied +Piper! Think about it. Watch the movie two times and you will notice it... i +watched it only one time, but i am German so naturally i am familiar with the +legend and noticed it on my first watch. The policeman even mentions it in the +beginning. Later on you have that creepy dude who appears to be the manager or +personal child molester of the "Dessert" kids (-: You see him towards the end of +the movie with a black mask on his head, together with all the kids and cutting +their skin away and there are tons of chickens on the floor too that at that +moment actually look and squeak like rats. This creepy guy is the Pied Piper! He +uses the secret code you see in the poster to lure the kids in. And only +children can learn his secret code (simple because only kids would be obsessed +enough with "Dessert" to actually figure the code out). -So here goes the spoiler: Yes, it's a cultural thing... and yes it's true that only people with a certain cultural background can understand it. But the cultural background required is not Japanese, it's German! To put it blunt, this movie is nothing more than a gory, modernized, Japanese version of the Pied Piper! Think about it. Watch the movie two times and you will notice it... i watched it only one time, but i am German so naturally i am familiar with the legend and noticed it on my first watch. The policeman even mentions it in the beginning. Later on you have that creepy dude who appears to be the manager or personal child molester of the "Dessert" kids (-: You see him towards the end of the movie with a black mask on his head, together with all the kids and cutting their skin away and there are tons of chickens on the floor too that at that moment actually look and squeak like rats. This creepy guy is the Pied Piper! He uses the secret code you see in the poster to lure the kids in. And only children can learn his secret code (simple because only kids would be obsessed enough with "Dessert" to actually figure the code out). +Once they have the code and use it on the door to the backstage area they are +basically brainwashed into committing suicide as you can see with the girl at +the end or with the policemen who is made to commit suicide. There are MANY +references to the Pied Piper... probably much more than what i spotted. And just +like in the Pied Piper, he first makes the children follow him using his music +and his magic (in this case J-Pop and his "magic" code) then he leads the +children away, and then he leads them to their death. Even the method of death +is the same. in the Pied Piper he makes them jump down a cave in the hills. In +"Suicide Club" he makes them jump down a school, or some other building or makes +them jump down the subway... anyway they always jump down something and always +collectively, just like in the Pied Piper. Also the subway itself could simply +be a modernized version of the cave in the original German story. Also the +subway is located just outside Tokyo just like the cave is located just outside +Hamelin. Also like in the Pied Piper the adults are to blame for not watching +over their children. In the Pied Piper the adults attend church and leave the +kids unsupervised. In Suicide Club the police (who are also the adults and the +parents) try hard to solve the case. But they don't pay any attention to their +kids and family and it's precisely that lack of attention for their children +that prevents them from actually solving the case and from protecting their +children. -Once they have the code and use it on the door to the backstage area they are basically brainwashed into committing suicide as you can see with the girl at the end or with the policemen who is made to commit suicide. There are MANY references to the Pied Piper... probably much more than what i spotted. And just like in the Pied Piper, he first makes the children follow him using his music and his magic (in this case J-Pop and his "magic" code) then he leads the children away, and then he leads them to their death. Even the method of death is the same. in the Pied Piper he makes them jump down a cave in the hills. In "Suicide Club" he makes them jump down a school, or some other building or makes them jump down the subway... anyway they always jump down something and always collectively, just like in the Pied Piper. Also the subway itself could simply be a modernized version of the cave in the original German story. Also the subway is located just outside Tokyo just like the cave is located just outside Hamelin. -Also like in the Pied Piper the adults are to blame for not watching over their children. In the Pied Piper the adults attend church and leave the kids unsupervised. In Suicide Club the police (who are also the adults and the parents) try hard to solve the case. But they don't pay any attention to their kids and family and it's precisely that lack of attention for their children that prevents them from actually solving the case and from protecting their children. +PS: I spotted yet another Pied Piper reference. In the Pied Piper there is one +kid who is able to tell the adults about it because he is blind and crippled and +so he couldn't catch up with the others - but precisely because he is blind he +couldn't see where they went. You have the same thing in Suicide Club where the +girl calling herself "the bat" tries to inform the police while being +blindfolded. You can even see that she is unable to describe where she is +precisely because she can't see so that wanna be Charles Mansion types it for +her. And the fact that she is put in a sack makes her move around like being +crippled... -PS: I spotted yet another Pied Piper reference. In the Pied Piper there is one kid who is able to tell the adults about it because he is blind and crippled and so he couldn't catch up with the others - but precisely because he is blind he couldn't see where they went. You have the same thing in Suicide Club where the girl calling herself "the bat" tries to inform the police while being blindfolded. You can even see that she is unable to describe where she is precisely because she can't see so that wanna be Charles Mansion types it for her. And the fact that she is put in a sack makes her move around like being crippled... +And yet another one: in the first video we see of "Dessert" when they are +wearing the clothes with the numbers on them that are relevant for the secret +code... those clothes consist of many brightly colored stripes (pied)... just +like the colorful stripes the pied piper is wearing in the original German +legend. -And yet another one: in the first video we see of "Dessert" when they are wearing the clothes with the numbers on them that are relevant for the secret code... those clothes consist of many brightly colored stripes (pied)... just like the colorful stripes the pied piper is wearing in the original German legend. +And yet another one: as someone already pointed out "Dessert" is sometimes +spelled "Dessert" and sometimes "Desert" in the movie. So this is another hint +to how the parents have deserted their children making it possible for the Pied +Piper to exploit that carelessness just like he does in the original story. -And yet another one: as someone already pointed out "Dessert" is sometimes spelled "Dessert" and sometimes "Desert" in the movie. So this is another hint to how the parents have deserted their children making it possible for the Pied Piper to exploit that carelessness just like he does in the original story. - -And yet another one... man once you know what to look for it really never stops... i was wondering about the dates...the movie always tells you what date it is and i couldn't figure out the significance of that. Also we are only told the day and the month - not the year. The film starts on May 26th with the subway suicides. In the original story the Pied Piper starts leading the children away on June 26th. I know it a month off, but it's the 26th nonetheless so i don't think it's a coincidence. +And yet another one... man once you know what to look for it really never +stops... i was wondering about the dates...the movie always tells you what date +it is and i couldn't figure out the significance of that. Also we are only told +the day and the month - not the year. The film starts on May 26th with the +subway suicides. In the original story the Pied Piper starts leading the +children away on June 26th. I know it a month off, but it's the 26th nonetheless +so i don't think it's a coincidence. diff --git a/articles/de.org b/articles/de.org index fbe347b2c5160ae3b7de102767a875f271288589..edfd95b8f71e03730703f9ea2f1bc81c7fbb3be0 100644 --- a/articles/de.org +++ b/articles/de.org @@ -1,4 +1,4 @@ -% Building your own desktop environment +#+TITLE: Building your own desktop environment All-in-one desktop environments (DE) such as Gnome, KDE and those found in popular systems get in the way of the power user in that they do not leave much @@ -6,25 +6,28 @@ room for customization, it restricts the potential a computer can provide. Moreover most popular DE have been plagued for decades with unpractical, non-automated and limited window management. -Positioning and resizing windows does not take much time _per se_, but it stacks +Positioning and resizing windows does not take much time /per se/, but it stacks up. Think about how many windows you fire up every day (which, for some obscure reasons, seem to be spawned at random places all the time on many popular systems...). Add up all the seconds you spend managing them, moving around, resizing... Wouldn't it be nice if this were done for us? After all, a computer -is an augmented _automata_, it should be able to automate this. +is an augmented /automata/, it should be able to automate this. Another side-effect: the classic desktop environment (DE) will bundle more than you want... As such, they will always require more disk space and virtual memory than you actually need. The following advises will make things lighter and faster. -# Dynamic Tiling Window Managers +* Dynamic Tiling Window Managers + :PROPERTIES: + :CUSTOM_ID: dynamic-tiling-window-managers + :END: There are ways to automate window management: tiling window managers. Once we have gotten used to them, it gets hard to bear with the waste of time incurred by the "regular, user-friendly" window managers. (Like those featured in Gnome and KDE.) The latter are called stacking window managers. I find the concept of -stacking fairly questionable: why would the user want to _cover_ part of other +stacking fairly questionable: why would the user want to /cover/ part of other windows? Either you want to have a full look at some of the currently opened windows, or you just want to focus on one windows. Which means either tile the windows or go fullscreen. In other words, unused screen space is wasted screen @@ -32,7 +35,7 @@ space. I know how badly we like to look at our wallpaper, but that is not the primary use we should make of our computer. This leads us to the antagonists of the stacking window managers: the tiling -window managers. Windows do not "float" anymore, i.e. their position and size is +window managers. Windows do not "float" anymore, i.e. their position and size is not controlled manually but automatically so that it fits the screen space best. Understand that tiling managers do no deprive you from manual control over the windows: they are, by definition, a superset of stacking WMs, and thus you can @@ -45,16 +48,15 @@ stacking windows, so all applications go fullscreen. TWMs allow the user to relay full window management to the WM independently of the application capabilities. For instance a good WM will let you go fullscreen -or use tabs with any program. This is part of the [Unix philosophy][unix]: -instead of duplicating functionalities across programs, write it once and for -all where it is meant to be. File browsers and Internet browsers should not have -tabs: WMs will do a better job (grouping, sorting) while keeping consistent -behaviour and key bindings across programs. +or use tabs with any program. This is part of the +[[http://en.wikipedia.org/wiki/Unix_philosophy][Unix philosophy]]: instead of +duplicating functionalities across programs, write it once and for all where it +is meant to be. File browsers and Internet browsers should not have tabs: WMs +will do a better job (grouping, sorting) while keeping consistent behaviour and +key bindings across programs. -[unix]: http://en.wikipedia.org/wiki/Unix_philosophy - -Last but not least, there is a subcategory among tiling WMs: the -dynamic ones. The tiling is done whenever a window is fired up. +Last but not least, there is a subcategory among tiling WMs: the dynamic ones. +The tiling is done whenever a window is fired up. In theory, the concept of a window manager is rather simple. In practice, they mainly differ by their configuration. Some WM will offer little tweaking @@ -63,25 +65,31 @@ The former is easier to get started with, while the latter will allow for the exact desired behaviour. In my opinion, the most powerful WM would be a DTWM with a Turing-complete -configuration. Such popular DTWMs include _Awesome_ and _i3_. Awesome's -configuration is in Lua, i.e. it allows virtually unlimited tweaking, while i3 -relies on external shell scripts. Another possibly more powerful example is -[EXWM](../emacs-everywhere/index.html) which is extensible in Emacs Lisp. +configuration. Such popular DTWMs include /Awesome/ and /i3/. Awesome's +configuration is in Lua, i.e. it allows virtually unlimited tweaking, while i3 +relies on external shell scripts. Another possibly more powerful example is +[[../emacs-everywhere/index.html][EXWM]] which is extensible in Emacs Lisp. -# Keyboard control +* Keyboard control + :PROPERTIES: + :CUSTOM_ID: keyboard-control + :END: The use of the mouse has become so widespread with the advent of desktop computers that it now seems "intuitive" to associate the use of a mouse together with desktop environments. All the desktop management (windows, firing up -programs, monitoring the system, etc.) are _discrete actions_, they do not +programs, monitoring the system, etc.) are /discrete actions/, they do not require the continuous movement of a mouse pointer. As such, everything can be bound to a keybinding. Go fullscreen? Keybinding. Change the window layout? Keybinding. -Keep in mind we say "keyboard shortcut": the word _shortcut_ is there for a -reason. See my article on [mastering the keyboard](../keymap/index.html). +Keep in mind we say "keyboard shortcut": the word /shortcut/ is there for a +reason. See my article on [[../keymap/index.html][mastering the keyboard]]. -# Building from bricks +* Building from bricks + :PROPERTIES: + :CUSTOM_ID: building-from-bricks + :END: Refraining from using a full-featured desktop environment means that you will have to install the components one by one and configure it all to your liking. @@ -89,7 +97,10 @@ Which is the only way to fully tailor a computer to your needs. While it may seem daunting at first, this is a one-time task: it is straightforward to re-use the configuration across systems when it is stored in text files. -## Text configuration +** Text configuration + :PROPERTIES: + :CUSTOM_ID: text-configuration + :END: Most desktop environments come with a configuration panel: a program that is meant to centralize configuration. In practice, this is an additional layer of @@ -98,9 +109,10 @@ complexity that is not really needed. Editing a text file has many advantages: - It is just as fast. Even faster with a good text editor. -- A GUI may not be exhaustive: a text file guarantees full access to the options. +- A GUI may not be exhaustive: a text file guarantees full access to the + options. - It is more reliable since it removes the extra layer of complexity. (The GUI -might introduce bugs.) + might introduce bugs.) - You keep control over configuration file updates. - The configuration can be saved and even put under version control. @@ -109,18 +121,21 @@ configuration changes (diffs); it also lets you deploy and sync a configuration across several computers in an instant. Last but not least, it lets you share the configuration! -## Independent power-apps +** Independent power-apps + :PROPERTIES: + :CUSTOM_ID: independent-power-apps + :END: Desktop environments push for developing "desktop-specific" applications, such -as the K* programs for KDE vs. the G* programs for Gnome. I find this very +as the K* programs for KDE vs. the G* programs for Gnome. I find this very embarrassing: why would we spend months and years developing a program for a sub-group of people? Why not spreading it to the world? Is it a form of geek-racism? Our favorite programs should not have to do with the desktop environment. If the latter goes out of fashion and gets unmaintained, does it mean we are supposed -to change the whole set of our programs, possibly losing some functionalities -in the process? +to change the whole set of our programs, possibly losing some functionalities in +the process? A "desktop" in my sense is just the WM with a launcher. The rest should be the user favorite programs: a terminal emulator, a desktop monitor, a text editor, @@ -129,10 +144,13 @@ icon? Add key bindings. Want to tweak the network configuration? Fire-up the appropriate tool with a keybinding, or go edit the text file full of endless possibilities... -See my [list of power-user applications](../power-apps/index.html) for some +See my [[../power-apps/index.html][list of power-user applications]] for some inspiration. -# Black & White +* Black & White + :PROPERTIES: + :CUSTOM_ID: black-white + :END: When the mere mortals have a glimpse at my computer, I would expect their first reaction to be "What system is it?" or "Why is it so barren?". No, it is @@ -141,13 +159,13 @@ invariably "Maaaah, why is it all black?!?" I find it interesting that black has become shocking. Black used to be the standard back in the days. The white-dominant background may have been popularized by the "user-friendliness" marketing of Apple and Microsoft in the -1980s as a reaction against the "hard-to-use" Unix and friends: the color was here -to mark the difference. The association of ideas was easily made: white is -brightness, it is light, as opposed to the uncomfortable, insecure darkness of Mr. -Black. +1980s as a reaction against the "hard-to-use" Unix and friends: the color was +here to mark the difference. The association of ideas was easily made: white is +brightness, it is light, as opposed to the uncomfortable, insecure darkness of +Mr. Black. -Now if we are rational, it is just a color. But there is more to it. Working on a -dark background will not strain the eye as much. The whole industry is warning +Now if we are rational, it is just a color. But there is more to it. Working on +a dark background will not strain the eye as much. The whole industry is warning its professional, intensive users against the possible eye strains and headaches resulting from staring at a screen for too many hours. diff --git a/articles/emacs-eshell.org b/articles/emacs-eshell.org index 1a24bdc5857a26f1050be7c21482238472d3aa99..1d3575ed12e8060d30326d703d6c9a3fe9e8e927 100644 --- a/articles/emacs-eshell.org +++ b/articles/emacs-eshell.org @@ -1,190 +1,201 @@ -% Eshell as a main shell +#+TITLE: Eshell as a main shell -*This article was initially posted on -[Reddit](https://www.reddit.com/r/emacs/comments/6y3q4k/yes_eshell_is_my_main_shell/). -I've revised some arguments by taking the community feedback into account.* +/This article was initially posted on +[[https://www.reddit.com/r/emacs/comments/6y3q4k/yes_eshell_is_my_main_shell/][Reddit]]. +I've revised some arguments by taking the community feedback into account./ -Since its inception, Eshell has suffered from an arguably low popularity. -See what the community says: +Since its inception, Eshell has suffered from an arguably low popularity. See +what the community says: -- https://www.reddit.com/r/emacs/comments/492edz/is_anybody_using_eshell_as_the_default_shell_any/ -- https://www.reddit.com/r/emacs/comments/48opk1/eshell_and_why_cant_i_convert_to_you/ +- https://www.reddit.com/r/emacs/comments/492edz/is\_anybody\_using\_eshell\_as\_the\_default\_shell\_any/ +- https://www.reddit.com/r/emacs/comments/48opk1/eshell\_and\_why\_cant\_i\_convert\_to\_you/ Hell, even the official manual gives a fair warning against its use: -- https://www.gnu.org/software/emacs/manual/html_mono/eshell.html#Introduction +- https://www.gnu.org/software/emacs/manual/html\_mono/eshell.html#Introduction I'm not arguing that those arguments are not valid. (To most extent, they are.) -I want to argue that the issue is not so much that Eshell is not a -proper _Unix shell_ (in the sense of POSIX sh and derivatives), but rather -perhaps we do not need a _Unix shell_ and all its historical limitations. +I want to argue that the issue is not so much that Eshell is not a proper /Unix +shell/ (in the sense of POSIX =sh= and derivatives), but rather perhaps we do +not need a /Unix shell/ and all its historical limitations. In other words: the non-Unix-ness of Eshell is more of a step forward than a -hindrance. Let's review the situation. +hindrance. Let's review the situation. - -# "State of the art" +* "State of the art" + :PROPERTIES: + :CUSTOM_ID: state-of-the-art + :END: Take this typical scenario: I've got a long running process which finally -outputs pages of relevant information. Only then I realize that it would have -been smart saving that somewhere. Now I want to copy it to the clipboard. How -do I do that? +outputs pages of relevant information. Only then I realize that it would have +been smart saving that somewhere. Now I want to copy it to the clipboard. How do +I do that? -_By clumsily mouse-selecting pages of terminal output, assuming my terminal can -handle it._ +/By clumsily mouse-selecting pages of terminal output, assuming my terminal can +handle it./ -> Why can't I (fuzzy-)search the shell output? +#+BEGIN_QUOTE + Why can't I (fuzzy-)search the shell output? +#+END_QUOTE -> Why can't I browse the different prompts of my shell session? +#+BEGIN_QUOTE + Why can't I browse the different prompts of my shell session? +#+END_QUOTE -> Why can't I copy/paste without a mouse? +#+BEGIN_QUOTE + Why can't I copy/paste without a mouse? +#+END_QUOTE Back in the days, VT-like terminals were our main mean of communicating with a -machine. Decades went by, our desktop computers can now handle gigabytes of +machine. Decades went by, our desktop computers can now handle gigabytes of buffering and display in 24-bit colors, and yet we still stick to terminal -_emulators_, that is, programs that emulate the _restrictions_ of those ancient +/emulators/, that is, programs that emulate the /restrictions/ of those ancient machines. - Cannot move the cursor around. - Limited colors. -- Terminal capabilities are neither simple nor portable. See termcap(5). +- Terminal capabilities are neither simple nor portable. See termcap(5). -- Formatting (e.g. colors) needs ugly escape codes that are not even portable. +- Formatting (e.g. colors) needs ugly escape codes that are not even portable. - More hacks than computer science history would have ever hoped for. Say I run this command: - $cat file-list.txt - /path/to/foo - /other/path/to/bar +#+BEGIN_EXAMPLE +$ cat file-list.txt + /path/to/foo + /other/path/to/bar +#+END_EXAMPLE -Now I want to see what's in _foo_. How do I do that? +Now I want to see what's in /foo/. How do I do that? 1. Either I mouse-select the file, hoping I'll not be off-by-one on one side. -Then I paste to the prompt. + Then I paste to the prompt. 2. Or I type the path myself, with a bit of luck completion will bring me there -soon enough. + soon enough. 3. Or I resort to re-call the command from a subcommand and program some output -manipulation to extract the filename and then open it with my editor. This is -not nearly universal nor even convenient. + manipulation to extract the filename and then open it with my editor. This is + not nearly universal nor even convenient. All this is terribly backwards and it is high time we moved on. - -# Terminals vs. shells +* Terminals vs. shells + :PROPERTIES: + :CUSTOM_ID: terminals-vs.shells + :END: It's important to understand that shells are not (or should not be) semantically -bound to terminal emulator restrictions. Shells are a textual interface to the -machine. They just need _input, evaluation, execution, output_. -GTK/Qt/Tk/etc. are absolutely capable of handling such a task. Not mentioning -Emacs... +bound to terminal emulator restrictions. Shells are a textual interface to the +machine. They just need /input, evaluation, execution, output/. GTK/Qt/Tk/etc. +are absolutely capable of handling such a task. Not mentioning Emacs... -So why do shells _de facto_ need a terminal? One thing: curses-like interfaces. -Those libraries need terminal capabilities to render. Do we still really need that? +So why do shells /de facto/ need a terminal? One thing: curses-like interfaces. +Those libraries need terminal capabilities to render. Do we still really need +that? -- It's most limited in terms of user experience. Mouse selection simply won't -do. +- It's most limited in terms of user experience. Mouse selection simply won't + do. - It's not beautiful. -- It cannot render anything beside text. (Without clumsy hacks at least.) +- It cannot render anything beside text. (Without clumsy hacks at least.) - "It works on low-end systems": there is not much hardware out there that -cannot take Wayland or X. Anyways, my case is about desktop computers. + cannot take Wayland or X. Anyways, my case is about desktop computers. - It's not even fast: in fact terminals can be pretty slow at rendering. -(Emulators also emulate the baud-rate, etc.) + (Emulators also emulate the baud-rate, etc.) - Since it's character-based, it cannot render fonts of different sizes nor, -say, thin lines. - -- "Efficiency on remote systems": that's almost a fair statement. But then... -TRAMP! ;) + say, thin lines. +- "Efficiency on remote systems": that's almost a fair statement. But then... + TRAMP! ;) -# Enter Eshell +* Enter Eshell + :PROPERTIES: + :CUSTOM_ID: enter-eshell + :END: Back to my initial example of the long-running process: want to copy that output -somewhere? 3 key-strokes with Eshell. And better: could be even 1 if you want -to program it your way. The _interface_ is extensible. (Terminals usually -aren't, URxvt and its Perl scripts are not a proud example of _extensibility_.) - - +somewhere? 3 key-strokes with Eshell. And better: could be even 1 if you want to +program it your way. The /interface/ is extensible. (Terminals usually aren't, +URxvt and its Perl scripts are not a proud example of /extensibility/.) Eshell breaks up with the "terminal-interface" paradigm: the user interface of -the Eshell _commandline-interface_ is Emacs it self. The pager is the Emacs -buffer. The input is Emacs keystrokes. The extension language is Elisp. +the Eshell /commandline-interface/ is Emacs it self. The pager is the Emacs +buffer. The input is Emacs keystrokes. The extension language is Elisp. Consequences: -- No need for pagers like less. You won't ever re-run a long-output command by - appending | less to it. +- No need for pagers like less. You won't ever re-run a long-output command by + appending =| less= to it. -- Little need for output ad-hoc minipulations (e.g. grep trial-and-error -queries--grep is not a search tool): output the result to an Emacs buffer, use -some Lisp functions, use Evil ex commands, iedit, swiper, helm-moccur or -multiple-cursors... +- Little need for output ad-hoc minipulations (e.g. =grep= trial-and-error + queries--=grep= is not a search tool): output the result to an Emacs buffer, + use some Lisp functions, use Evil ex commands, iedit, swiper, helm-moccur or + multiple-cursors... -- Eshell supports TRAMP! Which means you don't have to put aside your powerful -environment when switching to root or connecting to a remote host: all the power -of _your_ Emacs can be used anywhere, the shell included. +- Eshell supports TRAMP! Which means you don't have to put aside your powerful + environment when switching to root or connecting to a remote host: all the + power of /your/ Emacs can be used anywhere, the shell included. -- Find file/URL at point. Or list them all with Ivy/Helm and fuzzy-search them! +- Find file/URL at point. Or list them all with Ivy/Helm and fuzzy-search them! -- No need for [fzf](http://github.com/junegunn/fzf). "Hey! fzf is awesome!" -I know, but even fzf cannot compete with Emacs. +- No need for [[http://github.com/junegunn/fzf][fzf]]. "Hey! =fzf= is awesome!" + I know, but even =fzf= cannot compete with Emacs. -- At last, done with the clumsy Bash and sh-inspired shell languages! No more -nasty word-splitting, no more unexpected expansion. You don't even need calc -or bc when you have Elisp or Emacs' calc at hand. +- At last, done with the clumsy Bash and sh-inspired shell languages! No more + nasty word-splitting, no more unexpected expansion. You don't even need =calc= + or =bc= when you have Elisp or Emacs' calc at hand. - No need for half-implemented vi-bindings when you can have Evil to edit your -command-line. (Some shells like [fish](http://fishshell.com) can call an editor -to edit the commandline. Well, with Eshell it's one step less.) + command-line. (Some shells like [[http://fishshell.com][fish]] can call an + editor to edit the commandline. Well, with Eshell it's one step less.) - You can even edit the output directly! - You can redirect output to several buffers(!), they don't even need to be -existing files. + existing files. -- Eshell has loads of [fancy -features](https://www.masteringemacs.org/article/complete-guide-mastering-eshell) -for globbing, filtering, and editing. +- Eshell has loads of + [[https://www.masteringemacs.org/article/complete-guide-mastering-eshell][fancy + features]] for globbing, filtering, and editing. -- Programmable completion with pcomplete, which is arguably easier and yet -more powerful than its Bash/Zsh/fish counterparts. +- Programmable completion with =pcomplete=, which is arguably easier and yet + more powerful than its Bash/Zsh/fish counterparts. Finally, some Helm niceties: -- helm-find-files has M-e to switch to Eshell in the currently browsed -folder. Bye-bye cd, hello fuzzy-browsing! +- =helm-find-files= has =M-e= to switch to Eshell in the currently browsed + folder. Bye-bye =cd=, hello fuzzy-browsing! - The previous technique makes it very convenient to go up the hierarchy to some -parent folder (C-l by default). Seriously, bye-bye annoying little cd ... + parent folder (=C-l= by default). Seriously, bye-bye annoying little =cd ..=. -- Still with helm-find-files, M-p will trigger the "cd history", so you can -fuzzy-find the history of your shell paths. Make it persistent with -(add-to-list 'desktop-globals-to-save 'helm-ff-history)). +- Still with =helm-find-files=, =M-p= will trigger the "cd history", so you can + fuzzy-find the history of your shell paths. Make it persistent with + =(add-to-list 'desktop-globals-to-save 'helm-ff-history))=. -- Use Helm as a menu for completion. Awesome for completing big file names or -packages! +- Use Helm as a menu for completion. Awesome for completing big file names or + packages! -- helm-find-files has C-c i (and C-u C-c i, etc.) to insert the selected -relative/absolute file names at point, that is, in the commandline. This -sometimes proves more convenient than file completion. +- =helm-find-files= has =C-c i= (and =C-u C-c i=, etc.) to insert the selected + relative/absolute file names at point, that is, in the commandline. This + sometimes proves more convenient than file completion. +In terms of extensibility, it's key to understand that (as of +[[https://debbugs.gnu.org/cgi/bugreport.cgi?bug=27405][Emacs 26]]) all prompts +are properly /identified/ by Emacs. This opens the door to infinite +programmability of the shell interface: -In terms of extensibility, it's key to understand that (as of [Emacs -26](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=27405)) all prompts are -properly _identified_ by Emacs. This opens the door to infinite programmability -of the shell interface: - -- Browse all prompts with C-c C-n/C-c C-p. (Or simpler bindings...) +- Browse all prompts with =C-c C-n=/=C-c C-p=. (Or simpler bindings...) - Copy the output of a command with one keystroke. @@ -192,19 +203,21 @@ of the shell interface: - Use hs-minor-mode to fold the outputs. - - -# There is more to come +* There is more to come + :PROPERTIES: + :CUSTOM_ID: there-is-more-to-come + :END: It's undeniable however that, as of Emacs 25.2, Eshell has numerous rough edges. Quite a few bug reports have been sent on debbugs and it's getting better. -Native completion is limited considering very little effort was ever made. The -community can help changing that. In the meantime, I've implemented [a fallback -to Bash/fish completion](https://github.com/Ambrevar/emacs-fish-completion) so -that I can keep completing just as much as I could with Bash or fish. +Native completion is limited considering very little effort was ever made. The +community can help changing that. In the meantime, I've implemented +[[https://github.com/Ambrevar/emacs-fish-completion][a fallback to Bash/fish +completion]] so that I can keep completing just as much as I could with Bash or +fish. -Need that curses-based program? Get a (much more) powerful Emacs alternative: +Need that curses-based program? Get a (much more) powerful Emacs alternative: - git (CLI), gitk, tig, etc. -> magit @@ -229,90 +242,88 @@ features: - Configure/extend in Lisp. -- (Fuzzy-)search the buffer. Copy anything to the clipboard. +- (Fuzzy-)search the buffer. Copy anything to the clipboard. -- No need to change the color theme, it's already using your Emacs theme. No -need for dircolors, lesspipe, Xresources... +- No need to change the color theme, it's already using your Emacs theme. No + need for dircolors, lesspipe, Xresources... -- Emacs bindings (Evil?). All configurable. No more painful context-switch -between Emacs-based and vi-based bindings. +- Emacs bindings (Evil?). All configurable. No more painful context-switch + between Emacs-based and vi-based bindings. In the meantime, there might still be that annoying curses-based program you -cannot replace. Eshell allows you to run those programs by transparently -switching to a *term* buffer. See eshell-visual-commands. - +cannot replace. Eshell allows you to run those programs by transparently +switching to a =*term*= buffer. See =eshell-visual-commands=. Curses-based programs are not the only ones that won't run well on Eshell: -[REPL](https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop) are -known for working poorly too. REPLs are poorly designed though: from a -Unix-philosophy standpoint, only the E is required for every language, while -re-implementing the R, P, and L, i.e. the user interface, for each one of -them is simply re-inventing the wheel. Instead of that, Emacs has -comint-mode, an REPL-lie user interface that can be plugged to any language. -With all the powerful features of Emacs that all those specialized REPLs are -lacking. For a start, you can write code in Emacs and execute any piece of code -from a single keystroke. It's configurable and extensible in a consistent manner -across all languages. +[[https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop][REPL]] are +known for working poorly too. REPLs are poorly designed though: from a +Unix-philosophy standpoint, only the =E= is required for every language, while +re-implementing the =R=, =P=, and =L=, i.e. the user interface, for each one of +them is simply re-inventing the wheel. Instead of that, Emacs has =comint-mode=, +an REPL-lie user interface that can be plugged to any language. With all the +powerful features of Emacs that all those specialized REPLs are lacking. For a +start, you can write code in Emacs and execute any piece of code from a single +keystroke. It's configurable and extensible in a consistent manner across all +languages. Similarly, "environment setups", as commonly used by programming language tools -such as Python's virtualenv or Ruby rvm, expect the user to be running a -specific shell. This is broken by design. Let me quote _/u/rpdillon_ as I -could not have said better: - -> To OP's point, though, yes, these environments do make assumptions that you'll -> be using some kind of bash-compatible shell, which tells you a little bit -> about how sub-optimal the state-of-the-art is. I suppose the only alternative -> I can think of is to have the language itself manage many versions of itself -> so that every developer's environment wouldn't have to be customized/extended -> to negotiate the maze. - -The [Go](https://golang.org/) programming language for instance does not have +such as Python's =virtualenv= or Ruby =rvm=, expect the user to be running a +specific shell. This is broken by design. Let me quote //u/rpdillon/ as I could +not have said better: + +#+BEGIN_QUOTE + To OP's point, though, yes, these environments do make assumptions that you'll + be using some kind of bash-compatible shell, which tells you a little bit + about how sub-optimal the state-of-the-art is. I suppose the only alternative + I can think of is to have the language itself manage many versions of itself + so that every developer's environment wouldn't have to be customized/extended + to negotiate the maze. +#+END_QUOTE + +The [[https://golang.org/][Go]] programming language for instance does not have such a tool as I know of. - - A long-standing critique of Eshell is that it does not support input -redirection. This is hardly a major hindrance since -you can always use a work-around such as +redirection. This is hardly a major hindrance since you can always use a +work-around such as - cat file | command +#+BEGIN_EXAMPLE + cat file | command +#+END_EXAMPLE Another common argument against Eshell is that it cannot run the widely popular -Tmux. The latter is used for many things, out of which: +Tmux. The latter is used for many things, out of which: - Window management, tabs, etc.: you don't need that if you have Emacs. - Leave long-running processes in the background after the user has logged out: -this feature is easily provided with nohup or dtach. - - + this feature is easily provided with =nohup= or =dtach=. -All that being said, should Eshell not do the job for you, then maybe give M-x -shell a try. It has a similar design: it is a shell emancipated from any +All that being said, should Eshell not do the job for you, then maybe give +=M-x shell= a try. It has a similar design: it is a shell emancipated from any terminal; instead of being Lisp based, it runs an existing shell as an -interpreter. Bash and zsh are known to work. It has all the advantages an -Emacs buffer has over terminal emulators. The main difference then lies in the -language: [Emacs](https://www.gnu.org/software/emacs/emacs-paper.html) -[Lisp](http://paulgraham.com/avg.html) vs. [Bash and related -languages](http://mywiki.wooledge.org/BashPitfalls). +interpreter. =Bash= and =zsh= are known to work. It has all the advantages an +Emacs buffer has over terminal emulators. The main difference then lies in the +language: [[https://www.gnu.org/software/emacs/emacs-paper.html][Emacs]] +[[http://paulgraham.com/avg.html][Lisp]] vs. +[[http://mywiki.wooledge.org/BashPitfalls][Bash and related languages]]. - - -# An open door to Emacs-Everywhere +* An open door to Emacs-Everywhere + :PROPERTIES: + :CUSTOM_ID: an-open-door-to-emacs-everywhere + :END: Switching to Eshell marked a milestone for me: from then on I dropped all my crufty curses-based programs and switched to much more powerful Emacs -alternatives. I now use [Emacs everywhere](../emacs-everywhere) to the point -that it even is [my window manager](https://github.com/ch11ng/exwm). -Having a consistent environment well glued together really empowers you. +alternatives. I now use [[../emacs-everywhere][Emacs everywhere]] to the point +that it even is [[https://github.com/ch11ng/exwm][my window manager]]. Having a +consistent environment well glued together really empowers you. Ironically, the environment is so powerful that I ended up not using any shell -so much anymore, only for a few commands here and there. For which I use -Eshell, obviously. - - +so much anymore, only for a few commands here and there. For which I use Eshell, +obviously. Eshell might not be perfect (yet), a little beta perhaps, but this can only be helped by a growing community: the more people join, the more feedback -developers get, the better it will become. Hopefully this pamphlet will -help raise your curiosity, if not more esteem for this brilliant project. +developers get, the better it will become. Hopefully this pamphlet will help +raise your curiosity, if not more esteem for this brilliant project. diff --git a/articles/emacs-everywhere.org b/articles/emacs-everywhere.org index f0e97111aa938da7514d1824b98043228fc8c849..6b17b103e37aca6372bd62e672dc37aee6f42567 100644 --- a/articles/emacs-everywhere.org +++ b/articles/emacs-everywhere.org @@ -1,97 +1,102 @@ -% Emacs Everywhere +#+TITLE: Emacs Everywhere -*This article was initially posted on -[Reddit](https://www.reddit.com/r/emacs/comments/74hetz/emacs_everywhere/). -I've revised some arguments by taking the community feedback into account.* +/This article was initially posted on +[[https://www.reddit.com/r/emacs/comments/74hetz/emacs_everywhere/][Reddit]]. +I've revised some arguments by taking the community feedback into account./ For many years I refrained from using Emacs everywhere because I clung to the following Unix philosophical principle: "Make each program do one thing well." It did not make sense to me then to use a text editor as an e-mail client or a -music library manager. I used to favour well-established ncurses programs like +music library manager. I used to favour well-established ncurses programs like Mutt and cmus respectively. -When I started using [Eshell as my main shell](../emacs-eshell), the benefits of -the Emacs _interface_ became increasingly obvious. Maybe my initial reasoning -was not well founded after all. Since then I successfully moved on to using -this user-interface-on-steroids everywhere. Looking back, it feels like I had -been missing out and wasted my time for that many years. +When I started using [[../emacs-eshell][Eshell as my main shell]], the benefits +of the Emacs /interface/ became increasingly obvious. Maybe my initial reasoning +was not well founded after all. Since then I successfully moved on to using this +user-interface-on-steroids everywhere. Looking back, it feels like I had been +missing out and wasted my time for that many years. This realization is what leads me to write a pamphlet on the matter to help the user community move forward out of a loop of misconception that does not help -but waste everybody's time. I hope to help informing you with strong arguments +but waste everybody's time. I hope to help informing you with strong arguments as to why you should or you should not use Emacs extensively. Too often the same fallacious misconceptions stir endless debates: -> Configuring Emacs cost thousands of hours just to tweak the editor, so sending -> e-mails with Emacs is simply not within my reach. +#+BEGIN_QUOTE + Configuring Emacs cost thousands of hours just to tweak the editor, so sending + e-mails with Emacs is simply not within my reach. +#+END_QUOTE or, even more famously -> Emacs is a great operating system, lacking only a decent editor. +#+BEGIN_QUOTE + Emacs is a great operating system, lacking only a decent editor. +#+END_QUOTE While obviously sarcastic, it might be worth point out that no, Emacs' intent is -not to be an operating system. That being said, it is true that Emacs is not -_only_ an editor. From a broader perspective, it would be best described as -a _programmable, text-oriented user-interface_ (containing, among others, an +not to be an operating system. That being said, it is true that Emacs is not +/only/ an editor. From a broader perspective, it would be best described as a +/programmable, text-oriented user-interface/ (containing, among others, an editor). As such it is erroneous to discard Emacs special modes for the sole reason that -an editor should not do anything but editing text. If you think of Emacs as a -user interface, then it covers the same ground as Qt, GTK, Tk or _curses_ and +an editor should not do anything but editing text. If you think of Emacs as a +user interface, then it covers the same ground as Qt, GTK, Tk or /curses/ and the Unix-philosophical argument falls flat. As a matter of fact, using Emacs to send e-mails or listen to music is usually -_much simpler_ than configuring the editor. This is because an editor is +/much simpler/ than configuring the editor. This is because an editor is intrisically richer in terms of features and configuration possibilities. Emacs might not suit everybody's needs everywhere, although I believe that more -often than not it will. Hopefully the insights of this pamphlet will add enough +often than not it will. Hopefully the insights of this pamphlet will add enough fuel to the fire to help nuance the various views held by the community. - - -## Emacs vs. the others +** Emacs vs. the others + :PROPERTIES: + :CUSTOM_ID: emacs-vs.the-others + :END: The power features Emacs offers and that are lacking in other "common" interfaces (GTK, Qt, Tk, EFL, cocoa, curses, etc.) include: -- Everything is text, everything is searchable and copy-able. Even better, you -can fuzzy-search anything. Enter Helm, Ivy and others. +- Everything is text, everything is searchable and copy-able. Even better, you + can fuzzy-search anything. Enter Helm, Ivy and others. - It can be fully keyboard-controlled (not very common with GTK and friends), -while mouse controls are supported too (which sucks in ncurses). + while mouse controls are supported too (which sucks in ncurses). - It works both in graphical and textual environments, mostly out-of-the-box. -Nevertheless you should prefer the less limited graphical Emacs: all keyboard -modifiers are supported, various font sizes can be displayed, and... pictures! + Nevertheless you should prefer the less limited graphical Emacs: all keyboard + modifiers are supported, various font sizes can be displayed, and... pictures! -- Configuration is done in Emacs Lisp. Which is the best language ever, as [we -all know](http://www.paulgraham.com/avg.html). At least when it comes to -[extensibility](https://www.gnu.org/software/emacs/emacs-paper.html). And even -if you don't agree with that, it sucks less than most of its competitors. +- Configuration is done in Emacs Lisp. Which is the best language ever, as + [[http://www.paulgraham.com/avg.html][we all know]]. At least when it comes to + [[https://www.gnu.org/software/emacs/emacs-paper.html][extensibility]]. And + even if you don't agree with that, it sucks less than most of its competitors. - Configuration, as a consequence of being Lisp, can be tracked by version -control systems. - - + control systems. -## What Emacs really does +** What Emacs really does + :PROPERTIES: + :CUSTOM_ID: what-emacs-really-does + :END: -Now let's move on to the core of the question: Is it wise to have _everything_ +Now let's move on to the core of the question: Is it wise to have /everything/ run from within Emacs? A common misconception when thinking of "Emacs as an OS" is to assume that Emacs -special modes are re-inventing the wheel. They are not (for most of them), -Emacs and its modes focus on the _user interface_ side of things. The backends -are almost always separate programs. This is precisely where the Unix -philosophy still stands strong. Using Emacs as an interface for everything is -merely equivalent to using GTK-only applications. (Only much better, -obviously.) - -As opposed to other user interfaces Emacs is a _programmable_ environment: any +special modes are re-inventing the wheel. They are not (for most of them), Emacs +and its modes focus on the /user interface/ side of things. The backends are +almost always separate programs. This is precisely where the Unix philosophy +still stands strong. Using Emacs as an interface for everything is merely +equivalent to using GTK-only applications. (Only much better, obviously.) + +As opposed to other user interfaces Emacs is a /programmable/ environment: any structure, interface element and even code can be passed around and combined -between the different interfaces to various programs. Consider those canonical +between the different interfaces to various programs. Consider those canonical features: - Buffers @@ -133,134 +138,147 @@ And many more. Emacs does not lure developers into reinventing the wheel, quite the opposite: it shines at reusing and combining features in the most unexpected ways. - - -## The perks of Emacs as a user interface +** The perks of Emacs as a user interface + :PROPERTIES: + :CUSTOM_ID: the-perks-of-emacs-as-a-user-interface + :END: There is more to it: - Since Emacs can display pictures: EMMS can display album covers, e-mails can -display inline attachments. + display inline attachments. - Configuration consistency: Bindings, color themes and other interface elements -are consistently shared across the various special modes. No need to waste time -_syncing the different configuration files of your different programs_ (in -different configuration languages). + are consistently shared across the various special modes. No need to waste + time /syncing the different configuration files of your different programs/ + (in different configuration languages). - Configure, extend, fix: With Emacs, everything is configurable, even what was -not foreseen by its developers. All the Lisp source code is available at hand. -Want to add a feature? It's usually as simple as adding a few Elisp lines to -the configuration. Something is broken? After reporting it upstream, you don't -have to wait for the next release, you can hot-patch the bug from your -configuration. - -- Universality. Emacs is somewhat easy to compile. It runs on virtually all -desktop platforms you could think of. As such, running everything from Emacs -effectively abstracts away the OS user interface, which makes it possible to use -your personal configuration on any system. This is especially useful when you -are forced to a famous crappy OS. + not foreseen by its developers. All the Lisp source code is available at hand. + Want to add a feature? It's usually as simple as adding a few Elisp lines to + the configuration. Something is broken? After reporting it upstream, you don't + have to wait for the next release, you can hot-patch the bug from your + configuration. + +- Universality. Emacs is somewhat easy to compile. It runs on virtually all + desktop platforms you could think of. As such, running everything from Emacs + effectively abstracts away the OS user interface, which makes it possible to + use your personal configuration on any system. This is especially useful when + you are forced to a famous crappy OS. - OS-independent package manager: This provides the user with cutting-edge -packages even on (rusty) conservative distributions or when the user has no -privileges. - -- Flatter learning-curve for new programs: Emacs documentation system is (more or -less) consistently used among all Emacs modes, which makes the learning process -of a new mode somewhat easier. No need to figure out where the static -documentation is (HTML page? man page?), Emacs lets you (fuzzy-)search the -configuration variables and the bindings. - -- Lightweight, efficient: When I replaced all my super-lightweight _curses_ -programs with their Emacs counterparts, I did not notice a significant change in -disk usage. With the difference that ELPA packages have both the source code -and the byte-compiled files installed. For programmers and tinkerers, having -source code at hand is a boon. In terms of performance, graphical Emacs is not -limited by the restrictions born by terminal emulators and inflicted upon -_curses_ programs. - - - -## Side effects and other niceties - -- Another perk of using Emacs as everything is the _ubiquitous global state_ of -all Emacs special modes. Say you want to switch transmission to turtle mode, -there is no need to switch to transmission first, you can invoke the command or -its associated binding directly, regardless of the buffers curretly under focus. -Same goes for music with EMMS, or anything else running in the background. + packages even on (rusty) conservative distributions or when the user has no + privileges. + +- Flatter learning-curve for new programs: Emacs documentation system is (more + or less) consistently used among all Emacs modes, which makes the learning + process of a new mode somewhat easier. No need to figure out where the static + documentation is (HTML page? man page?), Emacs lets you (fuzzy-)search the + configuration variables and the bindings. + +- Lightweight, efficient: When I replaced all my super-lightweight /curses/ + programs with their Emacs counterparts, I did not notice a significant change + in disk usage. With the difference that ELPA packages have both the source + code and the byte-compiled files installed. For programmers and tinkerers, + having source code at hand is a boon. In terms of performance, graphical Emacs + is not limited by the restrictions born by terminal emulators and inflicted + upon /curses/ programs. + +** Side effects and other niceties + :PROPERTIES: + :CUSTOM_ID: side-effects-and-other-niceties + :END: + +- Another perk of using Emacs as everything is the /ubiquitous global state/ of + all Emacs special modes. Say you want to switch transmission to turtle mode, + there is no need to switch to transmission first, you can invoke the command + or its associated binding directly, regardless of the buffers curretly under + focus. Same goes for music with EMMS, or anything else running in the + background. - If you use Eshell, you don't need that lengthy, clunky bash/zsh/fish -configuration anymore. + configuration anymore. -- Other cumbersome configurations can go: dircolors, lesspipe, -Xresources... Even fzf can be disposed of. +- Other cumbersome configurations can go: =dircolors=, =lesspipe=, + =Xresources=... Even =fzf= can be disposed of. - No need to duplicate your dotfiles for the root user or on remote machines: -use TRAMP! - -- "Apps" for web services: debbugs (to browse the Emacs bug tracker), -mediawiki-mode to browse and edit MediaWiki-based pages, sx to work with -StackExchange pages... + use TRAMP! +- "Apps" for web services: =debbugs= (to browse the Emacs bug tracker), + =mediawiki-mode= to browse and edit MediaWiki-based pages, =sx= to work with + StackExchange pages... -## EXWM to rule them all +** EXWM to rule them all + :PROPERTIES: + :CUSTOM_ID: exwm-to-rule-them-all + :END: -_Warning: Aficionados of sparkles and glitter, behold! EXWM is visually as -barren as Emacs gets!_ +/Warning: Aficionados of sparkles and glitter, behold! EXWM is visually as +barren as Emacs gets!/ EXWM was for me the last milestone in the course of The Big Shift to a fully Emacs-based environment. I've been an avid user of AwesomeWM for years, but with time I grew tired of "losing windows" among the excess of workspaces (tags in Awesome terminology) or -screens. I wish I could have just fuzzy-searched them with fzf or something -similar. I never managed to implement the idea. Until I discovered EXWM. +screens. I wish I could have just fuzzy-searched them with =fzf= or something +similar. I never managed to implement the idea. Until I discovered EXWM. EXWM has all the benefits of being Emacs-based with the particularity that all X windows are buffers and as such are part of Emacs buffer list, which makes for -the capability to fuzzy-select windows with Helm or Ivy! "Lost windows" belong -to the past. When opening several windows at once, you can configure how to -display them. (This is a [recent addition to -Helm](https://github.com/emacs-helm/helm/commit/a66849bfc008bc0082b0a97caa3ac75652f0a8ad).) -A nice use-case is to first narrow down some windows using Helm patterns, -display them all in an "Exposé" fashion, and finally select your desired windows -visually. +the capability to fuzzy-select windows with Helm or Ivy! "Lost windows" belong +to the past. When opening several windows at once, you can configure how to +display them. (This is a +[[https://github.com/emacs-helm/helm/commit/a66849bfc008bc0082b0a97caa3ac75652f0a8ad][recent +addition to Helm]].) A nice use-case is to first narrow down some windows using +Helm patterns, display them all in an "Exposé" fashion, and finally select your +desired windows visually. Concretely, I have the following bindings: -- s-b: helm-mini -- M-a: helm-mark-all -- s-: windmove-* +- =s-b=: =helm-mini= +- =M-a=: =helm-mark-all= +- =s-=: =windmove-*= The "Exposé" binding sequence: - s-b M-a RET - s- RET +#+BEGIN_EXAMPLE + s-b M-a RET + s- RET +#+END_EXAMPLE Say I have a few Wikipedia pages of which I want to want to select one which title I do not remember: - s-b wikipedia M-a RET +#+BEGIN_EXAMPLE + s-b wikipedia M-a RET +#+END_EXAMPLE Or usually it's enough with - s-b wk M-a RET +#+BEGIN_EXAMPLE + s-b wk M-a RET +#+END_EXAMPLE then - s-l s-j RET +#+BEGIN_EXAMPLE + s-l s-j RET +#+END_EXAMPLE if if's the second window to the right then down. Since the window management is extensible, you can write your own -helm-window-show-buffers-function to fine-tune your needs: +=helm-window-show-buffers-function= to fine-tune your needs: - Always display the compile buffer at the same spot with a specific height. -- If all buffers are web browser windows, pop them up in a mosaic tiling. If -not, use a master-slave tiling. +- If all buffers are web browser windows, pop them up in a mosaic tiling. If + not, use a master-slave tiling. - Behave differently depending on the number of buffers or depending on the -content or the modes. + content or the modes. - Keep a column of "tools" to the right (calendar, calculator, etc.). @@ -268,28 +286,30 @@ It is so convenient to lookup buffers with EXWM+Helm that I've quit using workspaces (a.k.a. tags) altogether in favour of a more global approach. Maybe one the most noticeable benefit on a daily basis is that it lifts up some -weight off the _cognitive burden_ of having to manage windows both from Emacs -and from an external window manager. With EXWM, there is no more need to +weight off the /cognitive burden/ of having to manage windows both from Emacs +and from an external window manager. With EXWM, there is no more need to remember two sets of bindings and windowing rules, the windowing being -effectively fully centralized. For instance I used to reserve the super key +effectively fully centralized. For instance I used to reserve the =super= key for all windowing bindings with AwesomeWM; now I reserve it to all Emacs windowing operations, and there is no need for other bindings. Adepts of the suckless philosophy would argue that Emacs should leave all windowing, including its own, to an external specialized program, the window -manager. But from the perspective that Emacs is a user interface, it is not -that much of a heresy to consider that Emacs is precisely _that window manager_. +manager. But from the perspective that Emacs is a user interface, it is not that +much of a heresy to consider that Emacs is precisely /that window manager/. Having Emacs as a window manager has some additional benefits, namely that it is fully aware of the content of the Emacs buffer, which allows for specializing -windowing actions depending of the different buffers. This is much harder to do +windowing actions depending of the different buffers. This is much harder to do with any other window manager. +** Examples and list of programs + :PROPERTIES: + :CUSTOM_ID: examples-and-list-of-programs + :END: +- See my [[../][home page]] for a link to my dotfiles which implement much of + these ideas. -## Examples and list of programs - -- See my [home page](../) for a link to my dotfiles which implement much of these ideas. - -- I keep track of my [favourite power-user applications](../power-apps), including -commandline, text-mode and Emacs equivalents. +- I keep track of my [[../power-apps][favourite power-user applications]], + including commandline, text-mode and Emacs equivalents. diff --git a/articles/emacs.org b/articles/emacs.org index bf999e1d8de83ad7a52d8033dad161f23fe69583..b7c872ad05ed148a8e0f5a51399b9e0f0e668bec 100644 --- a/articles/emacs.org +++ b/articles/emacs.org @@ -1,16 +1,18 @@ -% Emacs Pro-tips +#+TITLE: Emacs Pro-tips Emacs is a wild beast that takes a while to tame. Read upon the official documentation, learn Elisp, check out for the best third-party packages and learn from other users. Here follows a selection of subtle best practices on how to set up Emacs as well -as a selection of my favorite Emacs features. See my [dotfiles][] for my -complete configuration. +as a selection of my favorite Emacs features. See my +[[https://github.com/ambrevar/dotfiles][dotfiles]] for my complete +configuration. -[dotfiles]: https://github.com/ambrevar/dotfiles - -# Emacsclient and Emacs server (daemon) +* Emacsclient and Emacs server (daemon) + :PROPERTIES: + :CUSTOM_ID: emacsclient-and-emacs-server-daemon + :END: With time the configuration may grow and Emacs can take a while to load, especially when using heavy packages. @@ -22,62 +24,65 @@ state between instances. The simplest way to start the daemon (if not already started) and the instances is to set your editor as this script: - sh -emacsclient -c -a "" "$@" - +#+BEGIN_SRC sh + emacsclient -c -a "" "$@" +#+END_SRC -The -a "" option lets Emacs start a daemon if none was already started. The +The =-a ""= option lets Emacs start a daemon if none was already started. The command will spawn a new Emacs frame and return immediately, which is usually -what we want; this has some shortcomings however. While forking Emacs in the +what we want; this has some shortcomings however. While forking Emacs in the background is generally a good idea, there are times when it should wait (writing e-mails with Mutt, git commit message, web browser fields), others when it should fork. We can create links to it and parameterize the waiting behaviour depending on the name of the script. Which leads us to the following script: -em forks and does not return while the emw and emc links (windowed and +=em= forks and does not return while the =emw= and =emc= links (windowed and console version respectively) return to their caller. - sh -if [ "${0##*/}" = "emc" ]; then - ## Force terminal mode - param="-t" -else - ## If Emacs cannot start in graphical mode, -c will act just like -t. - param="-c" - if [ "${0##*/}" != "emw" ] && [ -n "$DISPLAY" ] && [ "$(emacs --batch -Q --eval='(message (if (fboundp '"'"'tool-bar-mode) "X" "TTY"))' 2>&1)" = X ]; then - ## Don't wait if not called with "emw" and if Emacs can start in graphical mode. - ## The Emacs batch test checks whether it was compiled with GUI suppport. - param="$param -n" - fi -fi - -emacsclient "$param" -a "" "$@" - +#+BEGIN_SRC sh + if [ "${0##*/}" = "emc" ]; then + ## Force terminal mode + param="-t" + else + ## If Emacs cannot start in graphical mode, -c will act just like -t. + param="-c" + if [ "${0##*/}" != "emw" ] && [ -n "$DISPLAY" ] && [ "$(emacs --batch -Q --eval='(message (if (fboundp '"'"'tool-bar-mode) "X" "TTY"))' 2>&1)" = X ]; then + ## Don't wait if not called with "emw" and if Emacs can start in graphical mode. + ## The Emacs batch test checks whether it was compiled with GUI suppport. + param="$param -n" + fi + fi + + emacsclient "$param" -a "" "$@" +#+END_SRC The windowed version of Emacs is not restricted by any terminal limitation; nonetheless the console version has the advantage of not spawning a new window when already running in console. -You can then set your environment to your liking, e.g. export EDITOR=em and -VISUAL=emw: some programs use VISUAL and wait for it to return, while others +You can then set your environment to your liking, e.g. export =EDITOR=em= and +=VISUAL=emw=: some programs use VISUAL and wait for it to return, while others use EDITOR and need not waiting. This is barely a convention though and some applications will require local adjustments. -If you want to pass additional parameters to Emacs, create an emacs wrapper -and place it first in your PATH. For instance, say you want to use ---no-site-file to enforce a vanilla setting on invasive distributions, you can +If you want to pass additional parameters to Emacs, create an =emacs= wrapper +and place it first in your PATH. For instance, say you want to use +=--no-site-file= to enforce a vanilla setting on invasive distributions, you can write the following wrapper: - -#!/bin/sh -exec /usr/bin/emacs --no-site-file "$@" - +#+BEGIN_EXAMPLE + #!/bin/sh + exec /usr/bin/emacs --no-site-file "$@" +#+END_EXAMPLE This will apply to both the daemon and stand-alone instances. -# Initialization file architecture +* Initialization file architecture + :PROPERTIES: + :CUSTOM_ID: initialization-file-architecture + :END: It is also wise to reduce the loading time to a minimum, should you start some -daemon-independent Emacs instance (e.g. for development purposes). +daemon-independent Emacs instance (e.g. for development purposes). There is not one and only way to architecture Emacs initialization files, but surely so there are some good practices. My personal design tenets: @@ -94,39 +99,46 @@ To minimize the startup time, we need to lazy-load the configuration depending on the running modes. Everything that is not part of the global configuration can be conditionally loaded. -## Major-modes configuration package +** Major-modes configuration package + :PROPERTIES: + :CUSTOM_ID: major-modes-configuration-package + :END: -Every major-mode related configuration can be moved to its own configuration file -which can be loaded: +Every major-mode related configuration can be moved to its own configuration +file which can be loaded: -- just in time when it is needed thanks to with-eval-after-load, -- and only _once_ thanks to the require function. +- just in time when it is needed thanks to =with-eval-after-load=, +- and only /once/ thanks to the =require= function. -In practice, it boils down to a simple line in init.el, e.g. for the C mode: +In practice, it boils down to a simple line in =init.el=, e.g. for the C mode: - (with-eval-after-load 'cc-mode (require 'init-cc)) +#+BEGIN_EXAMPLE + (with-eval-after-load 'cc-mode (require 'init-cc)) +#+END_EXAMPLE -The first time a C buffer is created, the c-mode autoloaded function -requires the cc-mode.el. When this one is loaded, the -with-eval-after-load kicks in and requires our additional init-cc. -The form of with-eval-after-load is evaluated everytime a C buffer is loaded, -thus it is important to rely on require instead of load so that we load our +The first time a C buffer is created, the =c-mode= autoloaded function +=require=s the =cc-mode.el=. When this one is loaded, the =with-eval-after-load= +kicks in and =requires= our additional =init-cc=. The form of +=with-eval-after-load= is evaluated everytime a C buffer is loaded, thus it is +important to rely on =require= instead of =load= so that we load our configuration only once. -The init-cc.el file should contain a C-specific global configuration: +The =init-cc.el= file should contain a C-specific global configuration: variables, function definitions, skeletons, etc. - (setq semanticdb-default-save-directory (concat emacs-cache-folder "semanticdb")) - (semantic-mode 1) - (local-set-key (kbd "") (recompile)) - ... +#+BEGIN_EXAMPLE + (setq semanticdb-default-save-directory (concat emacs-cache-folder "semanticdb")) + (semantic-mode 1) + (local-set-key (kbd "") (recompile)) + ... - ; Need to end with provide' so that require' does not load the file twice. - (provide 'init-cc) + ; Need to end with provide' so that require' does not load the file twice. + (provide 'init-cc) +#+END_EXAMPLE -Note that local-set-key generally sets the mode map globally and is _not_ +Note that =local-set-key= generally sets the mode map globally and is /not/ buffer-local. If it is, it means that the mode is not using the standard mode -API or it hasn't called use-local-map. You should probably report the issue +API or it hasn't called =use-local-map=. You should probably report the issue upstream. Some of your configuration might need to be buffer-local, in which case you must @@ -134,54 +146,63 @@ add it to the mode hook. Cluttering hooks will slow down buffer creation and can become a source of confusion, so it is advised to stick to only what requires a hook. - (defun go-setup () - (setq indent-tabs-mode t) - (set (make-local-variable 'compile-command) (concat "go run " (shell-quote-argument buffer-file-name))) - (add-hook 'before-save-hook #'gofmt-before-save nil t)) - (add-hook go-mode-hook go-setup) +#+BEGIN_EXAMPLE + (defun go-setup () + (setq indent-tabs-mode t) + (set (make-local-variable 'compile-command) (concat "go run " (shell-quote-argument buffer-file-name))) + (add-hook 'before-save-hook #'gofmt-before-save nil t)) + (add-hook go-mode-hook go-setup) +#+END_EXAMPLE This last example shows three types of relevant hook use: - Set a buffer-local variable. (Those variables whose documentation shows -"Automatically becomes buffer-local when set.", like indent-tabs-mode). If not -added to a hook, the change would apply to the current buffer only. Global -variables can be permanently made buffer-local with the -make-variable-buffer-local command. + "Automatically becomes buffer-local when set.", like =indent-tabs-mode=). If + not added to a hook, the change would apply to the current buffer only. Global + variables can be permanently made buffer-local with the + =make-variable-buffer-local= command. - Set a variable to be buffer-local for this mode only and set its value. -compile-command is global by default: making it buffer-local in the mode hook -allows for setting different compile commands for the various buffers in this -mode while other modes will keep dealing with a global compile command. + =compile-command= is global by default: making it buffer-local in the mode + hook allows for setting different compile commands for the various buffers in + this mode while other modes will keep dealing with a global compile command. -- Make a buffer-local change to a hook thanks to the LOCAL parameter of the -add-hook function. Adding this hook change to the mode hook will effectively -apply the hook change to all buffers in this mode while leaving it untouched for -other modes. +- Make a buffer-local change to a hook thanks to the =LOCAL= parameter of the + =add-hook= function. Adding this hook change to the mode hook will effectively + apply the hook change to all buffers in this mode while leaving it untouched + for other modes. Last but not least, refrain from using lambdas in hooks: it makes the documentation and the intention harder to understand, while making it much -trickier to use the remove-hook function, should you need to alter the hook at +trickier to use the =remove-hook= function, should you need to alter the hook at runtime. -## Package management +** Package management + :PROPERTIES: + :CUSTOM_ID: package-management + :END: Third party packages, major modes or not, can be loaded similarly depending on their availability: if the package is not installed, there is no need to parse -it's configuration. The procedure is the same: +it's configuration. The procedure is the same: - (with-eval-after-load 'lua-mode (require 'init-lua)) +#+BEGIN_EXAMPLE + (with-eval-after-load 'lua-mode (require 'init-lua)) +#+END_EXAMPLE If you want to make a mode immediately available on start-up: - (when (require 'helm-config nil t) (require 'init-helm)) - +#+BEGIN_EXAMPLE + (when (require 'helm-config nil t) (require 'init-helm)) +#+END_EXAMPLE -# Helm +* Helm + :PROPERTIES: + :CUSTOM_ID: helm + :END: -[Helm][] is a UI revolution: It will add incremental narrowing (fuzzy) -search to... everything! - -[helm]: https://emacs-helm.github.io/helm/ +[[https://emacs-helm.github.io/helm/][Helm]] is a UI revolution: It will add +incremental narrowing (fuzzy) search to... everything! The concept: instead of listing and selecting, it will list and narrow down as you type, while sorting by the most relevant results first. Beside, the search @@ -189,69 +210,71 @@ can be fuzzy, which makes it practical to find things when you do not know the exact name. You can lookup buffers, commands, documentation, files, and more: pretty much -anything that requires a _lookup_. See this [article][helmintro] for a more -exhaustive presentation. - -[helmintro]: https://tuhdo.github.io/helm-intro.html +anything that requires a /lookup/. See this +[[https://tuhdo.github.io/helm-intro.html][article]] for a more exhaustive +presentation. The one killer feature is the ability to search text in your whole project or -file tree. Helm comes with a few _greppers_: grep itself, but it also supports -the current version control grepper and other tools such as [ag][] and [pt][]. +file tree. Helm comes with a few /greppers/: grep itself, but it also supports +the current version control grepper and other tools such as +[[http://geoff.greer.fm/ag/][ag]] and +[[https://github.com/monochromegane/the_platinum_searcher][pt]]. -[ag]: http://geoff.greer.fm/ag/ -[pt]: https://github.com/monochromegane/the_platinum_searcher - -The VCS grepper is usually faster than grep. I have set the bindings to use -the VCS grepper first and to fallback to pt when no file in the current folder +The VCS grepper is usually faster than =grep=. I have set the bindings to use +the VCS grepper first and to fallback to =pt= when no file in the current folder is versioned: - (defun call-process-to-string (program &rest args) - "Call PROGRAM with ARGS and return output." - (with-output-to-string - (with-current-buffer - standard-output - (apply 'call-process program nil t nil args)))) - - (defun helm-grep-git-or-ag (arg) - "Run helm-grep-do-git-grep' if possible; fallback to helm-do-grep-ag' otherwise." - (interactive "P") - (require 'vc) - (if (and (vc-find-root default-directory ".git") - (or arg (split-string (call-process-to-string "git" "ls-files" "-z") "\0" t))) - (helm-grep-do-git-grep arg) - (helm-do-grep-ag arg))) - - (define-key mickey-minor-mode-map (kbd "C-x G") 'helm-grep-git-or-ag) - +#+BEGIN_EXAMPLE + (defun call-process-to-string (program &rest args) + "Call PROGRAM with ARGS and return output." + (with-output-to-string + (with-current-buffer + standard-output + (apply 'call-process program nil t nil args)))) + + (defun helm-grep-git-or-ag (arg) + "Run helm-grep-do-git-grep' if possible; fallback to helm-do-grep-ag' otherwise." + (interactive "P") + (require 'vc) + (if (and (vc-find-root default-directory ".git") + (or arg (split-string (call-process-to-string "git" "ls-files" "-z") "\0" t))) + (helm-grep-do-git-grep arg) + (helm-do-grep-ag arg))) + + (define-key mickey-minor-mode-map (kbd "C-x G") 'helm-grep-git-or-ag) +#+END_EXAMPLE Other features of Helm: - Lookup global variables and functions in current buffer with -helm-semantic-or-imenu, or for all buffers with helm-imenu-in-all-buffers. + =helm-semantic-or-imenu=, or for all buffers with =helm-imenu-in-all-buffers=. -- To enable proper fuzzy finding when finding files recursively (helm-find), -set helm-findutils-search-full-path to non-nil. +- To enable proper fuzzy finding when finding files recursively (=helm-find=), + set =helm-findutils-search-full-path= to non-nil. -- Lookup files in the Git project with third-party helm-ls-git. +- Lookup files in the Git project with third-party =helm-ls-git=. -- Use yank to lookup last region. +- Use =yank= to lookup last region. -- Use the universal argument to include more to your lookup (e.g. subfolders). +- Use the universal argument to include more to your lookup (e.g. subfolders). -- Use C-c C-f to activate follow mode and navigate through the results to -display a complete context. +- Use =C-c C-f= to activate follow mode and navigate through the results to + display a complete context. -- Save some helm sessions with C-x C-s for later re-use. Edit grep buffers -with wgrep and apply the changes all at once. +- Save some helm sessions with =C-x C-s= for later re-use. Edit =grep= buffers + with =wgrep= and apply the changes all at once. -- Or resume last helm session with C-x c b. +- Or resume last helm session with =C-x c b=. -- I like to replace M-s o with helm-occur, C-x C-x with -helm-all-mark-rings, M-y with helm-show-kill-ring, etc. +- I like to replace =M-s o= with =helm-occur=, =C-x C-x= with + =helm-all-mark-rings=, =M-y= with =helm-show-kill-ring=, etc. -- Lookup completion suggestions with helm-company. +- Lookup completion suggestions with =helm-company=. -# Update to latest Emacs version +* Update to latest Emacs version + :PROPERTIES: + :CUSTOM_ID: update-to-latest-emacs-version + :END: You might like Emacs enough you want it everywhere. And yet sometimes you are forced to use an outdated, crappy system on which you have no administrative @@ -263,94 +286,120 @@ and packages rely on recent Emacs. Thankfully it's easy enough to compile the latest Emacs thanks to its extreme portability and can be installed within the user home folder. -# Cache folder +* Cache folder + :PROPERTIES: + :CUSTOM_ID: cache-folder + :END: (Or how to keep your configuration folder clean.) -Many modes store their cache files in ~/.emacs.d. I prefer to keep those -ephemeral files in ~/.cache/emacs. +Many modes store their cache files in =~/.emacs.d=. I prefer to keep those +ephemeral files in =~/.cache/emacs=. - (defvar emacs-cache-folder "~/.cache/emacs" - "Cache folder is everything we do not want to track along with - the configuration files.") - (if (not (file-directory-p emacs-cache-folder)) - (make-directory emacs-cache-folder t)) +#+BEGIN_EXAMPLE + (defvar emacs-cache-folder "~/.cache/emacs" + "Cache folder is everything we do not want to track along with + the configuration files.") + (if (not (file-directory-p emacs-cache-folder)) + (make-directory emacs-cache-folder t)) +#+END_EXAMPLE Some common settings: - (setq save-place-file (expand-file-name "saveplace" emacs-cache-folder)) - (setq url-cookie-file (expand-file-name "url.cookies" emacs-cache-folder)) - (setq bookmark-default-file (expand-file-name "emacs.bmk" emacs-cache-folder)) - (setq recentf-save-file (expand-file-name "recentf" emacs-cache-folder)) - (setq backup-directory-alist ((".*" . ,(expand-file-name "backups/" emacs-cache-folder)))) - (setq eshell-directory-name (expand-file-name "eshell" emacs-cache-folder)) - -The cache folder must exist for desktop-mode: - - (setq desktop-dirname (expand-file-name "desktop" emacs-cache-folder)) - (unless (file-directory-p desktop-dirname) - (make-directory desktop-dirname t)) - (setq desktop-path (,desktop-dirname)) - -Semantic must be started _after_ setting the db folder: - - (setq semanticdb-default-save-directory (expand-file-name "semanticdb" emacs-cache-folder)) - (semantic-mode 1) - -# Streamline indentation - -I think Emacs has too many options for indentation. Since I have a -strong [opinion on always using tabs to indent](../indentation/index.html) -(except for Lisp), I "redirect" with defvaralias the mode-specific indentation -levels to only one variable, namely tab-width. - - (defvaralias 'standard-indent 'tab-width) - (setq-default indent-tabs-mode t) - - ;; Lisp should not use tabs. - (mapcar - (lambda (hook) - (add-hook - hook - (lambda () - (setq indent-tabs-mode nil)))) - '(lisp-mode-hook emacs-lisp-mode-hook)) - - ;; This needs to be set globally since they are defined as local variables and - ;; Emacs does not know how to set an alias on a local variable. - (defvaralias 'c-basic-offset 'tab-width) - (defvaralias 'sh-basic-offset 'tab-width) - -Add the following to sh-mode-hook: - - (defvaralias 'sh-indentation 'sh-basic-offset) - -The case of _C_ and _sh_ are special for historical reasons. -Other modes can be corrected as follows: - - (defvaralias 'js-indent-level 'tab-width) - (defvaralias 'lua-indent-level 'tab-width) - (defvaralias 'perl-indent-level 'tab-width) - -# Elisp 'go to definition' - -Elisp has the find-variable-at-point and the find-function-at-point -functions, yet it does not have a proper go to definition command. Not for +#+BEGIN_EXAMPLE + (setq save-place-file (expand-file-name "saveplace" emacs-cache-folder)) + (setq url-cookie-file (expand-file-name "url.cookies" emacs-cache-folder)) + (setq bookmark-default-file (expand-file-name "emacs.bmk" emacs-cache-folder)) + (setq recentf-save-file (expand-file-name "recentf" emacs-cache-folder)) + (setq backup-directory-alist ((".*" . ,(expand-file-name "backups/" emacs-cache-folder)))) + (setq eshell-directory-name (expand-file-name "eshell" emacs-cache-folder)) +#+END_EXAMPLE + +The cache folder must exist for =desktop-mode=: + +#+BEGIN_EXAMPLE + (setq desktop-dirname (expand-file-name "desktop" emacs-cache-folder)) + (unless (file-directory-p desktop-dirname) + (make-directory desktop-dirname t)) + (setq desktop-path (,desktop-dirname)) +#+END_EXAMPLE + +Semantic must be started /after/ setting the db folder: + +#+BEGIN_EXAMPLE + (setq semanticdb-default-save-directory (expand-file-name "semanticdb" emacs-cache-folder)) + (semantic-mode 1) +#+END_EXAMPLE + +* Streamline indentation + :PROPERTIES: + :CUSTOM_ID: streamline-indentation + :END: + +I think Emacs has too many options for indentation. Since I have a strong +[[../indentation/index.html][opinion on always using tabs to indent]] (except +for Lisp), I "redirect" with =defvaralias= the mode-specific indentation levels +to only one variable, namely =tab-width=. + +#+BEGIN_EXAMPLE + (defvaralias 'standard-indent 'tab-width) + (setq-default indent-tabs-mode t) + + ;; Lisp should not use tabs. + (mapcar + (lambda (hook) + (add-hook + hook + (lambda () + (setq indent-tabs-mode nil)))) + '(lisp-mode-hook emacs-lisp-mode-hook)) + + ;; This needs to be set globally since they are defined as local variables and + ;; Emacs does not know how to set an alias on a local variable. + (defvaralias 'c-basic-offset 'tab-width) + (defvaralias 'sh-basic-offset 'tab-width) +#+END_EXAMPLE + +Add the following to =sh-mode-hook=: + +#+BEGIN_EXAMPLE + (defvaralias 'sh-indentation 'sh-basic-offset) +#+END_EXAMPLE + +The case of /C/ and /sh/ are special for historical reasons. Other modes can be +corrected as follows: + +#+BEGIN_EXAMPLE + (defvaralias 'js-indent-level 'tab-width) + (defvaralias 'lua-indent-level 'tab-width) + (defvaralias 'perl-indent-level 'tab-width) +#+END_EXAMPLE + +* Elisp 'go to definition' + :PROPERTIES: + :CUSTOM_ID: elisp-go-to-definition + :END: + +Elisp has the =find-variable-at-point= and the =find-function-at-point= +functions, yet it does not have a proper =go to definition= command. Not for long: - -(define-key lisp-mode-shared-map "\M-." 'find-symbol-at-point) +#+BEGIN_EXAMPLE + (define-key lisp-mode-shared-map "\M-." 'find-symbol-at-point) -(defun find-symbol-at-point () - "Find directly the symbol at point, i.e. go to definition." - (interactive) - (let ((sym (symbol-at-point))) - (if (boundp sym) - (find-variable sym) - (find-function sym)))) - + (defun find-symbol-at-point () + "Find directly the symbol at point, i.e. go to definition." + (interactive) + (let ((sym (symbol-at-point))) + (if (boundp sym) + (find-variable sym) + (find-function sym)))) +#+END_EXAMPLE -# Smart compilation +* Smart compilation + :PROPERTIES: + :CUSTOM_ID: smart-compilation + :END: Emacs has a compilation mode that comes in very handy to run arbitrary commands over your buffer and navigate the errors back to the source code. @@ -359,12 +408,12 @@ It is not only useful for compilers but also for, say, browsing your own programs' debug messages, a linter, etc. Emacs standard behaviour is to store the last used compile command in the global -variable compile-command. Similarly, compile-history remembers all compile +variable =compile-command=. Similarly, =compile-history= remembers all compile commands used globally. This is useful if you jump from buffer to buffer and want to run the same compile command for your project without having to switch back to a specific buffer. -Another approach is to make compile-command buffer-local. You'll have to be in +Another approach is to make =compile-command= buffer-local. You'll have to be in a specific buffer to run the desired command. In practice, I find myself having to run several buffer-specific commands per project (documentation, linting, library builds, executable builds, etc.). @@ -372,177 +421,191 @@ library builds, executable builds, etc.). To use the buffer-local approach, add this to your init file before configuring the modes: - (eval-after-load 'compile (make-variable-buffer-local 'compile-command)) +#+BEGIN_EXAMPLE + (eval-after-load 'compile (make-variable-buffer-local 'compile-command)) +#+END_EXAMPLE The compile command can be modified per buffer upon request. If you use the -desktop mode to save your session, each buffer's command can be restored as +=desktop= mode to save your session, each buffer's command can be restored as well. - (add-to-list 'desktop-locals-to-save 'compile-command) +#+BEGIN_EXAMPLE + (add-to-list 'desktop-locals-to-save 'compile-command) +#+END_EXAMPLE Emacs provides two compilation commands: -- (compile COMMAND &optional COMINT) prompts for the command to run when -called interactively. A user-defined command can do that with -(call-interactively 'compile). Make compile-command local to the function -scope if you want to run a temporary command. +- =(compile COMMAND &optional COMINT)= prompts for the command to run when + called interactively. A user-defined command can do that with + =(call-interactively 'compile)=. Make =compile-command= local to the function + scope if you want to run a temporary command. -- (recompile &optional EDIT-COMMAND) is handy to recall last command without -prompting the user. It has some shortcomings when using a buffer-local -compile-command: +- =(recompile &optional EDIT-COMMAND)= is handy to recall last command without + prompting the user. It has some shortcomings when using a buffer-local + =compile-command=: - - compile-history remains untouched unless we do some manual bookkeeping. + - =compile-history= remains untouched unless we do some manual bookkeeping. - - It uses a global compilation-directory, thus calling recompile in - another buffer will fail if the target file is in a different folder. We can - make that variable buffer-local, but that would only work if we never use - compile. In such a scenario, compile-history is unused. + - It uses a global =compilation-directory=, thus calling =recompile= in + another buffer will fail if the target file is in a different folder. We can + make that variable buffer-local, but that would only work if we never use + =compile=. In such a scenario, =compile-history= is unused. -In short: when using a buffer-local compile-command, we are better off -sticking to compile and leaving recompile aside. +In short: when using a buffer-local =compile-command=, we are better off +sticking to =compile= and leaving =recompile= aside. Let's add some bindings for convenience: - (defun compile-last-command () (interactive) (compile compile-command)) - (global-set-key (kbd "C-") 'compile) - (global-set-key (kbd "") 'compile-last-command) +#+BEGIN_EXAMPLE + (defun compile-last-command () (interactive) (compile compile-command)) + (global-set-key (kbd "C-") 'compile) + (global-set-key (kbd "") 'compile-last-command) +#+END_EXAMPLE -Here follows a complete example for C: it will look for the closest Makefile -in the parent folders and set the command to make -C /path/to/makefile or else +Here follows a complete example for C: it will look for the closest =Makefile= +in the parent folders and set the command to =make -C /path/to/makefile= or else fallback on some dynamically set values depending on the language (C or C++) and the environment (GCC, Clang, etc.). The linker flags are configurable on a -per-buffer basis thanks to the buffer-local cc-ldlibs and cc-ldflags variables. - - -(defvar-local cc-ldlibs "-lm -pthread" - "Custom linker flags for C/C++ linkage.") - -(defvar-local cc-ldflags "" - "Custom linker libs for C/C++ linkage.") - -(defun cc-set-compiler (&optional nomakefile) - "Set compile command to be nearest Makefile or a generic command. -The Makefile is looked up in parent folders. If no Makefile is -found (or if NOMAKEFILE is non-nil or if function was called with -universal argument), then a configurable commandline is -provided." - (interactive "P") - (hack-local-variables) - ;; Alternatively, if a Makefile is found, we could change default directory - ;; and leave the compile command to "make". Changing default-directory' - ;; could have side effects though. - (let ((makefile-dir (locate-dominating-file "." "Makefile"))) - (if (and makefile-dir (not nomakefile)) - (setq compile-command (concat "make -k -C " (shell-quote-argument (file-name-directory makefile-dir)))) - (setq compile-command - (let - ((c++-p (eq major-mode 'c++-mode)) - (file (file-name-nondirectory buffer-file-name))) - (format "%s %s -o '%s' %s %s %s" - (if c++-p - (or (getenv "CXX") "g++") - (or (getenv "CC") "gcc")) - (shell-quote-argument file) - (shell-quote-argument (file-name-sans-extension file)) - (if c++-p - (or (getenv "CXXFLAGS") "-Wall -Wextra -Wshadow -DDEBUG=9 -g3 -O0") - (or (getenv "CFLAGS") "-ansi -pedantic -std=c11 -Wall -Wextra -Wshadow -DDEBUG=9 -g3 -O0")) - (or (getenv "LDFLAGS") cc-ldflags) - (or (getenv "LDLIBS") cc-ldlibs))))))) - -(defun cc-clean () - "Find Makefile and call the clean' rule. If no Makefile is -found, no action is taken. The previous compile' command is -restored." - (interactive) - (let (compile-command - (makefile-dir (locate-dominating-file "." "Makefile"))) - (when makefile-dir - (compile (format "make -k -C %s clean" (shell-quote-argument makefile-dir)))))) - -(dolist (map (list c-mode-map c++-mode-map)) - (define-key map "" 'cc-clean)) - -(dolist (hook '(c-mode-hook c++-mode-hook)) - (add-hook hook 'cc-set-compiler)) - - -# C pretty format - -I use [uncrustify][] to format my C code automatically. See my -[indentation rationale](../indentation/index.html). - -[uncrustify]: http://uncrustify.sourceforge.net/ +per-buffer basis thanks to the buffer-local =cc-ldlibs= and =cc-ldflags= +variables. + +#+BEGIN_EXAMPLE + (defvar-local cc-ldlibs "-lm -pthread" + "Custom linker flags for C/C++ linkage.") + + (defvar-local cc-ldflags "" + "Custom linker libs for C/C++ linkage.") + + (defun cc-set-compiler (&optional nomakefile) + "Set compile command to be nearest Makefile or a generic command. + The Makefile is looked up in parent folders. If no Makefile is + found (or if NOMAKEFILE is non-nil or if function was called with + universal argument), then a configurable commandline is + provided." + (interactive "P") + (hack-local-variables) + ;; Alternatively, if a Makefile is found, we could change default directory + ;; and leave the compile command to "make". Changing default-directory' + ;; could have side effects though. + (let ((makefile-dir (locate-dominating-file "." "Makefile"))) + (if (and makefile-dir (not nomakefile)) + (setq compile-command (concat "make -k -C " (shell-quote-argument (file-name-directory makefile-dir)))) + (setq compile-command + (let + ((c++-p (eq major-mode 'c++-mode)) + (file (file-name-nondirectory buffer-file-name))) + (format "%s %s -o '%s' %s %s %s" + (if c++-p + (or (getenv "CXX") "g++") + (or (getenv "CC") "gcc")) + (shell-quote-argument file) + (shell-quote-argument (file-name-sans-extension file)) + (if c++-p + (or (getenv "CXXFLAGS") "-Wall -Wextra -Wshadow -DDEBUG=9 -g3 -O0") + (or (getenv "CFLAGS") "-ansi -pedantic -std=c11 -Wall -Wextra -Wshadow -DDEBUG=9 -g3 -O0")) + (or (getenv "LDFLAGS") cc-ldflags) + (or (getenv "LDLIBS") cc-ldlibs))))))) + + (defun cc-clean () + "Find Makefile and call the clean' rule. If no Makefile is + found, no action is taken. The previous compile' command is + restored." + (interactive) + (let (compile-command + (makefile-dir (locate-dominating-file "." "Makefile"))) + (when makefile-dir + (compile (format "make -k -C %s clean" (shell-quote-argument makefile-dir)))))) + + (dolist (map (list c-mode-map c++-mode-map)) + (define-key map "" 'cc-clean)) + + (dolist (hook '(c-mode-hook c++-mode-hook)) + (add-hook hook 'cc-set-compiler)) +#+END_EXAMPLE + +* C pretty format + :PROPERTIES: + :CUSTOM_ID: c-pretty-format + :END: + +I use [[http://uncrustify.sourceforge.net/][uncrustify]] to format my C code +automatically. See my [[../indentation/index.html][indentation rationale]]. I can call it from Emacs with the following function: - -(defun cc-fmt () - "Run uncrustify(1) on current buffer or region." - (interactive) - (let (status - start end - (formatbuf (get-buffer-create "*C format buffer*"))) - (if (use-region-p) - (setq start (region-beginning) end (region-end)) - (setq start (point-min) end (point-max))) - (setq status - (call-process-region start end "uncrustify" nil formatbuf nil "-lc" "-q" "-c" (concat (getenv "HOME") "/.uncrustify.cfg"))) - (if (/= status 0) - (error "error running uncrustify") - (delete-region start end) - (insert-buffer formatbuf) - (kill-buffer formatbuf)))) - +#+BEGIN_EXAMPLE + (defun cc-fmt () + "Run uncrustify(1) on current buffer or region." + (interactive) + (let (status + start end + (formatbuf (get-buffer-create "*C format buffer*"))) + (if (use-region-p) + (setq start (region-beginning) end (region-end)) + (setq start (point-min) end (point-max))) + (setq status + (call-process-region start end "uncrustify" nil formatbuf nil "-lc" "-q" "-c" (concat (getenv "HOME") "/.uncrustify.cfg"))) + (if (/= status 0) + (error "error running uncrustify") + (delete-region start end) + (insert-buffer formatbuf) + (kill-buffer formatbuf)))) +#+END_EXAMPLE We could add this to the save hook to auto-format my code at all times, but that would be bad practice when working with source code using different formatting rules. -# Magit - -[Magit][] makes Git management a bliss. The most evident feature would be the -easy hunk selection when staging code. This simple feature together with a few -others will make a drastic change to your workflow. +* Magit + :PROPERTIES: + :CUSTOM_ID: magit + :END: -[magit]: https://magit.vc/ +[[https://magit.vc/][Magit]] makes Git management a bliss. The most evident +feature would be the easy hunk selection when staging code. This simple feature +together with a few others will make a drastic change to your workflow. -# Multiple cursors +* Multiple cursors + :PROPERTIES: + :CUSTOM_ID: multiple-cursors + :END: -See this [video][mc] for a short introduction of this very powerful editing -framework. - -[mc]: http://emacsrocks.com/e13.html +See this [[http://emacsrocks.com/e13.html][video]] for a short introduction of +this very powerful editing framework. As of Septermber 2016, multiple cursors does not support searching, so I use -phi-search that automatically adds support to it. - - -(when (require 'multiple-cursors nil t) - (setq mc/list-file (concat emacs-cache-folder "mc-lists.el")) - ;; Load the file at the new location. - (load mc/list-file t) - (global-unset-key (kbd "C-")) - (define-key mickey-minor-mode-map (kbd "C-") 'mc/add-cursor-on-click) - (define-key mickey-minor-mode-map (kbd "C-x M-r") 'mc/edit-lines) - (define-key mickey-minor-mode-map (kbd "C-x M-m") 'mc/mark-more-like-this-extended) - (define-key mickey-minor-mode-map (kbd "C-x M-l") 'mc/mark-all-like-this-dwim) - ;; Search compatible with mc. - (require 'phi-search nil t)) - - -If you are an Evil user, multiple-cursors will not work. Use the -dedicated evil-mc instead. - -# Org Mode - -Last but not least, the famous [Org Mode][orgmode]. It offers some impressive -features, such as seamless table manipulation (swap columns with a keystroke...) -and formula computation. From the manual: - - Finally, just to whet your appetite for what can be done with the - fantastic calc.el' package, here is a table that computes the Taylor - series of degree n' at location x' for a couple of functions. +=phi-search= that automatically adds support to it. + +#+BEGIN_EXAMPLE + (when (require 'multiple-cursors nil t) + (setq mc/list-file (concat emacs-cache-folder "mc-lists.el")) + ;; Load the file at the new location. + (load mc/list-file t) + (global-unset-key (kbd "C-")) + (define-key mickey-minor-mode-map (kbd "C-") 'mc/add-cursor-on-click) + (define-key mickey-minor-mode-map (kbd "C-x M-r") 'mc/edit-lines) + (define-key mickey-minor-mode-map (kbd "C-x M-m") 'mc/mark-more-like-this-extended) + (define-key mickey-minor-mode-map (kbd "C-x M-l") 'mc/mark-all-like-this-dwim) + ;; Search compatible with mc. + (require 'phi-search nil t)) +#+END_EXAMPLE + +If you are an Evil user, =multiple-cursors= will not work. Use the dedicated +=evil-mc= instead. + +* Org Mode + :PROPERTIES: + :CUSTOM_ID: org-mode + :END: + +Last but not least, the famous [[http://orgmode.org/][Org Mode]]. It offers some +impressive features, such as seamless table manipulation (swap columns with a +keystroke...) and formula computation. From the manual: + +#+BEGIN_EXAMPLE + Finally, just to whet your appetite for what can be done with the + fantastic calc.el' package, here is a table that computes the Taylor + series of degree n' at location x' for a couple of functions. |---+-------------+---+-----+--------------------------------------| | | Func | n | x | Result | @@ -555,6 +618,7 @@ and formula computation. From the manual: | * | tan(x) | 3 | x | 0.0175 x + 1.77e-6 x^3 | |---+-------------+---+-----+--------------------------------------| #+TBLFM: $5=taylor($2,$4,$3);n3 +#+END_EXAMPLE Note that the last column is computed automatically! Formulae can be computed using the Calc mode, Elisp, or even external programs such as R or PARI/GP. @@ -562,10 +626,10 @@ Possibilities are endless. Finally, you can export the end result to LaTeX, HTML, etc. -[orgmode]: http://orgmode.org/ - - -# References +* References + :PROPERTIES: + :CUSTOM_ID: references + :END: Aggregator wikis: diff --git a/articles/filesystems.org b/articles/filesystems.org index 9c4ef129da7f7b20221679812be43f018d832e24..6d6c12ae7f37699234d1ae1d198f5698fc890f31 100644 --- a/articles/filesystems.org +++ b/articles/filesystems.org @@ -1,4 +1,4 @@ -% Filesystems unraveled +#+TITLE: Filesystems unraveled A filesystem is the structure of some data storage. They allow for storing file hierarchically (folders), remembering metadata (timestamps, owners, etc.). @@ -7,65 +7,76 @@ The mechanics lying behind filesystems is often misunderstood. As a consequence, installing an operating systems is often perceived as a complex operation. A few enlightening explanations might help a great deal to alleviate this fear. - -# Structure +* Structure + :PROPERTIES: + :CUSTOM_ID: structure + :END: The most important thing to grasp is that every computer storage device, from a hardware point of view, is a continuous segment of memory. The way data gets organized by partitions, folders, metadata, etc. is defined logically by tools and operating systems. -## Boot sectors and partition tables +** Boot sectors and partition tables + :PROPERTIES: + :CUSTOM_ID: boot-sectors-and-partition-tables + :END: -There are 2 types of boot sectors: the legacy _Master Boot Record_ (MBR) and the -newer _GUID Partition Table_ (GPT). GPT has less limitations in regard to the +There are 2 types of boot sectors: the legacy /Master Boot Record/ (MBR) and the +newer /GUID Partition Table/ (GPT). GPT has less limitations in regard to the number and the size of the partitions. The boot sector and the partition table typically reside at the very beginning of the disk. They are not on any partition. This would not make sense since the partition table defines the partition layout. On Linux, hard disk drives are -typically referenced by the path /dev/sdX, where X is a letter, and their -partitions by the path /dev/sdXN, where N a number. +typically referenced by the path =/dev/sdX=, where X is a letter, and their +partitions by the path =/dev/sdXN=, where N a number. The OS and programs running on it identify partitions by the sector address (or -_logical block address_, a.k.a. LBA) stored in the partition table. The standard +/logical block address/, a.k.a. LBA) stored in the partition table. The standard starting sector for partitions is at byte 2048. In the past, the first partition used to be written at byte 63, which may cause performance issues since it does not align with the physical sector size of the drive. See the references for more explanations. Tools for creating and manipulating the boot sector and the partition table -include dd, syslinux, grub, fdisk, gdisk and more. A tool like fdisk +include =dd=, =syslinux=, =grub=, =fdisk=, =gdisk= and more. A tool like =fdisk= will manipulate the partition table found at the beginning of the designated -storage media. As such, it usually only makes sense to call fdisk over a hard -disk drive, such as fdisk /dev/sdX, and not over a partition. +storage media. As such, it usually only makes sense to call =fdisk= over a hard +disk drive, such as =fdisk /dev/sdX=, and not over a partition. -The MBR has different partition types: _primary_, _extended_ and _logical_. See +The MBR has different partition types: /primary/, /extended/ and /logical/. See this -[Arch Wiki article](https://wiki.archlinux.org/index.php/Partitioning#Master_Boot_Record) -for more details. +[[https://wiki.archlinux.org/index.php/Partitioning#Master_Boot_Record][Arch +Wiki article]] for more details. GPT has only one partition type. -## Partitions +** Partitions + :PROPERTIES: + :CUSTOM_ID: partitions + :END: Partitions need to be initialized before they can be used by the OS, that is, -the _header_ must be created. This header has different names depending on the -filesystem type (e.g. _table of content_, _superblock_). The header will usually +the /header/ must be created. This header has different names depending on the +filesystem type (e.g. /table of content/, /superblock/). The header will usually occupy the first sectors of the partition. -Tools such as mkfs can be used to initialize partitions. +Tools such as =mkfs= can be used to initialize partitions. As mentioned before, partitions are purely logical: it is possible to write data -across partitions with dd. Although that would probably destroy the logical +across partitions with =dd=. Although that would probably destroy the logical integrity of some partitions. If you remove the partition entry N, then partition N won't exist in the eyes of -the OS. But dd can force reading data at any position on disk, and thus +the OS. But =dd= can force reading data at any position on disk, and thus recover data from lost partitions. If you re-add the partition entry with the same LBA addresses, then the partition will be accessible just like before. -## Bootloaders +** Bootloaders + :PROPERTIES: + :CUSTOM_ID: bootloaders + :END: The bootloader is a program that resides partly on a partition and partly on the boot sector. @@ -75,59 +86,67 @@ MBR or a GPT in the first sectors and run the executable code of the boot loader. This code can be configured to boot an OS located at a specific partition. - -# Disk usage and apparent size +* Disk usage and apparent size + :PROPERTIES: + :CUSTOM_ID: disk-usage-and-apparent-size + :END: Every file on the system has 2 "size" properties: the disk usage and the apparent size. The apparent size is the number of bytes contained in a file. It represents the information held by the file, and as such it is the same across different file -systems. It can be queried with ls -l or du -b (GNU) / du -A (BSD). +systems. It can be queried with =ls -l= or =du -b= (GNU) / =du -A= (BSD). -Disk usage is highly dependent on file systems. It can be queried with ls -s -or du. Disk usage accounts for several properties of the file: +Disk usage is highly dependent on file systems. It can be queried with =ls -s= +or =du=. Disk usage accounts for several properties of the file: -- A file has attributes on the filesystem (e.g. timestamp, owner, etc.). Thus -it usually requires some additional bytes. +- A file has attributes on the filesystem (e.g. timestamp, owner, etc.). Thus it + usually requires some additional bytes. - A file can be fragmented, have indirect blocks, have unused space in some -blocks, and the like. + blocks, and the like. -- A file can be _sparse_. It means that it has big chunks of zeros. Modern -filesystems make use of this property to save space, that is, they do not write -the zeros on the disk and only tell the filesystem that _there are zeros from -byte M to byte N_. +- A file can be /sparse/. It means that it has big chunks of zeros. Modern + filesystems make use of this property to save space, that is, they do not + write the zeros on the disk and only tell the filesystem that /there are zeros + from byte M to byte N/. The disk usage is usually higher than the apparent size because of metadata and fragmentation, but it can also be smaller if the file is sparse. Let's experiment: - $dd of=sparse-file bs=1k seek=5120 count=0 - 0+0 records in - 0+0 records out - 0 bytes (0 B) copied, 5.668e-05 s, 0.0 kB/s +#+BEGIN_EXAMPLE +$ dd of=sparse-file bs=1k seek=5120 count=0 + 0+0 records in + 0+0 records out + 0 bytes (0 B) copied, 5.668e-05 s, 0.0 kB/s - $du sparse-file - 0 +$ du sparse-file + 0 - $du -b sparse-file - 5242880 +$ du -b sparse-file + 5242880 +#+END_EXAMPLE Alternatively we can also use - truncate -s 5M sparse-file +#+BEGIN_EXAMPLE + truncate -s 5M sparse-file +#+END_EXAMPLE The file is full of zeros and requires only 4 bytes on the filesystem, although it contains 5242880 information bytes. - -# Online resizing +* Online resizing + :PROPERTIES: + :CUSTOM_ID: online-resizing + :END: If the kernel and the filesystem support it, it is possible to resize online -partitions, e.g. the system partition. Note that while extending a partition -is not problematic, shrinking a partition can cause data loss. +partitions, e.g. the system partition. Note that while extending a partition is +not problematic, shrinking a partition can cause data loss. Let's see how this works on an ext4 filesystem. @@ -135,19 +154,24 @@ Warning: The whole process should not be interrupted. Back up your partition table and the data if possible. Make sure the computer is powered by a battery or a UPS. -- Delete the partition N from the partition table (e.g. with fdisk), and -recreate it immediately with the same starting sector and the desired new size. +- Delete the partition N from the partition table (e.g. with =fdisk=), and + recreate it immediately with the same starting sector and the desired new + size. -- Run resize2fs /dev/sdXN on partition N of disk X. +- Run =resize2fs /dev/sdXN= on partition N of disk X. And there is no need to restart the computer! See (8)resize2fs for more options. - -# References - -* [Arch Wiki/Partitioning](https://wiki.archlinux.org/index.php/Partitioning) -* [Wikipedia/GUID Partition Table](https://en.wikipedia.org/wiki/GUID_Partition_Table) -* [Wikipedia/Master boot record](https://en.wikipedia.org/wiki/Master_boot_record) -* [Wikipedia/Sparse file](http://en.wikipedia.org/wiki/Sparse_file) +* References + :PROPERTIES: + :CUSTOM_ID: references + :END: + +- [[https://wiki.archlinux.org/index.php/Partitioning][Arch Wiki/Partitioning]] +- [[https://en.wikipedia.org/wiki/GUID_Partition_Table][Wikipedia/GUID Partition + Table]] +- [[https://en.wikipedia.org/wiki/Master_boot_record][Wikipedia/Master boot + record]] +- [[http://en.wikipedia.org/wiki/Sparse_file][Wikipedia/Sparse file]] diff --git a/articles/git.org b/articles/git.org index 057c22ed6c332233f0824bdfd7d7fba3825bd3b7..1743da264f28710189ccfb93b6dbe9852993a469 100644 --- a/articles/git.org +++ b/articles/git.org @@ -1,6 +1,9 @@ -% Git Pro-tips +#+TITLE: Git Pro-tips -# Add hunks selectively +* Add hunks selectively + :PROPERTIES: + :CUSTOM_ID: add-hunks-selectively + :END: One of Git's best practice is to limit every commit to an atomic change. Should your contribution require several changes to the code, it should result in just @@ -12,19 +15,29 @@ independent changes have occurend into the same file. Git has a nice command that lets the user select individual hunks for staging: - git add --patch +#+BEGIN_EXAMPLE + git add --patch +#+END_EXAMPLE This might prove daunting from the command-line. A UI can be very practical for this. -# Converting repositories from another VCS +* Converting repositories from another VCS + :PROPERTIES: + :CUSTOM_ID: converting-repositories-from-another-vcs + :END: Want to revamp a project using another VCS? No problem, Git can import many VCS, -such as Subversion. See git-svn(1) for the gory details. +such as Subversion. See =git-svn(1)= for the gory details. -# Stashes +* Stashes + :PROPERTIES: + :CUSTOM_ID: stashes + :END: - git stash +#+BEGIN_EXAMPLE + git stash +#+END_EXAMPLE In the middle of an intense hacking session, stashes allow you to save some chunks, revert to a clean workspace, keep on hacking, re-apply the previous @@ -33,37 +46,57 @@ chunk, revert, etc. All without commiting anything. It is also possible to stash the current hack and to keep working on it. A temporary save, in a way. - git stash && git stash apply +#+BEGIN_EXAMPLE + git stash && git stash apply +#+END_EXAMPLE -# Custom logs +* Custom logs + :PROPERTIES: + :CUSTOM_ID: custom-logs + :END: Logs decoration can be heavily customized. For instance: - git log --graph --decorate --pretty=format:'%C(yellow)%h %Cgreen%cd%C(bold red)%d%Creset %s' --abbrev-commit --date=short +#+BEGIN_EXAMPLE + git log --graph --decorate --pretty=format:'%C(yellow)%h %Cgreen%cd%C(bold red)%d%Creset %s' --abbrev-commit --date=short +#+END_EXAMPLE -# Diffing +* Diffing + :PROPERTIES: + :CUSTOM_ID: diffing + :END: To highlight differences to the word: - git diff --color-words +#+BEGIN_EXAMPLE + git diff --color-words +#+END_EXAMPLE Most importantly, use a UI for diff'ing between any commits, it helps a lot. -# Merging with Ediff +* Merging with Ediff + :PROPERTIES: + :CUSTOM_ID: merging-with-ediff + :END: -Add this to .gitconfig: +Add this to =.gitconfig=: - [mergetool.ediff] - cmd = emacs --eval \" (progn (defun ediff-write-merge-buffer () (let ((file ediff-merge-store-file)) (set-buffer ediff-buffer-C) (write-region (point-min) (point-max) file) (message \\\"Merge buffer saved in: %s\\\" file) (set-buffer-modified-p nil) (sit-for 1))) (setq ediff-quit-hook 'kill-emacs ediff-quit-merge-hook 'ediff-write-merge-buffer) (ediff-merge-files-with-ancestor \\\"$LOCAL\\\" \\\"$REMOTE\\\" \\\"$BASE\\\" nil \\\"$MERGED\\\"))\" +#+BEGIN_EXAMPLE + [mergetool.ediff] + cmd = emacs --eval \" (progn (defun ediff-write-merge-buffer () (let ((file ediff-merge-store-file)) (set-buffer ediff-buffer-C) (write-region (point-min) (point-max) file) (message \\\"Merge buffer saved in: %s\\\" file) (set-buffer-modified-p nil) (sit-for 1))) (setq ediff-quit-hook 'kill-emacs ediff-quit-merge-hook 'ediff-write-merge-buffer) (ediff-merge-files-with-ancestor \\\"$LOCAL\\\" \\\"$REMOTE\\\" \\\"$BASE\\\" nil \\\"$MERGED\\\"))\" - [merge] - tool = ediff + [merge] + tool = ediff +#+END_EXAMPLE -# Attributes +* Attributes + :PROPERTIES: + :CUSTOM_ID: attributes + :END: Some text files may not be "versionable" in that they are hardly human-readable, -it not at all (e.g. generated content). A few good examples include .ps, -.svg, and most XML-based files. +it not at all (e.g. generated content). A few good examples include =.ps=, +=.svg=, and most XML-based files. Git will treat all text files as such and store the diffs when necessary. This will increase disk-space consumption (and bandwidth) and slow down the @@ -71,81 +104,114 @@ repository. You can tell Git to ignore the diff's for specific extensions: - $cat .gitattributes - *.pdf -diff - *.ps -diff - *.svg -diff - *.xml -diff +#+BEGIN_EXAMPLE +$ cat .gitattributes + *.pdf -diff + *.ps -diff + *.svg -diff + *.xml -diff +#+END_EXAMPLE With such a setting, a Git repository can serve as a powerful cloud-based storage service. -# Maintenance +* Maintenance + :PROPERTIES: + :CUSTOM_ID: maintenance + :END: Run - git gc +#+BEGIN_EXAMPLE + git gc +#+END_EXAMPLE and be surprised by how much your repository just shrinked! -# Rebase interactively +* Rebase interactively + :PROPERTIES: + :CUSTOM_ID: rebase-interactively + :END: Rebasing can be a hassle. Thankfully Git comes with the very handy ---interactive option to overview the whole operation from your favorite text +=--interactive= option to overview the whole operation from your favorite text editor: - git rebase -i +#+BEGIN_EXAMPLE + git rebase -i +#+END_EXAMPLE -# Delete file from history +* Delete file from history + :PROPERTIES: + :CUSTOM_ID: delete-file-from-history + :END: Sometimes we need to carry a big binary blob with us. And more often than not we regret our decision! -It is never too late, and assuming the history is not public, you can rewrite it, -alleviating it from its binary burden. +It is never too late, and assuming the history is not public, you can rewrite +it, alleviating it from its binary burden. A few initial checks are in order: - Check for the largest files: - git verify-pack -v .git/objects/pack/${pack-name}.idx | sort -k 3 -n | tail -5 + #+BEGIN_EXAMPLE + git verify-pack -v .git/objects/pack/${pack-name}.idx | sort -k 3 -n | tail -5 + #+END_EXAMPLE - Check what files those are: - git rev-list --objects --all | grep ${object} + #+BEGIN_EXAMPLE + git rev-list --objects --all | grep${object} + #+END_EXAMPLE -Time to remove the file ${FILENAME} from the revision: Action! +Time to remove the file =${FILENAME}= from the revision: Action! - git filter-branch --index-filter 'git rm --cached --ignore-unmatch ${FILENAME}' +#+BEGIN_EXAMPLE + git filter-branch --index-filter 'git rm --cached --ignore-unmatch${FILENAME}' +#+END_EXAMPLE Next, some clean-up might be necessary: - Remove Git's backup - rm -rf .git/refs/original/ + #+BEGIN_EXAMPLE + rm -rf .git/refs/original/ + #+END_EXAMPLE - Expire all the loose objects: - git reflog expire --all --expire='0 days' + #+BEGIN_EXAMPLE + git reflog expire --all --expire='0 days' + #+END_EXAMPLE - Check if there are any loose objects: - git fsck --full --unreachable + #+BEGIN_EXAMPLE + git fsck --full --unreachable + #+END_EXAMPLE - Repack everything, leaving loose objects behind: - git repack -A -d + #+BEGIN_EXAMPLE + git repack -A -d + #+END_EXAMPLE - At last, remove the loose objects: - git prune + #+BEGIN_EXAMPLE + git prune + #+END_EXAMPLE -# UI +* UI + :PROPERTIES: + :CUSTOM_ID: ui + :END: -While the git CLI is great and will get you far, I recommend using a dedicated -UI such as the excellent [Magit][]. It will ease many operations: - -[magit]: https://magit.vc/ +While the =git= CLI is great and will get you far, I recommend using a dedicated +UI such as the excellent [[https://magit.vc/][Magit]]. It will ease many +operations: - Adding hunks of code. - Choosing commits for diffing, etc. diff --git a/articles/go.org b/articles/go.org index cfb4aa5affff150e206b2a3fcc2c8effb59e79ab..da3678b0ac74b5d6fe64baedf694d782bc02379e 100644 --- a/articles/go.org +++ b/articles/go.org @@ -1,185 +1,207 @@ -% Go: A Short Review +#+TITLE: Go: A Short Review This article relates my opinion on the pros and cons of the Go programming language. In short: the pros are overwhelming while the cons are merely points of contentions. -# Official documentation +* Official documentation + :PROPERTIES: + :CUSTOM_ID: official-documentation + :END: -The [official documentation](https://golang.org/doc/) is fairly complete and +The [[https://golang.org/doc/][official documentation]] is fairly complete and well organized. It is recommended to go through the following documentation, in order: -- A Tour of Go (a.k.a. _go-tour_) +- A Tour of Go (a.k.a. /go-tour/) - Effective Go - Frequently Asked Questions (FAQ) -The _go-tour_ is an interactive introduction that covers most of the concepts of +The /go-tour/ is an interactive introduction that covers most of the concepts of the language. It features a few challenging exercises. One of them is a virtual web-crawler: the challenge is to implement a recursive goroutine that needs to -synchronize with all its instances. [Here](crawler.go) is my suggested solution. +synchronize with all its instances. [[file:crawler.go][Here]] is my suggested +solution. -# Pros +* Pros + :PROPERTIES: + :CUSTOM_ID: pros + :END: -- Almost no _dark corners_ (see below). +- Almost no /dark corners/ (see below). -- Minimalist syntax (e.g. no more parenthesis for if and for, optional -semicolons). +- Minimalist syntax (e.g. no more parenthesis for =if= and =for=, optional + semicolons). - The syntax is rigid enough to put an end to the never-ending C-style wars: the -opening curly brace _must_ be on the same line, one-line control structures -_must_ have braces. + opening curly brace /must/ be on the same line, one-line control structures + /must/ have braces. - Local control structure declarations, e.g. - if _, status := f(); status { ... + #+BEGIN_EXAMPLE + if _, status := f(); status { ... + #+END_EXAMPLE - Universal UTF-8 support. - Strong typing. Type names are clear, size can be relative or absolute (e.g. -int or int8), the C char is the more sensical Go byte, rune is used -for unicode characters. + =int= or =int8=), the C =char= is the more sensical Go =byte=, =rune= is used + for unicode characters. -- Slices, maps, and channels are compound types (i.e. they are actually -pointers), all the other base types are _atomic_ (strings included). C-style -strings are still possible using static arrays. These simple rules makes it easy -to grasp memory usage. +- Slices, maps, and channels are compound types (i.e. they are actually + pointers), all the other base types are /atomic/ (strings included). C-style + strings are still possible using static arrays. These simple rules makes it + easy to grasp memory usage. - Garbage collector: nothing very new here, except that the simple -aforementioned rules on compound types makes it easy to exploit it correctly and -efficiently. + aforementioned rules on compound types makes it easy to exploit it correctly + and efficiently. - Nearly everything is explicit; as for C, we can almost guess the generated -assembly code in most cases. + assembly code in most cases. - The standard library is very complete: checksums, file system management, -serialization, big numbers, etc. + serialization, big numbers, etc. -- External calls à-la execve are safe (it does not make use of a subshell, it -uses a string list to pass arguments) and support both redirections and return -code. Many languages have this feature, but this is important to point out since -it is a big issue for those which do not, like Lua or POSIX shell. +- External calls à-la =execve= are safe (it does not make use of a subshell, it + uses a string list to pass arguments) and support both redirections and return + code. Many languages have this feature, but this is important to point out + since it is a big issue for those which do not, like Lua or POSIX shell. - Multiple return values are supported. This is a big plus for error management. - No exceptions (in general): very big plus for error management. -- The defer keyword: an elegant goto that allows execution of code before a -function returns (such as closing file descriptors). It is essential for writing -a clear and clean function termination. +- The =defer= keyword: an elegant =goto= that allows execution of code before a + function returns (such as closing file descriptors). It is essential for + writing a clear and clean function termination. - Package import is as simple as it can be, as is the hierarchy of projects. No -protected members, no headers, function order does not matter, and so on. + =protected= members, no headers, function order does not matter, and so on. - Performance: Go is a serious competitor to C and Fortran! - No object orientation! Perhaps one of the strongest point of the language. - Interface: allows for the genericity of object orientation with only one new -_type-concept_, and without hierarchy (flat relations through composition). + /type-concept/, and without hierarchy (flat relations through composition). - Concurrency: one of the main goal of Go is to offer a seamless support for -asynchronous concurrency via its goroutines and channels. This remains to me the -simplest concurrent programming experience I've had. A very welcomed feature. - -- Distribution tools: go can download source packages, compile, install and -execute. - - * It is possible to run go files with go run just as if it were an - interpreter. The source file actually gets compiled and the resulting - executable gets deleted upon termination. - - * go install analyses the imports: for every package that is not to be found - locally, it will automatically download it if it is a URL, then compile and - install. Various version control systems such as _git_ and _Mercurial_ are - supported. In a way, go is also a package manager (like Luarocks, pip, - rubygems, etc.). Having a standard package manager is a big advantage for - centralizing community contributions. The lack of such a tool has long been - a major flaw of Lua and still is a burden for C. - - * go only compiles source files that are more recent than binaries. Farewell, - Makefiles! Something less to care about. - - * go links statically by default. It is quite opposed to the current trend. - See [cat-v.org](http://harmful.cat-v.org/software/dynamic-linking/) and - [here](http://port70.net/~nsz/32_dynlink.html) for an (anti-)rationale. - - * A Go tree is very simple: - - /bin - /pkg///.a ## static libs - /src//.go - - * gofmt is an automatic code formatter. The fact that it is part of the - standard distribution ensures some universality in style. It puts an end to - never-ending debates on style issues that have no right solution. - - * godoc is a simple yet powerful documentation system. The lack of markup - makes it easy to use. - - * go test is a testing system. Again, very simple: run go test over a - package and every filename ending in _test.go will be run. This keeps the - file hierarchy simple. Having this part of the standard distribution makes - it easier and more automatic to integrate tests. - -- [Go 1 Compatibility Guarantee](https://golang.org/doc/go1compat) ensures that -Go 1 program will always build with the latest Go distribution. Considering this -together with static binaries, a Go 1 program will never fail to start either. -This is a gift in a world where programs tend to be very fragile in regard to -library updates. ([Python](https://docs.python.org/dev/whatsnew/3.5.html) is one -of numerous examples.) - -# Points of contention and other dark corners - -- Debugging capabilities are yet to improve. (See .) -Nonetheless the standard distribution features some nice tracing and profiling -tools. Most importantly, Go binaries support displaying a backtrace on crashes, -thus freeing Go from one of the main needs for a debugger. - -- Strings are defined as _immutable byte slices_ and behave as such except for -the range keyword which loops over the _runes_ of the string, and not the -bytes. This behaviour has been adopted for performance reasons by many modern -languages. Indeed, rune indexing needs linear time while byte indexing runs in -constant time; on the other hand, a range is performed linearly over the whole -slice, and thus no time is wasted returning the runes instead of the less useful -bytes. While this design choice is optimal in my opinion, the documentation -should emphasize this to the beginners as it can be misleading at first. + asynchronous concurrency via its goroutines and channels. This remains to me + the simplest concurrent programming experience I've had. A very welcomed + feature. + +- Distribution tools: =go= can download source packages, compile, install and + execute. + + - It is possible to run go files with =go run= just as if it were an + interpreter. The source file actually gets compiled and the resulting + executable gets deleted upon termination. + + - =go install= analyses the imports: for every package that is not to be found + locally, it will automatically download it if it is a URL, then compile and + install. Various version control systems such as /git/ and /Mercurial/ are + supported. In a way, =go= is also a package manager (like Luarocks, pip, + rubygems, etc.). Having a standard package manager is a big advantage for + centralizing community contributions. The lack of such a tool has long been + a major flaw of Lua and still is a burden for C. + + - =go= only compiles source files that are more recent than binaries. + Farewell, Makefiles! Something less to care about. + + - =go= links statically by default. It is quite opposed to the current trend. + See [[http://harmful.cat-v.org/software/dynamic-linking/][cat-v.org]] and + [[http://port70.net/~nsz/32_dynlink.html][here]] for an (anti-)rationale. + + - A Go tree is very simple: + + #+BEGIN_EXAMPLE + /bin + /pkg///.a ## static libs + /src//.go + #+END_EXAMPLE + + - =gofmt= is an automatic code formatter. The fact that it is part of the + standard distribution ensures some universality in style. It puts an end to + never-ending debates on style issues that have no right solution. + + - =godoc= is a simple yet powerful documentation system. The lack of markup + makes it easy to use. + + - =go test= is a testing system. Again, very simple: run =go test= over a + package and every filename ending in =_test.go= will be run. This keeps the + file hierarchy simple. Having this part of the standard distribution makes + it easier and more automatic to integrate tests. + +- [[https://golang.org/doc/go1compat][Go 1 Compatibility Guarantee]] ensures + that Go 1 program will always build with the latest Go distribution. + Considering this together with static binaries, a Go 1 program will never fail + to start either. This is a gift in a world where programs tend to be very + fragile in regard to library updates. + ([[https://docs.python.org/dev/whatsnew/3.5.html][Python]] is one of numerous + examples.) + +* Points of contention and other dark corners + :PROPERTIES: + :CUSTOM_ID: points-of-contention-and-other-dark-corners + :END: + +- Debugging capabilities are yet to improve. (See + [[http://golang.org/doc/gdb]].) Nonetheless the standard distribution features + some nice tracing and profiling tools. Most importantly, Go binaries support + displaying a backtrace on crashes, thus freeing Go from one of the main needs + for a debugger. + +- Strings are defined as /immutable byte slices/ and behave as such except for + the =range= keyword which loops over the /runes/ of the string, and not the + bytes. This behaviour has been adopted for performance reasons by many modern + languages. Indeed, rune indexing needs linear time while byte indexing runs in + constant time; on the other hand, a =range= is performed linearly over the + whole slice, and thus no time is wasted returning the runes instead of the + less useful bytes. While this design choice is optimal in my opinion, the + documentation should emphasize this to the beginners as it can be misleading + at first. - Members starting with an uppercase letter are automatically exported. On the -one hand it exposes the visibility of the objects very clearly, everywhere. It -also enforces a naming style. One the other hand, if you want to change the -visibility of a member, you need to refactor all the calls to it in the package. -An export keyword would have saved the effort. + one hand it exposes the visibility of the objects very clearly, everywhere. It + also enforces a naming style. One the other hand, if you want to change the + visibility of a member, you need to refactor all the calls to it in the + package. An =export= keyword would have saved the effort. - Go has no operator overloading ability. This has led to long debates and it is -hard to tell what the right choice is. It makes Go not so convenient to write -code for big numbers and matrices. + hard to tell what the right choice is. It makes Go not so convenient to write + code for big numbers and matrices. - The Go standard library boasts a simple and elegant design overall: only the -most useful and hard to implement methods are present. The rest can be easily -implemented if needs be, possibly using those initial methods. -However, [a few packages][go2] show little use and could have been left out -(e.g. [list](https://golang.org/pkg/container/list/)), while some have too many -methods in my opinion (e.g. [strings](https://golang.org/pkg/strings/)) + most useful and hard to implement methods are present. The rest can be easily + implemented if needs be, possibly using those initial methods. However, [a few + packages][go2] show little use and could have been left out (e.g. + [[https://golang.org/pkg/container/list/][list]]), while some have too many + methods in my opinion (e.g. [[https://golang.org/pkg/strings/][strings]]) - Go allows shadowing functions and types, even the basic ones. On the one hand, -it removes any name restriction on short-lived variables within restricted -scopes (functions, blocks...). On the other hand, it paves the way for obscure -compilation errors. + it removes any name restriction on short-lived variables within restricted + scopes (functions, blocks...). On the other hand, it paves the way for obscure + compilation errors. - Go's strict typing is a blessing and does not allow for automatic -conversions... but for one exception: + conversions... but for one exception: - > x’s type V and T have identical underlying types and at least one of V or T -is not a named type. + #+BEGIN_QUOTE + x's type V and T have identical underlying types and at least one of V or T + is not a named type. + #+END_QUOTE - [D. Honnef][implconv] mentions this oddity and provides one explanation. + [[http://dominik.honnef.co/posts/2012/12/go__on_implicit_type_conversions__type_identity_and_a_little_gotcha/][D. + Honnef]] mentions this oddity and provides one explanation. -[implconv]: http://dominik.honnef.co/posts/2012/12/go__on_implicit_type_conversions__type_identity_and_a_little_gotcha/ - -# Conclusion +* Conclusion + :PROPERTIES: + :CUSTOM_ID: conclusion + :END: In my opinion Go is a versatile language that is suitable to most contexts. One exception would be very low-level programming with important memory constraints @@ -191,8 +213,8 @@ rigorous and reliable, supported by very smart tools, modern in the way that it features much of the useful progress made in language expressiveness. To top it all, its implementation is extremely efficient. -I am not sure how convenient it is in a context of math or computer graphics: -it is debatable that the lack of operator overloading is detrimental. +I am not sure how convenient it is in a context of math or computer graphics: it +is debatable that the lack of operator overloading is detrimental. Go's main goal is not expressiveness though: languages such as Lua and Lisp are much more capable in this field thanks to a powerful introspection design. (Lua @@ -201,31 +223,47 @@ treacherous at the same time and I would not recommend it for team development: the risk to write unreadable code is just too high. The lack of advanced introspection capabilities makes of Go a very solid language by design. -# References +* References + :PROPERTIES: + :CUSTOM_ID: references + :END: - Rationales, in short: - - [Less is more (Rob Pike)](https://commandcenter.blogspot.in/2012/06/less-is-exponentially-more.html) - - [Why not Go? (Andrew Binstock _via_ Dr.Dobb's, 2012)](http://www.drdobbs.com/open-source/why-not-go/240005062) - - [Why I Program in Go (Tahir Hashmi, 2013)](https://tech.t9i.in/2013/01/05/why-program-in-go/) - - [Why Go? (João Henrique Machado _via_ Hackernoon, 2016)](https://hackernoon.com/why-go-ef8850dc5f3c) + - [[https://commandcenter.blogspot.in/2012/06/less-is-exponentially-more.html][Less + is more (Rob Pike)]] + - [[http://www.drdobbs.com/open-source/why-not-go/240005062][Why not Go? + (Andrew Binstock /via/ Dr.Dobb's, 2012)]] + - [[https://tech.t9i.in/2013/01/05/why-program-in-go/][Why I Program in Go + (Tahir Hashmi, 2013)]] + - [[https://hackernoon.com/why-go-ef8850dc5f3c][Why Go? (João Henrique Machado + /via/ Hackernoon, 2016)]] -- [Rationale, the official, long version](https://talks.golang.org/2012/splash.article) +- [[https://talks.golang.org/2012/splash.article][Rationale, the official, long + version]] - The cases of Go: - - [The Business Case for Go (_golang-nuts_)](https://groups.google.com/forum/#!topic/golang-nuts/Y-ka9lTRRPU) - - [Data science gophers (Daniel Whitenack)](https://www.oreilly.com/ideas/data-science-gophers) - - On Data analysis: [How Python Makes Working With Data More Difficult in the Long Run (Jeff Knupp _via_ Hackernoon)](https://hackernoon.com/how-python-makes-working-with-data-more-difficult-in-the-long-run-8da7c8e083fe) + - [[https://groups.google.com/forum/#!topic/golang-nuts/Y-ka9lTRRPU][The + Business Case for Go (/golang-nuts/)]] + - [[https://www.oreilly.com/ideas/data-science-gophers][Data science gophers + (Daniel Whitenack)]] + - On Data analysis: + [[https://hackernoon.com/how-python-makes-working-with-data-more-difficult-in-the-long-run-8da7c8e083fe][How + Python Makes Working With Data More Difficult in the Long Run (Jeff Knupp + /via/ Hackernoon)]] -- [Go Proverbs](http://go-proverbs.github.io/) +- [[http://go-proverbs.github.io/][Go Proverbs]] -- [List of success stories](https://github.com/golang/go/wiki/SuccessStories) +- [[https://github.com/golang/go/wiki/SuccessStories][List of success stories]] -- [Introducing Go 2.0 (Dave Cheney)][go2] -[go2]: https://dave.cheney.net/2016/10/25/introducing-go-2-0 +- [Introducing Go 2.0 (Dave Cheney)][go2] [go2]: + https://dave.cheney.net/2016/10/25/introducing-go-2-0 - And some rants: - - On Go assembly: [Golang is Trash](http://dtrace.org/blogs/wesolows/2014/12/29/golang-is-trash/) - - On polymorphism: [Why Go Is Not Good (Will Yager)](http://yager.io/programming/go.html) + - On Go assembly: + [[http://dtrace.org/blogs/wesolows/2014/12/29/golang-is-trash/][Golang is + Trash]] + - On polymorphism: [[http://yager.io/programming/go.html][Why Go Is Not Good + (Will Yager)]] diff --git a/articles/homogeneous.org b/articles/homogeneous.org index 12f216edcd4bdaa7fb037dbb899c082da26af701..334ba0ffd955c607630922d21dff0a314eaea19c 100644 --- a/articles/homogeneous.org +++ b/articles/homogeneous.org @@ -1,72 +1,81 @@ -% Homogeneous Coordinates -% A Concrete Explanation +#+TITLE: Homogeneous Coordinates -Computer graphics programs usually work with *homogeneous coordinates*, whose +#+AUTHOR: A Concrete Explanation +Computer graphics programs usually work with /homogeneous coordinates/, whose main purpose is to allow for some operations like translation to be linear and thus vectorizable. Vectorized operations can be heavily parallelized on the -processing unit which results in a tremendous performance boost. Video games make -heavy use of this technique. +processing unit which results in a tremendous performance boost. Video games +make heavy use of this technique. -Homogeneous coordinates are often introduced as a *trick*. I think this is a +Homogeneous coordinates are often introduced as a /trick/. I think this is a misconception and it paves the way for a lot of misunderstandings and errors when working with computer graphics. In the following article I will detail a more natural approach to the concept. -# Notations +* Notations + :PROPERTIES: + :CUSTOM_ID: notations + :END: -In the following we will consider canonical Euclidean planes and spaces. $$x, -$$ y and $$z are the canonical Euclidean dimensions or the vector components -depending on the context. +In the following we will consider canonical Euclidean planes and spaces. =$$ x=, +=$$y= and =$$ z= are the canonical Euclidean dimensions or the vector +components depending on the context. The initial space is called the projective space. All the transformations are supposed to be endomorphisms. -# Usual transformations +* Usual transformations + :PROPERTIES: + :CUSTOM_ID: usual-transformations + :END: -Linear transformations in a Euclidean space can be written as a matrix $$M so -that the transformed point $$ P' of $$P is: +Linear transformations in a Euclidean space can be written as a matrix =$$ M= so +that the transformed point =$$P'= of =$$ P= is: - math -P' = M \cdot P - +#+BEGIN_EXAMPLE + P' = M \cdot P +#+END_EXAMPLE -In 2D, the matrix is of size $$(2,2). A few examples follow: +In 2D, the matrix is of size =$$ (2,2)=. A few examples follow: -- 2D scaling $$(x,y) \to (sx, sy): +- 2D scaling =$$ (x,y) \to (sx, sy)=: - math -\begin{pmatrix} -s & 0 \\ -0 & s \\ -\end{pmatrix} - +#+BEGIN_EXAMPLE + \begin{pmatrix} + s & 0 \\ + 0 & s \\ + \end{pmatrix} +#+END_EXAMPLE -- Rotation of angle $$\theta: +- Rotation of angle =$$ \theta=: - In 2D: + In 2D: -  math - \begin{pmatrix} - \cos(\theta) & -\sin(\theta) \\ - \sin(\theta) & \cos(\theta) \\ - \end{pmatrix} -  + #+BEGIN_EXAMPLE + \begin{pmatrix} + \cos(\theta) & -\sin(\theta) \\ + \sin(\theta) & \cos(\theta) \\ + \end{pmatrix} + #+END_EXAMPLE - In 3D, around the $$z-axis: + In 3D, around the =$$ z=-axis: -  math - \begin{pmatrix} - \cos(\theta) & -\sin(\theta) & 0 \\ - \sin(\theta) & \cos(\theta) & 0 \\ - 0 & 0 & 1 \\ - \end{pmatrix} -  + #+BEGIN_EXAMPLE + \begin{pmatrix} + \cos(\theta) & -\sin(\theta) & 0 \\ + \sin(\theta) & \cos(\theta) & 0 \\ + 0 & 0 & 1 \\ + \end{pmatrix} + #+END_EXAMPLE Other linear transformations include shearing and mirroring. -# Non-linear operations +* Non-linear operations + :PROPERTIES: + :CUSTOM_ID: non-linear-operations + :END: In the following section we will consider the projective space to be a 2D plane. This is only to make the explanation more visual, since this can be easily @@ -76,10 +85,10 @@ Some operations are not linear in the projective plane. But what if we consider the projective plane as being a plane in a 3D space? Then we can apply any 3D transformation on the points of this plane. -We say that we *promote* the plane to the next dimension. +We say that we /promote/ the plane to the next dimension. -The *fundamental principle* here is that the higher dimension allows for more -complex transformations, e.g. some specific shearing operations in 3D can be +The /fundamental principle/ here is that the higher dimension allows for more +complex transformations, e.g. some specific shearing operations in 3D can be seen as a translation on the plane. After the desired transformations have been applied, we project the resulting @@ -91,42 +100,53 @@ There are some parameters we need to specify: - How do we project the transformation result back to the plane? -In order to answer this we need to analyze how some non-linear operations behave. +In order to answer this we need to analyze how some non-linear operations +behave. -## Translation operation +** Translation operation + :PROPERTIES: + :CUSTOM_ID: translation-operation + :END: -### Promotion +*** Promotion + :PROPERTIES: + :CUSTOM_ID: promotion + :END: Translations are not linear in the 2D-Euclidean space. Now if we promote the input points to a 3D space, they will all lie in a plane. It is obviously -simpler to keep the $$x and $$ y vectors identical. Then the only parameter for -defining the plane is $$z. +simpler to keep the =$$ x= and =$$y= vectors identical. Then the only parameter +for defining the plane is =$$ z=. -Let's have a look at the following shearing of some point $$P=(i,j,k) by the -scalars $$ a and $$b: +Let's have a look at the following shearing of some point =$$ P=(i,j,k)= by the +scalars =$$a= and =$$ b=: - math -\begin{pmatrix} -1 & 0 & a \\ -0 & 1 & b \\ -0 & 0 & 1 \\ -\end{pmatrix} \cdot -(i, j, k) = -(i+ka, j+kb, k) - +#+BEGIN_EXAMPLE + \begin{pmatrix} + 1 & 0 & a \\ + 0 & 1 & b \\ + 0 & 0 & 1 \\ + \end{pmatrix} \cdot + (i, j, k) = + (i+ka, j+kb, k) +#+END_EXAMPLE -If $$k=1, then the transformed point is $$ (i+a, j+b, 1), which is a -translation on the plane $$z=1. +If =$$ k=1=, then the transformed point is =$$(i+a, j+b, 1)=, which is a +translation on the plane =$$ z=1=. -Considering the projective plane at $$z=1 makes it easy to write a translation -matrix. +Considering the projective plane at =$$ z=1= makes it easy to write a +translation matrix. Conclusion: the points from a projective space of any dimension can be promoted to the next dimension by adding a coordinate with the scalar value 1. -![Translation of a promoted point](promotion.svg) +#+CAPTION: Translation of a promoted point +[[file:promotion.svg]] -### Projection +*** Projection + :PROPERTIES: + :CUSTOM_ID: projection + :END: If the transformed points lie in the plane, the projection back to the projective space is trivial: we simply cut off the last dimension. @@ -138,179 +158,201 @@ several ways of doing this. - One would be to simply cut off the last dimension. - Another solution would be to divide all dimensions by the last one when it is -non-zero. The last dimension would then always be 1, which matches the position -of the plane $$z=1 as we defined it previously. + non-zero. The last dimension would then always be 1, which matches the + position of the plane =$$ z=1= as we defined it previously. -First let's consider some point $$P=(i,j,k) and let's see what happens if we +First let's consider some point =$$ P=(i,j,k)= and let's see what happens if we apply the same transformation as before: - math -\begin{pmatrix} -1 & 0 & a \\ -0 & 1 & b \\ -0 & 0 & 1 \\ -\end{pmatrix} -\cdot -(i,j,k) = -(i+ka, j+ka, k) = P' - - -- If we project by cutting off the last coordinate, the projection of $$P is -$$ (i, j), the projection of $$P' is $$ (i+ka, j+kb). The transformation is -generally *not* the translation by $$(a, b). - -- Using the division by the last coordinate, the projection of $$ P is -$$(i/k, j/k), the projection of $$ P' is $$(i/k+a, j/k+b). This time the -transformation is the translation by $$ (a, b). +#+BEGIN_EXAMPLE + \begin{pmatrix} + 1 & 0 & a \\ + 0 & 1 & b \\ + 0 & 0 & 1 \\ + \end{pmatrix} + \cdot + (i,j,k) = + (i+ka, j+ka, k) = P' +#+END_EXAMPLE + +- If we project by cutting off the last coordinate, the projection of =$$P= is + =$$ (i, j)=, the projection of =$$P'= is =$$ (i+ka, j+kb)=. The + transformation is generally /not/ the translation by =$$(a, b)=. + +- Using the division by the last coordinate, the projection of =$$ P= is + =$$(i/k, j/k)=, the projection of =$$ P'= is =$$(i/k+a, j/k+b)=. This time + the transformation is the translation by =$$ (a, b)=. Conclusion: the projection back to the projective space is done by dividing all dimensions by the last one when non-zero. This allows for translations in the projective space. -## Perspective operation +** Perspective operation + :PROPERTIES: + :CUSTOM_ID: perspective-operation + :END: We have seen one example that covers it all. But what if the above analysis was good enough for translations only? Let us have a look at a very common operation in computer graphics: perspective correction. In this section we consider a 3D projective space. The higher dimension is 4D -and the 4th coordinates we be written $$w. +and the 4th coordinates we be written =$$ w=. -When the camera and the screen are centered on the $$z-axis, perspective -projection results from simple triangular relations, i.e. by dividing $$ x and -$$y by $$ \alpha+\beta\cdot z, where $$\alpha and $$ \beta are scalars depending -on the distance from the camera and the screen to the origin. +When the camera and the screen are centered on the =$$z=-axis, perspective +projection results from simple triangular relations, i.e. by dividing =$$ x= and +=$$y= by =$$ \alpha+\beta\cdot z=, where =$$\alpha= and =$$ \beta= are scalars +depending on the distance from the camera and the screen to the origin. -![Perspective relations](perspective.svg) +#+CAPTION: Perspective relations +[[file:perspective.svg]] -In the previous scheme, $$i'=i\frac{z_s-z_c}{k-z_c} and $$ j'=j\frac{z_s-z_c}{k-z_c}. +In the previous scheme, =$$i'=i\frac{z_s-z_c}{k-z_c}= and +=$$ j'=j\frac{z_s-z_c}{k-z_c}=. -In the simplest case the camera is at the origin and the screen is at $$z=1, as -above. We get the simple transformation where $$ \alpha=1 and $$\beta=1, that is, -in our example, $$ i'=i/k and $$j'=j/k. +In the simplest case the camera is at the origin and the screen is at =$$ z=1=, +as above. We get the simple transformation where =$$\alpha=1= and =$$ \beta=1=, +that is, in our example, =$$i'=i/k= and =$$ j'=j/k=. But there is no way a matrix multiplication can induce a division by one of the dimension. After all, matrices are a notation for linear transformations. There is one way out though: a division by one dimension is performed during the -projection back to the projective space. As such, our lead is to "inject" $$z -into $$ w: - - math -\begin{pmatrix} -1 & 0 & 0 & 0 \\ -0 & 1 & 0 & 0 \\ -0 & 0 & 1 & 0 \\ -0 & 0 & \beta & \alpha \\ -\end{pmatrix} \cdot -(i, j, k, 1) = -(i, j, k, \alpha+\beta k) - +projection back to the projective space. As such, our lead is to "inject" =$$z= +into =$$ w=: + +#+BEGIN_EXAMPLE + \begin{pmatrix} + 1 & 0 & 0 & 0 \\ + 0 & 1 & 0 & 0 \\ + 0 & 0 & 1 & 0 \\ + 0 & 0 & \beta & \alpha \\ + \end{pmatrix} \cdot + (i, j, k, 1) = + (i, j, k, \alpha+\beta k) +#+END_EXAMPLE If we project the result back to the projective space, we get the 3D point -$$(\frac{i}{\alpha+\beta k}, \frac{j}{\alpha+\beta k}, \frac{k}{\alpha+\beta -k}). If we project the result on the plane $$ z=z_s, we obtain the perspective -projection of $$(i, j, k). +=$$ (\frac{i}{\alpha+\beta k}, \frac{j}{\alpha+\beta k}, \frac{k}{\alpha+\beta k})=. +If we project the result on the plane =$$z=z_s=, we obtain the perspective +projection of =$$ (i, j, k)=. Conclusion: one more time, the division by the last coordinate is a projection technique that allows for operations in the next dimension to be properly mapped to the projective space. +* Homogeneous coordinates + :PROPERTIES: + :CUSTOM_ID: homogeneous-coordinates + :END: -# Homogeneous coordinates - -The above analysis issued a relation between points in the projective space and points -in the next dimension. +The above analysis issued a relation between points in the projective space and +points in the next dimension. Making use of the above reasoning, homogeneous coordinates offer an alternative -*notation* to cartesian coordinates. We insist on the word 'notation' as there -are no *new points* nor *new spaces*. It means that $$(i,j,k) and $$ (i,j,k,1)_h -specify the same point in 3D. +/notation/ to cartesian coordinates. We insist on the word 'notation' as there +are no /new points/ nor /new spaces/. It means that =$$(i,j,k)= and +=$$ (i,j,k,1)_h= specify the same point in 3D. -We use the $$_h index to avoid confusion between homogeneous coordinates and +We use the =$$ _h= index to avoid confusion between homogeneous coordinates and cartesian coordinates in a higher dimension. The mapping between the two coordinate systems is central: the cartesian coordinates are obtained by dividing all dimensions by the homogeneous component if not null, and discarding this last component. This has several consequences: -- Infinity can be represented using finite coordinates, e.g. $$(x, y, 0)_h in 2D. +- Infinity can be represented using finite coordinates, e.g. =$$ (x, y, 0)_h= in + 2D. -- The $$0_h vector is ignored. +- The =$$ 0_h= vector is ignored. - The origin is the vector where all components are 0 but the last one is 1, - e.g. $$(0,0,1)_h in 2D. + e.g. =$$ (0,0,1)_h= in 2D. -- All points on a line going through $$0_h represent the same point. A non-null -scalar multiplication over homogeneous coordinates does not change the point. -For instance $$ (i, j) = (i, j, 1)_h = (2i, 2j, 2)_h = (\alpha i, \alpha j, -\alpha)_h for all scalars $$\alpha. +- All points on a line going through =$$ 0_h= represent the same point. A + non-null scalar multiplication over homogeneous coordinates does not change + the point. For instance + =$$(i, j) = (i, j, 1)_h = (2i, 2j, 2)_h = (\alpha i, \alpha j, \alpha)_h= for + all scalars =$$ \alpha=. -# Examples +* Examples + :PROPERTIES: + :CUSTOM_ID: examples + :END: -## Rendering pipeline +** Rendering pipeline + :PROPERTIES: + :CUSTOM_ID: rendering-pipeline + :END: In computer graphics, homogeneous coordinates play an important role as they allow for perspective transformations in 3D. -The 3D rendering pipeline can be summarized as follows (*C* stands for -*cartesian*, *H* stands for *homogeneous*): +The 3D rendering pipeline can be summarized as follows (/C/ stands for +/cartesian/, /H/ stands for /homogeneous/): -> Vertex data (3D C) → Object data (3D H) → Eye coordinates (3D H) -> → -> Clip coordinates (3D H) → Normalized device coordinates (3D C) → -> Window coordinates (2D C) +#+BEGIN_QUOTE + Vertex data (3D C) → Object data (3D H) → Eye coordinates (3D H) → Clip + coordinates (3D H) → Normalized device coordinates (3D C) → Window coordinates + (2D C) +#+END_QUOTE - Vertex data are 3D vectors. The graphic pipeline uses homogeneous coordinates -for object data, with the homogeneous component being 1, for the -reason detailed above. + for object data, with the homogeneous component being 1, for the reason + detailed above. -- Object data is then translated and rotated as needed via $$(4,4)_h-matrices. -This yields the homogeneous eye coordinates. +- Object data is then translated and rotated as needed via + =$$ (4,4)_h=-matrices. This yields the homogeneous eye coordinates. -- A $$(4,4)_h-perspective matrix is applied on the eye coordinates, which leads -to the homogeneous clip coordinates. +- A =$$ (4,4)_h=-perspective matrix is applied on the eye coordinates, which + leads to the homogeneous clip coordinates. - The homogeneous clip coordinates are converted to cartesian coordinates, -a.k.a. normalized device coordinates (NDC), by dividing every coordinate by the -last one, as usual. + a.k.a. normalized device coordinates (NDC), by dividing every coordinate by + the last one, as usual. - The NDC are converted to the 2D window coordinates by removing the third -dimension and applying some scaling. + dimension and applying some scaling. -## 2D perspective correction +** 2D perspective correction + :PROPERTIES: + :CUSTOM_ID: d-perspective-correction + :END: Let us consider the following problem: we want to rectify the perspective of a -picture, e.g. we want a tilted building facade picture to appear as if it had +picture, e.g. we want a tilted building facade picture to appear as if it had been taken orthogonally. We proceed with control points: we select pixels in the input picture and tell the program where we would like it to be on the transformed picture. -A $$(2,2)-matrix will not do the trick as it is not able to handle translations -for instance. +A =$$ (2,2)=-matrix will not do the trick as it is not able to handle +translations for instance. Using homogeneous coordinates, we can transform the entire plane to the -orthogonal picture we would like. In 3D, this can be seen as tilting the -plane to make it match the screen. - - - -This transformation can be embodied within a $$(3,3)_h-matrix $$ M. The answer to -our problem is $$M. - -Let us consider a set of known point pairs $$ \langle \text{ control point } -(i,j), \text{ target point } (I,J)\rangle, so that - - math -\begin{pmatrix} -m_{11} & m_{12} & m_{13} \\ -m_{21} & m_{22} & m_{23} \\ -m_{31} & m_{32} & m_{33} \\ -\end{pmatrix} \cdot -(i, j, 1)_h = -(\alpha I, \alpha J, \alpha)_h - +orthogonal picture we would like. In 3D, this can be seen as tilting the plane +to make it match the screen. + +#+BEGIN_HTML + +#+END_HTML + +This transformation can be embodied within a =$$(3,3)_h=-matrix =$$ M=. The +answer to our problem is =$$M=. + +Let us consider a set of known point pairs +=$$ \langle \text{ control point } (i,j), \text{ target point } (I,J)\rangle=, +so that + +#+BEGIN_EXAMPLE + \begin{pmatrix} + m_{11} & m_{12} & m_{13} \\ + m_{21} & m_{22} & m_{23} \\ + m_{31} & m_{32} & m_{33} \\ + \end{pmatrix} \cdot + (i, j, 1)_h = + (\alpha I, \alpha J, \alpha)_h +#+END_EXAMPLE We could be tempted to think we only need the first 2 rows of the matrix: this is wrong! The result of the matrix multiplication yields the homogeneous @@ -320,94 +362,99 @@ point. The previous matrix operation yields the following equations: - math -\left\{ -\begin{array}{l l} -m_{11}i + m_{12}j + m_{13} &= \alpha I \\ -m_{21}i + m_{22}j + m_{23} &= \alpha J \\ -m_{31}i + m_{32}j + m_{33} &= \alpha \\ -\end{array} -\right. - +#+BEGIN_EXAMPLE + \left\{ + \begin{array}{l l} + m_{11}i + m_{12}j + m_{13} &= \alpha I \\ + m_{21}i + m_{22}j + m_{23} &= \alpha J \\ + m_{31}i + m_{32}j + m_{33} &= \alpha \\ + \end{array} + \right. +#+END_EXAMPLE or, cancelling the scale factor: - math -\left\{ -\begin{array}{l l} -\frac{m_{11}i + m_{12}j + m_{13}}{m_{31}i + m_{32}j + m_{33}} &= I \\ -\frac{m_{21}i + m_{22}j + m_{23}}{m_{31}i + m_{32}j + m_{33}} &= J \\ -\end{array} -\right. - +#+BEGIN_EXAMPLE + \left\{ + \begin{array}{l l} + \frac{m_{11}i + m_{12}j + m_{13}}{m_{31}i + m_{32}j + m_{33}} &= I \\ + \frac{m_{21}i + m_{22}j + m_{23}}{m_{31}i + m_{32}j + m_{33}} &= J \\ + \end{array} + \right. +#+END_EXAMPLE -The matrix has 9 unknown coefficients, but since it is homogeneous, it is defined -up to a scale factor. For instance, we have $$M=M/m_{11} if $$ m_{11}≠0. As such, -only 8 coefficient determines a homogeneous matrix uniquely. +The matrix has 9 unknown coefficients, but since it is homogeneous, it is +defined up to a scale factor. For instance, we have =$$M=M/m_{11}= if +=$$ m_{11}≠0=. As such, only 8 coefficient determines a homogeneous matrix +uniquely. We can solve the system of 8 linearly independent equations constructed as above from 4 well chosen points. The above equation pair yields - math -\left\{ -\begin{array}{l l} -m_{11}i + m_{12}j + m_{13} - m_{31}iI - m_{32}jI - m_{33}I &= 0 \\ -m_{21}i + m_{22}j + m_{23} - m_{31}iJ - m_{32}jJ - m_{33}J &= 0 \\ -\end{array} -\right. - +#+BEGIN_EXAMPLE + \left\{ + \begin{array}{l l} + m_{11}i + m_{12}j + m_{13} - m_{31}iI - m_{32}jI - m_{33}I &= 0 \\ + m_{21}i + m_{22}j + m_{23} - m_{31}iJ - m_{32}jJ - m_{33}J &= 0 \\ + \end{array} + \right. +#+END_EXAMPLE which can be written as a multiplication - math -\left\{ -\begin{array}{l l} -(i, j, 1, 0, 0, 0, -iI, -jI, -I) \cdot m^T &= 0 \\ -(0, 0, 0, i, j, 1, -iJ, -jJ, -J) \cdot m^T &= 0 \\ -\end{array} -\right. - +#+BEGIN_EXAMPLE + \left\{ + \begin{array}{l l} + (i, j, 1, 0, 0, 0, -iI, -jI, -I) \cdot m^T &= 0 \\ + (0, 0, 0, i, j, 1, -iJ, -jJ, -J) \cdot m^T &= 0 \\ + \end{array} + \right. +#+END_EXAMPLE -with $$m=(m_{11}, m_{12}, m_{13}, m_{21}, m_{22}, m_{23}, m_{31}, m_{32}, m_{33}). +with +=$$ m=(m_{11}, m_{12}, m_{13}, m_{21}, m_{22}, m_{23}, m_{31}, m_{32}, m_{33})=. We can build the following matrix from 4 point sets: - math -S = \begin{pmatrix} -i_1 & j_1 & 1 & 0 & 0 & 0 & -i_1I_1 & -j_1I_1 & -I_1 \\ -0 & 0 & 0 & i_1 & j_1 & 1 & -i_1J_1 & -j_1J_1 & -J_1 \\ -... \\ -i_4 & j_4 & 1 & 0 & 0 & 0 & -i_4I_4 & -j_4I_4 & -I_4 \\ -0 & 0 & 0 & i_4 & j_4 & 1 & -i_4J_4 & -j_4J_4 & -J_4 \\ -\end{pmatrix} - +#+BEGIN_EXAMPLE + S = \begin{pmatrix} + i_1 & j_1 & 1 & 0 & 0 & 0 & -i_1I_1 & -j_1I_1 & -I_1 \\ + 0 & 0 & 0 & i_1 & j_1 & 1 & -i_1J_1 & -j_1J_1 & -J_1 \\ + ... \\ + i_4 & j_4 & 1 & 0 & 0 & 0 & -i_4I_4 & -j_4I_4 & -I_4 \\ + 0 & 0 & 0 & i_4 & j_4 & 1 & -i_4J_4 & -j_4J_4 & -J_4 \\ + \end{pmatrix} +#+END_EXAMPLE -If $$det(S) ≠ 0, then the result can be found by solving +If =$$ det(S) ≠ 0=, then the result can be found by solving - math -S\cdot m = 0 - +#+BEGIN_EXAMPLE + S\cdot m = 0 +#+END_EXAMPLE -which is equivalent to finding the eigenvalues of $$S. An SVD will numerically -find $$ m. +which is equivalent to finding the eigenvalues of =$$S=. An SVD will +numerically find =$$ m=. -# Final note +* Final note + :PROPERTIES: + :CUSTOM_ID: final-note + :END: The present article does not claim to provide any new results when it comes to -homogeneous coordinates. It only aims at explaining the theory more *visually*, +homogeneous coordinates. It only aims at explaining the theory more /visually/, and thus making it easier to understand and remember. It all makes perfect sense, it is not just a trick. Mathematically speaking, the key argument that I -introduced above is to *consider the projective space as an affine hyperplane in -the next dimension*. Then everything falls into place: +introduced above is to /consider the projective space as an affine hyperplane in +the next dimension/. Then everything falls into place: - the link between cartesian and homogeneous coordinates; - why this actually works; - when we should manipulate homogeneous coordinates, when we should project back -to the projective space. + to the projective space. The central idea can be seen as a hook in the process: we momentarily promote our data to the next dimension, manipulate it in there, then project the result -back to the original dimension. It generally *does not make sense* to exploit a +back to the original dimension. It generally /does not make sense/ to exploit a result in homogeneous coordinates. diff --git a/articles/ide.org b/articles/ide.org index b3af38d244537e53360d7a3f3423e9e5de605965..a0d61b0297816d79592942eb951e66ccd41d3885 100644 --- a/articles/ide.org +++ b/articles/ide.org @@ -1,4 +1,4 @@ -% Integrated Development Environments: Debunking the Myth +#+TITLE: Integrated Development Environments: Debunking the Myth IDEs provide the user with a lot of neat features beside pure text editing capabilities: auto-completion, refactoring, project management, etc. What are @@ -32,79 +32,83 @@ permanent solution resides in avoiding the use of diamond inheritance in the first place. Nothing good can result from not knowing the objects you manipulate. - - -# The myth of the _too-good-an-editor_ - -[Emacs][] and [Vim][] have the reputation of the being the geek's best friends, -_the tools to rule them all_. And yet many professional programmers discard them -from their toolbox to fallback to the "reputable corporate programs." Why is -that so? - -[emacs]: https://www.gnu.org/software/emacs/ -[vim]: http://www.vim.org/ - -To quote [Paul Graham](http://www.paulgraham.com/avg.html): - -> "Lisp is worth learning for the profound enlightenment experience you will -> have when you finally get it; that experience will make you a better -> programmer for the rest of your days, even if you never actually use Lisp -> itself a lot." -> -> This is the same argument you tend to hear for learning Latin. It won't get -> you a job, except perhaps as a classics professor, but it will improve your -> mind, and make you a better writer in languages you do want to use, like -> English. -> -> But wait a minute. This metaphor doesn't stretch that far. The reason Latin -> won't get you a job is that no one speaks it. If you write in Latin, no one -> can understand you. But Lisp is a computer language, and computers speak -> whatever language you, the programmer, tell them to. -> -> So if Lisp makes you a better programmer, like he says, why wouldn't you want -> to use it? If a painter were offered a brush that would make him a better -> painter, it seems to me that he would want to use it in all his paintings, -> wouldn't he? I'm not trying to make fun of Eric Raymond here. On the whole, -> his advice is good. What he says about Lisp is pretty much the conventional -> wisdom. But there is a contradiction in the conventional wisdom: Lisp will -> make you a better programmer, and yet you won't use it. +* The myth of the /too-good-an-editor/ + :PROPERTIES: + :CUSTOM_ID: the-myth-of-the-too-good-an-editor + :END: + +[[https://www.gnu.org/software/emacs/][Emacs]] and [[http://www.vim.org/][Vim]] +have the reputation of the being the geek's best friends, /the tools to rule +them all/. And yet many professional programmers discard them from their toolbox +to fallback to the "reputable corporate programs." Why is that so? + +To quote [[http://www.paulgraham.com/avg.html][Paul Graham]]: + +#+BEGIN_QUOTE + "Lisp is worth learning for the profound enlightenment experience you will + have when you finally get it; that experience will make you a better + programmer for the rest of your days, even if you never actually use Lisp + itself a lot." + + This is the same argument you tend to hear for learning Latin. It won't get + you a job, except perhaps as a classics professor, but it will improve your + mind, and make you a better writer in languages you do want to use, like + English. + + But wait a minute. This metaphor doesn't stretch that far. The reason Latin + won't get you a job is that no one speaks it. If you write in Latin, no one + can understand you. But Lisp is a computer language, and computers speak + whatever language you, the programmer, tell them to. + + So if Lisp makes you a better programmer, like he says, why wouldn't you want + to use it? If a painter were offered a brush that would make him a better + painter, it seems to me that he would want to use it in all his paintings, + wouldn't he? I'm not trying to make fun of Eric Raymond here. On the whole, + his advice is good. What he says about Lisp is pretty much the conventional + wisdom. But there is a contradiction in the conventional wisdom: Lisp will + make you a better programmer, and yet you won't use it. +#+END_QUOTE Questions: -- Why would the big IDEs be more corporate-friendly than the venerable Emacs -and Vim? +- Why would the big IDEs be more corporate-friendly than the venerable Emacs and + Vim? - The users of the corporate IDEs argue that Emacs/Vim users sacrifice the -ergonomy and the technical advance of "real" IDEs for the sake of saying -"Look!, I use Emacs/Vim!". + ergonomy and the technical advance of "real" IDEs for the sake of saying + "Look!, I use Emacs/Vim!". Answers: - A lot of myths surround us. - Using Emacs/Vim to its full potential implies learning how to customize it. -The learning curve might be steep. Thus it is not so apparent _at first_ how -Emacs/Vim can overtake the big IDEs. A naive use of those editors will seem very -tedious and unproductive. + The learning curve might be steep. Thus it is not so apparent /at first/ how + Emacs/Vim can overtake the big IDEs. A naive use of those editors will seem + very tedious and unproductive. - One of the strongest selling point for IDEs in corporate environments is that -they are supposed to have an out-of-the-box UI, thus avoiding costly training -programs for using the more geeky editors. + they are supposed to have an out-of-the-box UI, thus avoiding costly training + programs for using the more geeky editors. - As Paul Graham hints it, there is a popular misconception that too powerful -tools can be nice fooling around with, but they better stay out of the way of -the professional case. + tools can be nice fooling around with, but they better stay out of the way of + the professional case. I can recall the times when I was studying, our dean would say (and write in the syllabus): -> - Nano: a simple editor -> - Vim: an advanced editor -> - Emacs: too-powerful an editor - +#+BEGIN_QUOTE + - Nano: a simple editor + - Vim: an advanced editor + - Emacs: too-powerful an editor +#+END_QUOTE -# Fundamental editing +* Fundamental editing + :PROPERTIES: + :CUSTOM_ID: fundamental-editing + :END: What is the one feature we need from a text editor? @@ -119,19 +123,21 @@ user's productivity by implementing every single corner case of editing capabilities is not possible. As such, an editor needs just one core feature to boost the productivity of -_every_ user: +/every/ user: -> Be an extensible editor. +#+BEGIN_QUOTE + Be an extensible editor. +#+END_QUOTE That's it. Well, let's add some frivolities to the list: - Any action can be bound to a keyboard shortcut. -Touch-typing is what will make you write code _fast_, not fancy IDE features. +Touch-typing is what will make you write code /fast/, not fancy IDE features. Mouse actions will disrupt the flow which is why a keyboard-oriented editor will -allow for a faster flow. See [Mastering the keyboard](../keymap/index.html). +allow for a faster flow. See [[../keymap/index.html][Mastering the keyboard]]. -- Capable of running in text mode (e.g. TTY) is a plus for historical reasons. +- Capable of running in text mode (e.g. TTY) is a plus for historical reasons. Text mode is not strictly required, but as of 2016 it is still prevalent in many contexts in which you really want to keep your favorite tool at hand. @@ -150,27 +156,28 @@ instance: - Auto-completion. - Spell checking. - Fully keyboard controlled, without relying on arrows. (This would disrupt the -flow when the hands are on the homerow.) + flow when the hands are on the homerow.) - Fast file navigation (function, paragraph, word, matching parenthesis, page, -search, etc.). -- [Multiple cursors](http://emacsrocks.com/e13.html). -- Shell piping (output to buffer), call shell commands on files and process output. + search, etc.). +- [[http://emacsrocks.com/e13.html][Multiple cursors]]. +- Shell piping (output to buffer), call shell commands on files and process + output. - Display editing windows side-by-side, including of the same document. - Templates / snippets. - Whitespace management, automatic code formatting. - Fast buffer switching. (Tabs are slow to browse...) - Fullscreen text editing area. This is where you'll spend most of your time -reading and writing. Paradoxically, some IDEs forget they should be _editors_ -before anything else and bundle so many features in side windows that the text -area is less than half the screen space by default. + reading and writing. Paradoxically, some IDEs forget they should be /editors/ + before anything else and bundle so many features in side windows that the text + area is less than half the screen space by default. Text editors that are extensible, keyboard controlled and work in text mode are -not legions. The most popular are, by far, the ancient, legendary Emacs and -Vim. TextAdept is another one that offers some interesting feats. +not legions. The most popular are, by far, the ancient, legendary Emacs and Vim. +TextAdept is another one that offers some interesting feats. -Vim inherits from vi, a historical text editor that is very minimalist. -Both editors are sometimes mixed up, in so far as some systems actually symlink -vi to Vim. This is dangerous so let's stress this out: if you find yourself +Vim inherits from =vi=, a historical text editor that is very minimalist. Both +editors are sometimes mixed up, in so far as some systems actually symlink =vi= +to Vim. This is dangerous so let's stress this out: if you find yourself wondering why so many people follow the Cult of Vim while it seems to be the most limited editor you've ever encountered, you might be using the wrong piece of software. @@ -187,7 +194,10 @@ Another difference is that Emacs has a greedy tendency for bundling its community extensions (for the sake of GPL3 protection?) while Vim keeps it slightly lighter. -## Learning curve +** Learning curve + :PROPERTIES: + :CUSTOM_ID: learning-curve + :END: An editor is not the easiest tool to master. It takes time getting used to the bindings. It will only start proving efficient once the bindings have been @@ -198,11 +208,14 @@ comfortable at customizing the editor, that is, learning the scripting language and the editor API to make it fit all your needs. And your needs will become legions over the years. -## Bindings +** Bindings + :PROPERTIES: + :CUSTOM_ID: bindings + :END: -The venerable vi has left its mark in computer history: its modal approach has +The venerable =vi= has left its mark in computer history: its modal approach has inspired many unrelated programs to use its bindings. Getting familiar with the -basics of vi will help using many programs smoothly. +basics of =vi= will help using many programs smoothly. Emacs has generated a similar phenomenon, maybe a tad more limited. Some tools (shells, Matlab) also offer Emacs key bindings. @@ -210,28 +223,32 @@ Emacs has generated a similar phenomenon, maybe a tad more limited. Some tools This had become prevalent to the point that bindings can be classified into families: -* vi -* Emacs -* CUA +- vi +- Emacs +- CUA CUA are the bindings that IBM had laid down in the 1980 and that are now (2016) -considered "standard": Ctrl-c, Ctrl-v and the like. If you think that Emacs -and vi go "against the rule", note that they were first... +considered "standard": =Ctrl-c=, =Ctrl-v= and the like. If you think that Emacs +and =vi= go "against the rule", note that they were first... Switching from a keybinding family to another can be disturbing: I personally try to stick to one as much as possible. Which also means that it might be a good idea not to change the default bindings of the most common actions within Emacs and Vim. That is more a matter of personal taste though. -## Versionable configuration +** Versionable configuration + :PROPERTIES: + :CUSTOM_ID: versionable-configuration + :END: Last but not least, an extensible text editor means you will want to save and sync your configuration across machines. If the configuration is not saved in a versionable format, the editor is not an acceptable tool for the purpose... - - -# Project editing +* Project editing + :PROPERTIES: + :CUSTOM_ID: project-editing + :END: Using a text editor a programmer is often bound to managing big sets of files for each projects. This has set the need for IDEs. But not so fast. What is most @@ -249,7 +266,9 @@ tree) within or outside a text editor. More on that later. What you will need most in big projects: -> Surfing the whole code base as fast as possible. +#+BEGIN_QUOTE + Surfing the whole code base as fast as possible. +#+END_QUOTE It is not only about finding files, it is about finding chunks of code. With little typing and little cognitive effort. When you write code, you actually @@ -260,7 +279,10 @@ spend a lot of time reading the existing code: - Checking out algorithms. - Looking for files as a whole (head comment and content). -## Incremental narrowing search +** Incremental narrowing search + :PROPERTIES: + :CUSTOM_ID: incremental-narrowing-search + :END: Above we mentioned the following thinking process: "list first, then think about what you want, then select". This is counter-productive since you should know @@ -272,18 +294,17 @@ The "classic" UI design bound the the former thinking process is made of lists of objects the user is looking for: file lists, buffer lists, text search lists, etc. The user browses them and picks the desired items. -_Incremental narrowing search_ is a much more productive design reflecting the +/Incremental narrowing search/ is a much more productive design reflecting the latter mindset: the list of results is filtered as you type while it is ordered by best match. The typed text needs only be approximate. Which makes it very fast to look for anything, should you know the exact name, remember part of it, or just be looking for some keywords. -Implementations of such UI designs include [fzf][] and [Emacs Helm][helm]. fzf -runs from the shell, it will let you search files, shell history and so on. -Emacs Helm is similar but runs for any lookup action within Emacs. - -[fzf]: https://github.com/junegunn/fzf -[helm]: https://emacs-helm.github.io/helm/ +Implementations of such UI designs include +[[https://github.com/junegunn/fzf][fzf]] and +[[https://emacs-helm.github.io/helm/][Emacs Helm]]. =fzf= runs from the shell, +it will let you search files, shell history and so on. Emacs Helm is similar but +runs for any lookup action within Emacs. The strong point of this design is its universality: it is not specific to one action, it can be used pretty much anywhere the user can make a request with @@ -298,33 +319,38 @@ words: - Looking for an entry in some command history or in the copy-paste clipboard. - You name it! -[Linus Torvalds on git](http://www.youtube.com/watch?v=4XpnKHJAok8): +[[http://www.youtube.com/watch?v=4XpnKHJAok8][Linus Torvalds on git]]: -> It's the content that matter, it's not actually the files. +#+BEGIN_QUOTE + It's the content that matter, it's not actually the files. +#+END_QUOTE It is most important to be able to browse your project by content as it should not matter if a portion of code got moved to another file. It is also a cheap way to find references: forcing exact matches with a regexp -like \ will make it reasonably accurate. +like =\= will make it reasonably accurate. -It is an inefficient go to definition however, which only proves useful if you +It is an inefficient =go to definition= however, which only proves useful if you don't have any semantic analysis at hand for your programming language. -## Go to definition +** Go to definition + :PROPERTIES: + :CUSTOM_ID: go-to-definition + :END: I used to work for a company that was working on a massive code base. On my first day I was told "disable auto-completion and all those cumbersome, slow -features, then activate go to definition." As a matter of fact, all +features, then activate =go to definition=." As a matter of fact, all programmers in the company were using and abusing of this one feature, almost exclusively. Since the code base was immense and the documentation somewhat lacking (don't be surprised...), contributing to the code-base implied reading big portions of the existing code and thus going down the huge hierarchy of structures and functions. -Go to definition proved to be by far the most useful navigation function. If +=Go to definition= proved to be by far the most useful navigation function. If the code base is well written, a glimpse at the definition will give you a full understanding of the functions you are calling and the data structures you are manipulating, both from comments and code. When badly written (lacking in -clarity or comments...), you can jump straight to the code to decrypt what -it truly does! +clarity or comments...), you can jump straight to the code to decrypt what it +truly does! diff --git a/articles/indentation.org b/articles/indentation.org index bfd1cd5e547beb1bd33698c88ce450ed9169042b..eae143c16d05461eb154fa867ea23258fd432ec7 100644 --- a/articles/indentation.org +++ b/articles/indentation.org @@ -1,11 +1,11 @@ -% Indentation: A Rationale +#+TITLE: Indentation: A Rationale This article sketches a rationale on source code indentation. Indentation is a never-ending holy war between programmers. The divergence that resulted led to the development of many complex tools (GNU indent, Uncrustify, -to name a few) and every decent text editor to implement a complex -indentation engine. +to name a few) and every decent text editor to implement a complex indentation +engine. Teams have to reach an agreement on indentation. If you happen to be part of several teams, chances are high that you have to work with different @@ -14,9 +14,9 @@ conventions. This pushes editors to implement yet another complex and unreliable feature: indentation guessing. -Versioning and diff tools bring yet more trouble to the case. A simple +Versioning and =diff= tools bring yet more trouble to the case. A simple open/save on a file with a different editor than the original can lead to a huge -diff. +=diff=. Discussions on how to solve this problem have proven to inevitably reach a dead-end. The following sketch of rationale does not claim to solve it either, @@ -24,102 +24,138 @@ however I would like to point out how simple measures taken right from the inception of a programming language can help reducing all the hassle the indentation question creates. -# Rationale +* Rationale + :PROPERTIES: + :CUSTOM_ID: rationale + :END: Main arguments: -* Readability matters. Code is read more than it is written. -* Code changes should be versioning-friendly. -* Freedom of form for clarity where that matters. Tools should not constrain you. +- Readability matters. Code is read more than it is written. +- Code changes should be versioning-friendly. +- Freedom of form for clarity where that matters. Tools should not constrain + you. Lesser arguments: -* The less bytes the better. -* The indentation engine should be short and simple. +- The less bytes the better. +- The indentation engine should be short and simple. -# The Go example +* The Go example + :PROPERTIES: + :CUSTOM_ID: the-go-example + :END: Let us consider the Go language (first issued in 2010). The official -implementation ships with an indentation/formatting tool called gofmt. From the -[official website](http://blog.golang.org/go-fmt-your-code): - -> Gofmt'd code is: -> -> * easier to write: never worry about minor formatting concerns while hacking away, -> * easier to read: when all code looks the same you need not mentally convert others' formatting style into something you can understand. -> * easier to maintain: mechanical changes to the source don't cause unrelated changes to the file's formatting; diffs show only the real changes. -> * uncontroversial: never have a debate about spacing or brace position ever again! +implementation ships with an indentation/formatting tool called =gofmt=. From +the [[http://blog.golang.org/go-fmt-your-code][official website]]: + +#+BEGIN_QUOTE + Gofmt'd code is: + + - easier to write: never worry about minor formatting concerns while hacking + away, + - easier to read: when all code looks the same you need not mentally convert + others' formatting style into something you can understand. + - easier to maintain: mechanical changes to the source don't cause unrelated + changes to the file's formatting; diffs show only the real changes. + - uncontroversial: never have a debate about spacing or brace position ever + again! +#+END_QUOTE This basically brings an end to any of the aforementioned conflicts on indentation. -gofmt is a tool for both indenting and style. Important things to note however, -it will not +=gofmt= is a tool for both indenting and style. Important things to note +however, it will not -* touch your comments; -* split nor join lines; -* change empty lines. +- touch your comments; +- split nor join lines; +- change empty lines. However, it will: -* change spacing between tokens; -* indent your code conforming to one precise style (more or less in K&R style); -* tabify the code (no spaces, no alignment). +- change spacing between tokens; +- indent your code conforming to one precise style (more or less in K&R style); +- tabify the code (no spaces, no alignment). Let us analyze these choices. -* Comment style is extremely varied, and there is no clear winner. gofmt is a -_standard_ tool, and as such it does not claim to impose a comment style. - -* Line splitting depends on line width (which value is left to the programmer's -choice), and _where it feels right_. This can obviously not be determined -computationally. So it seems wise not to touch it. However it does merge multiple -empty lines between two comments. I am not sure why it is being inconsistent here. +- Comment style is extremely varied, and there is no clear winner. =gofmt= is a + /standard/ tool, and as such it does not claim to impose a comment style. -* Empty lines are used to separate parts of the program, again, _where it feels -right_. This should not be changed either. +- Line splitting depends on line width (which value is left to the programmer's + choice), and /where it feels right/. This can obviously not be determined + computationally. So it seems wise not to touch it. However it does merge + multiple empty lines between two comments. I am not sure why it is being + inconsistent here. -* It is very hard for a human being to be consistent with token spacing. Rules -can vary here. All that matters is that the end result is consistent and -readable. +- Empty lines are used to separate parts of the program, again, /where it feels + right/. This should not be changed either. -* Same thing for indentation. +- It is very hard for a human being to be consistent with token spacing. Rules + can vary here. All that matters is that the end result is consistent and + readable. -* The use of tabulations over spaces can be argued. Same thing for alignment, -despite being more subject to personal opinions. I will discuss this in later -sections. +- Same thing for indentation. +- The use of tabulations over spaces can be argued. Same thing for alignment, + despite being more subject to personal opinions. I will discuss this in later + sections. Now let us come back to our rationale. -* > Readability matters. Code is read more than it is written. +- + + #+BEGIN_QUOTE + Readability matters. Code is read more than it is written. + #+END_QUOTE + + The choice of style for indentation and token spacing is good enough: it is + close to K&R, a style which has been widely accepted and source of inspiration + for many other styles. + +- - The choice of style for indentation and token spacing is good enough: it is -close to K&R, a style which has been widely accepted and source of inspiration -for many other styles. + #+BEGIN_QUOTE + Code changes should be versioning-friendly. + #+END_QUOTE -* > Code changes should be versioning-friendly. + =gofmt= ensures this. - gofmt ensures this. +- -* > Freedom of form for clarity where that matters. Tools should not constrain you. + #+BEGIN_QUOTE + Freedom of form for clarity where that matters. Tools should not constrain + you. + #+END_QUOTE - gofmt leaves you freedom on how you separate parts of your code with -linebreaks (except for comments) and how you format comments. What cannot be -decided computationally is left to the author. + =gofmt= leaves you freedom on how you separate parts of your code with + linebreaks (except for comments) and how you format comments. What cannot be + decided computationally is left to the author. -* > The less bytes the better. +- - The use of tabs and no alignment help. + #+BEGIN_QUOTE + The less bytes the better. + #+END_QUOTE -* > The indentation engine should be short and simple. + The use of tabs and no alignment help. - Rules are just a few. No alignment helps. +- + #+BEGIN_QUOTE + The indentation engine should be short and simple. + #+END_QUOTE -# A solution + Rules are just a few. No alignment helps. -gofmt choices look reasonable in regard to our rationale. But the quality of +* A solution + :PROPERTIES: + :CUSTOM_ID: a-solution + :END: + +=gofmt= choices look reasonable in regard to our rationale. But the quality of the style are not worth discussing in face of the invaluable advantages it provides. Its ultimate goal, as stated in the above quote, is not to format code, it is to relieve the programmer from the burden of style and indentation, @@ -129,7 +165,7 @@ Being part of the standard distribution helps a great deal to bring an end to the style and indentation wars. The solution to the style and indentation problem is not lying in a helpless -attempt to find the best rules, it lies in providing a _standard_ formatting +attempt to find the best rules, it lies in providing a /standard/ formatting tool for a language that follows some rational criteria. If a language has been out for years without a formatting tool, as it is the @@ -147,157 +183,206 @@ effort. In the next sections, I will argue more on specific indentation choices. -# Tabulations +* Tabulations + :PROPERTIES: + :CUSTOM_ID: tabulations + :END: The use of tabs over spaces is a source of flaming debates. However it seems -like there is a (not so overwhelming) majority of programmers using _tabs only_ -or _tabs + spaces for alignment_. +like there is a (not so overwhelming) majority of programmers using /tabs only/ +or /tabs + spaces for alignment/. Pros: -* It uses less bytes as long as indentation is >1. Who uses an indentation of 1 -anyway? +- It uses less bytes as long as indentation is >1. Who uses an indentation of 1 + anyway? -* The tab width is adaptable live. This it _the one argument_ for tabs against -spaces. Values between 2 and 8 are customary. 2 leaves more space on screen -which can be handy in restricted conditions. But 8 allows for more clarity, in particular -to distinguish big nested blocks. The [Linux kernel style][linuxstyle] provides a -rationale for using a tab width of 8. +- The tab width is adaptable live. This it /the one argument/ for tabs against + spaces. Values between 2 and 8 are customary. 2 leaves more space on screen + which can be handy in restricted conditions. But 8 allows for more clarity, in + particular to distinguish big nested blocks. The + [[https://www.kernel.org/doc/Documentation/CodingStyle][Linux kernel style]] + provides a rationale for using a tab width of 8. - When using spaces, if one uses 2-spaces indentation rule, the code can not be -made more readable without actually modifying the file. + When using spaces, if one uses 2-spaces indentation rule, the code can not be + made more readable without actually modifying the file. Cons: -* The code will not display identically across different configurations. But in -regard to the previous point, this is a feature, not an issue. - +- The code will not display identically across different configurations. But in + regard to the previous point, this is a feature, not an issue. Back to the rationale: -* > Readability matters. Code is read more than it is written. +- - Tabs allow for flexibility in readability. + #+BEGIN_QUOTE + Readability matters. Code is read more than it is written. + #+END_QUOTE -* > Code changes should be versioning-friendly. + Tabs allow for flexibility in readability. - Changing the tab width does not change the file. Changing space indentation -does. +- -* > Freedom of form for clarity where that matters. Tools should not constrain you. + #+BEGIN_QUOTE + Code changes should be versioning-friendly. + #+END_QUOTE - Tab width is left to the user taste and needs. + Changing the tab width does not change the file. Changing space indentation + does. -* > The less bytes the better. +- - Tab use less bytes than spaces. + #+BEGIN_QUOTE + Freedom of form for clarity where that matters. Tools should not constrain + you. + #+END_QUOTE -* > The indentation engine should be short and simple. + Tab width is left to the user taste and needs. - N/A. +- -# Alignment + #+BEGIN_QUOTE + The less bytes the better. + #+END_QUOTE -Alignment is mostly seen in tables and in function calls: + Tab use less bytes than spaces. + +- - foo = { bar, - baz } + #+BEGIN_QUOTE + The indentation engine should be short and simple. + #+END_QUOTE - foo ( bar, - baz ) + N/A. +* Alignment + :PROPERTIES: + :CUSTOM_ID: alignment + :END: + +Alignment is mostly seen in tables and in function calls: + +#+BEGIN_EXAMPLE + foo = { bar, + baz } + + foo ( bar, + baz ) +#+END_EXAMPLE Pro: -* It makes structured code look clear. +- It makes structured code look clear. Cons: -* Alignment, as for style, is determined by many parameters. The question of -what part of the code should be aligned is hard to answer both for humans and -machines. +- Alignment, as for style, is determined by many parameters. The question of + what part of the code should be aligned is hard to answer both for humans and + machines. - Body only: + Body only: - int foo[3] = { bar, - baz, - qux - }; + #+BEGIN_EXAMPLE + int foo[3] = { bar, + baz, + qux + }; + #+END_EXAMPLE - Body + openers/closers: + Body + openers/closers: - int foo[3] = { bar, - baz, - qux - }; + #+BEGIN_EXAMPLE + int foo[3] = { bar, + baz, + qux + }; + #+END_EXAMPLE -* Alignment is hard to implement in formatters and editors. All the parameters -must be configurable, or else the tool is constraining the formatting. +- Alignment is hard to implement in formatters and editors. All the parameters + must be configurable, or else the tool is constraining the formatting. - For example, if alignment yields an ugly result as in + For example, if alignment yields an ugly result as in - foobar('arg1', function () - print('foobar') - end, - 'arg3' - ) + #+BEGIN_EXAMPLE + foobar('arg1', function () + print('foobar') + end, + 'arg3' + ) + #+END_EXAMPLE - one can "fix" this by changing the outline: + one can "fix" this by changing the outline: - foobar('arg1', - function () - print('foobar') - end, - 'arg3' - ) + #+BEGIN_EXAMPLE + foobar('arg1', + function () + print('foobar') + end, + 'arg3' + ) + #+END_EXAMPLE - In that case you cannot control clarity, you are forced to obey to the limitation -of your editor. + In that case you cannot control clarity, you are forced to obey to the + limitation of your editor. Alignment can bring some clarity. As such, our rationale sounds very favourable: -> Readability matters. Code is read more than it is written. +#+BEGIN_QUOTE + Readability matters. Code is read more than it is written. +#+END_QUOTE However it should be noted that unaligned but indented code is notwithstanding readable: - foo = { - bar, - baz - } +#+BEGIN_EXAMPLE + foo = { + bar, + baz + } +#+END_EXAMPLE And the other parts of the rationale are not as positive: -> Code changes should be versioning-friendly. +#+BEGIN_QUOTE + Code changes should be versioning-friendly. +#+END_QUOTE This is never the case with alignment: +#+BEGIN_EXAMPLE + foo = { bar, + baz + } - foo = { bar, - baz - } + foobar = { bar, + baz + } - foobar = { bar, - baz - } + foo = { + bar, + baz + } +#+END_EXAMPLE - foo = { - bar, - baz - } - - -> Freedom of form for clarity where that matters. Tools should not constrain you. +#+BEGIN_QUOTE + Freedom of form for clarity where that matters. Tools should not constrain + you. +#+END_QUOTE This is not the case if your editor is not customizable enough to let you have fine-grained control over the alignment rules. -> The less bytes the better. +#+BEGIN_QUOTE + The less bytes the better. +#+END_QUOTE Alignment is always using more bytes than unaligned code. -> The indentation engine should be short and simple. +#+BEGIN_QUOTE + The indentation engine should be short and simple. +#+END_QUOTE Alignment is obviously harder to implement than no alignment at all; moreover, it can be quite tricky to implement a generic alignment engine that is @@ -306,44 +391,64 @@ customizable enough so as not to impede freedom of form. In practice, I believe that the fanciness alignment provides is not worth its downsides. -## Other considerations - -### Inner alignment - - map { - key = value1, - longerkey = value2, - } - -Inner alignment is independent of indentation, and should use spaces only. -It often increases clarity in long and complex structure. The downside is that -as soon as you add an entry that does not fit the alignment, you will have to +** Other considerations + :PROPERTIES: + :CUSTOM_ID: other-considerations + :END: + +*** Inner alignment + :PROPERTIES: + :CUSTOM_ID: inner-alignment + :END: + +#+BEGIN_EXAMPLE + map { + key = value1, + longerkey = value2, + } +#+END_EXAMPLE + +Inner alignment is independent of indentation, and should use spaces only. It +often increases clarity in long and complex structure. The downside is that as +soon as you add an entry that does not fit the alignment, you will have to re-align everything. This is a bit tricky to do automatically with the most advanced editors, while being impossible with all the others. -### End-of-line alignment +*** End-of-line alignment + :PROPERTIES: + :CUSTOM_ID: end-of-line-alignment + :END: This is usually restricted to comments. One should never spread end-of-line comments over different indentation levels. In the following example with a tab width of 2, - if (foo) { // Long comment - bar; // spread over - } // Multiple line +#+BEGIN_EXAMPLE + if (foo) { // Long comment + bar; // spread over + } // Multiple line +#+END_EXAMPLE switching to a tab width of 4 will break the alignment. - if (foo) { // Long comment - bar; // spread over - } // Multiple line - +#+BEGIN_EXAMPLE + if (foo) { // Long comment + bar; // spread over + } // Multiple line +#+END_EXAMPLE -# Tools +* Tools + :PROPERTIES: + :CUSTOM_ID: tools + :END: The only help you can find for incurable languages like C lies in formatting tools. They will help a team working with a consistently formatted code base. -## GNU indent +** GNU indent + :PROPERTIES: + :CUSTOM_ID: gnu-indent + :END: It uses the GNU style by default, and every option is meant to change its behaviour starting from there. This is hardly transparent or convenient. @@ -352,221 +457,281 @@ GNU indent comes with the possibility to split or join lines. If this option is set, and maximum line-width is set to, let us say 70 characters, it will transform - if (long_condition1 && long_condition2 && long_condition3 && long_condition4) { +#+BEGIN_EXAMPLE + if (long_condition1 && long_condition2 && long_condition3 && long_condition4) { +#+END_EXAMPLE to - if (long_condition1 - && long_condition2 - && long_condition3 - && long_condition4) { +#+BEGIN_EXAMPLE + if (long_condition1 + && long_condition2 + && long_condition3 + && long_condition4) { +#+END_EXAMPLE If line splitting is off, then it will force joining lines! I.e. it will transform - if (long_condition1 - && long_condition2 - && long_condition3 - && long_condition4) { +#+BEGIN_EXAMPLE + if (long_condition1 + && long_condition2 + && long_condition3 + && long_condition4) { +#+END_EXAMPLE to - if (long_condition1 && long_condition2 && long_condition3 && long_condition4) { - -and does not leave the formatting to the user. It can be very annoying for long lines. +#+BEGIN_EXAMPLE + if (long_condition1 && long_condition2 && long_condition3 && long_condition4) { +#+END_EXAMPLE +and does not leave the formatting to the user. It can be very annoying for long +lines. -## Astyle +** Astyle + :PROPERTIES: + :CUSTOM_ID: astyle + :END: As for GNU indent, Astyle comes with a default formatting (this is not a good thing). Beside it cannot toggle alignment off. Its options have some yet unseen intricacies: -> Using the k&r option may cause problems because of the &. This can be resolved -> by enclosing the k&r in quotes (e.g. --style="k&r") or by using one of the -> alternates --style=kr or --style=k/r. +#+BEGIN_QUOTE + Using the k&r option may cause problems because of the &. This can be resolved + by enclosing the k&r in quotes (e.g. --style="k&r") or by using one of the + alternates --style=kr or --style=k/r. +#+END_QUOTE -The [documentation](http://astyle.sourceforge.net/astyle.html) is of debatable quality: +The [[http://astyle.sourceforge.net/astyle.html][documentation]] is of debatable +quality: -> Also known as Kernel Normal Form (KNF) style, this is the style used in the -> Linux kernel. +#+BEGIN_QUOTE + Also known as Kernel Normal Form (KNF) style, this is the style used in the + Linux kernel. +#+END_QUOTE -KNF is used for the [*BSD kernels](https://en.wikipedia.org/wiki/Kernel_Normal_Form). -(See the external links.) +KNF is used for the [[https://en.wikipedia.org/wiki/Kernel_Normal_Form][*BSD +kernels]]. (See the external links.) -> "One True Brace Style" formatting/indenting uses linux brackets and adds -> brackets to unbracketed one line conditional statements. +#+BEGIN_QUOTE + "One True Brace Style" formatting/indenting uses linux brackets and adds + brackets to unbracketed one line conditional statements. +#+END_QUOTE The Linux kernel style already enforces the use of brackets on one-line -statements. Note that it is not only restricted to conditional statements. -See the [Linux kernel coding style][linuxstyle]. +statements. Note that it is not only restricted to conditional statements. See +the [[https://www.kernel.org/doc/Documentation/CodingStyle][Linux kernel coding +style]]. -To put it together, it is maintained with [Subversion](http://harmful.cat-v.org/software/svn/). +To put it together, it is maintained with +[[http://harmful.cat-v.org/software/svn/][Subversion]]. - -## Uncrustify +** Uncrustify + :PROPERTIES: + :CUSTOM_ID: uncrustify + :END: It is much more complete than its competitors and comes with an option for virtually everything. It will not let you run it without specifying an option file. As such it is totally transparent. Alignment is customizable. - -# Algorithm +* Algorithm + :PROPERTIES: + :CUSTOM_ID: algorithm + :END: One of our points in the rationale is dedicated to the simplicity of the algorithm. We will investigate a few implementation challenges to help us understand why additional features such as alignment can impede simplicity. -## Terms - -* Opener: a token that increases the indentation of the following part. -Typically an opening parenthesis/bracket, or a token signaling the beginning of -a statement. -* Closer: the counter part of an opener. Typically a closing -parenthesis/bracket, or a token signaling the end of a statement. -* Middler: a key word found between an opener and a closer. It can impact -indentation under specific circumstance. Typically keywords such as then or else. -* Continuation: a line is said to be continuing if it is semantically attached to -the previous line. For instance - - foo = bar + - baz - - foo = "string on\ - multiple lines" - -## Edge cases +** Terms + :PROPERTIES: + :CUSTOM_ID: terms + :END: + +- Opener: a token that increases the indentation of the following part. + Typically an opening parenthesis/bracket, or a token signaling the beginning + of a statement. +- Closer: the counter part of an opener. Typically a closing + parenthesis/bracket, or a token signaling the end of a statement. +- Middler: a key word found between an opener and a closer. It can impact + indentation under specific circumstance. Typically keywords such as =then= or + =else=. +- Continuation: a line is said to be continuing if it is semantically attached + to the previous line. For instance + + #+BEGIN_EXAMPLE + foo = bar + + baz + + foo = "string on\ + multiple lines" + #+END_EXAMPLE + +** Edge cases + :PROPERTIES: + :CUSTOM_ID: edge-cases + :END: Before going any further, let us review a few generic edge cases. -### Stacked openers and closers +*** Stacked openers and closers + :PROPERTIES: + :CUSTOM_ID: stacked-openers-and-closers + :END: -What if several openers appear on the same line? Shall we indent once -or stack the indent values? Indenting once looks fine here: +What if several openers appear on the same line? Shall we indent once or stack +the indent values? Indenting once looks fine here: - if cond1 { if cond2 { - // code - }} +#+BEGIN_EXAMPLE + if cond1 { if cond2 { + // code + }} +#+END_EXAMPLE but not right there: - if cond1 { if cond2 { - // code - } - } +#+BEGIN_EXAMPLE + if cond1 { if cond2 { + // code + } + } +#+END_EXAMPLE Besides it breaks the rule of versioning-friendliness from the rationale: - - if cond1 { - if cond2 { - // code - }} +#+BEGIN_EXAMPLE + if cond1 { + if cond2 { + // code + }} +#+END_EXAMPLE This leads us to the closers: should all closers appearing on the same line unindent once? The choice is this - - if cond1 { if cond2 { if cond3 { - // code - }}} +#+BEGIN_EXAMPLE + if cond1 { if cond2 { if cond3 { + // code + }}} +#+END_EXAMPLE versus that: - if cond1 { if cond2 { if cond3 { - // code - }}} +#+BEGIN_EXAMPLE + if cond1 { if cond2 { if cond3 { + // code + }}} +#+END_EXAMPLE It is hard to see after 2 indentation levels which statement the closers are actually closing. Lisp hackers will probably not mind. - However, it should be noted that closers should unindent their own line only if they appear first. It is important for the indentation to be meaningful in -regard to block nesting. In the following it is clearer that foo() is called -inside the condition on cond2: +regard to block nesting. In the following it is clearer that =foo()= is called +inside the condition on =cond2=: - if cond1 then - if cond2 - then foo() end end +#+BEGIN_EXAMPLE + if cond1 then + if cond2 + then foo() end end +#+END_EXAMPLE than here: - if cond1 then - if cond2 - then foo() end end +#+BEGIN_EXAMPLE + if cond1 then + if cond2 + then foo() end end +#+END_EXAMPLE or there in C: - if (cond1) { - if (cond2) - { foo() } } - +#+BEGIN_EXAMPLE + if (cond1) { + if (cond2) + { foo() } } +#+END_EXAMPLE -### Continuation +*** Continuation + :PROPERTIES: + :CUSTOM_ID: continuation + :END: Line continuation is one of the trickiest part of indentation. It is considered good practice to keep line width within a reasonable range, typically around 80. -Some lines will eventually end up being too long to fit within the desired width, -and will have to get split. +Some lines will eventually end up being too long to fit within the desired +width, and will have to get split. Determining if a line is continuing or not involves some analysis. A naive approach would be: -* Check both the last token of previous line and the first -token of current line. At least one of them must be a continuation token -(boolean logic, newline escape, etc.). +- Check both the last token of previous line and the first token of current + line. At least one of them must be a continuation token (boolean logic, + newline escape, etc.). -* If current line is continuing and previous line is not, increase indentation. +- If current line is continuing and previous line is not, increase indentation. -* If current line is not continuing and previous line is, decrease indentation. +- If current line is not continuing and previous line is, decrease indentation. Continuing lines should typically be indented one level up. - foo = bar + - baz +#+BEGIN_EXAMPLE + foo = bar + + baz +#+END_EXAMPLE For statement conditions, both the opener and the continuation will increase the indentation level: - if foo && - bar - then ... +#+BEGIN_EXAMPLE + if foo && + bar + then ... +#+END_EXAMPLE The double indentation is not really desired. The only reason for considering -if as an opener is for cases like this: +=if= as an opener is for cases like this: - if - foo && - bar - then ... +#+BEGIN_EXAMPLE + if + foo && + bar + then ... +#+END_EXAMPLE -We can easily solve this issue by considering then as an opener and if as a +We can easily solve this issue by considering =then= as an opener and =if= as a continuation token. -In Go, the curly braces are obvious openers and closers. The if and for -keywords should be continuation tokens. -In C, the parenthesis adds up to the indentation, so the problem is still not solved: +In Go, the curly braces are obvious openers and closers. The =if= and =for= +keywords should be continuation tokens. In C, the parenthesis adds up to the +indentation, so the problem is still not solved: - if (foo && - bar) { - } +#+BEGIN_EXAMPLE + if (foo && + bar) { + } +#+END_EXAMPLE Workaround suggestions: -* Leave it as it is. -* Add an exception. -* Feel free to send me a smarter solution! +- Leave it as it is. +- Add an exception. +- Feel free to send me a smarter solution! Another tricky problem: - a = foo + - bar(baz, - barbaz - ) + - foobar - +#+BEGIN_EXAMPLE + a = foo + + bar(baz, + barbaz + ) + + foobar +#+END_EXAMPLE After the function call, the line is not seen as continuing anymore, so indentation gets decreased by 1 level, while being increased by 1 level because @@ -574,74 +739,86 @@ of the opener (so no change overall). It gets even worse when nested: - a = foo + - bar(baz + - bazbaz - ) + - foobar - - b = foo + - bar(baz( - bazbaz - ) - ) + - foobar +#+BEGIN_EXAMPLE + a = foo + + bar(baz + + bazbaz + ) + + foobar + + b = foo + + bar(baz( + bazbaz + ) + ) + + foobar +#+END_EXAMPLE The solution to this problem involves a more complex continuation algorithm. We need to use a stack to remember the level of indentation of the nested continuations. -* Indent line. +- Indent line. -* Check if continuing: either the end token on previous line or the start token -on current line must be a continuation token. +- Check if continuing: either the end token on previous line or the start token + on current line must be a continuation token. -* If current indentation does not match the stack and if continuing, increase -indent and push the new indentation on the stack. +- If current indentation does not match the stack and if continuing, increase + indent and push the new indentation on the stack. -* If not continuing and current indentation is < stack indentation, pop the -stack and decrease indent. +- If not continuing and current indentation is < stack indentation, pop the + stack and decrease indent. -The cases not matched by the previous conditions do not change the indentation level. +The cases not matched by the previous conditions do not change the indentation +level. +** An example algorithm + :PROPERTIES: + :CUSTOM_ID: an-example-algorithm + :END: -## An example algorithm - -The following algorithm yields an indentation engine close in practice to the one -found in gofmt. The indentation is computed by looking at current and previous -lines only, thus being efficient. +The following algorithm yields an indentation engine close in practice to the +one found in =gofmt=. The indentation is computed by looking at current and +previous lines only, thus being efficient. What if previous line indentation is wrong? It does not matter, as indenting the whole file at once will make sure every line is indented properly. -_Unmatched_ means _not matched on the same line_. Openers match their respective +/Unmatched/ means /not matched on the same line/. Openers match their respective closer forward and vice-versa. -*First pass:* +/First pass:/ - On current line: - -1 on every closer until an unmatched opener is met, if first token is a closer. - -1 if first token is a middler. +#+BEGIN_EXAMPLE + On current line: + -1 on every closer until an unmatched opener is met, if first token is a closer. + -1 if first token is a middler. - On previous line: - + current indentation. - +1 on every unmatched opener. - +1 if first token is a middler. - -1 on every unmatched closer if first token is not a closer. + On previous line: + + current indentation. + +1 on every unmatched opener. + +1 if first token is a middler. + -1 on every unmatched closer if first token is not a closer. +#+END_EXAMPLE For this last case, it is worth noting that if the first token is a middler, then the first unmatched closer is compensating the indentation change. -*Second pass:* - - + continuation indentation. +/Second pass:/ -For the continuation indentation, we use the algorithm from the previous section. +#+BEGIN_EXAMPLE + + continuation indentation. +#+END_EXAMPLE -# References +For the continuation indentation, we use the algorithm from the previous +section. -* [EmacsWiki: Tabs are evil](http://www.emacswiki.org/emacs/TabsAreEvil) -* [Wikipedia: Indent style](https://en.wikipedia.org/wiki/Indent_style) -* [Linux kernel coding style][linuxstyle] +* References + :PROPERTIES: + :CUSTOM_ID: references + :END: -[linuxstyle]: https://www.kernel.org/doc/Documentation/CodingStyle +- [[http://www.emacswiki.org/emacs/TabsAreEvil][EmacsWiki: Tabs are evil]] +- [[https://en.wikipedia.org/wiki/Indent_style][Wikipedia: Indent style]] +- [[https://www.kernel.org/doc/Documentation/CodingStyle][Linux kernel coding + style]] diff --git a/articles/keymap.org b/articles/keymap.org index 8a0b0ef4aa5ed17fc8c15176f553430bbee61821..e69452d09c2d82f21307b06570596f4e64242b0a 100644 --- a/articles/keymap.org +++ b/articles/keymap.org @@ -1,10 +1,12 @@ -% Mastering the keyboard +#+TITLE: Mastering the keyboard The following article deals with techniques to optimize comfort and speed using a computer. - -# Touch typing +* Touch typing + :PROPERTIES: + :CUSTOM_ID: touch-typing + :END: Writing text holds a central part in our use of computers: e-mails, searching, web browsing, programming, configuring, etc. @@ -15,15 +17,17 @@ believe it to be a big time saver, while alleviating the frustration of too much stuttering and too many typos. Touch typing can be trained in a fairly short amount of time. One way would be -to use a trainer program such as _GNU Typist_. It is straightforward to go +to use a trainer program such as /GNU Typist/. It is straightforward to go through the various lessons and the result will be immediately noticeable. -Touch typing is highly dependent on the keyboard layout (a.k.a. _keymap_), so you -might want to choose the keymap wisely before starting the training. Read on for -tips on keymaps. - +Touch typing is highly dependent on the keyboard layout (a.k.a. /keymap/), so +you might want to choose the keymap wisely before starting the training. Read on +for tips on keymaps. -# Mouse-less control +* Mouse-less control + :PROPERTIES: + :CUSTOM_ID: mouse-less-control + :END: The mouse use has increased tremendously since the rise of graphical user interfaces. Mostly for illegitimate reasons. Do we really need a mouse to select @@ -31,21 +35,22 @@ objects or to toggle buttons? The mouse proves to be poor at selecting text. How many times have you tried to select a sentence and missed the last letter? It is equally bad at selecting -objects. -When it comes to interact with the user interface, it is usually faster to use -keyboard shortcuts. Well, we call them "shortcuts" for a reason... +objects. When it comes to interact with the user interface, it is usually faster +to use keyboard shortcuts. Well, we call them "shortcuts" for a reason... The use of a mouse makes sense when there is a need for a continuous input, such as in graphics design, video games, etc. +* Home row vs. arrows + :PROPERTIES: + :CUSTOM_ID: home-row-vs.arrows + :END: -# Home row vs. arrows - -The _home row_ refers to the center row of the alphabetical part of the -keyboard, that is, the characters asdf...jkl; on a QWERTY keyboard. -The standard position is when the index fingers rest on the characters f and -j on a QWERTY keyboard. Those letters usually come with a bump to make -them distinguishable without looking. +The /home row/ refers to the center row of the alphabetical part of the +keyboard, that is, the characters =asdf...jkl;= on a QWERTY keyboard. The +standard position is when the index fingers rest on the characters =f= and =j= +on a QWERTY keyboard. Those letters usually come with a bump to make them +distinguishable without looking. Moving hands from the home row to the arrows back and forth can be a small waste of time that quickly stacks up. The time required for the "context switch" of @@ -56,16 +61,16 @@ interface navigation. Now there are various changes we can make to the environment so that the navigation bindings stick around the home row. First of, you may consider switching your text editor for one that allows -navigation without arrows. Famous examples include _Emacs_ and _Vim_. +navigation without arrows. Famous examples include /Emacs/ and /Vim/. The window manager can have limited bindings to such an extent that it forces the use of arrows or the mouse. Decent window managers usually feature full -keyboard control. Popular examples include _Awesome_ and _i3_. +keyboard control. Popular examples include /Awesome/ and /i3/. Web browsers have become more and more dominant in our use of computers. The way the World Wide Web was designed has put emphasis on the mouse, so that it is now -almost impossible to browse the web without a mouse. Which might be a sign for poor -design from the ground up. But let's not drift off too much. It is still +almost impossible to browse the web without a mouse. Which might be a sign for +poor design from the ground up. But let's not drift off too much. It is still possible to use a graphical web browser while making best use of the keyboard thanks to the "hint" feature. Many Webkit-based browsers offer this feature. It is also possible to edit any field using your favourite editor, which greatly @@ -75,22 +80,25 @@ If you have got the chance to witness a hardcore geek with proper touch typing skills and a keyboard / home row centered environment, you will be amazed by how many actions per minute that geek can perform! - -# Caps-Lock switch +* Caps-Lock switch + :PROPERTIES: + :CUSTOM_ID: caps-lock-switch + :END: The Caps-Lock key is very accessible albeit little used. On the other hand, the -use frequency of keys such as Control or Escape is much higher (in +use frequency of keys such as =Control= or =Escape= is much higher (in particular when using the Emacs or Vim text editors). -Therefore it is very recommended to swap -Caps-Lock with the key you use most. There are several ways of doing this, read -on for an example. +Therefore it is very recommended to swap Caps-Lock with the key you use most. +There are several ways of doing this, read on for an example. This is one of the keymap tweaks that will save you most from wrecking your hands with some carpal tunnel syndrome. - -# International custom keymaps +* International custom keymaps + :PROPERTIES: + :CUSTOM_ID: international-custom-keymaps + :END: Users of languages using a Latin-based alphabet should be familiar with the existence of a variety of "standard" keymaps out there: QWERTY (US, UK, ...), @@ -115,48 +123,54 @@ characters. Besides, it is possible to base the new keymap on QWERTY US, which has some inherent benefits: - Matching parentheses are next to each other. Punctuation tends to be -reasonably accessible (e.g. , and . are unshifted). + reasonably accessible (e.g. =,= and =.= are unshifted). - It is the most widespread keymap, so when somebody wants to use your computer, -chances are high they can type something. + chances are high they can type something. - Most importantly, some programs are ergonomically optimized for QWERTY US, -such as Emacs and Vim. + such as Emacs and Vim. Bonus for scientists: it is possible to add some common mathematical characters, -such as ≠ or ⋅, which can be a big time saver when it comes to writing +such as =≠= or =⋅=, which can be a big time saver when it comes to writing scientific documents. - -## Custom Xkb keymaps +** Custom Xkb keymaps + :PROPERTIES: + :CUSTOM_ID: custom-xkb-keymaps + :END: Let's move on to the details on how to load a custom layout for the X window system without needing administrative rights. -In the following, we will use the xkb folder as our workspace. This folder is -arbitrary. Replace the keymap name custom with any unused name you like. +In the following, we will use the =xkb= folder as our workspace. This folder is +arbitrary. Replace the keymap name =custom= with any unused name you like. -Create an xkb/custom.xkb file: +Create an =xkb/custom.xkb= file: - xkb_keymap { - xkb_keycodes { include "evdev+aliases(qwerty)" }; - xkb_types { include "complete" }; - xkb_compat { include "complete" }; - xkb_symbols { include "pc+custom+inet(evdev)" }; +#+BEGIN_EXAMPLE + xkb_keymap { + xkb_keycodes { include "evdev+aliases(qwerty)" }; + xkb_types { include "complete" }; + xkb_compat { include "complete" }; + xkb_symbols { include "pc+custom+inet(evdev)" }; - // Geometry is completely optional. - // xkb_geometry { include "pc(pc104)" }; - }; + // Geometry is completely optional. + // xkb_geometry { include "pc(pc104)" }; + }; +#+END_EXAMPLE -Fill in xkb/symbols/custom. This file is formatted just like every other Xkb -symbol files, usually located in X11/xkb/symbols/ under /usr/share or -/usr/local/share. +Fill in =xkb/symbols/custom=. This file is formatted just like every other Xkb +symbol files, usually located in =X11/xkb/symbols/= under =/usr/share= or +=/usr/local/share=. Finally, load the new keymap with - xkbcomp -I"xkb" "xkb/custom.xkb" $DISPLAY +#+BEGIN_EXAMPLE + xkbcomp -I"xkb" "xkb/custom.xkb"$DISPLAY +#+END_EXAMPLE For a concrete example, see -[my personal keymap](https://github.com/ambrevar/dotfiles/tree/master/.xkb/). +[[https://github.com/ambrevar/dotfiles/tree/master/.xkb/][my personal keymap]]. It is a superset of QWERTY US which covers almost every language in western Europe, with Caps-Lock and Control swapped. diff --git a/articles/power-apps.org b/articles/power-apps.org index dec0dc2907c3ae3910744c85539aadae65bf4a3a..84753402938424787fc3ea53e5e4bba64cbf7fa2 100644 --- a/articles/power-apps.org +++ b/articles/power-apps.org @@ -1,348 +1,502 @@ -% Power Apps +#+TITLE: Power Apps -The following programs mostly target power users and programmers. Recurring +The following programs mostly target power users and programmers. Recurring characteristics: FOSS, lightweight, fast, versionable configuration (plain text, -non-XML), keyboard-driven, _Unix_ design. Or simply an Emacs package. +non-XML), keyboard-driven, /Unix/ design. Or simply an Emacs package. +**** 3D graphics: Blender + :PROPERTIES: + :CUSTOM_ID: d-graphics-blender + :END: -#### 3D graphics: Blender +**** Archivers: atool, bzip2, cabextract, cdrtools, cpio, gzip, lha, lrzip, lz4, +lzop, p7zip, unace, unarj, unrar, unshield, unzip, upx, xz + :PROPERTIES: + :CUSTOM_ID: archivers-atool-bzip2-cabextract-cdrtools-cpio-gzip-lha-lrzip-lz4-lzop-p7zip-unace-unarj-unrar-unshield-unzip-upx-xz + :END: -#### Archivers: atool, bzip2, cabextract, cdrtools, cpio, gzip, lha, lrzip, lz4, lzop, p7zip, unace, unarj, unrar, unshield, unzip, upx, xz +- =atool= is a wrapper around many tools to unify compression and extraction + from command-line. -* atool is a wrapper around many tools to unify compression and extraction from -command-line. +- =cdrtools= can process ISO files. -* cdrtools can process ISO files. +- =xz= offers a good compression ratio together with great decompression + performance. Compression is quite demanding. =bzip2= has little to offer + compared to =xz=. =p7zip= bundles the features =tar=, =xz= and =ccrypt= in one + program. -* xz offers a good compression ratio together with great decompression -performance. Compression is quite demanding. bzip2 has little to offer -compared to xz. p7zip bundles the features tar, xz and ccrypt in one -program. +- =lrzip= is at its best on large files and with multiple CPUs available. -* lrzip is at its best on large files and with multiple CPUs available. +- =upx= can compress executables. -* upx can compress executables. +- The rest is for decompression. -* The rest is for decompression. +**** Assembly: NASM, Yasm + :PROPERTIES: + :CUSTOM_ID: assembly-nasm-yasm + :END: -#### Assembly: NASM, Yasm +**** Audio editors: Audacity + :PROPERTIES: + :CUSTOM_ID: audio-editors-audacity + :END: -#### Audio editors: Audacity +**** Binary file tools: chrpath, DHEX, Emacs (nhexl-mode), ltrace, nm, od, +strace + :PROPERTIES: + :CUSTOM_ID: binary-file-tools-chrpath-dhex-emacs-nhexl-mode-ltrace-nm-od-strace + :END: -#### Binary file tools: chrpath, DHEX, Emacs (nhexl-mode), ltrace, nm, od, strace +**** Calculation: bc, calc, Emacs (calc, Lisp), Maxima, PARI/GP, Octave + :PROPERTIES: + :CUSTOM_ID: calculation-bc-calc-emacs-calc-lisp-maxima-parigp-octave + :END: -#### Calculation: bc, calc, Emacs (calc, Lisp), Maxima, PARI/GP, Octave +- =bc= and =calc= are simple arbitrary-precision calculators. bc is lighter but + calc has somewhat more features. -* bc and calc are simple arbitrary-precision calculators. bc is lighter but -calc has somewhat more features. +- =PARI/GP= is an extremely fast and advanced algebra system for number theory. + Great for prime numbers and such. -* PARI/GP is an extremely fast and advanced algebra system for number theory. -Great for prime numbers and such. +- =Octave= will serve as a Unix-designed Matlab. -* Octave will serve as a Unix-designed Matlab. +**** Camera capture: Guvcview + :PROPERTIES: + :CUSTOM_ID: camera-capture-guvcview + :END: -#### Camera capture: Guvcview +**** Clipboard tools: xsel + :PROPERTIES: + :CUSTOM_ID: clipboard-tools-xsel + :END: -#### Clipboard tools: xsel +Using [[../emacs-everywhere][Emacs everywhere]], xsel is barely useful. I only +use it to yank the filename of pictures from sxiv. -Using [Emacs everywhere](../emacs-everywhere), xsel is barely useful. -I only use it to yank the filename of pictures from sxiv. +**** Contacts: Abook, Emacs (org-contacts) + :PROPERTIES: + :CUSTOM_ID: contacts-abook-emacs-org-contacts + :END: -#### Contacts: Abook, Emacs (org-contacts) +- Abook is a stand-alone curses contact manager. Contacts are stored in plain + text and as such they are versionable, but the numbering makes up for huge + diffs. -* Abook is a stand-alone curses contact manager. Contacts are stored in plain -text and as such they are versionable, but the numbering makes up for huge -diffs. - -* org-contacts is much more powerful thank Abook: free form, arbitrary fields, -contacts can be fuzzy-searched/retrieved from anywhere within Emacs. +- org-contacts is much more powerful thank Abook: free form, arbitrary fields, + contacts can be fuzzy-searched/retrieved from anywhere within Emacs. Versioning your contacts is a great way to centralize them, instead of spreading them across your mail agent contacts, CSV files, etc. Plain text contacts also -means it is easy to write a converter from a CSV file (e.g. using AWK). +means it is easy to write a converter from a CSV file (e.g. using =AWK=). -#### Cryptography: ccrypt, encfs, GnuPG, pass, Pwgen +**** Cryptography: ccrypt, encfs, GnuPG, pass, Pwgen + :PROPERTIES: + :CUSTOM_ID: cryptography-ccrypt-encfs-gnupg-pass-pwgen + :END: -* GnuPG is overkill when it comes to encrypting files for personal use. +- =GnuPG= is overkill when it comes to encrypting files for personal use. -* ccrypt is good at encrypting single files. +- =ccrypt= is good at encrypting single files. -* encfs can encrypt folders and mount them as an encrypted file system. Files -can be browsed transparently without being ever written in clear to the disk. It -uses the fuse backend which makes it portable across systems at the expense of -speed. +- =encfs= can encrypt folders and mount them as an encrypted file system. Files + can be browsed transparently without being ever written in clear to the disk. + It uses the fuse backend which makes it portable across systems at the expense + of speed. -#### Dictionaries: aspell +**** Dictionaries: aspell + :PROPERTIES: + :CUSTOM_ID: dictionaries-aspell + :END: -#### Disk utilities: gparted, ncdu, parted, rmlint, shred, trash-cli, tree, udiskie, wipe +**** Disk utilities: gparted, ncdu, parted, rmlint, shred, trash-cli, tree, +udiskie, wipe + :PROPERTIES: + :CUSTOM_ID: disk-utilities-gparted-ncdu-parted-rmlint-shred-trash-cli-tree-udiskie-wipe + :END: -* ncdu is a very fast and convenient disk usage analyzer. +- =ncdu= is a very fast and convenient disk usage analyzer. -* trash-cli is a command-line interface implementing FreeDesktop.org's Trash -specification. It can be combined with a file browser for easier use. Emacs -also has the delete-by-moving-to-trash variable. +- =trash-cli= is a command-line interface implementing FreeDesktop.org's Trash + specification. It can be combined with a file browser for easier use. Emacs + also has the =delete-by-moving-to-trash= variable. -* wipe can delete folders securely while shred can only process files. +- =wipe= can delete folders securely while =shred= can only process files. -#### Document processing: groff, lilypond, Markdown, pdf2svg, Texinfo, TeX Live, txt2man +**** Document processing: groff, lilypond, Markdown, pdf2svg, Texinfo, TeX Live, +txt2man + :PROPERTIES: + :CUSTOM_ID: document-processing-groff-lilypond-markdown-pdf2svg-texinfo-tex-live-txt2man + :END: -#### Document readers and tools: antiword, apvlv, catdvi, catdoc, docx2txt, Emacs (pdf-tools), evince, ghostscript, odt2txt, mcomix, poppler, pstotext, unrtf, wv, xchm, zathura +**** Document readers and tools: antiword, apvlv, catdvi, catdoc, docx2txt, +Emacs (pdf-tools), evince, ghostscript, odt2txt, mcomix, poppler, pstotext, +unrtf, wv, xchm, zathura + :PROPERTIES: + :CUSTOM_ID: document-readers-and-tools-antiword-apvlv-catdvi-catdoc-docx2txt-emacs-pdf-tools-evince-ghostscript-odt2txt-mcomix-poppler-pstotext-unrtf-wv-xchm-zathura + :END: -* apvlv, pdf-tools and zathura are light and keyboard-driven. zathura -supports PostScript .ps files and SyncTeX. apvlv supports UMD, HTML and .txt -files. pdf-tools supports SyncTeX and has good text search and selection -facilities, it enables many Emacs features (Helm, Ivy). +- =apvlv=, =pdf-tools= and =zathura= are light and keyboard-driven. =zathura= + supports PostScript .ps files and SyncTeX. =apvlv= supports UMD, HTML and .txt + files. =pdf-tools= supports SyncTeX and has good text search and selection + facilities, it enables many Emacs features (Helm, Ivy). -* Evince is far too heavy but can fill PDF forms. (Yet another ill-conceived -PDF feature...) +- =Evince= is far too heavy but can fill PDF forms. (Yet another ill-conceived + PDF feature...) -* Poppler has numerous PDF converters (e.g. _pdftotext_). +- =Poppler= has numerous PDF converters (e.g. /pdftotext/). -* ghostscript can convert PDF to and from PS. +- =ghostscript= can convert PDF to and from PS. -* Most of these programs can be used for previewing files from some -applications, e.g. ranger. +- Most of these programs can be used for previewing files from some + applications, e.g. =ranger=. -#### E-mail clients: Emacs (mu4e), Mutt +**** E-mail clients: Emacs (mu4e), Mutt + :PROPERTIES: + :CUSTOM_ID: e-mail-clients-emacs-mu4e-mutt + :END: -* Mutt is extensible but somewhat hard to configure. Each e-mail account setup -requires a good deal of manual configuration. +- Mutt is extensible but somewhat hard to configure. Each e-mail account setup + requires a good deal of manual configuration. -* mu4e is moderately easy to configure, extremely extensible with Emacs Lisp and -suffers from far less limitations than Mutt. You can fuzzy-search contacts, -e-mails, preview HTML, display embedded pictures and much more. +- mu4e is moderately easy to configure, extremely extensible with Emacs Lisp and + suffers from far less limitations than Mutt. You can fuzzy-search contacts, + e-mails, preview HTML, display embedded pictures and much more. -#### File browsers: Emacs (dired, Helm), ranger +**** File browsers: Emacs (dired, Helm), ranger + :PROPERTIES: + :CUSTOM_ID: file-browsers-emacs-dired-helm-ranger + :END: -* ranger is configurable and extensible. It can preview any kind of file in -the way you want. It can run an arbitrary command on any file selection and it -remembers the selection in every folder. It can run various powerful commands -conveniently, such as recursive hardlink creation or batch-renaming. +- =ranger= is configurable and extensible. It can preview any kind of file in + the way you want. It can run an arbitrary command on any file selection and it + remembers the selection in every folder. It can run various powerful commands + conveniently, such as recursive hardlink creation or batch-renaming. -* Emacs Helm makes for a revolutionary file browser: typing anything will -fuzzy-filter the current directory, or even subdirectories. The filtered -results can be browsed with special keys: you can select files from different -directories and apply arbitrary actions to them. +- Emacs Helm makes for a revolutionary file browser: typing anything will + fuzzy-filter the current directory, or even subdirectories. The filtered + results can be browsed with special keys: you can select files from different + directories and apply arbitrary actions to them. -#### File synchronization: hsync, rsync +**** File synchronization: hsync, rsync + :PROPERTIES: + :CUSTOM_ID: file-synchronization-hsync-rsync + :END: -* Read the documentation carefully. rsync has a lot of useful options, like ---append-verify, --progress and --size-only. +- Read the documentation carefully. =rsync= has a lot of useful options, like + =--append-verify=, =--progress= and =--size-only=. -#### Finders: Emacs (Helm, Ivy), fzf, tre +**** Finders: Emacs (Helm, Ivy), fzf, tre + :PROPERTIES: + :CUSTOM_ID: finders-emacs-helm-ivy-fzf-tre + :END: -* fzf puts any shell on steroids. To such an extent it can replace any file -browser with no hesitation. It allows you to fuzzy-find anything: files, folders -to cd into, command history, folder history, completion... +- =fzf= puts any shell on steroids. To such an extent it can replace any file + browser with no hesitation. It allows you to fuzzy-find anything: files, + folders to cd into, command history, folder history, completion... -* tre has agrep. +- =tre= has =agrep=. -* If you live in Emacs, fzf and tre are completely superseded by Helm. +- If you live in Emacs, fzf and tre are completely superseded by Helm. -#### FTP clients: curlftpfs, Emacs (TRAMP), NcFTP +**** FTP clients: curlftpfs, Emacs (TRAMP), NcFTP + :PROPERTIES: + :CUSTOM_ID: ftp-clients-curlftpfs-emacs-tramp-ncftp + :END: Emacs' TRAMP allows you to work on remote files (move, delete, download) and edit them transparently: first they are automatically downloaded, then all edits are done locally within Emacs, and last the file is uploaded upon saving. -#### FTP servers: vsftpd - -#### Gaming emulators: DGen, DOSBox, Gens/GS, Mupen64Plus, PCSX-Reloaded, MAME, Wine - -#### Gaming tools: DriConf, TiMidity++, xosd - -* osd_cat from xosd can display text on screen, such as FPS or network -traffic. - -#### Music: Beets, clyrics, cmus, Demlo, Emacs (EMMS), mps-youtube - -* clyrics can display the lyrics of the song currently playing in cmus. - -* cmus is extremely fast at updating the library. Its UI makes it convenient -to browse the library and to create playlists. - -* Emacs' EMMS is similar to cmus with the bonus that it is extensible, it can -fetch lyrics, it can resume state upon restart and it can display album covers -within the music library tree. - -* mps-youtube can build albums from Youtube links automatically and save the -resulting playlist locally. - -#### Network monitors: Aircrack-ng, ngrep, nmap, Tcpdump, Wireshark - -#### News readers: Emacs (elfeed), Newsbeuter - -* Elfeed is better at rendering HTML feeds than newsbeuter, it supports image -display, it can fuzzy-search everything... - -#### Picture batch processors: dcraw, gifsicle GraphicsMagick, ImageMagick, optipng - -* dcraw can convert many camera raw formats. - -* GraphicsMagick and ImageMagick are very similar in feature, and they may -be complementary regarding performance. - -#### Picture editors: darktable, GIMP, Hugin, Metapixel, RawTherapee - -#### Picture viewers: feh, sxiv - -* feh can set the wallpaper. sxiv is very fast at loading and displaying big -pictures. It supports GIF animations unlike feh. - -#### Programming: cloc - -#### Programming in C: cppcheck, GDB, mcpp, musl, Splint, TCC, Valgrind, Uncrustify - -* cppcheck and Splint are static analyzers with overlapping features. - -* Uncrustify is much better engineered than GNU Indent and Astyle. See the - [Indentation rationale](../indentation/index.html). - -#### Screen capture: scrot - -#### Session locks: slock, vlock, xlockmore, xss-lock - -* slock is as simple as it can be but does not support PAM sessions unlike -xlockmore. - -* vlock is for TTY sessions. It is part of the kbd project. - -#### Shell: DASH, Emacs (Eshell), fish, shellcheck - -* DASH is a light, fast and POSIX compliant shell. It is quite limited for -interactive use but ideal for testing the POSIX-ness of scripts. - -* fish departs from the POSIX-derived shells. Bash suffers from the design -issues of the venerable Bourne shell (e.g. word-splitting). Zsh has tried to -unite all shell languages under one banner, thus becoming complicated beyond -reason to the point that the simplest configuration can be an Odyssey on its -own. Like rc, fish uses a clear syntax. It also has a straightforward API, -which makes it very straighforward to customize and extend. Last but not least, -its interactive features are efficient and to the point. - - The lack of POSIX-ness is no problem in practice: - - - Any POSIX shell script will be executed by the interpreter pointed by the -shebang. - - - Initialization files such as .profile can still be set up by sh at the -beginning of the session: use sh as your login shell and exec fish at the -end. - -* Further down the road of non-POSIX shells, Eshell lies even further: it uses -Emacs Lisp as a language which is arguably much more powerful than fish. See -[my article on Eshell](../emacs-eshell) for more good reasons why you should use -it. - - Eshell has a very powerful completion framework, pcomplete. As of 2017 the -limited popularity of Eshell result in limited support for completion. That -being said, it is possible to configure Eshell so that it falls back on the -completion of fish, which then makes for a very extensive completion support. - -* shellcheck is a static-analyzer for shell scripts. - -#### Spreadsheet: Emacs (Org-mode) - -* This Emacs mode lets you write plain text tables (track them with git!) -and apply arbitrary functions to cells. These functions are either pre-defined -or self-written in Lisp (with ubiquitous support for M-x calc and its -arbitrary precision arithmetic). From there you can use every Elisp feature, -and if that would not be enough (e.g. too slow) you can call external programs -to perform the task, say, your favourite scripting language. This makes the -tables infinitely programmable. - -#### System monitors: Emacs (proced, helm-top), htop, iftop, Iotop, lsof, NetHogs, PSmisc - -#### Task management: Emacs (Org-mode), Taskwarrior - -* They can be used as TODO managers, calendars, etc. - -* The Taskwarrior file format is plain text but hard to read. The editing is -far less convenient than with a proper text editor. This is where the power of -using an editor as a user interface really shines. Org-mode is not an Emacs -exclusivity, some other editors support it. - -#### Text editors: Emacs - -[Why you should use Emacs](https://www.youtube.com/watch?v=JWD1Fpdd4Pc). - -#### Touch-typing: GNU Typist - -#### Torrent clients: Transmission - -* Transmission is full-featured and offers various UIs: GTK, Qt, curses, -and... Emacs! (transmission.el) Beside not supporting magnet links, -rtorrent has a poor UI for selecting files and folders manually, which makes -it very impractical for large torrents. The Emacs interface brings in its load -of usual advantages: extensibility, keyboard-driven, fuzzy-search, macros, etc. - -#### Transcoding: cdparanoia, dvdbackup, FFmpeg, flac, Gaupol, id3v2, libvpx, MediaInfo, mkvtoolnix, opus, vorbis-tools, wavpack, x264 +**** FTP servers: vsftpd + :PROPERTIES: + :CUSTOM_ID: ftp-servers-vsftpd + :END: -* FFmpeg is the swiss-army knife of transcoding: aspect ratio, concat, crop, -mixdown, remux, metadata, etc. It is much more efficient to use FFmpeg from a -smart custom script than using a GUI. +**** Gaming emulators: DGen, DOSBox, Gens/GS, Mupen64Plus, PCSX-Reloaded, MAME, +Wine + :PROPERTIES: + :CUSTOM_ID: gaming-emulators-dgen-dosbox-gensgs-mupen64plus-pcsx-reloaded-mame-wine + :END: -* mkvtoolnix can process mkv files in place, e.g. it can instantly change metadata. +**** Gaming tools: DriConf, TiMidity++, xosd + :PROPERTIES: + :CUSTOM_ID: gaming-tools-driconf-timidity-xosd + :END: -* cdparanoia rips audio CDs. +- =osd_cat= from =xosd= can display text on screen, such as FPS or network + traffic. + +**** Music: Beets, clyrics, cmus, Demlo, Emacs (EMMS), mps-youtube + :PROPERTIES: + :CUSTOM_ID: music-beets-clyrics-cmus-demlo-emacs-emms-mps-youtube + :END: -* dvdbackup decrypts VOB files. +- =clyrics= can display the lyrics of the song currently playing in =cmus=. -* Gaupol is a simple yet complete subtitle editor. +- =cmus= is extremely fast at updating the library. Its UI makes it convenient + to browse the library and to create playlists. -* MediaInfo displays the media property of pictures, audio and video files -(codecs, container, etc.). It overlaps a lot with FFprobe (from FFmpeg), but -still manages to provide some details that FFprobe misses. +- Emacs' EMMS is similar to cmus with the bonus that it is extensible, it can + fetch lyrics, it can resume state upon restart and it can display album covers + within the music library tree. -* The rest is a set of tools for containers and codecs. +- =mps-youtube= can build albums from Youtube links automatically and save the + resulting playlist locally. -#### Vector graphics: Asymptote, Graphviz, Inkscape +**** Network monitors: Aircrack-ng, ngrep, nmap, Tcpdump, Wireshark + :PROPERTIES: + :CUSTOM_ID: network-monitors-aircrack-ng-ngrep-nmap-tcpdump-wireshark + :END: -* Asymptote is a full-featured descriptive vector graphics renderer. It -features libraries for: plots, trees, 3D (with perspective!), and much more. The -language is much more convenient (C-style) and far less limited than its -competitors (TikZ, Metapost, PSTricks): it has double-precision arithmetic -support, classic control structures, data structures, custom structures, etc. It -also supersedes Gnuplot. +**** News readers: Emacs (elfeed), Newsbeuter + :PROPERTIES: + :CUSTOM_ID: news-readers-emacs-elfeed-newsbeuter + :END: -* Graphviz is a smart graph drawing tool that will decide automatically of the -best arrangement for the vertices and edges. +- Elfeed is better at rendering HTML feeds than newsbeuter, it supports image + display, it can fuzzy-search everything... -* Inkscape can export in LaTeX, which is useful to improve typography or make -the font consistent in your document. +**** Picture batch processors: dcraw, gifsicle GraphicsMagick, ImageMagick, +optipng + :PROPERTIES: + :CUSTOM_ID: picture-batch-processors-dcraw-gifsicle-graphicsmagick-imagemagick-optipng + :END: + +- =dcraw= can convert many camera raw formats. + +- =GraphicsMagick= and =ImageMagick= are very similar in feature, and they may + be complementary regarding performance. + +**** Picture editors: darktable, GIMP, Hugin, Metapixel, RawTherapee + :PROPERTIES: + :CUSTOM_ID: picture-editors-darktable-gimp-hugin-metapixel-rawtherapee + :END: + +**** Picture viewers: feh, sxiv + :PROPERTIES: + :CUSTOM_ID: picture-viewers-feh-sxiv + :END: + +- =feh= can set the wallpaper. =sxiv= is very fast at loading and displaying big + pictures. It supports GIF animations unlike =feh=. + +**** Programming: cloc + :PROPERTIES: + :CUSTOM_ID: programming-cloc + :END: + +**** Programming in C: cppcheck, GDB, mcpp, musl, Splint, TCC, Valgrind, +Uncrustify + :PROPERTIES: + :CUSTOM_ID: programming-in-c-cppcheck-gdb-mcpp-musl-splint-tcc-valgrind-uncrustify + :END: + +- =cppcheck= and =Splint= are static analyzers with overlapping features. + +- =Uncrustify= is much better engineered than =GNU Indent= and =Astyle=. See the + [[../indentation/index.html][Indentation rationale]]. + +**** Screen capture: scrot + :PROPERTIES: + :CUSTOM_ID: screen-capture-scrot + :END: + +**** Session locks: slock, vlock, xlockmore, xss-lock + :PROPERTIES: + :CUSTOM_ID: session-locks-slock-vlock-xlockmore-xss-lock + :END: + +- =slock= is as simple as it can be but does not support PAM sessions unlike + =xlockmore=. + +- =vlock= is for TTY sessions. It is part of the =kbd= project. + +**** Shell: DASH, Emacs (Eshell), fish, shellcheck + :PROPERTIES: + :CUSTOM_ID: shell-dash-emacs-eshell-fish-shellcheck + :END: + +- =DASH= is a light, fast and POSIX compliant shell. It is quite limited for + interactive use but ideal for testing the POSIX-ness of scripts. + +- =fish= departs from the POSIX-derived shells. =Bash= suffers from the design + issues of the venerable Bourne shell (e.g. word-splitting). =Zsh= has tried to + unite all shell languages under one banner, thus becoming complicated beyond + reason to the point that the simplest configuration can be an Odyssey on its + own. Like =rc=, =fish= uses a clear syntax. It also has a straightforward API, + which makes it very straighforward to customize and extend. Last but not + least, its interactive features are efficient and to the point. + + The lack of POSIX-ness is no problem in practice: -#### Version control: git, Emacs (Magit) + - Any POSIX shell script will be executed by the interpreter pointed by the + shebang. + + - Initialization files such as =.profile= can still be set up by =sh= at the + beginning of the session: use =sh= as your login shell and =exec fish= at + the end. + +- Further down the road of non-POSIX shells, Eshell lies even further: it uses + Emacs Lisp as a language which is arguably much more powerful than =fish=. See + [[../emacs-eshell][my article on Eshell]] for more good reasons why you should + use it. + + Eshell has a very powerful completion framework, =pcomplete=. As of 2017 the + limited popularity of Eshell result in limited support for completion. That + being said, it is possible to configure Eshell so that it falls back on the + completion of =fish=, which then makes for a very extensive completion + support. + +- =shellcheck= is a static-analyzer for shell scripts. + +**** Spreadsheet: Emacs (Org-mode) + :PROPERTIES: + :CUSTOM_ID: spreadsheet-emacs-org-mode + :END: + +- This =Emacs= mode lets you write plain text tables (track them with =git=!) + and apply arbitrary functions to cells. These functions are either pre-defined + or self-written in Lisp (with ubiquitous support for =M-x calc= and its + arbitrary precision arithmetic). From there you can use every Elisp feature, + and if that would not be enough (e.g. too slow) you can call external programs + to perform the task, say, your favourite scripting language. This makes the + tables infinitely programmable. + +**** System monitors: Emacs (proced, helm-top), htop, iftop, Iotop, lsof, +NetHogs, PSmisc + :PROPERTIES: + :CUSTOM_ID: system-monitors-emacs-proced-helm-top-htop-iftop-iotop-lsof-nethogs-psmisc + :END: + +**** Task management: Emacs (Org-mode), Taskwarrior + :PROPERTIES: + :CUSTOM_ID: task-management-emacs-org-mode-taskwarrior + :END: + +- They can be used as TODO managers, calendars, etc. + +- The =Taskwarrior= file format is plain text but hard to read. The editing is + far less convenient than with a proper text editor. This is where the power of + using an editor as a user interface really shines. Org-mode is not an Emacs + exclusivity, some other editors support it. + +**** Text editors: Emacs + :PROPERTIES: + :CUSTOM_ID: text-editors-emacs + :END: + +[[https://www.youtube.com/watch?v=JWD1Fpdd4Pc][Why you should use Emacs]]. + +**** Touch-typing: GNU Typist + :PROPERTIES: + :CUSTOM_ID: touch-typing-gnu-typist + :END: + +**** Torrent clients: Transmission + :PROPERTIES: + :CUSTOM_ID: torrent-clients-transmission + :END: + +- =Transmission= is full-featured and offers various UIs: GTK, Qt, curses, + and... Emacs! (=transmission.el=) Beside not supporting magnet links, + =rtorrent= has a poor UI for selecting files and folders manually, which makes + it very impractical for large torrents. The Emacs interface brings in its load + of usual advantages: extensibility, keyboard-driven, fuzzy-search, macros, + etc. + +**** Transcoding: cdparanoia, dvdbackup, FFmpeg, flac, Gaupol, id3v2, libvpx, +MediaInfo, mkvtoolnix, opus, vorbis-tools, wavpack, x264 + :PROPERTIES: + :CUSTOM_ID: transcoding-cdparanoia-dvdbackup-ffmpeg-flac-gaupol-id3v2-libvpx-mediainfo-mkvtoolnix-opus-vorbis-tools-wavpack-x264 + :END: + +- =FFmpeg= is the swiss-army knife of transcoding: aspect ratio, concat, crop, + mixdown, remux, metadata, etc. It is much more efficient to use FFmpeg from a + smart custom script than using a GUI. + +- =mkvtoolnix= can process mkv files in place, e.g. it can instantly change + metadata. + +- =cdparanoia= rips audio CDs. + +- =dvdbackup= decrypts VOB files. + +- =Gaupol= is a simple yet complete subtitle editor. + +- =MediaInfo= displays the media property of pictures, audio and video files + (codecs, container, etc.). It overlaps a lot with =FFprobe= (from =FFmpeg=), + but still manages to provide some details that =FFprobe= misses. -#### Video: mpv, subdl +- The rest is a set of tools for containers and codecs. + +**** Vector graphics: Asymptote, Graphviz, Inkscape + :PROPERTIES: + :CUSTOM_ID: vector-graphics-asymptote-graphviz-inkscape + :END: + +- =Asymptote= is a full-featured descriptive vector graphics renderer. It + features libraries for: plots, trees, 3D (with perspective!), and much more. + The language is much more convenient (C-style) and far less limited than its + competitors (TikZ, Metapost, PSTricks): it has double-precision arithmetic + support, classic control structures, data structures, custom structures, etc. + It also supersedes Gnuplot. + +- =Graphviz= is a smart graph drawing tool that will decide automatically of the + best arrangement for the vertices and edges. + +- =Inkscape= can export in LaTeX, which is useful to improve typography or make + the font consistent in your document. -* mpv is a fork of mplayer with fewer dependencies and some additions such -as an on-screen display, support for resuming and chapter markers. Both mpv -and mplayer allow for very fast video rendering, which can render 1080p -videos on lower-end machines where VLC would stutter. +**** Version control: git, Emacs (Magit) + :PROPERTIES: + :CUSTOM_ID: version-control-git-emacs-magit + :END: + +**** Video: mpv, subdl + :PROPERTIES: + :CUSTOM_ID: video-mpv-subdl + :END: + +- =mpv= is a fork of =mplayer= with fewer dependencies and some additions such + as an on-screen display, support for resuming and chapter markers. Both =mpv= + and =mplayer= allow for very fast video rendering, which can render 1080p + videos on lower-end machines where =VLC= would stutter. -* subdl will often fetch the right subtitles for the desired language. When it -fails to pick the right one, it is still possible to select it manually. +- =subdl= will often fetch the right subtitles for the desired language. When it + fails to pick the right one, it is still possible to select it manually. -#### Virtual machine hypervisors: QEMU +**** Virtual machine hypervisors: QEMU + :PROPERTIES: + :CUSTOM_ID: virtual-machine-hypervisors-qemu + :END: -#### Web browsers: Emacs (eww), qutebrowser, surfraw, w3m +**** Web browsers: Emacs (eww), qutebrowser, surfraw, w3m + :PROPERTIES: + :CUSTOM_ID: web-browsers-emacs-eww-qutebrowser-surfraw-w3m + :END: -* While eww is text-based, it can render variable width/height fonts as well -as pictures. +- While =eww= is text-based, it can render variable width/height fonts as well + as pictures. -#### Web tools: curl, Wget, youtube-dl +**** Web tools: curl, Wget, youtube-dl + :PROPERTIES: + :CUSTOM_ID: web-tools-curl-wget-youtube-dl + :END: -* curl and Wget are overlapping but also very complementary. +- =curl= and =Wget= are overlapping but also very complementary. -* youtube-dl, as the name does not imply, is not restricted to YouTube. +- =youtube-dl=, as the name does not imply, is not restricted to YouTube. -#### Window managers: Awesome, Emacs (EXWM), i3 +**** Window managers: Awesome, Emacs (EXWM), i3 + :PROPERTIES: + :CUSTOM_ID: window-managers-awesome-emacs-exwm-i3 + :END: -* Awesome is extensible in Lua. +- =Awesome= is extensible in Lua. -* i3 is lighter than Awesome but relies on externals scripts for -extensibility. +- =i3= is lighter than =Awesome= but relies on externals scripts for + extensibility. -* EXWM is a WM where all windows are Emacs buffers. Consequence: you can -fuzzy-select your windows, tile your selection, delete the complementary -selection, etc. EXWM is obviously extensible in Emacs Lisp. +- =EXWM= is a WM where all windows are Emacs buffers. Consequence: you can + fuzzy-select your windows, tile your selection, delete the complementary + selection, etc. EXWM is obviously extensible in Emacs Lisp. diff --git a/articles/real-time.org b/articles/real-time.org index 737863ec844a7d36a4383dff3b5cffcbd0a4e015..ac8900cc2864578570781df05861e7c3ff770d3a 100644 --- a/articles/real-time.org +++ b/articles/real-time.org @@ -1,49 +1,68 @@ -% Real-Time Kernels and Scheduling -% Just go ahead and write your own multitasking multiuser os! Worked for me all the times. -% Linus Torvalds +#+TITLE: Real-Time Kernels and Scheduling -# Abstract +#+AUTHOR: Just go ahead and write your own multitasking multiuser os! Worked for me all +the times. +#+DATE: Linus Torvalds -This article is mainly based on the French book [_Solution temps réel sous -Linux_](http://www.blaess.fr/christophe/livres/solutions-temps-reel-sous-linux/) -by C. Blaess, chapters 1 to 8. I have summarized some results of my -experiments while I was playing with the real-time capabilities of the Linux -kernel. I am far from being an expert in the field, so feel free to [send me any -correction/suggestion](https://bitbucket.org/ambrevar/ambrevar.bitbucket.org/issues). +* Abstract + :PROPERTIES: + :CUSTOM_ID: abstract + :END: + +This article is mainly based on the French book +[[http://www.blaess.fr/christophe/livres/solutions-temps-reel-sous-linux/][/Solution +temps réel sous Linux/]] by C. Blaess, chapters 1 to 8. I have summarized some +results of my experiments while I was playing with the real-time capabilities of +the Linux kernel. I am far from being an expert in the field, so feel free to +[[https://bitbucket.org/ambrevar/ambrevar.bitbucket.org/issues][send me any +correction/suggestion]]. Most of the references mentioned throughout this article refer to the book. -# Playground +* Playground + :PROPERTIES: + :CUSTOM_ID: playground + :END: -I wrote a small [test program](rt.c) to display some real-time properties. It -should be straightforward to compile: +I wrote a small [[file:rt.c][test program]] to display some real-time +properties. It should be straightforward to compile: - ${CC} -pthread -o rt rt.c +#+BEGIN_EXAMPLE +${CC} -pthread -o rt rt.c +#+END_EXAMPLE -Have a look at the comments and the user options: tweaking them will show -a different behaviour. +Have a look at the comments and the user options: tweaking them will show a +different behaviour. -# Problem overview +* Problem overview + :PROPERTIES: + :CUSTOM_ID: problem-overview + :END: Real-time programming becomes necessary in a time-constrained context with -various tasks executing in parallel. -It can be useful in various contexts such as simulation, video games, media -stream decoding, etc. +various tasks executing in parallel. It can be useful in various contexts such +as simulation, video games, media stream decoding, etc. -# Priorities & niceness +* Priorities & niceness + :PROPERTIES: + :CUSTOM_ID: priorities-niceness + :END: -## Priorities +** Priorities + :PROPERTIES: + :CUSTOM_ID: priorities + :END: In real-time, priorities are scaled from 1 to 99 by default, but this is configurable. Shared time always has lower priority than real time, as if it would be of priority 0. -In shared time, the distinction between _niceness_ and _priority_ is somewhat +In shared time, the distinction between /niceness/ and /priority/ is somewhat fuzzy. The niceness goes from -20 to 19, and has a value of 0 by default. A higher -niceness means a lower priority. Be aware that priority can be tricky to use: the -scheduler can penalize programs with higher priorities since they run more +niceness means a lower priority. Be aware that priority can be tricky to use: +the scheduler can penalize programs with higher priorities since they run more often, which leads to programs with lower priority having access to some resources before. @@ -53,189 +72,236 @@ this would allow a non-privileged user to paralyze the machine. Note however that Linux includes a guard on real-time processes. By default, lower priority processes still have a little share of time to run. This is made -so that it is impossible to paralyze the whole system with a simple while(1);; +so that it is impossible to paralyze the whole system with a simple =while(1);=; albeit severely slowed down, the system can still kill the greedy task. This behaviour can be disabled with: - echo -1 > /proc/sys/kernel/sched_rt_runtime_us +#+BEGIN_EXAMPLE + echo -1 > /proc/sys/kernel/sched_rt_runtime_us +#+END_EXAMPLE -## Configuration +** Configuration + :PROPERTIES: + :CUSTOM_ID: configuration + :END: -System resources such as priorities or niceness can be configured in different places. +System resources such as priorities or niceness can be configured in different +places. -* Globally in /etc/security/limits.conf. -* setrlimit() for a single task. -* ulimit to show some values. +- Globally in =/etc/security/limits.conf=. +- =setrlimit()= for a single task. +- =ulimit= to show some values. See the respective man pages for more details. -# Schedulers +* Schedulers + :PROPERTIES: + :CUSTOM_ID: schedulers + :END: There are several scheduling policies. -* Shared time - * Other (Linux default) - * Batch (Linux only) - * Idle (Linux only) -* Real time - * FIFO - * Round-Robin +- Shared time +- Other (Linux default) +- Batch (Linux only) +- Idle (Linux only) +- Real time +- FIFO +- Round-Robin Linux < 2.6.23 default scheduler was known as the O(1) scheduler, using priority -lists. -The Completely Fair Schedular (CFS) was merged in Linux 2.6.23: it uses a +lists. The Completely Fair Schedular (CFS) was merged in Linux 2.6.23: it uses a red-black tree to choose the next task to run. Some processus are CPU-bound (intensive computation), other are I/O-bound (a lot of time is spent on processing the input/output). The scheduling of tasks in shared time depends on their use. -* Interactive tasks need a low response time for best user experience. +- Interactive tasks need a low response time for best user experience. -* Batch processes run in the background; they perform a lot of computation with -little I/O. +- Batch processes run in the background; they perform a lot of computation with + little I/O. -* Real-time processes need low response time, minimal variance and should not be -blocked when avoidable. +- Real-time processes need low response time, minimal variance and should not be + blocked when avoidable. Preemption happens when one of the following condition is met: -* A hardware interrupt was received. -* The end of the time quantum was reached. -* A process with higher priority is active. +- A hardware interrupt was received. +- The end of the time quantum was reached. +- A process with higher priority is active. -## Shared-time +** Shared-time + :PROPERTIES: + :CUSTOM_ID: shared-time + :END: Various commands: -* chrt -m: show current values. -* chrt -ap PID: display PID's policy and priority, as well as for its current -threads if any. -* ps ma -o command,ni,pri displays the priority and the niceness for various -processes and threads. See the ps(1) man page. +- =chrt -m=: show current values. +- =chrt -ap PID=: display PID's policy and priority, as well as for its current + threads if any. +- =ps ma -o command,ni,pri= displays the priority and the niceness for various + processes and threads. See the =ps(1)= man page. - +#+BEGIN_HTML + +#+END_HTML -## RT FIFO +** RT FIFO + :PROPERTIES: + :CUSTOM_ID: rt-fifo + :END: This is the simplest: first in, first out. As soon as current task goes into a waiting state, the next task immediately goes into running state. It is ideal for a sequential execution. -## RT Round Robin +** RT Round Robin + :PROPERTIES: + :CUSTOM_ID: rt-round-robin + :END: -This allows for _parallel_ execution of several tasks. Unlike for the FIFO +This allows for /parallel/ execution of several tasks. Unlike for the FIFO scheduler, CPU time slices play a role here (see page 154). For instance, on IPC synchronization, time gets wasted if synchronization happens at the beginning of -the time slice. The solution is to use sched_yield() to control the scheduling. +the time slice. The solution is to use =sched_yield()= to control the +scheduling. - +#+BEGIN_HTML + +#+END_HTML -# Interrupts +* Interrupts + :PROPERTIES: + :CUSTOM_ID: interrupts + :END: Hardware interrupts cannot be software-managed: it is necessary to work at the kernel level. One solution is to write a dedicated module (driver). The main advantage of hardware interrupts is to allow for fine-tuning of the execution order. -Interrupts happen in two steps: the _top half_ and the _bottom half_. The first +Interrupts happen in two steps: the /top half/ and the /bottom half/. The first is for the IRQs that are immediately processed and give information to the latter on how to execute. This second step can be deferred. The purpose of this separation is to avoid blocking the processor on an interrupt for which the bottom half processing is long. -The bottom half processing happens through _tasklets_, _workqueues_, -_soft-IRQs_, or _threaded interrupts_. +The bottom half processing happens through /tasklets/, /workqueues/, +/soft-IRQs/, or /threaded interrupts/. -Interrupts can be displayed from the file -/proc/interrupts. More precisely, it can be interesting to display its -evolution over time: +Interrupts can be displayed from the file =/proc/interrupts=. More precisely, it +can be interesting to display its evolution over time: - watch -n 1 cat /proc/interrupts +#+BEGIN_EXAMPLE + watch -n 1 cat /proc/interrupts +#+END_EXAMPLE (See pages 24, 28, 172.) Software interrupts happen as soon as a specific instruction runs on the -processor (e.g. division by 0, segmentation fault). In the general case a signal +processor (e.g. division by 0, segmentation fault). In the general case a signal is sent to the calling process. In any case, from the point of view of the process, interrupts consume CPU time. The time lapse between the moment when they might send a signal and when the process will actually receive it depends on when the process goes back to its running state. Thus it depends on the scheduler. To make sure that the process -_handler_ will run before anything else, we must make sure that the process has +/handler/ will run before anything else, we must make sure that the process has maximum priority. -# Multiprocessors +* Multiprocessors + :PROPERTIES: + :CUSTOM_ID: multiprocessors + :END: A major challenge for scheduling is to handle the multi-CPU case: it is much -more complex to control (i.e. to foresee) the running time of the different +more complex to control (i.e. to foresee) the running time of the different processes. When working in real time, we are better off to force the application to work on the same CPU by setting the affinity. This suggestion does not always apply, of course. Some interesting functions worth knowing: -* _Linux_: sysconf(_SC_NPROCESSORS_ONLN): return the number of online processors. -* _GNU_: sched_setaffinity(): change the affinity of a task - (process or thread). The equivalent for threads is available in the GNU -* _pthread_ library as an extension (and thus non-POSIX). +- /Linux/: =sysconf(_SC_NPROCESSORS_ONLN)=: return the number of online + processors. +- /GNU/: =sched_setaffinity()=: change the affinity of a task (process or + thread). The equivalent for threads is available in the GNU +- /pthread/ library as an extension (and thus non-POSIX). -From a terminal, the affinity can be tweaked with taskset -pc CPUNO PID. +From a terminal, the affinity can be tweaked with =taskset -pc CPUNO PID=. -# Linux, Linux-rt, and others +* Linux, Linux-rt, and others + :PROPERTIES: + :CUSTOM_ID: linux-linux-rt-and-others + :END: -## Linux +** Linux + :PROPERTIES: + :CUSTOM_ID: linux + :END: -The standard kernel has been through several evolutions. Part of the linux-rt +The standard kernel has been through several evolutions. Part of the =linux-rt= patch set has been merged into the 2.6 branch, so that the kernel now has a real-time scheduling policy as well as preemtible system calls. Kernels before 2.6 are not preemtible. This can be checked with: - zcat /proc/config.gz | grep CONFIG_PREEMPT +#+BEGIN_EXAMPLE + zcat /proc/config.gz | grep CONFIG_PREEMPT +#+END_EXAMPLE On a 2.6+ kernel: - CONFIG_PREEMPT_RCU=y - CONFIG_PREEMPT_NOTIFIERS=y - # CONFIG_PREEMPT_NONE is not set - # CONFIG_PREEMPT_VOLUNTARY is not set - CONFIG_PREEMPT=y - CONFIG_PREEMPT_COUNT=y - # CONFIG_PREEMPT_TRACER is not set - -The command uname -a also give some hints on the preemption capabilities. -See page 129. - -There is no difference in the ABI between the regular _Linux_ kernel and -_Linux-rt_. It means that there is full binary compatibility. -Why not merging the Linux-rt branch entirely? Performance for a personal use -happens to be worse. - -## Linux-rt +#+BEGIN_EXAMPLE + CONFIG_PREEMPT_RCU=y + CONFIG_PREEMPT_NOTIFIERS=y + # CONFIG_PREEMPT_NONE is not set + # CONFIG_PREEMPT_VOLUNTARY is not set + CONFIG_PREEMPT=y + CONFIG_PREEMPT_COUNT=y + # CONFIG_PREEMPT_TRACER is not set +#+END_EXAMPLE + +The command =uname -a= also give some hints on the preemption capabilities. See +page 129. + +There is no difference in the ABI between the regular /Linux/ kernel and +/Linux-rt/. It means that there is full binary compatibility. Why not merging +the Linux-rt branch entirely? Performance for a personal use happens to be +worse. + +** Linux-rt + :PROPERTIES: + :CUSTOM_ID: linux-rt + :END: This patch set only changes some kernel parameters and makes system calls even more preemtible. Interrupts are as such even more under control, and above all they have less -impact on real-time processes. For instance, on a Linux machine -where all processors are being used by a real-time application, a _ping flood_ -would halt the application. With Linux-rt, the application would continue -(perhaps more slowly). See page 166. +impact on real-time processes. For instance, on a Linux machine where all +processors are being used by a real-time application, a /ping flood/ would halt +the application. With Linux-rt, the application would continue (perhaps more +slowly). See page 166. The main purpose of Linux-rt lies in predictability. See chapter 8. -## RTLinux, Xenomai +** RTLinux, Xenomai + :PROPERTIES: + :CUSTOM_ID: rtlinux-xenomai + :END: -Linux and Linux-rt have interesting capabilities in term of _soft real-time_. -However, they are almost worthless in a more constrained environment as we cannot -_prove_ the execution time. Some Linux-based _hard real-time_ projects do exist, -though. +Linux and Linux-rt have interesting capabilities in term of /soft real-time/. +However, they are almost worthless in a more constrained environment as we +cannot /prove/ the execution time. Some Linux-based /hard real-time/ projects do +exist, though. -* _RTLinux_ is a micro kernel running the entire Linux kernel as a process, thus -it is fully preemptible. The development of this proprietary kernel has been -discontinued and the source have been released under GPL2. +- /RTLinux/ is a micro kernel running the entire Linux kernel as a process, thus + it is fully preemptible. The development of this proprietary kernel has been + discontinued and the source have been released under GPL2. -* [_Xenomai_](http://xenomai.org/) is one of the most active project in the field. It is free and open-source. +- [[http://xenomai.org/][/Xenomai/]] is one of the most active project in the + field. It is free and open-source. diff --git a/articles/vcs.org b/articles/vcs.org index ac0b00c773bd0a3aac0cd1325a9f5b8fed8122d0..c123a5a74487ae4a8ca41f01bab7f5af267427c7 100644 --- a/articles/vcs.org +++ b/articles/vcs.org @@ -1,24 +1,25 @@ -% Version Control System: The One Computer Skill +#+TITLE: Version Control System: The One Computer Skill The first thing we should teach about driving a vehicle is where the brakes are. -I believe that the same reasoning applies to computers. Before teaching any sort of -programming or web editing, wouldn't it be reasonable to teach how to _save your -work_? +I believe that the same reasoning applies to computers. Before teaching any sort +of programming or web editing, wouldn't it be reasonable to teach how to /save +your work/? There is something essential that could be taught first hand and which I believe -to be the number one skill that will remain invaluable for the rest of your -life as a computer user: _version control systems_ (VCS). +to be the number one skill that will remain invaluable for the rest of your life +as a computer user: /version control systems/ (VCS). -From [Wikipedia][def]: +From +[[https://en.wikipedia.org/w/index.php?title=Version_control&oldid=740941888][Wikipedia]]: -> Version control [...] is the management of changes to documents, computer -> programs, large web sites, and other collections of information. +#+BEGIN_QUOTE + Version control [...] is the management of changes to documents, computer + programs, large web sites, and other collections of information. +#+END_QUOTE -[def]: https://en.wikipedia.org/w/index.php?title=Version_control&oldid=740941888 - -A concrete and straightforward example would the [history][] of a Wikipedia page. - -[history]: https://en.wikipedia.org/w/index.php?title=Version_control&action=history +A concrete and straightforward example would the +[[https://en.wikipedia.org/w/index.php?title=Version_control&action=history][history]] +of a Wikipedia page. A VCS is: @@ -51,32 +52,32 @@ A VCS will hold much of your project information: - version tags, - authoring. -Thus the VCS metadata should be stored in an _open format_, that is, in a format +Thus the VCS metadata should be stored in an /open format/, that is, in a format that is open for viewing and editing at any time in the future. Should the VCS become un-maintained, the OS support be dropped, or any other catastrophe occur, the project information can be recovered. VCSes should be decentralized. See this -[Tech Talk: Linux Torvalds on git](http://www.youtube.com/watch?v=4XpnKHJAok8) +[[http://www.youtube.com/watch?v=4XpnKHJAok8][Tech Talk: Linux Torvalds on git]] for a rationale. Other features: - Permanent online access for files hosted on a server (public or private). - Offline access: decentralization means you do not need to be connected to -perform any sort of VCS management. + perform any sort of VCS management. - Content guarantee: content is check-summed, which means that no -data-corruption is possible without acknowledgement. + data-corruption is possible without acknowledgement. - Show differences across versions. - Handle conflicts. - Clear control of the content that is under version control: no spurious file -(temp file), no wrong save of a file, clear view at what is being synced (up and -down). + (temp file), no wrong save of a file, clear view at what is being synced (up + and down). "User-friendly" sharing tools like Dropbox have the terrible drawback of hiding the syncing operations: when offline, if you change the content then go online again, what happens? Will Dropbox discard the changes or upload them? What if -there is a conflict, i.e. some file got changed in the Dropbox repository in the +there is a conflict, i.e. some file got changed in the Dropbox repository in the meantime? If you care about the content of your files, do not use such tools. It is @@ -99,16 +100,15 @@ save on bandwidth. Avoid *Office documents: they are archived-files, a VCS will treat them as raw data, it won't be able to "see" the content. -For book or blog authoring, prefer text formats such as [Markdown][] and [LaTeX][]. - -[markdown]: http://daringfireball.net/projects/markdown/ -[latex]: https://www.latex-project.org/ +For book or blog authoring, prefer text formats such as +[[http://daringfireball.net/projects/markdown/][Markdown]] and +[[https://www.latex-project.org/][LaTeX]]. Text-formats can have some more exotic use: - Spreadsheets with Org-mode. -- Vector graphics with Asymptote: for geomatrical or scientific drawings, this is -much more efficient than manual drawings with Inkscape and the like. +- Vector graphics with Asymptote: for geomatrical or scientific drawings, this + is much more efficient than manual drawings with Inkscape and the like. Last but not least, a VCS can be somewhat technical and challenging to master. Which is why I believe a proper introductory course to be necessary. diff --git a/articles/xii.org b/articles/xii.org index 2e5043dda6bdb6b5fff7b0a955e203f1bf0c2b8d..f65e7b9047306b5b842527c06b777e57566e2032 100644 --- a/articles/xii.org +++ b/articles/xii.org @@ -1,108 +1,126 @@ -% xii.tex: Deciphering a TeX Puzzle +#+TITLE: xii.tex: Deciphering a TeX Puzzle -xii.tex is a famous TeX puzzle by David Carlisle. The document source is +=xii.tex= is a famous TeX puzzle by David Carlisle. The document source is terribly obfuscated. The following document is an attempt at deciphering the -puzzle, i.e. an explanation of the various processes and TeX tricks. +puzzle, i.e. an explanation of the various processes and TeX tricks. -The original puzzle can be found at [CTAN/xii](https://www.ctan.org/pkg/xii). +The original puzzle can be found at [[https://www.ctan.org/pkg/xii][CTAN/xii]]. Here is a copy: - \let~\catcode~76~A13~F1~j00~P2jdefA71F~7113jdefPALLF - PA''FwPA;;FPAZZFLaLPA//71F71iPAHHFLPAzzFenPASSFthP;A$$FevP - A@@FfPARR717273F737271P;ADDFRgniPAWW71FPATTFvePA**FstRsamP - AGGFRruoPAqq71.72.F717271PAYY7172F727171PA??Fi*LmPA&&71jfi - Fjfi71PAVVFjbigskipRPWGAUU71727374 75,76Fjpar71727375Djifx - :76jelse&U76jfiPLAKK7172F71l7271PAXX71FVLnOSeL71SLRyadR@oL - RrhC?yLRurtKFeLPFovPgaTLtReRomL;PABB71 72,73:Fjif.73.jelse - B73:jfiXF71PU71 72,73:PWs;AMM71F71diPAJJFRdriPAQQFRsreLPAI - I71Fo71dPA!!FRgiePBt'el@ lTLqdrYmu.Q.,Ke;vz vzLqpip.Q.,tz; - ;Lql.IrsZ.eap,qn.i. i.eLlMaesLdRcna,;!;h htLqm.MRasZ.ilk,% - s;z zLqs'.ansZ.Ymi,/sx ;LYegseZRyal,@i;@ TLRlogdLrDsW,@;G - LcYlaDLbJsW,SWXJW ree @rzchLhzsW,;WERcesInW qt.'oL.Rtrul;e - doTsW,Wk;Rri@stW aHAHHFndZPpqar.tridgeLinZpe.LtYer.W,:jbye - -# Catcodes - -The first step is to understand the TeX _catcodes_. It stands for _category -codes_, i.e. the semantic of the various ASCII characters. - -- 0: Escape character (default \) -- 1: Beginning of group (default {) -- 2: End of group (default }) -- 3: Math shift (default ) -- 4: Alignment tab (default &) -- 5: End of line (default ^^M ≡ ASCII return) -- 6: Macro parameter (default #) -- 7: Superscript (default ^ and ^^K) -- 8: Subscript (default _ and ^^A) -- 9: Ignored character (default ^^@ ≡ ASCII null) -- 10: Space (default ^^I ≡ ASCII horizontal tab) -- 11: Letter (default [A-Za-z]) +#+BEGIN_EXAMPLE + \let~\catcode~76~A13~F1~j00~P2jdefA71F~7113jdefPALLF + PA''FwPA;;FPAZZFLaLPA//71F71iPAHHFLPAzzFenPASSFthP;A$$FevP + A@@FfPARR717273F737271P;ADDFRgniPAWW71FPATTFvePA**FstRsamP + AGGFRruoPAqq71.72.F717271PAYY7172F727171PA??Fi*LmPA&&71jfi + Fjfi71PAVVFjbigskipRPWGAUU71727374 75,76Fjpar71727375Djifx + :76jelse&U76jfiPLAKK7172F71l7271PAXX71FVLnOSeL71SLRyadR@oL + RrhC?yLRurtKFeLPFovPgaTLtReRomL;PABB71 72,73:Fjif.73.jelse + B73:jfiXF71PU71 72,73:PWs;AMM71F71diPAJJFRdriPAQQFRsreLPAI + I71Fo71dPA!!FRgiePBt'el@ lTLqdrYmu.Q.,Ke;vz vzLqpip.Q.,tz; + ;Lql.IrsZ.eap,qn.i. i.eLlMaesLdRcna,;!;h htLqm.MRasZ.ilk,% + s$;z zLqs'.ansZ.Ymi,/sx ;LYegseZRyal,@i;@ TLRlogdLrDsW,@;G + LcYlaDLbJsW,SWXJW ree @rzchLhzsW,;WERcesInW qt.'oL.Rtrul;e + doTsW,Wk;Rri@stW aHAHHFndZPpqar.tridgeLinZpe.LtYer.W,:jbye +#+END_EXAMPLE + +* Catcodes + :PROPERTIES: + :CUSTOM_ID: catcodes + :END: + +The first step is to understand the TeX /catcodes/. It stands for /category +codes/, i.e. the semantic of the various ASCII characters. + +- 0: Escape character (default =\=) +- 1: Beginning of group (default ={=) +- 2: End of group (default =}=) +- 3: Math shift (default =$=) +- 4: Alignment tab (default =&=) +- 5: End of line (default =^^M= ≡ ASCII return) +- 6: Macro parameter (default =#=) +- 7: Superscript (default =^= and =^^K=) +- 8: Subscript (default =_= and =^^A=) +- 9: Ignored character (default =^^@= ≡ ASCII null) +- 10: Space (default =^^I= ≡ ASCII horizontal tab) +- 11: Letter (default =[A-Za-z]=) - 12: Other character (everything not listed above or below) -- 13: Active character (default ~ and ^^L ≡ ASCII form feed) -- 14: Comment character (default %) -- 15: Invalid character (default ^^? ≡ ASCII delete) +- 13: Active character (default =~= and =^^L= ≡ ASCII form feed) +- 14: Comment character (default =%=) +- 15: Invalid character (default =^^?= ≡ ASCII delete) The first line - \let~\catcode~76~A13~F1~j00~P2 +#+BEGIN_EXAMPLE + \let~\catcode~76~A13~F1~j00~P2 +#+END_EXAMPLE applies the following catcodes: - ~ = \catcode -- j = \\ (space) +- j = \ (space) - 7 = # - A = active - F = { - P = } - -# Command defining commands +* Command defining commands + :PROPERTIES: + :CUSTOM_ID: command-defining-commands + :END: Next we have - jdefA71F~7113jdefP +#+BEGIN_EXAMPLE + jdefA71F~7113jdefP +#+END_EXAMPLE which, in proper TeX, would be - \def A #1{\catcode #1=13 \def} +#+BEGIN_EXAMPLE + \def A #1{\catcode #1=13 \def} +#+END_EXAMPLE -From then on, A will turn the first next character to an active character -and define the second next character to what follows. +From then on, =A= will turn the first next character to an active character and +define the second next character to what follows. -Example with the first application of A: +Example with the first application of =A=: - ALLF - P +#+BEGIN_EXAMPLE + ALLF + P +#+END_EXAMPLE -(The linefeed matters.) -It gets expanded to +(The linefeed matters.) It gets expanded to - \catcode L=13 \def L { - } +#+BEGIN_EXAMPLE + \catcode L=13 \def L { + } +#+END_EXAMPLE -So L is now a space. +So =L= is now a space. The same process gets repeated throughout the following lines: - A''FwPA;;FPAZZFLaLPA//71F71iPAHHFLPAzzFenPASSFthP;A$$FevP - A@@FfPARR717273F737271P;ADDFRgniPAWW71FPATTFvePA**FstRsamP - AGGFRruoPAqq71.72.F717271PAYY7172F727171PA??Fi*LmPA&&71jfi - Fjfi71PAVVFjbigskipRPWGAUU71727374 75,76Fjpar71727375Djifx - :76jelse&U76jfiPLAKK7172F71l7271PAXX71FVLnOSeL71SLRyadR@oL - RrhC?yLRurtKFeLPFovPgaTLtReRomL;PABB71 72,73:Fjif.73.jelse - B73:jfiXF71PU71 72,73:PWs;AMM71F71diPAJJFRdriPAQQFRsreLPAI - I71Fo71dPA!!FRgieP +#+BEGIN_EXAMPLE + A''FwPA;;FPAZZFLaLPA//71F71iPAHHFLPAzzFenPASSFthP;A$$FevP + A@@FfPARR717273F737271P;ADDFRgniPAWW71FPATTFvePA**FstRsamP + AGGFRruoPAqq71.72.F717271PAYY7172F727171PA??Fi*LmPA&&71jfi + Fjfi71PAVVFjbigskipRPWGAUU71727374 75,76Fjpar71727375Djifx + :76jelse&U76jfiPLAKK7172F71l7271PAXX71FVLnOSeL71SLRyadR@oL + RrhC?yLRurtKFeLPFovPgaTLtReRomL;PABB71 72,73:Fjif.73.jelse + B73:jfiXF71PU71 72,73:PWs;AMM71F71diPAJJFRdriPAQQFRsreLPAI + I71Fo71dPA!!FRgieP +#+END_EXAMPLE To sum up: -- L: { } (_Space_) +- L: { } (/Space/) - ': {w} -- ;: {} (_Nothing, useful to avoid globing_) +- ;: {} (/Nothing, useful to avoid globing/) - Z: { a } - /: #1{#1i} -- H: { } (_Space, used for last sentence shift from "a" to "and a"._) +- H: { } (/Space, used for last sentence shift from "a" to "and a"./) - z: {en} - S: {th} - $: {ev} @@ -110,8 +128,8 @@ To sum up: - @: {f} - R: #1#2#3{#3#2#1} -- D {ing} (_Note: Rgni = ing_) -- W: #1{} (_Globing_) +- D {ing} (/Note: Rgni = ing/) +- W: #1{} (/Globing/) - T: {ve} - *: {stmas} - G: {our} @@ -120,20 +138,23 @@ To sum up: - Y: #1#2{#2#1#1} - ?: {istmas m} -- &: #1\fi{\fi#1} (_Key command_) +- &: #1\fi{\fi#1} (/Key command/) - V: {\bigskip R} -- W: G (_Useless command_) +- W: G (/Useless command/) -- U: #1#2#3#4 #5,#6{\par#1#2#3#5ing \ifx:#6 \else &U#6\fi} (_Key command_) +- U: #1#2#3#4 #5,#6{ + \par + #1#2#3#5ing \ifx:#6 \else &U#6\fi} (/Key command/) -- L: (_Useless command_) +- L: (/Useless command/) - K: #1#2{#1l#2#1} -- X: #1{\bigskip On the #1th day of Christmas my true love gave to me} (_Key command_) +- X: #1{\bigskip On the #1th day of Christmas my true love gave to me} (/Key + command/) -- B: #1 #2,#3:{\if.#3.\else B #3:\fi X{#1} U#1 #2,#3:} (_Key command_) +- B: #1 #2,#3:{\if.#3.\else B #3:\fi X{#1} U#1 #2,#3:} (/Key command/) -- W: s; (_Useless command_) +- W: s; (/Useless command/) - M: #1 {#1di} @@ -142,114 +163,137 @@ To sum up: - I: #1{o#1d} - !: {eig} -# Core text +* Core text + :PROPERTIES: + :CUSTOM_ID: core-text + :END: The above definitions did not produce any text, they merely served for laying down the foundations of the obfuscation. The remaining text will generate the song: - Bt'el@ lTLqdrYmu.Q.,Ke;vz vzLqpip.Q.,tz; - ;Lql.IrsZ.eap,qn.i. i.eLlMaesLdRcna,;!;h htLqm.MRasZ.ilk,% - s$;z zLqs'.ansZ.Ymi,/sx ;LYegseZRyal,@i;@ TLRlogdLrDsW,@;G - LcYlaDLbJsW,SWXJW ree @rzchLhzsW,;WERcesInW qt.'oL.Rtrul;e - doTsW,Wk;Rri@stW aHAHHFndZPpqar.tridgeLinZpe.LtYer.W,:jbye +#+BEGIN_EXAMPLE + Bt'el@ lTLqdrYmu.Q.,Ke;vz vzLqpip.Q.,tz; + ;Lql.IrsZ.eap,qn.i. i.eLlMaesLdRcna,;!;h htLqm.MRasZ.ilk,% + s$;z zLqs'.ansZ.Ymi,/sx ;LYegseZRyal,@i;@ TLRlogdLrDsW,@;G + LcYlaDLbJsW,SWXJW ree @rzchLhzsW,;WERcesInW qt.'oL.Rtrul;e + doTsW,Wk;Rri@stW aHAHHFndZPpqar.tridgeLinZpe.LtYer.W,:jbye +#+END_EXAMPLE -# Generative commands +* Generative commands + :PROPERTIES: + :CUSTOM_ID: generative-commands + :END: The fundamental idea behind the text generation is that one command will loop over a list of sentences and run another command over each sentence. -The loop command is B. It recurses over the argument list using the , +The loop command is =B=. It recurses over the argument list using the =,= separator. -The sentence building command is X. +The sentence building command is =X=. -There is one last trick: the U command is used to propagate itself recursively +There is one last trick: the =U= command is used to propagate itself recursively on the #6 argument. Since there is no delimiter for #6, only one character is -passed to U. The use of the & command shifts expands into another U called -right after itself. Thus the rest of the #3 argument from B is successively -passed to U . +passed to =U=. The use of the =&= command shifts expands into another =U= called +right after itself. Thus the rest of the #3 argument from =B= is successively +passed to =U= . -# Readable version +* Readable version + :PROPERTIES: + :CUSTOM_ID: readable-version + :END: This version uses the same algorithm with human readable names. - % This is used to define active characters. - \catcode@13 - \def@#1{\catcode#113\def} - - % Nothing, useful to prevent globing. - @;;{} +#+BEGIN_EXAMPLE + % This is used to define active characters. + \catcode@13 + \def@#1{\catcode#113\def} - % Space, used for last sentence shift from "a" to "and a". - @HH{\ } + % Nothing, useful to prevent globing. + @;;{} - % Character globing; used to glob extra commas, "ing" in some verses and "th" - % in ordinal numbers. - @WW#1{} + % Space, used for last sentence shift from "a" to "and a". + @HH{\ } - % Note that '#1' is everything from '&' to '\fi'. In our case, it is 'U#6'. - @&\fi{\fi#1} + % Character globing; used to glob extra commas, "ing" in some verses and "th" + % in ordinal numbers. + @WW#1{} - @UU#1#2#3#4 #5,#6{\par#1#2#3#5{ing}\if:#6\else &U#6\fi} + % Note that '#1' is everything from '&' to '\fi'. In our case, it is 'U#6'. + @&\fi{\fi#1} - \def\day#1{\bigskip On the #1{th} day of Christmas my true love gave to me} + @UU#1#2#3#4 #5,#6{\par#1#2#3#5{ing}\if:#6\else &U#6\fi} - \def\gen#1 #2,#3:{\if.#3.\else \gen#3:\fi\day{#1}U#1 #2,#3:} + \def\day#1{\bigskip On the #1{th} day of Christmas my true love gave to me} - % Note that comments at the end of the lines are mandatory. - \gen - twelf lve drummers drumm,% - eleven ven pipers pip,% - ten ; lords a leap,% - nin e ladies danc,% - eigh ht maids a milk,% - seven en swans a swimm,% - six ; geese a lay,% - ;fif ve gold ringsW,% - four r calling birdsW,% - ;th{ird}W ree french hensW,% - ;;;secondW two turtle dovesW,% - ;;;firstW aH@HH{nd a }partridge in a pear tree.W,: + \def\gen#1 #2,#3:{\if.#3.\else \gen#3:\fi\day{#1}U#1 #2,#3:} - \bye + % Note that comments at the end of the lines are mandatory. + \gen + twelf lve drummers drumm,% + eleven ven pipers pip,% + ten ; lords a leap,% + nin e ladies danc,% + eigh ht maids a milk,% + seven en swans a swimm,% + six ; geese a lay,% + ;fif ve gold ringsW,% + four r calling birdsW,% + ;th{ird}W ree french hensW,% + ;;;secondW two turtle dovesW,% + ;;;firstW aH@HH{nd a }partridge in a pear tree.W,: + \bye +#+END_EXAMPLE -# Simple version +* Simple version + :PROPERTIES: + :CUSTOM_ID: simple-version + :END: This version uses an idea similar to the original, but without all the catcode intricacies. - - - \def\endverse{} - - \def\verse#1 #2,#3:{\if.#3.\par #2{}\else\par#2\verse#3:\fi} - - \def\day#1{\bigskip On the #1 day of Christmas my true love gave to me} - - \def\gen#1 #2,#3:{\if.#3.\else \gen#3:\fi\day{#1}\verse#1 #2,#3:} - - \gen - twelfth twelve drummers drumming,% - eleven eleven pipers piping,% - tenth ten lords a leaping,% - ninth nine ladies dancing,% - eighth eight maids a milk,% - seventh seven swans a swimm,% - sixth six geese a lay,% - fifth five gold rings,% - fourth fourr calling birds,% - third three french hens,% - second two turtle doves,% - first \endverse\def\endverse{and }a partridge in a pear tree.,: - - \bye - - -# References - -- [The TeXbook](http://www.ctan.org/pkg/texbook), Donald E. Knuth -- [TeX by Topic](http://www.eijkhout.net/texbytopic/texbytopic.html), Victor Eijkhout -- [TeX for the Impatient](http://www.ctan.org/tex-archive/info/impatient/), Paul Abra­hams, Kathryn Har­g­reaves, Karl Berry +#+BEGIN_HTML + +#+END_HTML + +#+BEGIN_EXAMPLE + \def\endverse{} + + \def\verse#1 #2,#3:{\if.#3.\par #2{}\else\par#2\verse#3:\fi} + + \def\day#1{\bigskip On the #1 day of Christmas my true love gave to me} + + \def\gen#1 #2,#3:{\if.#3.\else \gen#3:\fi\day{#1}\verse#1 #2,#3:} + + \gen + twelfth twelve drummers drumming,% + eleven eleven pipers piping,% + tenth ten lords a leaping,% + ninth nine ladies dancing,% + eighth eight maids a milk,% + seventh seven swans a swimm,% + sixth six geese a lay,% + fifth five gold rings,% + fourth fourr calling birds,% + third three french hens,% + second two turtle doves,% + first \endverse\def\endverse{and }a partridge in a pear tree.,: + + \bye +#+END_EXAMPLE + +* References + :PROPERTIES: + :CUSTOM_ID: references + :END: + +- [[http://www.ctan.org/pkg/texbook][The TeXbook]], Donald E. Knuth +- [[http://www.eijkhout.net/texbytopic/texbytopic.html][TeX by Topic]], Victor + Eijkhout +- [[http://www.ctan.org/tex-archive/info/impatient/][TeX for the Impatient]], + Paul Abra­hams, Kathryn Har­g­reaves, Karl Berry diff --git a/projects/demlo.org b/projects/demlo.org index e69f1aed8ac2497c6858a639d7775ecb0545eff8..0f316ea380d33d0ec3421ca3ce64b139e885b172 100644 --- a/projects/demlo.org +++ b/projects/demlo.org @@ -1,68 +1,83 @@ -% Demlo -% A dynamic and extensible music library organizer +#+TITLE: Demlo -# About +#+AUTHOR: A dynamic and extensible music library organizer +* About + :PROPERTIES: + :CUSTOM_ID: about + :END: Demlo organizes your music library automatically and dynamically. It runs a chain of user-defined Lua scripts using variables such as tags and file properties. This way it yields virtually unlimited customization power to the user. -# Preview +* Preview + :PROPERTIES: + :CUSTOM_ID: preview + :END: Here follows a sample output showing the 'before-after' differences. -$ demlo fantasie_impromptu.flac - :: Load config: /home/johndoe/.config/demlo/demlorc - :: Load script: /usr/share/demlo/scripts/tag.lua - :: Load script: /usr/share/demlo/scripts/sub.lua - :: Load script: /usr/share/demlo/scripts/case.lua - :: Load script: /usr/share/demlo/scripts/punctuation.lua - :: Load script: /usr/share/demlo/scripts/encoding.lua - :: Load script: /usr/share/demlo/scripts/path.lua - :: Load script: /usr/share/demlo/scripts/cover.lua - :: Load script: /home/johndoe/.config/demlo/scripts/lossy.lua - ==> fantasie_impromptu.flac - - === FILE === - [/home/johndoe/fantasie_impromptu.flac] | path | [/home/johndoe/music/Chopin/The Best Ever Piano ] - | | [Classics (John Doe, 2014)/Fantasie-Impromptu in] - | | [ C Sharp Minor, Op. 66.ogg] - [flac] | format | [ogg] - [bitrate=320000] | parameters | [[-c:a libvorbis -q:a 10]] - === TAGS === - [john doe's classical collection II] | album | [John Doe's Classical Collection II] - [] | album_artist | [Chopin] - [] | artist | [Chopin] - [chopin] | composer | [] - [02/13/2014] | date | [2014] - [Classics] | genre | [Classical] - [John_Doe ] | performer | [John Doe] - [Fantasie-Impromptu in c sharp MInor , Op.66] | title | [Fantasie-Impromptu in C Sharp Minor, Op. 66] - === COVERS === - ['cover.jpg' [500x500] ] | external | [/home/johndoe/music/Chopin/The Best Ever Piano ] - | | [Classics (John Doe, 2014)/Cover.jpg] - -# Scenario +#+BEGIN_EXAMPLE + \$ demlo fantasie_impromptu.flac + :: Load config: /home/johndoe/.config/demlo/demlorc + :: Load script: /usr/share/demlo/scripts/tag.lua + :: Load script: /usr/share/demlo/scripts/sub.lua + :: Load script: /usr/share/demlo/scripts/case.lua + :: Load script: /usr/share/demlo/scripts/punctuation.lua + :: Load script: /usr/share/demlo/scripts/encoding.lua + :: Load script: /usr/share/demlo/scripts/path.lua + :: Load script: /usr/share/demlo/scripts/cover.lua + :: Load script: /home/johndoe/.config/demlo/scripts/lossy.lua + ==> fantasie_impromptu.flac + + === FILE === + [/home/johndoe/fantasie_impromptu.flac] | path | [/home/johndoe/music/Chopin/The Best Ever Piano ] + | | [Classics (John Doe, 2014)/Fantasie-Impromptu in] + | | [ C Sharp Minor, Op. 66.ogg] + [flac] | format | [ogg] + [bitrate=320000] | parameters | [[-c:a libvorbis -q:a 10]] + === TAGS === + [john doe's classical collection II] | album | [John Doe's Classical Collection II] + [] | album_artist | [Chopin] + [] | artist | [Chopin] + [chopin] | composer | [] + [02/13/2014] | date | [2014] + [Classics] | genre | [Classical] + [John_Doe ] | performer | [John Doe] + [Fantasie-Impromptu in c sharp MInor , Op.66] | title | [Fantasie-Impromptu in C Sharp Minor, Op. 66] + === COVERS === + ['cover.jpg' [500x500] ] | external | [/home/johndoe/music/Chopin/The Best Ever Piano ] + | | [Classics (John Doe, 2014)/Cover.jpg] +#+END_EXAMPLE + +* Scenario + :PROPERTIES: + :CUSTOM_ID: scenario + :END: Demlo can address any of these recurring music library issues (and much more): - Lack of folder structure. -- Tags are not homogeneous, case is wrong, some entries are filled and other not. +- Tags are not homogeneous, case is wrong, some entries are filled and other + not. - Bad audio quality. - Audio codecs are not always consistent. (Lossless/lossy...) - mp3's id3tags hell... - Multiple covers, embedded and/or external, bad quality, etc. -# Features +* Features + :PROPERTIES: + :CUSTOM_ID: features + :END: A few scripts are provided by default. They give a good example of the possibilities offered by Demlo. Pick which one to load by default, rewrite some of them, or write new ones from scratch. You can preview changes to make sure everything looks satisfactory. -Call Demlo over a complete music library and it will process everything -in one single run. +Call Demlo over a complete music library and it will process everything in one +single run. Scripts can be chained: this feature makes Demlo extremely flexible and powerful. @@ -70,55 +85,73 @@ powerful. - Move all files according to a unique, yet dynamic folder hierarchy. For instance, setting - output.path = table.concat {library, '/', o.artist, '/', - o.album ~= nil and (o.date ~= nil and o.album .. '/' or o.date .. ' - ' .. o.album .. '/') or '', - track_padded ~= nil and '' or track_padded .. ' - ', - o.title} + #+BEGIN_EXAMPLE + output.path = table.concat {library, '/', o.artist, '/', + o.album ~= nil and (o.date ~= nil and o.album .. '/' or o.date .. ' - ' .. o.album .. '/') or '', + track_padded ~= nil and '' or track_padded .. ' - ', + o.title} + #+END_EXAMPLE - will move john doe - 1. intro.ogg to + will move =john doe - 1. intro.ogg= to - - ~/music/John Doe/2013 - Great Album/01 - Intro.ogg if there is an album and a date, - - ~/music/John Doe/Great Album/01 - Intro.ogg if there is an album but no date, - - ~/music/John Doe/01 - Intro.ogg if there is no album. + - =~/music/John Doe/2013 - Great Album/01 - Intro.ogg= if there is an album + and a date, + - =~/music/John Doe/Great Album/01 - Intro.ogg= if there is an album but no + date, + - =~/music/John Doe/01 - Intro.ogg= if there is no album. - Case checking: the default script 'case.lua' will turn everything to title -case or sentence case. It supports special cases such as Roman numerals, -McDonald, etc. It also supports a list of exceptions, e.g. 'DJ' and 'feat'. + case or sentence case. It supports special cases such as Roman numerals, + McDonald, etc. It also supports a list of exceptions, e.g. 'DJ' and 'feat'. - Encoding: the file format and codec is exposed to user scripts so that you can -conditionally re-encode any file. The encoding is performed by FFmpeg. The -encoding parameters can be fully customized. (Format, codec, bitrate, etc.) + conditionally re-encode any file. The encoding is performed by FFmpeg. The + encoding parameters can be fully customized. (Format, codec, bitrate, etc.) - Covers: Demlo makes it possible to automatically remove embedded covers, skip -duplicates, skip covers below a quality threshold, resize big covers down, etc. + duplicates, skip covers below a quality threshold, resize big covers down, + etc. - Different kinds of music: classical, soundtrack, and band music usually have -different types of tags and different folder structures. It is possible to -manage different music libraries easily by setting the desired root folder from -command-line. + different types of tags and different folder structures. It is possible to + manage different music libraries easily by setting the desired root folder + from command-line. - Demlo supports both external and embedded cuesheets. It is also able to parse -"non-standard" sheets, which are not so uncommon. + "non-standard" sheets, which are not so uncommon. - Demlo can be interfaced with any other program, both ways. (X calls Demlo or -Demlo calls X.) For instance you can edit tags with your favorite text editor. + Demlo calls X.) For instance you can edit tags with your favorite text editor. - Demlo has MusicBrainz and Cover Archive support for online tagging and cover -fetching. + fetching. -# Installation +* Installation + :PROPERTIES: + :CUSTOM_ID: installation + :END: -See [the development page][dev]. +See [[https://github.com/ambrevar/demlo][the development page]]. -# Usage +* Usage + :PROPERTIES: + :CUSTOM_ID: usage + :END: -See demlo -h and the [manual][]. +See =demlo -h= and the [[http://godoc.org/github.com/ambrevar/demlo][manual]]. -# License +* License + :PROPERTIES: + :CUSTOM_ID: license + :END: -See the LICENSE file in [the source code][src]. +See the LICENSE file in [[https://github.com/ambrevar/demlo/releases][the source +code]]. -# The Go rewrite +* The Go rewrite + :PROPERTIES: + :CUSTOM_ID: the-go-rewrite + :END: Demlo used to be written in pure Lua. This has led to numerous issues, such as poor parallelism, slow processing, unreliable dependencies, race conditions, @@ -128,77 +161,79 @@ Starting from version 2 onward, the core has been rewritten in Go to address all these issues. Many changes have been undertaken in the process: - Performance: Faster JSON (un)marshaling, faster checksums (no subprocess, -partial checksums), better parallelization. + partial checksums), better parallelization. - CLI flags: Reduced the number of flags and made them simpler. Some flags can -be used more than once, which makes it possible to call scripts with spaces in -their names. + be used more than once, which makes it possible to call scripts with spaces in + their names. - Portability: Core detection now works on all OSes. - Race conditions: Remove all race conditions (cache, screen output, file -access, temp files). + access, temp files). -- tags are no longer stripped away from input.streams[i].tags nor -input.format.tags. +- =tags= are no longer stripped away from =input.streams[i].tags= nor + =input.format.tags=. - Cuesheet: Multi-file support. - Preview: Changes in path, format, parameters and covers are displayed using -the same formatting than tags. Long lines are wrapped. Formatted preview is off -when stdout is redirected, JSON formatting is used instead. + the same formatting than tags. Long lines are wrapped. Formatted preview is + off when stdout is redirected, JSON formatting is used instead. -- Full unicode support in scripts (string.len, string.gsub, etc.). Fix -issues with [XY] regexps where X and Y are non-ASCII. +- Full unicode support in scripts (=string.len=, =string.gsub=, etc.). Fix + issues with =[XY]= regexps where X and Y are non-ASCII. - Scripts can be parameterized through global variables. -- Config file is defined from the DEMLORC environment variable or default -path, no more from a CLI flag. It greatly alleviates the complexity of flag -parsing. +- Config file is defined from the =DEMLORC= environment variable or default + path, no more from a CLI flag. It greatly alleviates the complexity of flag + parsing. - Sandboxes accept more functions from the Lua standard library, such as -os.getenv. + =os.getenv=. -- input and output structures have changed. (See compatibility notice.) +- =input= and =output= structures have changed. (See compatibility notice.) - Lua patterns have been replaced by Go regexps. (See compatibility notice.) - Extra audio streams and non-cover streams are stripped out. - Misc. fixes: Index creation for multi-track files, non-integer cover -resolution, etc. + resolution, etc. - MusicBrainz: Clean queries, improved heuristic. The official artist name is -used instead of the official album artist name. (It enforces homogeneity.) + used instead of the official album artist name. (It enforces homogeneity.) -- The titlecase function has been moved to the script side and split into -'setcase' and 'punctuation'. +- The =titlecase= function has been moved to the script side and split into + 'setcase' and 'punctuation'. - Support for in-place tagging. This dramatically speeds up the process when the --rmsrc option is active. + =-rmsrc= option is active. -## Compatibility notice +** Compatibility notice + :PROPERTIES: + :CUSTOM_ID: compatibility-notice + :END: -User scripts still rely on input and output only. These structures have +User scripts still rely on =input= and =output= only. These structures have remained identical but for a few elements: -- filename has been renamed to path. -- output.bitrate has been replaced by the output.parameters array. Use it to -pass arbitrary arguments to FFmpeg (codec, bitrate, etc.). +- =filename= has been renamed to =path=. +- =output.bitrate= has been replaced by the =output.parameters= array. Use it to + pass arbitrary arguments to FFmpeg (codec, bitrate, etc.). - Cover options have been fully ravamped. The Lua patterns of the string library have been replaced by -[Go regexps](https://github.com/google/re2/wiki/Syntax). - -# Links - -- [Development page][dev] -- [Issue tracker](https://github.com/ambrevar/demlo/issues) -- [Source code][src] -- [Manual][manual] -- [Arch Linux package (AUR)](https://aur.archlinux.org/packages/demlo/) - -[dev]: https://github.com/ambrevar/demlo -[manual]: http://godoc.org/github.com/ambrevar/demlo -[src]: https://github.com/ambrevar/demlo/releases +[[https://github.com/google/re2/wiki/Syntax][Go regexps]]. + +* Links + :PROPERTIES: + :CUSTOM_ID: links + :END: + +- [[https://github.com/ambrevar/demlo][Development page]] +- [[https://github.com/ambrevar/demlo/issues][Issue tracker]] +- [[https://github.com/ambrevar/demlo/releases][Source code]] +- [[http://godoc.org/github.com/ambrevar/demlo][Manual]] +- [[https://aur.archlinux.org/packages/demlo/][Arch Linux package (AUR)]] diff --git a/projects/hsync.org b/projects/hsync.org index e47bdea7d2a9ac2ae0a25016616c1ac64c364b25..cba8a3f488f88f1e80c9e95534c63f7bcbe1c1c7 100644 --- a/projects/hsync.org +++ b/projects/hsync.org @@ -1,9 +1,12 @@ -% hsync -% A filesystem hierarchy synchronizer +#+TITLE: hsync -# About +#+AUTHOR: A filesystem hierarchy synchronizer +* About + :PROPERTIES: + :CUSTOM_ID: about + :END: -hsync SOURCE TARGET renames files in TARGET so that identical files found in +=hsync SOURCE TARGET= renames files in TARGET so that identical files found in SOURCE and TARGET have the same relative path. The main goal of the program is to make folders synchronization faster by @@ -12,9 +15,9 @@ synchronization programs that lack this capability. By default, files are not renamed and a preview is printed to standard output. -False positives can happen, e.g. if two different files in SOURCE and TARGET are -the only ones of this size. Use the preview to spot false positives and make sure -all files get renamed properly. +False positives can happen, e.g. if two different files in SOURCE and TARGET are +the only ones of this size. Use the preview to spot false positives and make +sure all files get renamed properly. You can redirect the preview to a file. If you run the program using this preview file as SOURCE, the analysis will be skipped. This is useful if you want @@ -25,54 +28,74 @@ Notes: - Duplicate files in either folder are skipped. - Only regular files are processed. In particular, empty folders and symbolic -links are ignored. + links are ignored. -# Examples +* Examples + :PROPERTIES: + :CUSTOM_ID: examples + :END: -_hsync_ can easily be associated with _rsync_ (or any synchronization for that +/hsync/ can easily be associated with /rsync/ (or any synchronization for that matters) to speed up mirroring: - hsync -p /path/to/storage /path/to/mirror - rsync -livr --size-only --delete-excluded /path/to/storage/ /path/to/mirror +#+BEGIN_EXAMPLE + hsync -p /path/to/storage /path/to/mirror + rsync -livr --size-only --delete-excluded /path/to/storage/ /path/to/mirror +#+END_EXAMPLE -_hsync_ also has some interesting side uses. +/hsync/ also has some interesting side uses. -- Since _hsync_ will display a warning on duplicates, it can be used to get the -list of duplicates in a folder: +- Since /hsync/ will display a warning on duplicates, it can be used to get the + list of duplicates in a folder: - hsync /path/to/analyze /dev/null > ~/dups.log + #+BEGIN_EXAMPLE + hsync /path/to/analyze /dev/null > ~/dups.log + #+END_EXAMPLE -- If SOURCE is a [JSON](http://json.org/) file with the following structure: +- If SOURCE is a [[http://json.org/][JSON]] file with the following structure: - { - "oldpath1": "newpath1", - "oldpath2": "newpath2", - ... - "oldpathN": "newpathN" - } + #+BEGIN_EXAMPLE + { + "oldpath1": "newpath1", + "oldpath2": "newpath2", + ... + "oldpathN": "newpathN" + } + #+END_EXAMPLE - _hsync_ will rename files in TARGET accordingly. This JSON file can be - generated by hand or dynamically from any other program. If generated from - an _hsync_ preview, TARGET needs not be the one used for previewing the - renames. This is useful for complex renaming operations containing cycles - (e.g. a->b->c->a) or chains (e.g. a->b->c->d). + /hsync/ will rename files in TARGET accordingly. This JSON file can be + generated by hand or dynamically from any other program. If generated from an + /hsync/ preview, TARGET needs not be the one used for previewing the renames. + This is useful for complex renaming operations containing cycles (e.g. + =a->b->c->a=) or chains (e.g. =a->b->c->d=). -# Installation +* Installation + :PROPERTIES: + :CUSTOM_ID: installation + :END: -See [the development page][dev]. +See [[https://github.com/ambrevar/hsync][the development page]]. -# Usage +* Usage + :PROPERTIES: + :CUSTOM_ID: usage + :END: -See hsync -h. +See =hsync -h=. -# License +* License + :PROPERTIES: + :CUSTOM_ID: license + :END: See the LICENSE in [the source code][src]. -# Links +* Links + :PROPERTIES: + :CUSTOM_ID: links + :END: -- [Development page][dev] (Please file reports there.) -- [Implementation details](http://godoc.org/github.com/ambrevar/hsync) -- [Arch Linux package (AUR)](https://aur.archlinux.org/packages/hsync/) - -[dev]: https://github.com/ambrevar/hsync +- [[https://github.com/ambrevar/hsync][Development page]] (Please file reports + there.) +- [[http://godoc.org/github.com/ambrevar/hsync][Implementation details]] +- [[https://aur.archlinux.org/packages/hsync/][Arch Linux package (AUR)]] diff --git a/projects/inprogen.org b/projects/inprogen.org index 00362363307cb07566c904a6f71cfa8d031164d1..0df0e8b5b7d71cfd8321998bcb7e728fd2ab0150 100644 --- a/projects/inprogen.org +++ b/projects/inprogen.org @@ -1,13 +1,19 @@ -% Inprogen -% An inverse procedural generator +#+TITLE: Inprogen -### About +#+AUTHOR: An inverse procedural generator +*** About + :PROPERTIES: + :CUSTOM_ID: about + :END: This is the home page of my research on inverse procedural generation applied to building facades. A quick presentation of the concept follows. I will update this page to keep track of my progress. -### Background +*** Background + :PROPERTIES: + :CUSTOM_ID: background + :END: In computer graphics it is often useful to generate the content automatically to ease the burden of drawing everything by hand. Common examples include cities, @@ -21,7 +27,10 @@ This is where procedural graphics prove useful: with a set of rules combined to some transformation functions, one can generate a whole city with its underlying randomness by specifying a few parameters. -### Goals +*** Goals + :PROPERTIES: + :CUSTOM_ID: goals + :END: The use of procedural generator can be overwhelming: they usually require of dozens of inter-related parameters. A tiny alteration in a parameter value may @@ -30,28 +39,37 @@ graphics is the understanding of the parameter set for every generator. To overcome the limitation of procedural generation, we could think of a different approach: from a real-world input, we can deduce the right parameter -set that generate the desired result. This is called _inverse procedural -generation_. +set that generate the desired result. This is called /inverse procedural +generation/. Nonetheless facade formalization might involve hundreds of parameters. Some of them may affect minor details in the output, others may transform the result -drastically. In other words, they have different psychophysical properties, i.e. -they affect the way we perceive the building unevenly. +drastically. In other words, they have different psychophysical properties, +i.e. they affect the way we perceive the building unevenly. Which one makes it look believable? Does it look old or sad? Does it belong to a specific neighbourhood of some city? A psychophysical study of the architectural properties is relevant to improve the generation process. -### Progress +*** Progress + :PROPERTIES: + :CUSTOM_ID: progress + :END: -#### Forward generation +**** Forward generation + :PROPERTIES: + :CUSTOM_ID: forward-generation + :END: The first logical move is to master the forward procedural generation. Thus I -have implemented a prototype in Lua/Cairo based on the papers _Instant -Architecture_ (Wonka et al.) and _Procedural Modeling of Buildings_ (Müller et +have implemented a prototype in Lua/Cairo based on the papers /Instant +Architecture/ (Wonka et al.) and /Procedural Modeling of Buildings/ (Müller et al.). -#### Inverse generation from a data structure +**** Inverse generation from a data structure + :PROPERTIES: + :CUSTOM_ID: inverse-generation-from-a-data-structure + :END: A facade can be seen as a tile of rectangular regions. This model can be conveniently stored into a data structure. @@ -60,39 +78,45 @@ Before working directly with pictures, we can feed our inverse data generator with a manually created data structure. As for the forward generator, I have implemented a Lua prototype with some basic -heuristics. The (only?) reference I have been using is _Inverse Procedural -Modeling of Facade Layouts_ (Fuzhang Wu et al.). +heuristics. The (only?) reference I have been using is /Inverse Procedural +Modeling of Facade Layouts/ (Fuzhang Wu et al.). The data structure and its associated functions are actually quite interesting to study. I will devote an article to it later. -#### Pre-processing +**** Pre-processing + :PROPERTIES: + :CUSTOM_ID: pre-processing + :END: The input data is real-world pictures. Some work is needed before we can get to the convenient data structure our inverse generator can use. -* [Perspective correction](../perspector/index.html): it is easier to work on -orthographic data. Real life pictures of facades are never perfectly aligned, -but this can be easily rectified. Even with some specialized hardware it is not -always possible to take orthographic pictures of facades, e.g. in narrow streets -with tall buildings. +- [[../perspector/index.html][Perspective correction]]: it is easier to work on + orthographic data. Real life pictures of facades are never perfectly aligned, + but this can be easily rectified. Even with some specialized hardware it is + not always possible to take orthographic pictures of facades, e.g. in narrow + streets with tall buildings. -* Occlusion handling: facade picture are usually not barebone, a lot of elements -cannot be taken away when taking the picture. This includes streets signs and -lights, bystanders, trees, and so on. They should not be included in the final -data structure, thus they should be removed. It is possible to handle most cases -automatically. +- Occlusion handling: facade picture are usually not barebone, a lot of elements + cannot be taken away when taking the picture. This includes streets signs and + lights, bystanders, trees, and so on. They should not be included in the final + data structure, thus they should be removed. It is possible to handle most + cases automatically. - To implement. It is still possible to work without it on ideal facades. + To implement. It is still possible to work without it on ideal facades. -* Lighting normalization: depending on the time of the day, the weather and the -cast shadows, not all parts of the facade might illuminated equally. This can -disturb the segmentation process, e.g. a shadow on a region can be seen as -different regions. +- Lighting normalization: depending on the time of the day, the weather and the + cast shadows, not all parts of the facade might illuminated equally. This can + disturb the segmentation process, e.g. a shadow on a region can be seen as + different regions. - To implement. + To implement. -#### Segmentation +**** Segmentation + :PROPERTIES: + :CUSTOM_ID: segmentation + :END: Once our images have been pre-processed, the segmentation can be run serenely. The quality of the result can hardly be garanteed. Indeed, there is no formal @@ -103,17 +127,27 @@ or split regions, resize them, group them, and so on. To implement. -### Future +*** Future + :PROPERTIES: + :CUSTOM_ID: future + :END: The current state of the art is fairly limited, yet certainly extensible to 3D. This is definitely some field to explore. -### References +*** References + :PROPERTIES: + :CUSTOM_ID: references + :END: -* Instant Architecture, Peter Wonka et al. -* Inverse Procedural Modeling of Facade Layouts, Fuzhang Wu et al. -* Procedural Modeling of Buildings, Pascal Müller et al. +- Instant Architecture, Peter Wonka et al. +- Inverse Procedural Modeling of Facade Layouts, Fuzhang Wu et al. +- Procedural Modeling of Buildings, Pascal Müller et al. -### Links +*** Links + :PROPERTIES: + :CUSTOM_ID: links + :END: -* [Perspector](../perspector/index.html) - a control-point-based perspective rectification tool. +- [[../perspector/index.html][Perspector]] - a control-point-based perspective + rectification tool. diff --git a/projects/inprogen/gallery.org b/projects/inprogen/gallery.org index 045103acbaac995a9d80e0e40c1e5f3ab253780c..0f7f6043078aba0f7deb0ad6e2043589f1c77080 100644 --- a/projects/inprogen/gallery.org +++ b/projects/inprogen/gallery.org @@ -1,24 +1,16 @@ -% [Perspector](index.html) -% Gallery +#+TITLE: [[file:index.html][Perspector]] -![Facade 1](1.jpg) -![Rectified facade 1](1-rect.jpg) +#+AUTHOR: Gallery +[[file:1.jpg]] [[file:1-rect.jpg]] -![Facade 2](2.jpg) -![Rectified facade 2](2-rect.jpg) +[[file:2.jpg]] [[file:2-rect.jpg]] -![Facade 3](3.jpg) -![Rectified facade 3](3-rect.jpg) +[[file:3.jpg]] [[file:3-rect.jpg]] -![Facade 4](4.jpg) -![Rectified facade 4](4-rect.jpg) +[[file:4.jpg]] [[file:4-rect.jpg]] -![Facade 5](5.jpg) -![Rectified facade 5](5-rect.jpg) +[[file:5.jpg]] [[file:5-rect.jpg]] -![Facade 6](6.jpg) -![Rectified facade 6](6-rect.jpg) - -![Facade 7](7.jpg) -![Rectified facade 7](7-rect.jpg) +[[file:6.jpg]] [[file:6-rect.jpg]] +[[file:7.jpg]] [[file:7-rect.jpg]] diff --git a/projects/perspector.org b/projects/perspector.org index 96b7d73a9a92edf6d17cf02ee572c16cbff97671..f063e314da1c834b08b2657c8c329cfc0f29d558 100644 --- a/projects/perspector.org +++ b/projects/perspector.org @@ -1,7 +1,10 @@ -% Perspector -% A control-point-based perspective rectification tool +#+TITLE: Perspector -# About +#+AUTHOR: A control-point-based perspective rectification tool +* About + :PROPERTIES: + :CUSTOM_ID: about + :END: Perspector is a perspective rectification tool. @@ -18,33 +21,46 @@ axis-aligned version, without being revolutionary. I developed this program as I could not find such a thing anywhere. I believe it would be great to see it implemented in a tool like Gimp. -# Preview +* Preview + :PROPERTIES: + :CUSTOM_ID: preview + :END: -![Facade 1](1.jpg) -![Rectified facade 1](1-rect.jpg) +[[file:1.jpg]] [[file:1-rect.jpg]] -See more in the [gallery](gallery.html). +See more in the [[file:gallery.html][gallery]]. -# Usage +* Usage + :PROPERTIES: + :CUSTOM_ID: usage + :END: Select four control points on the picture. When processed, the picture will be transformed so that the four control points become the corners of an axis-aligned rectangle. -# Installation +* Installation + :PROPERTIES: + :CUSTOM_ID: installation + :END: It should run out of the box on most Unix-based systems. -Fetch the [source code][dev] and read the README file for further -instructions. +Fetch the [[http://github.com/ambrevar/perspector][source code]] and read the +README file for further instructions. -# License +* License + :PROPERTIES: + :CUSTOM_ID: license + :END: Perspector is under MIT license. -# Links +* Links + :PROPERTIES: + :CUSTOM_ID: links + :END: -* [Arch Linux package (AUR)](https://aur.archlinux.org/packages/perspector/) -* [Development page][dev]: source code, bug reports - -[dev]: http://github.com/ambrevar/perspector +- [[https://aur.archlinux.org/packages/perspector/][Arch Linux package (AUR)]] +- [[http://github.com/ambrevar/perspector][Development page]]: source code, bug + reports