1465 lines
63 KiB
Org Mode
1465 lines
63 KiB
Org Mode
# -*- eval: (require 'ox-hugo) -*-
|
||
#+title: Phundrak’s Blog
|
||
#+author: Lucien “Phundrak” Cartier-Tilet
|
||
#+hugo_base_dir: ../
|
||
#+hugo_section: ./
|
||
#+hugo_categories: emacs linux conlanging orgmode
|
||
#+startup: content
|
||
|
||
* [EN] Open-Sourcing ALYS :ALYS:
|
||
:PROPERTIES:
|
||
:EXPORT_FILE_NAME: open-sourcing-alys
|
||
:EXPORT_DATE: 2021-12-15
|
||
:export_hugo_menu: :menu "main"
|
||
:END:
|
||
#+TOC: headlines 1 local
|
||
|
||
** Too Long, Didn’t Read
|
||
VoxWave no longer exists as a company, ALYS lives on as an open-source
|
||
project under the GPL-3.0 and the BY-CC-4.0 license. You can find it
|
||
at the following address:
|
||
+[[https://labs.phundrak.com/phundrak/ALYS]]+
|
||
|
||
EDIT: The main repository moved to [[https://labs.phundrak.com/ALYS/ALYS]]
|
||
and vocal libraries are now separated in different repositories linked
|
||
from the main one.
|
||
|
||
** What happened?
|
||
You might have noticed it, but VoxWave became quite silent over the
|
||
last months. This is because we at the head of VoxWave chose to close
|
||
the company, a decision which came in effect in early September 2021.
|
||
There’s not much else to say.
|
||
*However*, the good news is the rest still goes on! ALYS as a project is
|
||
still alive and well! As her creator, I decided to step in and
|
||
continue its technical support. Since the company no longer exists,
|
||
and as a free and open-source software supporter, I also decided to
|
||
open-source ALYS as much as possible. As a result:
|
||
|
||
#+begin_center
|
||
ALYS for Alter/Ego is now free software as in /free beer/.
|
||
|
||
ALYS for UTAU, including its previously unreleased UTAU prototype, is
|
||
now free as in /free beer/ and in /freedom/.
|
||
#+end_center
|
||
|
||
You can find the installer of ALYS for Alter/Ego on the repository
|
||
linked above as well as a free licence file. Regarding its UTAU
|
||
version, its prototype is already configured with ~oto.ini~ files, but
|
||
the source file for its Alter/Ego version are stripped of any
|
||
configuration.
|
||
|
||
** What’s New?
|
||
Therefore, ALYS is now available under three different licences:
|
||
- The character is now under the free [[https://creativecommons.org/licenses/by-nc/4.0/][CC-BY-NC-4.0 licence]]
|
||
- The UTAU vocal libraries are now under the [[https://www.gnu.org/licenses/gpl-3.0.en.html][GPL-3.0license ]] ([[https://choosealicense.com/licenses/gpl-3.0/][short
|
||
readable version]])
|
||
- The Alter/Ego is still under a proprietary licence under my name, but
|
||
it is now available free of charge
|
||
|
||
Basically, this means you can do whatever you wish with the character
|
||
as long as it is non-commercial, and you credit [[https://www.instagram.com/hsaphirya/][Saphirya]], ALYS’
|
||
designer. The UTAU vocal libraries can be used, modified, and
|
||
redistributed as much as you wish as long as it stays under the
|
||
GPL-3.0 licence. And you are free to use the Alter/Ego vocal libraries
|
||
as much as you wish, but you cannot redistribute or modify them.
|
||
|
||
I also decided to release ALYS’ very first, secret, unreleased,
|
||
unheard French vocal library. It was scrapped no long after recording
|
||
it due to quality issues and was replaced by its French UTAU prototype
|
||
people could hear through ALYS’ first songs. It is released more as a
|
||
way of preserving the fact it existed rather preserving a usable vocal
|
||
library. (I don’t even remember what it sounds like.)
|
||
|
||
If you have any question, you are free to email me at
|
||
[[mailto:lucien@phundrak.com][lucien@phundrak.com]] or open an issue on the repository mentioned
|
||
above.
|
||
* Conlanging :@conlang:
|
||
** TODO Writing my conlanging docs with Emacs :emacs:conlanging:
|
||
* Development :@dev:
|
||
** [FR] Mettre à niveau mes sites org-mode :dev:emacs:
|
||
:PROPERTIES:
|
||
:EXPORT_FILE_NAME: mettre-a-nivea-mes-sites-org-mode
|
||
:EXPORT_DATE: 2022-08-15
|
||
:export_hugo_menu: :menu "main"
|
||
:END:
|
||
*** Le Problème
|
||
Cela fait quelques temps que je réfléchis à une nouvelle manière de
|
||
gérer deux de mes sites web, [[https://conlang.phundrak.com][conlang.phundrak.com]] et
|
||
[[https://config.phundrak.com][config.phundrak.com]].
|
||
|
||
Les deux sites sont actuellement générés via un export depuis org-mode
|
||
(un des nombreux modes d’Emacs) directement vers du HTML. Sauf que
|
||
l’organisation du fichier HTML de sortie de me plaît pas, et depuis
|
||
plus de deux ans j’utilise un script rédigé en Dart et compilé vers
|
||
Javascript pour réorganiser les fichiers. En soit ce ne serait pas
|
||
trop grave si mes pages web n’étaient pas forcément lourdes. Mais
|
||
elles le sont! La plus lourde page de mon site de linguistique fait
|
||
232Ko (la page francophone sur le Proto-Ñyqy) et celle de mon site de
|
||
configuration fait 5,5Mo (configuration Emacs) ! Je parle bien de
|
||
fichiers HTML ! Il faut vraiment que ça change!
|
||
|
||
*** Nouveau Framework pour le front-end
|
||
À la base je m’étais lancé pour écrire un exporteur personnalisé pour
|
||
exporter mes fichiers org-mode vers des fichiers JSX qui seraient
|
||
utilisés par un projet [[https://reactjs.org/][Reac]]t, ou même [[https://nextjs.org/][Next.js]]. Mais j’ai récemment
|
||
découvert quelque chose qui pourrait être bien plus pratique pour
|
||
moi : [[https://vuejs.org/][Vue]] et tout particulièrement [[https://v3.nuxtjs.org/][Nuxt]] !
|
||
|
||
En effet, Nuxt lit le [[https://content.nuxtjs.org/guide/writing/mdc/][MDC]], ou Markdown Components. De fait, il est
|
||
possible avec MDC et Nuxt d’insérer dans du Markdown des composants
|
||
Vue soit en blocs soit en inline. Et pour moi, ça change tout ! Je
|
||
peux maintenant écrire un exporteur minimal qui se chargera simplement
|
||
d’exporter quelques éléments personnalisés vers des composants Vue,
|
||
voire même de simples macros org-mode pour exporter les composants
|
||
inline.
|
||
|
||
Et bien sûr, pour pallier au problème de fichiers HTML trop lourds, il
|
||
me faudra séparer mes fichiers actuels en plusieurs fichiers, mais
|
||
cela devrait être plus simple à gérer une fois la transition vers le
|
||
nouveau framework effectuée.
|
||
|
||
*** Et pour le backend ?
|
||
Mais ce n’est pas tout : un élément que j’aimerais ajouter à mon site
|
||
de linguistique serait un dictionnaire entre mes langues construites
|
||
et d’autres langues, qu’elles soient construites ou non. Ce
|
||
dictionnaire doit pouvoir être interactif, avec par exemple une
|
||
recherche, une page par mot, etc.
|
||
|
||
Je ne ferai certainement pas télécharger à mes utilisateurs
|
||
l’entièreté du dictionnaire à chaque recherche d’un mot dans le
|
||
dictionnaire, il ne peut donc pas être hébergé avec mon frontend, et
|
||
j’aurai besoin d’un backend avec une API REST pour gérer les requêtes
|
||
des visiteurs du site web. Maintenant la question est, quel type de
|
||
back-end ?
|
||
|
||
Tout d’abord, je vais complexifier un peu le problème : je suis un
|
||
grand amateur de org-mode. Je pourrais gérer ça via une base de
|
||
données classique, ajoutant chaque entrée manuellement, mais je vais
|
||
plutôt essayer de gérer tout ça via org-mode. Les fichiers texte sont
|
||
plus simples à versionner que des bases de données en un seul fichier
|
||
binaire. Du coup, il va falloir que je m’écrive un nouvel exporter,
|
||
mais lequel ?
|
||
|
||
Je pourrais rédiger un exporteur pour mon fichier ~dictionnaire.org~ qui
|
||
l’exporterait vers un fichier Json qui serait lu ensuite par mon
|
||
backend qui extraierait et enverrai à mes utilisateurs les
|
||
informations nécessaires. L’avantage serait de n’avoir quasiment pas
|
||
besoin de manipuler le Json et d’en envoyer tel quel. Mais l’ouverture
|
||
et fermeture constante du fichier n’est pas forcément la meilleure des
|
||
idées, quoi que cela pourrait permettre de remplacer le fichier
|
||
pendant que le backend tourne. Mais je suis sûr qu’on peut mieux
|
||
faire.
|
||
|
||
Ma solution suivante était d’utiliser EmacSQL, un paquet Emacs lui
|
||
permettant d’interagir avec des bases de données SQLite, PostgreSQL et
|
||
MySQL. Au moins ce serait une véritable base de données, avec
|
||
seulement un blob binaire à mettre à jour, et ce serait
|
||
potentiellement plus performant étant donné qu’il n’y aura qu’à ouvrir
|
||
une fois une connexion avec elle. Mais le problème est maintenant sa
|
||
mise à jour. Mince…
|
||
|
||
Vient enfin ma troisième solution qui, je pense, sera celle que je
|
||
vais adopter : utiliser une base de donnée type Firebase. L’idée d’un
|
||
verrouillage fournisseur ne me plaît pas franchement, donc j’ai décidé
|
||
d’utiliser une alternative open source et hébergeable : [[https://appwrite.io/][Appwrite]]! Je
|
||
peux écrire sur une de ses bases de données pendant que mes
|
||
utilisateurs peuvent la lire, donc la mise à jour n’est pas un
|
||
problème, et je n’ai rien à mettre en ligne, seulement une série de
|
||
requêtes à faire. Cependant, un problème reste : comment communiquer
|
||
avec Appwrite?
|
||
|
||
*** La quête pour un SDK Appwrite pour Emacs
|
||
Hélas, j’ai beau chercher, il n’existe aucun paquet pour Emacs
|
||
permettant une communication avec Appwrite. Mais ce n’est pas
|
||
franchement surprenant : Appwrite n’est pas encore extrêmement
|
||
répandu, et même Firebase ne dispose pas de paquet pour Emacs.
|
||
|
||
Bien heureusement, Appwrite dispose d’une API REST assez bien
|
||
documentée, et Emacs est capable de gérer des requêtes nativement via
|
||
sa bibliothèque ~url~, c’est donc naturellement que j’ai commencé à
|
||
travailler sur ~appwrite.el~, un SDK Appwrite pour du Emacs Lisp.
|
||
J’aurais pu utiliser ~request.el~, un paquet assez populaire pour Emacs
|
||
afin de gérer les requêtes HTTP, mais je ne suis pas grand fan de son
|
||
workflow et je préfère limiter au maximum le nombre de dépendances
|
||
dans mes paquets. Ce que ce paquet fait actuellement est une
|
||
transformation des paramètres nommés que mes fonctions acceptent en un
|
||
payload Json. Par exemple, ma fonction ~appwrite-stogare-list-buckets~
|
||
accepte les mot-clefs ~search~, ~limit~, ~offset~, ~cursor~, ~cursor-direction~
|
||
et ~order-type~. Ces arguments sont transformés en du Json via la
|
||
bibliothèque native d’Emacs afin de donner ceci :
|
||
#+begin_src js
|
||
{
|
||
"search": "my search request",
|
||
"limit": 30,
|
||
"offset": 0,
|
||
"cursor": "",
|
||
"cursorDirection": "before",
|
||
"orderType": "ASC",
|
||
}
|
||
#+end_src
|
||
|
||
Ce payload Json est enfin envoyé à l’API REST correspondante, en
|
||
l’occurrence ~/v1/storage/buckets~ comme on peut le voir [[https://appwrite.io/docs/server/storage?sdk=nodejs-default#storageListBuckets][sur la
|
||
documentation officielle]]. Bien sûr, les éléments optionels ne sont
|
||
pas nécessairement inclus afin d’éviter à avoir à envoyer trop
|
||
d’informations. Dans ce cas, tous les éléments du payload sont
|
||
optionels, ce qui ferait que le ~appwrite.el~ n’enverra que src_js{{}}
|
||
comme payload à l’API.
|
||
|
||
Pour l’instant, le projet en est encore à ses débuts, mais j’ai
|
||
commencé à travailler sur le SDK pour Appwrite que vous pouvez trouver
|
||
sur [[https://github.com/Phundrak/appwrite.el][ce dépôt Github]].
|
||
|
||
La question maintenant est : comment exporter mon dictionnaire vers
|
||
Appwrite ? La réponse me semble relativement simple ; je pourrai
|
||
écrire un exporteur org-mode dépendant de ~appwrite.el~ qui exportera
|
||
pour chaque mot qu’il rencontrera un payload Json vers mon instance
|
||
personnelle Appwrite. Et à la différence des exporteurs org-mode
|
||
habituels, ~ox-appwrite~ n’exportera aucun fichier sur mon système.
|
||
|
||
*** Conclusions
|
||
Au fur et à mesure de mon analyse du projet et de mes besoins, je me
|
||
suis rendu compte que j’aurai besoin d’outils plus intelligents que de
|
||
simples pages HTML exportées automatiquement via Emacs.
|
||
|
||
Ainsi, j’aurai besoin de créer un site web avec Nuxt, profitant ainsi
|
||
de sa capacité à rendre du Markdown avec du contenu interactif,
|
||
agissant en tant que frontend pour mon site web. Ce Markdown sera
|
||
exporté via org-mode à partir de mes fichiers déjà existants, bien
|
||
qu’à fragmenter afin de réduire la taille des fichiers de sortie.
|
||
|
||
Le backend sera une instance Appwrite que j’hébergerai moi-même sur
|
||
mes serveurs. Elle sera populée par un exporter org-mode custom via
|
||
Emacs, ce qui me permettra de continuer à gérer mes dictionnaires et
|
||
mes langues avec org-mode.
|
||
|
||
Ce projet est vraiment intéressant car cela m’a incité à explorer de
|
||
nombreuses possibilités et technologies différentes afin de trouver ce
|
||
qui correspond le mieux à mon besoin, notamment en me rendant compte
|
||
par exemple que React n’était pas forcément l’outil le plus adapté à
|
||
ce projet précisément. Cela me fera également travailler sur ma
|
||
capacité à interagir avec des backends et des API REST, tout autant du
|
||
côté front-end pour le site web que du côté SDK avec Emacs. Enfin, la
|
||
création de ce SDK ainsi que des exporteurs org-mode me sera bénéfique
|
||
afin d’approfondir ma connaissance d’Emacs et du Emacs Lisp.
|
||
|
||
Maintenant, au travail !
|
||
|
||
** [EN] Writing a Dynamic Array in C :dev:C:
|
||
:PROPERTIES:
|
||
:EXPORT_FILE_NAME: writing-dynamic-vector-c
|
||
:EXPORT_DATE: 2020-11-28
|
||
:export_hugo_menu: :menu "main"
|
||
:END:
|
||
/Edit on October 28th 2023:/ This article was written on November 28th
|
||
2020, almost three years ago. Since then, I have noticed issues with
|
||
the current implementation of my dynamic C array, as noted by some
|
||
readers in the comments below. I will probably rewrite a new dynamic
|
||
array in C some time in the future addressing these issues.
|
||
|
||
Although C is a very, very popular language, it is also known to be
|
||
quite tiny: memory is handled manually, and much of what is available
|
||
in its standard library is a given in all other languages. But C being
|
||
a low level language also means it lacks a lot of other stuff other
|
||
popular languages have; for instance, dynamic arrays are present in
|
||
the library of most popular languages, be it JavaScript, C++, Rust and
|
||
so on, but C’s simplicity forbids them from being there. If you want
|
||
it in C, you have to implement it –which is exactly what I did!
|
||
|
||
#+TOC: headlines 1 local
|
||
*** Introduction
|
||
When I wrote this library, I was mostly inspired by C++’s ~std::vector~
|
||
and Rust’s ~std::vec::Vec~, but my library lacks some features both
|
||
have: it’s still a simple one. Here is the list of what it is able to
|
||
do:
|
||
- Create a dynamic array, with or without an initial capacity
|
||
specified by the user
|
||
- Store a function pointer to the /destructor/ of the elements that will
|
||
be stored in the vector for when they are destroyed
|
||
- Append new elements at the end of the array
|
||
- Get elements by position, safely or not, or get the first and last
|
||
elements in the array
|
||
- Get the length of the vector as well as its capacity
|
||
- Shrink the size of the allocated array to the size of the vector
|
||
- Remove an element at a specific index, or the last element
|
||
- Completely destroy the vector and its elements
|
||
|
||
Elements that will be stored in the vector will need to be dynamically
|
||
allocated in memory since the vector will not store the elements
|
||
themselves, but rather pointers to them. This way, we avoid copying
|
||
data when inserting it to the vector, and handling these elements is
|
||
also a tad easier. And since we do not know what we will be storing,
|
||
we will be storing void pointers. The user will be able to cast them
|
||
to their desired type later on.
|
||
|
||
Before defining the vector, there are a few things I want to define.
|
||
First, there is an attribute I will often use with my functions:
|
||
#+NAME: vector-nonnull-h
|
||
#+BEGIN_SRC c
|
||
#indef NONNULL
|
||
# define NONNULL __attribute__((nonnull))
|
||
#endif
|
||
#+END_SRC
|
||
This will forbid passing to functions marked with this attribute ~NULL~
|
||
pointers, because we will use a lot of them.
|
||
|
||
We will also need to include some headers:
|
||
- ~assert.h~ :: so we can make sure memory is allocated and reallocated
|
||
correctly
|
||
- ~string.h~ :: for some memory operations such as ~memcpy~
|
||
#+NAME: vector-includes-c
|
||
#+BEGIN_SRC c
|
||
#include <assert.h>
|
||
#include <string.h>
|
||
#+END_SRC
|
||
|
||
We also need to define a type that will be used as the destructor
|
||
type. The functions we want to accept as destructors are functions
|
||
that accept a void pointer to an element and return nothing, hence
|
||
this definition:
|
||
#+NAME: vector-destructor-type-h
|
||
#+BEGIN_SRC c
|
||
typedef void (*Destructor)(void *element);
|
||
#+END_SRC
|
||
|
||
Now, onto the structure itself.
|
||
|
||
*** The Data Structure of the Vector
|
||
With our vector, we will need to keep track a couple of things:
|
||
- the size of the vector
|
||
- the capacity of the vector
|
||
- the destructor
|
||
- the array itself
|
||
With this, we can describe our structure for the vector:
|
||
#+NAME: vector-struct-def
|
||
#+BEGIN_SRC c
|
||
struct Vector_s {
|
||
size_t capacity;
|
||
size_t length;
|
||
void ** elements;
|
||
Destructor destructor;
|
||
};
|
||
typedef struct Vector_s Vector;
|
||
#+END_SRC
|
||
|
||
We have now four elements:
|
||
- ~elements~ :: an array of void pointers pointing themselves either to
|
||
elements stored in the vector or to nothing (initialized to ~NULL~)
|
||
(note this forbids storing ~NULL~ elements in the vector),
|
||
- ~length~ :: the number of elements currently stored in the vector,
|
||
- ~capacity~ :: the size of the allocated memory pointed to by ~elements~
|
||
divided by the size of a void pointer. This gives us the amount of
|
||
elements that can be stored in the vector without any reallocation /at most/,
|
||
- ~destructor~ :: pointer to the function used to free elements stored
|
||
in the vector
|
||
|
||
Now, onto the functions associated with this data structure. They are
|
||
all prefixed with ~vec_~ in order to avoid any collisions with other
|
||
libraries and functions.
|
||
|
||
*** Building Vectors
|
||
The first function for building vectors is ~vec_new()~. Here is its
|
||
definition:
|
||
#+NAME: vector-vec_new-h
|
||
#+BEGIN_SRC c
|
||
Vector *vec_new(Destructor const destructor);
|
||
#+END_SRC
|
||
|
||
It is quite straightforward: when creating a new, standard vector,
|
||
simply pass as its arguments a pointer to the destructor of this
|
||
vector, either a ~NULL~ pointer for trivial data types, or a pointer to
|
||
an existing function you declared somewhere. Once you do that, you get
|
||
yourself a pointer to the newly created vector with which you can now
|
||
store elements. Let’s see how it works under the hood:
|
||
#+NAME: vector-vec_new-c
|
||
#+BEGIN_SRC c
|
||
Vector *vec_new(Destructor const destructor)
|
||
{
|
||
Vector *self;
|
||
self = (Vector *)malloc(sizeof(Vector));
|
||
assert(self);
|
||
,*self = (Vector){.length = 0,
|
||
.capacity = VEC_INITIAL_CAPACITY,
|
||
.elements = (void *)malloc(sizeof(void *) * VEC_INITIAL_CAPACITY),
|
||
.destroy = destructor};
|
||
assert(self->elements);
|
||
return self;
|
||
}
|
||
#+END_SRC
|
||
|
||
A new pointer is created, which will be the pointer returned to the
|
||
user. To this pointer, we allocate enough memory to hold a vector.
|
||
Once that is done, we initialize this new memory buffer with an actual
|
||
vector, with its members initialized as described above. An assertion
|
||
is done in order to ensure both the vector but also its storage are
|
||
correctly allocated.
|
||
|
||
The second function, ~vec_with_capacity~, is quite similar though not
|
||
the same as ~vec_new~: it allows for an initialization of
|
||
~vec_with_capacity~ with a user-defined amount of capacity in the
|
||
storage of the vector. That is, if ~vec_with_capacity(14)~ is called,
|
||
the library will return a pointer to a vector which can contain and
|
||
has the size of precisely fourteen elements. That way, if the user
|
||
knows they’ll need a certain amount of elements to be stored in a
|
||
vector, they’ll be able to reserve that exactly and limit the amount
|
||
of reallocations when adding new elements. Its definition is the
|
||
following:
|
||
#+NAME: vector-vec_with_capacity-h
|
||
#+BEGIN_SRC c
|
||
Vector *vec_with_capacity(Destructor const destructor, size_t const capacity);
|
||
#+END_SRC
|
||
|
||
Under the hood, it calls ~vec_new~, then it will reallocate the memory
|
||
already allocated for the member ~elemements~.
|
||
#+NAME: vector-vec_with_capacity-c
|
||
#+BEGIN_SRC c
|
||
Vector *vec_with_capacity(Destructor const t_destructor,
|
||
size_t const t_capacity)
|
||
{
|
||
Vector *self = vec_new(t_destructor);
|
||
free(self->elements);
|
||
(*self).elements = (void *)malloc(sizeof(void *) * t_capacity);
|
||
assert(self->elements);
|
||
(*self).capacity = t_capacity;
|
||
return self;
|
||
}
|
||
#+END_SRC
|
||
|
||
*** Adding Data
|
||
The main feature of vectors is to hold data, so let’s make them able
|
||
to take new data from the user. But first, let me explain a bit how
|
||
this dynamic array which I call vector works in C.
|
||
|
||
As you saw earlier, a vector is initialized with a fixed amount of
|
||
memory allocated to the vector, so people can store their data in
|
||
these arrays. Now, imagine you have an array of four elements, and you
|
||
wish to add one more, what to do? You can reallocate your array with
|
||
~realloc~ with one more slot for your element, so now you have an array
|
||
for five elements with your four original elements and a free slot for
|
||
your fifth. Cool, now you can add new elements as you need them!
|
||
|
||
Except that if you want to add some tens of thousands of new elements,
|
||
you would end up calling some tens of thousands times ~realloc~, and
|
||
that is /*slow*/. Seriously, try it, you’ll understand what I mean. And
|
||
all these calls to ~realloc~ are an opportunity for it to fail. Let’s
|
||
limit calls to this function, OK ? If we end up short on slots in our
|
||
current array, let’s actually double the amount of slots in it. So, if
|
||
we have a four-slots array, let’s make it an eight-slots array, and
|
||
then a sixteen-slots array. And in a couple more calls to ~realloc~,
|
||
we’ll quickly reach our tens of thousands slots array, way faster than
|
||
by incrementing its capacity one by one.
|
||
/“But, we’ll end up with a lot of unused memory if we need just one more element than 2^{16} elements! We don’t need a 2^{17} elements array for 2^{16}+1 elements!”/
|
||
|
||
You’re completely right, but that’s a tradeoff. Would you rather have
|
||
a slow but memory-efficient program, or a fast but memory-hungry
|
||
software? Plus, as you’ll see later, there is a function to shrink the
|
||
size of the allocated array down to the actual amount of elements you
|
||
stored in it, making it possible to temporarily have a 2^{17} elements
|
||
array, and immediately after shrink it down to 2^{16}+1, once you know
|
||
you won’t be adding any other elements.
|
||
|
||
With this out of the way, let’s see how to add new elements to our
|
||
vector. First, let’s declare a static function that reallocates the
|
||
memory of a vector. Here is its declaration:
|
||
#+NAME: vector-vec_realloc-def-c
|
||
#+BEGIN_SRC c
|
||
static void vec_realloc(Vector *const self) NONNULL;
|
||
#+END_SRC
|
||
|
||
Its implementation is rather simple: double its capacity, and
|
||
reallocate its array twice its previous size. Of course, there is an
|
||
assertion on whether the arrays have been correctly reallocated to
|
||
ensure memory safety.
|
||
#+NAME: vector-vec_realloc-c
|
||
#+BEGIN_SRC c
|
||
void vec_realloc(Vector *const self)
|
||
{
|
||
self->capacity *= 2;
|
||
self->elements = realloc(self->elements, sizeof(void *) * vec_capacity(self));
|
||
assert(self->elements);
|
||
return;
|
||
}
|
||
#+END_SRC
|
||
|
||
Now, we can proceed to element insertion. Here is the definition of
|
||
~vec_push~, which adds a new element at the end of the vector:
|
||
#+NAME: vector-vec_push-h
|
||
#+BEGIN_SRC c
|
||
void *vec_push(Vector *const self, void *const element) NONNULL;
|
||
#+END_SRC
|
||
|
||
As you can see, it takes as its arguments a pointer to the vector (the
|
||
same returned by its constructor) as well as a pointer to the element
|
||
to be added to the vector. This is an important point: *the vector does
|
||
not store elements themselves, only their pointer*. If the function
|
||
detects there is not enough space for a new element, a call will be
|
||
made to ~vec_realloc~ described above. Once the function is done, it
|
||
will return a pointer to the newly inserted element.
|
||
#+NAME: vector-vec_push-c
|
||
#+BEGIN_SRC c
|
||
void *vec_push(Vector *const self, void *const t_element)
|
||
{
|
||
if (vec_length(self) >= vec_capacity(self)) {
|
||
vec_realloc(self);
|
||
}
|
||
self->elements[(*self).length++] = t_element;
|
||
return vec_last(self);
|
||
}
|
||
#+END_SRC
|
||
|
||
And this is it! There may be a function added later that will allow
|
||
the insertion of a new value in any valid position between the first
|
||
and last position of an array (not counting the unused slots of said
|
||
array), and if I implement this it will imply a reimplementation of
|
||
~vec_push~ so that ~vec_push~ relies on this potential new ~vec_insert~.
|
||
|
||
*** Retrieving Data
|
||
Two functions are available when retrieving data: ~vec_safe_at~ which
|
||
safely retrieves the element at a certain index, and ~vec_at~, which is
|
||
a bit more performant but without the safety of the former. Let’s see
|
||
the definition of both:
|
||
#+NAME: vector-vec_at-h
|
||
#+BEGIN_SRC c
|
||
void *vec_safe_at(Vector const *const self, size_t const index) NONNULL;
|
||
void *vec_at(Vector const *const self, size_t const index) NONNULL;
|
||
#+END_SRC
|
||
|
||
Both have the same arguments: the former is a pointer to the vector we
|
||
want to manipulate, and the latter is the index at which we want to
|
||
retrieve our data. To see the difference in how both work, let’s first
|
||
see the definition of ~vec_at~:
|
||
#+NAME: vector-vec_at-c
|
||
#+BEGIN_SRC c
|
||
void *vec_at(Vector const *const self, size_t const index)
|
||
{
|
||
return self->elements[index];
|
||
}
|
||
#+END_SRC
|
||
~vec_at~ is really straightforward and is just syntax sugar around the
|
||
vector’s ~elements~ member and will behave exactly like the square
|
||
brackets in standard C. However, ~vec_safe_at~ performs some additional
|
||
checks as you can see below:
|
||
#+NAME: vector-vec_safe_at-c
|
||
#+BEGIN_SRC c
|
||
void *vec_safe_at(Vector const *const self, size_t const t_index)
|
||
{
|
||
return (t_index >= vec_length(self)) ? NULL : vec_at(self, t_index);
|
||
}
|
||
#+END_SRC
|
||
|
||
If the requested index is larger than the furthest index possible, a
|
||
~NULL~ pointer will be returned, otherwise the pointer to the requested
|
||
element is. With this function, it is possible to check whether an
|
||
element has been returned or not while avoiding a possible segfault or
|
||
something similar. It could be used in a loop for instance in order to
|
||
check we only have valid elements.
|
||
|
||
It is also possible to retrieve directly the last element with
|
||
~vec_last~. Here is its definition:
|
||
#+NAME: vector-vec_last-h
|
||
#+BEGIN_SRC c
|
||
void *vec_last(Vector const *const self) NONNULL;
|
||
#+END_SRC
|
||
|
||
Just as the previous functions, its declaration is really straightforward:
|
||
#+NAME: vector-vec_last-c
|
||
#+BEGIN_SRC c
|
||
void *vec_last(Vector const *const self)
|
||
{
|
||
return vec_at(self, vec_length(self) - 1);
|
||
}
|
||
#+END_SRC
|
||
|
||
For the sake of the Object-Oriented Programming paradigm, two
|
||
functions were also declared in order to retrieve some data that could
|
||
otherwise be easily accessible:
|
||
#+NAME: vector-vec_length_capacity-h
|
||
#+BEGIN_SRC c
|
||
size_t vec_length(Vector const *const self) NONNULL;
|
||
size_t vec_capacity(Vector const *const self) NONNULL;
|
||
#+END_SRC
|
||
|
||
Their implementation is extremely trivial and doesn’t really need any
|
||
explanation.
|
||
#+NAME: vector-vec_length_capacity-c
|
||
#+BEGIN_SRC c
|
||
size_t vec_length(Vector const *const self)
|
||
{
|
||
return self->length;
|
||
}
|
||
|
||
size_t vec_capacity(Vector const *const self)
|
||
{
|
||
return self->capacity;
|
||
}
|
||
#+END_SRC
|
||
|
||
*** Deleting Data
|
||
While this chapter is about destroying data, this first function will
|
||
not exactly destroy data, or at least not data we care about:
|
||
~vec_shrink_to_fit~ will reallocate the memory in our vector to make it
|
||
so that the member ~elements~ is exactly large enough to store all of
|
||
our data with no more space than that. Here is its definition:
|
||
#+NAME: vector-shrink_to_fit-h
|
||
#+BEGIN_SRC c
|
||
void vec_shrink_to_fit(Vector *const self) NONNULL;
|
||
#+END_SRC
|
||
|
||
There’s nothing too exciting about its implementation: a simple
|
||
reallocation exactly the size of the number of elements currently
|
||
stored times the size of a void pointer, and we verify with an ~assert~
|
||
if it has been correctly reallocated. Nothing is returned.
|
||
#+NAME: vector-shrink_to_fit-c
|
||
#+BEGIN_SRC c
|
||
void vec_shrink_to_fit(Vector *const self)
|
||
{
|
||
if (self->length <= 0) {
|
||
return;
|
||
}
|
||
self->capacity = self->length;
|
||
self->elements = realloc(self->elements, sizeof(void *) * vec_capacity(self));
|
||
assert(self->elements);
|
||
return;
|
||
}
|
||
#+END_SRC
|
||
|
||
Notice that a check is done to see if the vector exists, because
|
||
otherwise calling ~shrink_to_fit~ on an empty vector would result in an
|
||
error while asserting the reallocation.
|
||
|
||
Next, we have two functions: ~vec_pop_at~ and ~vec_pop~. The latter relies
|
||
on the former, which can delete an element at any valid position.
|
||
*Beware*: these functions return /nothing/ and simply deletes the element.
|
||
Here is their definition:
|
||
#+NAME: vector-vec_pop-h
|
||
#+BEGIN_SRC c
|
||
void vec_pop_at(Vector *const self, size_t const index) NONNULL;
|
||
void vec_pop(Vector *const self) NONNULL;
|
||
#+END_SRC
|
||
|
||
In order to insure memory safety, a static function is declared in
|
||
~src/vector.c~ which will delete an element if a destructor has been
|
||
provided to the vector when it has been built. Its definition is the
|
||
following:
|
||
#+NAME: vector-vec_maybe_delete_element-def-c
|
||
#+BEGIN_SRC c
|
||
static void vec_maybe_delete_element(Vector const *self,
|
||
size_t const t_index) NONNULL;
|
||
#+END_SRC
|
||
|
||
Its implementation is quite simple: if a destructor exists, then the
|
||
element at the requested index will be destroyed through this
|
||
destructor. Otherwise, nothing is done with the destructor, hence the
|
||
name of the function ~vec_maybe_delete_element~. However, it should be
|
||
noted that the element will be freed from memory, so if the user needs
|
||
it before popping it, they need to retrieve it with something like
|
||
~vec_at~ and store it elsewhere.
|
||
#+NAME: vector-vec_maybe_delete_element-c
|
||
#+BEGIN_SRC c
|
||
void vec_maybe_delete_element(Vector const *self, size_t const t_index)
|
||
{
|
||
void *element = vec_at(self, t_index);
|
||
if (self->destroy) {
|
||
self->destroy(element);
|
||
}
|
||
free(element);
|
||
}
|
||
#+END_SRC
|
||
|
||
Now that we have this function sorted out, we can implement our pops.
|
||
Here is the implementation of ~vec_pop_at~:
|
||
#+NAME: vector-vec_pop_at-c
|
||
#+BEGIN_SRC c
|
||
void vec_pop_at(Vector *const t_self, size_t const t_index)
|
||
{
|
||
if (vec_safe_at(t_self, t_index) == NULL) {
|
||
return;
|
||
}
|
||
vec_maybe_delete_element(t_self, t_index);
|
||
if (t_index + 1 < vec_length(t_self)) {
|
||
memcpy(vec_at(t_self, t_index), vec_at(t_self, t_index + 1),
|
||
sizeof(void *) * (t_self->length - (t_index + 1)));
|
||
}
|
||
--(*t_self).length;
|
||
}
|
||
#+END_SRC
|
||
|
||
A check is performed at the beginning of the function: that the
|
||
element we want to pop actually exists. If it does not, the function
|
||
does nothing, otherwise the function deletes the element if needed.
|
||
The call to ~vec_maybe_delete_element~ will free the requested element.
|
||
Then, a check is performed to see if the requested element was at the
|
||
end of the array or not. If it was not, then the elements located
|
||
after the destroyed element are shifted one element closer to the
|
||
beginning of the array; otherwise, if the requested element was at the
|
||
end of the array, nothing is done particularly. Lastly, the count of
|
||
elements stored in the vector is decreased by one.
|
||
~vec_pop~ uses the above function in order to provide a simpler call if
|
||
we want to delete the last element of the array. We can see how it
|
||
relies on ~vec_pop_at~ in its implementation:
|
||
#+NAME: vector-vec_pop-c
|
||
#+BEGIN_SRC c
|
||
void vec_pop(Vector *const self)
|
||
{
|
||
vec_pop_at(self, vec_length(self));
|
||
}
|
||
#+END_SRC
|
||
|
||
Finally, ~vec_delete~ allows for the complete destruction and
|
||
deallocation of a vector, including all of its elements. Here is its
|
||
definition:
|
||
#+NAME: vector-vec_delete-h
|
||
#+BEGIN_SRC c
|
||
void vec_delete(Vector *const self) NONNULL;
|
||
#+END_SRC
|
||
|
||
In its implementation, we can see three distinct steps:
|
||
- The deletion of all its elements if a destructor exists
|
||
- The deletion of the array of the vector
|
||
- The deletion of the vector itself.
|
||
#+NAME: vector-vec_delete-c
|
||
#+BEGIN_SRC c
|
||
void vec_delete(Vector *const self)
|
||
{
|
||
if (self->destroy) {
|
||
for (size_t i = 0; i < vec_length(self); ++i) {
|
||
self->destroy(self->elements[i]);
|
||
}
|
||
}
|
||
free(self->elements);
|
||
free(self);
|
||
}
|
||
#+END_SRC
|
||
|
||
*** The Final Source Code
|
||
Finally, we can see the whole source code. Here is the header for the
|
||
library: ~vector.h~
|
||
#+BEGIN_SRC c :noweb yes
|
||
#ifndef VECTOR_H_
|
||
#define VECTOR_H_
|
||
|
||
<<vector-nonnull-h>>
|
||
|
||
<<vector-struct-def>>
|
||
|
||
<<vector-vec_new-h>>
|
||
<<vector-vec_with_capacity-h>>
|
||
<<vector-vec_push-h>>
|
||
<<vector-vec_at-h>>
|
||
<<vector-vec_last-h>>
|
||
<<vector-vec_length_capacity-h>>
|
||
<<vector-shrink_to_fit-h>>
|
||
<<vector-vec_pop-h>>
|
||
<<vector-vec_delete-h>>
|
||
|
||
#endif /* VECTOR_H_ */
|
||
#+END_SRC
|
||
|
||
And here is the implementation file: ~vector.c~
|
||
#+BEGIN_SRC c :noweb yes
|
||
#include "vector.h"
|
||
|
||
<<vector-includes-c>>
|
||
|
||
<<vector-vec_realloc-def-c>>
|
||
<<vector-vec_maybe_delete_element-def-c>>
|
||
|
||
<<vector-vec_new-c>>
|
||
|
||
<<vector-vec_with_capacity-c>>
|
||
|
||
<<vector-vec_realloc-c>>
|
||
|
||
<<vector-vec_push-c>>
|
||
|
||
<<vector-vec_at-c>>
|
||
|
||
<<vector-vec_safe_at-c>>
|
||
|
||
<<vector-vec_last-c>>
|
||
|
||
<<vector-vec_length_capacity-c>>
|
||
|
||
<<vector-shrink_to_fit-c>>
|
||
|
||
<<vector-vec_pop-c>>
|
||
|
||
<<vector-vec_maybe_delete_element-c>>
|
||
|
||
<<vector-vec_pop_at-c>>
|
||
|
||
<<vector-vec_pop-c>>
|
||
|
||
<<vector-vec_delete-c>>
|
||
#+END_SRC
|
||
|
||
And with that, we should be good! I used this library in a SOM
|
||
(Kohonen, 1982) implementation and ran it through valgrind, and there
|
||
were no memory leaks. If you find one though, don’t hesitate telling
|
||
me in the comments, through social media such as Twitter, or by email.
|
||
|
||
Happy programming!
|
||
|
||
* Emacs :@emacs:
|
||
** Emacs 29 is nigh! What can we expect? :dev:emacs:
|
||
:PROPERTIES:
|
||
:EXPORT_FILE_NAME: emacs-29-what-can-we-expect
|
||
:EXPORT_DATE: 2022-11-29
|
||
:EXPORT_OPTIONS: toc:2
|
||
:export_hugo_menu: :menu "main"
|
||
:END:
|
||
It [[https://lists.gnu.org/archive/html/emacs-devel/2022-11/msg01774.html][was announced a couple of hours ago]], Emacs 29’s branch is now cut
|
||
from the master branch! This means the ~emacs-29~ branch will from now
|
||
no longer receive any new feature, but only bug fixes.
|
||
|
||
So, what’s new with this new major release? I skimmed over the ~NEWS~
|
||
file, and here are the changes which I find interesting and even
|
||
exciting for some.
|
||
*Article updated on December 22nd at 14:05 UTC*
|
||
|
||
*** Major features
|
||
A couple of major improvements will be most likely present, here are
|
||
the ones that stand out the most for me.
|
||
|
||
**** Eglot is now part of Emacs core
|
||
During the last couple of years, LSP has given text editors incredible
|
||
capabilities, giving them IDE-like features relatively easily. Aside
|
||
from Elisp development, most of the code I write is now done with the
|
||
help of an LSP server, running along Emacs and analysing my code,
|
||
suggesting and performing changes and actions for me.
|
||
|
||
Several integrations of LSP exist for Emacs, such as [[https://emacs-lsp.github.io/lsp-mode/][LSP Mode]], [[https://github.com/joaotavora/eglot][Eglot]],
|
||
and [[https://github.com/manateelazycat/lsp-bridge][lsp-bridge]]. Among the three, Eglot is now part of Emacs core! No
|
||
longer do you need to install a package, simply register an LSP server
|
||
and autocompletion, documentation, error detection, and other features
|
||
will become available right away!
|
||
|
||
I must admit I don’t really know Eglot, I personally use LSP Mode, but
|
||
with this addition to Emacs core, I might attempt the switch.
|
||
|
||
**** Tree-Sitter is also part of Emacs core
|
||
In case you didn’t know, Emacs’ current syntax highlighting is
|
||
currently based on a system of regexes. Although it is not the /worst/
|
||
thing to use, it’s not the best either, and it can become quite slow
|
||
on larger files.
|
||
|
||
Tree-Sitter parses programming languages based into a concrete syntax
|
||
tree. From there, not only can syntax highlighting can be done at high
|
||
speed, but a much deeper analysis of the code is possible and actions
|
||
such as syntax manipulation can also be achieved since the syntax tree
|
||
itself is available as an object which can be manipulated!
|
||
|
||
In case you want some more information on Tree-Sitter itself, you can
|
||
check out the [[https://tree-sitter.github.io/tree-sitter/][official Tree-Sitter website]], or you can even check this
|
||
talk out given by Tree-Sitter’s creator, Max Brunsfeld.
|
||
|
||
#+begin_export html
|
||
<iframe width="560" height="315" src="https://www.youtube-nocookie.com/embed/Jes3bD6P0To" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
|
||
#+end_export
|
||
|
||
Well, this is now a native solution in Emacs! Currently, Emacs’
|
||
Tree-Sitter supports the current major modes :
|
||
- ~bash-ts-mode~
|
||
- ~c-ts-mode~
|
||
- ~c++-ts-mode~
|
||
- ~csharp-ts-mode~
|
||
- ~css-ts-mode~
|
||
- ~java-ts-mode~
|
||
- ~js-ts-mode~
|
||
- ~json-ts-mode~
|
||
- ~python-ts-mode~
|
||
- ~typescript-ts-mode~
|
||
|
||
Tree-Sitter also holds for now a special status in the new ~emacs-29~
|
||
branch since new features can still be added to it, as its merge with
|
||
the master branch is still recent. So we might see the list of major
|
||
modes for Emacs get a bit longer yet, especially considering
|
||
Tree-Sitter tries to make adding new languages relatively easy.
|
||
|
||
If you can’t wait to test Tree-Sitter, there is already [[https://emacs-tree-sitter.github.io/][another package]]
|
||
available for Emacs you can use right now. Just be aware this is not
|
||
the same package as the one that got integrated into Emacs.
|
||
|
||
**** Install packages from source with ~package.el~
|
||
If you use [[https://github.com/radian-software/straight.el][Straight]], you might be familiar with installing packages
|
||
directly from their Git repository. Well, good news, it is now
|
||
possible to install packages from Git using Emacs’ built-in packaging
|
||
system ~package.el~! It can be done with the new function
|
||
~package-vc-install~, and packages installed that way can be updated
|
||
with ~package-vc-update~ or ~package-vc-update-all~.
|
||
|
||
On the topic of ~package.el~, there is also the new function
|
||
~package-report-bug~ which allows Emacs users to report bugs to the
|
||
developers of a package directly from Emacs! Be aware though, it only
|
||
works for packages installed through ~package.el~. Since I’m a
|
||
[[https://github.com/jwiegley/use-package][~use-package~]] and ~straight.el~ user, there is no package listed when I
|
||
invoke the command.
|
||
|
||
**** Org mode 9.6
|
||
|
||
As confirmed by one of org-mode maintainers [[https://bzg.fr/][Bastien Guerry]] on [[https://lists.sr.ht/~bzg/emacsfr/%3C87bkophdzo.fsf%40phundrak.com%3E#%3C87tu2hqf4h.fsf@gnu.org%3E][a
|
||
French-speaking Emacs mailing list]], Org 9.6 is set to be part of
|
||
Emacs 29! There is an [[https://orgmode.org/Changes.html][official article]] on this release, which is
|
||
[[https://elpa.gnu.org/packages/org.html][already available on GNU ELPA]]!
|
||
|
||
**** use-package in Emacs core
|
||
It has also been confirmed on the [[https://lists.gnu.org/archive/html/emacs-devel/2022-11/msg01821.html][Emacs development mailing list]] that
|
||
[[https://github.com/jwiegley/use-package][~use-package~]], an awesome package manager, is set to be part of Emacs
|
||
29, although it initially wasn’t included in the ~emacs-29~ branch.
|
||
|
||
**** Pure GTK Emacs is here for Wayland!
|
||
One of the major issues Emacs had on Linux was its dependency on Xorg
|
||
when running in GUI mode. When running Xorg, it’s not really an issue,
|
||
but Wayland has become more and more common during the last years, and
|
||
even with the existence of XWayland, this became an annoyance.
|
||
|
||
Well, fear not, for pure GTK Emacs is here! It can now be built
|
||
Xorg-free and run natively in Wayland!
|
||
|
||
Be aware though that Wayland is basically the only use-case for pure
|
||
GTK Emacs. If you don’t use Wayland, Emacs will display a warning
|
||
message, as it will most likely cause issues if you are running Xorg.
|
||
In my case, I sometimes see some ghost text when the content of a
|
||
buffer updates (I still need pure GTK though, since I alternate
|
||
between Xorg and Wayland).
|
||
|
||
**** Compile EmacsLisp files ahead of time
|
||
With Emacs 28 came the ability to natively compile EmacsLisp if your
|
||
Emacs was built with the ability to do so, using GCC’s Just In Time
|
||
library. This results in quite the impressive boost in performance,
|
||
which made Emacs much snappier than it was before. The only issue I
|
||
had was Emacs would only compile its EmacsLisp files when they were
|
||
loaded for the first time.
|
||
|
||
This is no longer the case! If you now compile Emacs with
|
||
~--with-native-compilation=aot~, Emacs’ native EmacsLisp files will be
|
||
natively compiled along with Emacs itself! Be aware though, it can be
|
||
slow on most machines, so the time you save by not compiling these
|
||
files when launching Emacs for the first time is basically transferred
|
||
to when compiling Emacs itself. Is it worth your time? In my case, I
|
||
would say yes, because when I compile Emacs, I’m generally not in a
|
||
hurry. But in your case? Well, test it out and see for yourself.
|
||
|
||
**** Native access to SQLite databases
|
||
Emacs can now be built with native support for SQLite and the sqlite3
|
||
library. In fact, this is now a default behaviour, since you need to
|
||
pass ~--without-sqlite3~ to Emacs’ build configuration script in order
|
||
to prevent it.
|
||
|
||
This comes with a new ~sqlite-mode~ which allows you to explore SQLite
|
||
databases within Emacs and to interact with them. Check out the
|
||
~sqlite-mode-open-file~ function!
|
||
|
||
**** HaikuOS support
|
||
For all three HaikuOS users out there, good news, you now have access
|
||
to Emacs! (In all seriousness, I should check out HaikuOS one day)
|
||
|
||
Moreover, it also supports an optional window-system port to Haiku
|
||
with ~--with-be-app~. Be aware, you will need the Haiku Application Kit
|
||
development headers and a C++ compiler. Otherwise, Emacs will only run
|
||
in the terminal. If you want to also add Cairo to the mix, you can add
|
||
~--with-be-cairo~.
|
||
|
||
**** New major mode for C#
|
||
~csharp-mode~ is now a native major mode for Emacs and is based on ~cc-mode~.
|
||
|
||
*** Minor features
|
||
**** It’s easier to use Emacs in scripts!
|
||
If you like to write scripts and especially writing Lisp scripts,
|
||
Emacs now supports the option ~-x~ in order to execute scripts written
|
||
in EmacsLisp. When executing such a script with ~#!/usr/bin/emacs -x~ as
|
||
its shebang, Emacs will not read its init file (like with ~-Q~) and will
|
||
instead execute the Elisp code right away and return the last value to
|
||
the caller of the script (most likely the shell you called the script
|
||
from).
|
||
|
||
**** TRAMP natively supports Docker, Podman, and Kubernetes
|
||
Three new connections are now available for TRAMP:
|
||
- ~docker~
|
||
- ~podman~
|
||
- ~kubernetes~
|
||
You will now be able to access your containerized environment right
|
||
from Emacs without the need to write custom code.
|
||
|
||
**** Custom user directory
|
||
It is now easier to launch custom Emacs profiles without the need of
|
||
tools such as [[https://github.com/plexus/chemacs2][chemacs2]] with the addition of the flag ~--init-directory~.
|
||
This can set to any directory Emacs’ ~user-emacs-directory~ which
|
||
includes the ~init.el~ which comes along with it. Yet another reason for
|
||
me not to use a ~.emacs~ file, but the ~init.el~ file instead.
|
||
|
||
**** Support for Webp images
|
||
For quite some time, Emacs has been able to display images, but not
|
||
webp yet. Well, this is now fixed! And in fact, support for webp
|
||
images became the default behaviour, since you need to pass
|
||
~--without-webp~ to Emacs’ configuration script to disable webp support.
|
||
|
||
**** C++ mode now supports the C++20 standard
|
||
Yep. There’s nothing more to say, really. Happy coding!
|
||
|
||
**** Better handling of ~.pdmp~ files
|
||
Emacs has had for a few version the ability to dump its state into a
|
||
~pdmp~ file for faster startup time. Well now, when creating such a
|
||
file, it will include in its name a fingerprint of its current state,
|
||
although it will still prioritize an ~emacs.pdmp~ file if it exists.
|
||
|
||
**** Better mouse and touchpad support
|
||
Emacs now uses XInput 2, which enables Emacs to support more input
|
||
events, such as touchpad events. For instance, by default, a pinch
|
||
gesture on a touchpad increases or decreases the text size of the
|
||
current buffer. This is thanks to the new event ~pinch~, which comes
|
||
along with ~touch-end~.
|
||
|
||
**** Unicode 15.0 and emojis
|
||
Emacs now supports [[https://www.unicode.org/versions/Unicode15.0.0/][Unicode 15.0]], which is currently the latest Unicode
|
||
version. Although this is not directly related, quite a few new
|
||
emoji-related features have been introduced. The new prefix ~C-x 8 e~
|
||
now leads to a few new commands related to emojis:
|
||
- ~C-x 8 e e~ or ~C-x 8 e i~ :: Insert an emoji (~emoji-insert~)
|
||
- ~C-x 8 e s~ :: Search an emoji (~emoji-search~)
|
||
- ~C-x 8 e l~ :: List all emojis in a new buffer (~emoji-list~)
|
||
- ~C-x 8 e r~ :: Insert a recently inserted emoji (~emoji-recent~)
|
||
- ~C-x 8 e d~ :: Describe an emoji (~emoji-describe~)
|
||
- ~C-x 8 e +~ and ~C-x 8 e -~ :: Increase and decrease the size of any
|
||
character, but especially emojis (~emoji-zoom-increase~ and ~emoji-zoom-decrease~ respectively)
|
||
|
||
There is also the new input method ~emoji~ which allows you to type for
|
||
instance ~:grin:~ in order to get the emoji 😁.
|
||
|
||
**** True background transparency
|
||
Up until recently, if you wanted transparency with Emacs, you had no
|
||
choice but to make the whole frame transparent, including text and
|
||
images.
|
||
|
||
Thanks to the frame parameter ~alpha-background~ and its related
|
||
~alphaBackground~ X resource, it is now possible to set transparency
|
||
only for the frame’s background without affecting any of the other
|
||
elements on screen.
|
||
|
||
**** WebKit inspector in Emacs’ WebKit widget browser
|
||
You can now access the WebKit inspector when using the WebKit widget
|
||
browser in Emacs, given you are using a version of Emacs which has
|
||
been compiled with it. I wish there was a keybinding or at least a
|
||
function for it, but apparently you can only open it with a right
|
||
click and select /Inspect Element/. Still nice to have.
|
||
|
||
**** Some news for Windows
|
||
Although it has been available for Linux users since Emacs 26.1,
|
||
Windows finally has access to double-buffering to reduce display
|
||
flicker. If you wish to disable it, you can set the frame parameter
|
||
~inhibit-double-buffering~ to ~nil~.
|
||
|
||
Emacs also follows Windows’ dark mode with Windows 10 (version 1809)
|
||
and onwards.
|
||
|
||
Emacs also now uses Windows’ native API to render images. This
|
||
includes BMP, GIF, JPEG, PNG, and TIFF images. Other formats, however,
|
||
still rely on other dependencies and libraries to properly work, such
|
||
as Webp images.
|
||
|
||
*** What’s next?
|
||
With Emacs 29 being cut, development on the master branch will now go
|
||
towards Emacs 30. Is there anything we can expect yet?
|
||
|
||
It’s still very early to say, most stable features merged into master
|
||
went to Emacs 29, and only the ~feature/pkg~ and
|
||
~feature/improved-lock-narrowing~ branches seem to have received commits
|
||
less than a week prior to the day of writing this, and I do not know
|
||
the status of other branches that received commits during the past few
|
||
weeks such as ~feature/package+vc~ or ~feature/eglot2emacs~ (which I
|
||
assume both got merged).
|
||
+However, there are currently talks about including ~use-package~ into Emacs! I’m a bit disappointed it won’t make it into Emacs 29, but progress is being made on ~scratch/use-package~, and you can always check the mailing list to check its status such as [[https://lists.gnu.org/archive/html/emacs-devel/2022-11/msg01533.html][here]].+ *Update*: Rejoice! As mentioned above, ~use-package~ is actually set to land in Emacs 29!
|
||
|
||
** [EN] Automatic Meaningful Custom IDs for Org Headings :emacs:orgmode:dev:
|
||
:PROPERTIES:
|
||
:EXPORT_FILE_NAME: better-custom-ids-orgmode
|
||
:EXPORT_DATE: 2020-06-06
|
||
:export_hugo_menu: :menu "main"
|
||
:END:
|
||
Spoiler alert, I will just modify a bit of code that already exists,
|
||
go directly to the bottom if you want the solution, or read the whole
|
||
post if you are interested in how I got there.
|
||
|
||
#+TOC: headlines 1 local
|
||
|
||
**** Update 2021-11-22
|
||
I’ve put the code presented here as a complete package. You can find
|
||
it in [[https://labs.phundrak.com/phundrak/org-unique-id][this repository]] or in its [[https://github.com/Phundrak/org-unique-id][GitHub mirror]] (be aware the latter may
|
||
not be as up-to-date as the former is. Installation instructions are
|
||
in the README.
|
||
|
||
*** The issue
|
||
About two to three years ago, as I was working on a project that was
|
||
meant to be published on the internet, I looked for a solution to get
|
||
fixed anchor links to my various headings when I performed HTML
|
||
exports. As some of you may know, by default when an Org file is
|
||
exported to an HTML file, a random ID will be generated for each
|
||
header, and this ID will be used as their anchor. Here’s a quick
|
||
example of a simple org file:
|
||
|
||
#+caption: Example org file
|
||
#+begin_src org :exports code
|
||
,#+title: Sample org file
|
||
,* First heading
|
||
Reference to a subheading
|
||
,* Second heading
|
||
Some stuff written here
|
||
,** First subheading
|
||
Some stuff
|
||
,** Second subheading
|
||
Some other stuff
|
||
#+end_src
|
||
|
||
And this is the result once exported to HTML (with a lot of noise
|
||
removed from ~<head>~):
|
||
|
||
#+caption: Output HTML file
|
||
#+BEGIN_SRC html
|
||
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
|
||
|
||
<head>
|
||
<title>Sample org file</title>
|
||
<meta name="generator" content="Org mode" />
|
||
<meta name="author" content="Lucien Cartier-Tilet" />
|
||
</head>
|
||
|
||
<body>
|
||
<div id="content">
|
||
<h1 class="title">Sample org file</h1>
|
||
<div id="outline-container-orgd8e6238" class="outline-2">
|
||
<h2 id="orgd8e6238"><span class="section-number-2">1</span> First heading</h2>
|
||
<div class="outline-text-2" id="text-1">
|
||
<p>
|
||
Reference to a subheading
|
||
</p>
|
||
</div>
|
||
</div>
|
||
<div id="outline-container-org621c39a" class="outline-2">
|
||
<h2 id="org621c39a"><span class="section-number-2">2</span> Second heading</h2>
|
||
<div class="outline-text-2" id="text-2">
|
||
<p>
|
||
Some stuff written here
|
||
</p>
|
||
</div>
|
||
<div id="outline-container-orgae45d6b" class="outline-3">
|
||
<h3 id="orgae45d6b"><span class="section-number-3">2.1</span> First subheading</h3>
|
||
<div class="outline-text-3" id="text-2-1">
|
||
<p>
|
||
Some stuff
|
||
</p>
|
||
</div>
|
||
</div>
|
||
<div id="outline-container-org9301aa9" class="outline-3">
|
||
<h3 id="org9301aa9"><span class="section-number-3">2.2</span> Second subheading</h3>
|
||
<div class="outline-text-3" id="text-2-2">
|
||
<p>
|
||
Some other stuff
|
||
</p>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</body>
|
||
|
||
</html>
|
||
#+END_SRC
|
||
|
||
As you can see, all the anchors are in the format of ~org[a-f0-9]{7}~.
|
||
First, this is not really meaningful if you want to read the anchor
|
||
and guess where it will lead you. But secondly, these anchors will
|
||
change each time you export your Org file to HTML. If I want to share
|
||
a URL to my website and to a specific heading, … well I can’t, it will
|
||
change the next time I update the document. And I don’t want to have
|
||
to set a ~CUSTOM_ID~ property for each one of my headings manually. So,
|
||
what to do?
|
||
|
||
*** A first solution
|
||
A first solution I found came from [[https://writequit.org/articles/emacs-org-mode-generate-ids.html][this blog post]], where Lee Hinman
|
||
described the very same issue they had and wrote some Elisp code to
|
||
remedy that (it’s a great read, go take a look). And it worked, and
|
||
for some time I used their code in my Emacs configuration file in
|
||
order to generate unique custom IDs for my Org headers. Basically what
|
||
the code does is it detects if ~auto-id:t~ is set in an ~#+OPTIONS~
|
||
header. If it is, then it will iterate over all the Org headers, and
|
||
for each one of them it will insert a ~CUSTOM_ID~, which is made from a
|
||
UUID generated by Emacs. And tadah! we get for each header a
|
||
~h-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}~ custom
|
||
ID that won’t change next time we export our Org file to HTML when we
|
||
save our file, and only for headings which don’t already have a
|
||
~CUSTOM_ID~ property. Wohoo!
|
||
|
||
Except…
|
||
|
||
*** These headers are not meaningful
|
||
OK, alright, that’s still a huge step forward, we don’t have to type
|
||
any ~CUSTOM_ID~ property manually any more, it’s done automatically for
|
||
us. But, when I send someone a link like
|
||
~https://langue.phundrak.com/eittland#h-76fc0b91-e41c-42ad-8652-bba029632333~,
|
||
the first reaction to this URL is often something along the lines of
|
||
“What the fuck?”. And they’re right, this URL is unreadable when it
|
||
comes to the anchor. How am I supposed to guess it links to the
|
||
description of the vowels of the Eittlandic language? (That’s a
|
||
constructed language I’m working on, you won’t find anything about it
|
||
outside my website. Also, this link is dead now, it got simplified
|
||
thanks to Vuepres.)
|
||
|
||
So, I went back to my configuration file for Emacs, and through some
|
||
trial and error, I finally found a way to get a consistent custom ID
|
||
which is readable and automatically set. With the current state of my
|
||
code, what you get is the complete path of the Org heading, all spaces
|
||
replaced by underscores and headings separated by dashes, with a final
|
||
unique identifier taken from an Emacs-generated UUID. Now, the same
|
||
link as above will look like
|
||
~https://langue.phundrak.com/eittland#Aperçu_structurel-Inventaire_phonétique_et_orthographe-Voyelles_pures-84f05c2c~.
|
||
It won’t be more readable to you if you don’t speak French, but you
|
||
can guess it is way better than what we had before. I even added a
|
||
safety net by replacing all forward slashes with dashes. The last ID
|
||
is here to ensure the path will be unique in case we’d have two
|
||
identical paths in the org file for one reason or another.
|
||
|
||
The modifications I made to the first function ~eos/org-id-new~ are
|
||
minimal, where I just split the UUID and get its first part. This is
|
||
basically a way to simplify it.
|
||
#+BEGIN_SRC emacs-lisp
|
||
(defun eos/org-id-new (&optional prefix)
|
||
"Create a new globally unique ID.
|
||
|
||
An ID consists of two parts separated by a colon:
|
||
- a prefix
|
||
- a unique part that will be created according to
|
||
`org-id-method'.
|
||
|
||
PREFIX can specify the prefix, the default is given by the
|
||
variable `org-id-prefix'. However, if PREFIX is the symbol
|
||
`none', don't use any prefix even if `org-id-prefix' specifies
|
||
one.
|
||
|
||
So a typical ID could look like \"Org-4nd91V40HI\"."
|
||
(let* ((prefix (if (eq prefix 'none)
|
||
""
|
||
(concat (or prefix org-id-prefix)
|
||
"-"))) unique)
|
||
(if (equal prefix "-")
|
||
(setq prefix ""))
|
||
(cond
|
||
((memq org-id-method
|
||
'(uuidgen uuid))
|
||
(setq unique (org-trim (shell-command-to-string org-id-uuid-program)))
|
||
(unless (org-uuidgen-p unique)
|
||
(setq unique (org-id-uuid))))
|
||
((eq org-id-method 'org)
|
||
(let* ((etime (org-reverse-string (org-id-time-to-b36)))
|
||
(postfix (if org-id-include-domain
|
||
(progn
|
||
(require 'message)
|
||
(concat "@"
|
||
(message-make-fqdn))))))
|
||
(setq unique (concat etime postfix))))
|
||
(t (error "Invalid `org-id-method'")))
|
||
(concat prefix (car (split-string unique "-")))))
|
||
#+END_SRC
|
||
|
||
Next, we have here the actual generation of the custom ID. As you can
|
||
see, the ~let~ has been replaced by a ~let*~ which allowed me to create
|
||
the ID with the variables ~orgpath~ and ~heading~. The former concatenates
|
||
the path to the heading joined by dashes, and ~heading~ concatenates
|
||
~orgpath~ to the name of the current heading joined by a dash if ~orgpath~
|
||
is not empty. It will then create a slug out of the result, deleting
|
||
some elements such as forward slashes or tildes, and all whitespace is
|
||
replaced by underscores. It then passes ~heading~ as an argument to the
|
||
function described above to which the unique ID will be concatenated.
|
||
#+BEGIN_SRC emacs-lisp
|
||
(defun eos/org-custom-id-get (&optional pom create prefix)
|
||
"Get the CUSTOM_ID property of the entry at point-or-marker POM.
|
||
|
||
If POM is nil, refer to the entry at point. If the entry does not
|
||
have an CUSTOM_ID, the function returns nil. However, when CREATE
|
||
is non nil, create a CUSTOM_ID if none is present already. PREFIX
|
||
will be passed through to `eos/org-id-new'. In any case, the
|
||
CUSTOM_ID of the entry is returned."
|
||
(interactive)
|
||
(org-with-point-at pom
|
||
(let* ((orgpath (mapconcat #'identity (org-get-outline-path) "-"))
|
||
(heading (replace-regexp-in-string
|
||
"/\\|~\\|\\[\\|\\]" ""
|
||
(replace-regexp-in-string
|
||
"[[:space:]]+" "_" (if (string= orgpath "")
|
||
(org-get-heading t t t t)
|
||
(concat orgpath "-" (org-get-heading t t t t))))))
|
||
(id (org-entry-get nil "CUSTOM_ID")))
|
||
(cond
|
||
((and id
|
||
(stringp id)
|
||
(string-match "\\S-" id)) id)
|
||
(create (setq id (eos/org-id-new (concat prefix heading)))
|
||
(org-entry-put pom "CUSTOM_ID" id)
|
||
(org-id-add-location id
|
||
(buffer-file-name (buffer-base-buffer)))
|
||
id)))))
|
||
#+END_SRC
|
||
|
||
The rest of the code is unchanged, here it is anyway:
|
||
#+BEGIN_SRC emacs-lisp
|
||
(defun eos/org-add-ids-to-headlines-in-file ()
|
||
"Add CUSTOM_ID properties to all headlines in the current file
|
||
which do not already have one.
|
||
|
||
Only adds ids if the `auto-id' option is set to `t' in the file
|
||
somewhere. ie, #+OPTIONS: auto-id:t"
|
||
(interactive)
|
||
(save-excursion
|
||
(widen)
|
||
(goto-char (point-min))
|
||
(when (re-search-forward "^#\\+OPTIONS:.*auto-id:t"
|
||
(point-max)
|
||
t)
|
||
(org-map-entries (lambda ()
|
||
(eos/org-custom-id-get (point)
|
||
'create))))))
|
||
|
||
(add-hook 'org-mode-hook
|
||
(lambda ()
|
||
(add-hook 'before-save-hook
|
||
(lambda ()
|
||
(when (and (eq major-mode 'org-mode)
|
||
(eq buffer-read-only nil))
|
||
(eos/org-add-ids-to-headlines-in-file))))))
|
||
#+END_SRC
|
||
|
||
Note that you *will need* the package ~org-id~ to make this code work. You
|
||
simply need to add the following code before the code I shared above:
|
||
#+BEGIN_SRC emacs-lisp
|
||
(require 'org-id)
|
||
(setq org-id-link-to-org-use-id 'create-if-interactive-and-no-custom-id)
|
||
#+END_SRC
|
||
|
||
And that’s how my links are now way more readable *and* persistent! The
|
||
only downside I found to this is when you move headings and their path
|
||
is modified, or when you modify the heading itself, the custom ID is
|
||
not automatically updated. I could fix that by regenerating the custom
|
||
ID on each save, regardless of whether a custom ID already exists or
|
||
not, but it’s at the risk an ID manually set will get overwritten.
|
||
|
||
* Linux :@linux:
|
||
** [EN] My YouTube subscriptions as an RSS feed :linux:dev:tutorial:
|
||
:PROPERTIES:
|
||
:EXPORT_FILE_NAME: youtube-subscriptions-rss
|
||
:EXPORT_DATE: 2022-02-04
|
||
:export_hugo_menu: :menu "main"
|
||
:END:
|
||
*** The Problem
|
||
I’m sure you’ve been in the same situation before: you go on YouTube
|
||
because you want to watch a video, maybe two, from your subscriptions.
|
||
You open the first one. Oh great, an unskippable fifteen seconds ad.
|
||
And another one! OK, the video starts. It gets cut a couple of times
|
||
by other ads of varying length. Oh but what’s this? This recommended
|
||
video looks nice! And before you know it, your whole afternoon and
|
||
evening went by painfully watching videos on YouTube’s atrocious video
|
||
player. You lost focus.
|
||
|
||
*** My Solution: mpv + RSS
|
||
Wouldn’t it be nice if it were possible to watch these videos with a
|
||
full-fledged video player over which you have complete control? Which
|
||
could be customized to your heart’s content? Which won’t secretly
|
||
track what you watch?
|
||
|
||
Oh right, [[https://mpv.io/][mpv]]! It supports most video formats you can think of, and
|
||
thanks to its interoperability with [[https://github.com/ytdl-org/youtube-dl][youtube-dl]], you can also watch
|
||
videos from [[https://ytdl-org.github.io/youtube-dl/supportedsites.html][an extremely wide variety of websites]]! So why not YouTube?
|
||
|
||
Now, the question is how to get rid of YouTube’s interface. The answer
|
||
is actually quite simple: let’s use an RSS feed. With the RSS feeds
|
||
from YouTube, you will receive in your RSS reader the link of the
|
||
video with its thumbnail and its description. You can then copy from
|
||
there the link and open it with mpv with a command like this:
|
||
#+begin_src bash
|
||
mpv "https://www.youtube.com/watch?v=xym2R6_Qd7c"
|
||
#+end_src
|
||
|
||
**** Channel RSS
|
||
Now the question is how to get the RSS feed of a channel? The answer
|
||
is quite simple. The base URL for a YouTube channel RSS feed is
|
||
~https://www.youtube.com/feeds/videos.xml?channel_id=~ to which you
|
||
simply have to add the channel ID. For instance, if you want to follow
|
||
Tom Scott with this, you simply have to extract the part of the
|
||
channel after ~/channel/~ in his URL and append it to the URL mentioned
|
||
above, and TADAH! you get an RSS feed to his channel!
|
||
#+begin_src text
|
||
https://www.youtube.com/feeds/videos.xml?channel_id=UCBa659QWEk1AI4Tg--mrJ2A
|
||
#+end_src
|
||
|
||
Be careful to select the channel ID only if it is after a ~/channel/~
|
||
though! The part that is after a ~/c/~ will not work. If you end up on
|
||
the URL ~https://www.youtube.com/c/TomScottGo~, simply click on a random
|
||
video, then click on the channel’s name. This should bring you back to
|
||
the channel but with an important difference: the URL is now
|
||
~https://www.youtube.com/channel/UCBa659QWEk1AI4Tg--mrJ2A~.
|
||
|
||
The thing that is really nice with this setup is you don’t really need
|
||
to actually subscribe to a channel, your RSS feed already does that
|
||
for you! And with lots of RSS feed readers, you can categorize your
|
||
different feeds, meaning you can even categorize your subscriptions!
|
||
|
||
**** Playlist RSS
|
||
It is also possible to follow not only a channel but a playlist of
|
||
videos. For that, you will instead use
|
||
~https://www.youtube.com/feeds/videos.xml?playlist_id=~ as your base URL
|
||
to which you will add the ID of the playlist you want to follow. For
|
||
instance, with Tom Scott’s playlist for Citation Needed Season 7, the
|
||
URL of the playlist is
|
||
~https://www.youtube.com/playlist?list=PL96C35uN7xGI15-QbtUD-wJ5-G8oBI-tG~,
|
||
which means you need to keep the ~PL96C35uN7xGI15-QbtUD-wJ5-G8oBI-tG~
|
||
and put it into the URL like so:
|
||
#+begin_src text
|
||
https://www.youtube.com/feeds/videos.xml?playlist_id=PL96C35uN7xGI15-QbtUD-wJ5-G8oBI-tG
|
||
#+end_src
|
||
|
||
*** Which RSS reader to go with?
|
||
If you know me, you’ll know I am extremely biased towards Emacs, so of
|
||
course I’ll recommend Elfeed to any Emacs user ([[https://config.phundrak.com/emacs#Packages-Configuration-Applications-Elfeedoip0fl6184j0][my relevant
|
||
configuration is here]]). I even wrote an advice around
|
||
~elfeed-show-visit~ to ensure YouTube videos are open with mpv instead
|
||
of my web browser.
|
||
|
||
If you’re not into Emacs, or not /that/ into Emacs, you can also try
|
||
other alternatives such as [[https://gitlab.com/news-flash/news_flash_gtk][NewsFlash]], a very nice RSS reader written
|
||
in GTK for Linux –I may not always agree with DistroTube, but he made
|
||
a [[https://www.youtube.com/watch?v=KBAmviddh4A][very nice video]] presenting this piece of software. (Remember,
|
||
right-click and then ~mpv "the url here"~!)
|
||
|
||
The [[https://apps.nextcloud.com/apps/news][News app]] for Nextcloud is also very neat, I recommend you using it.
|
||
|
||
You can also get your RSS feed in your terminal with [[https://newsboat.org/][Newsboat]]. Not
|
||
really my cup of tea, but I can see why some people enjoy it.
|
||
|
||
*** Improving a bit the mpv tooling
|
||
You might have heard it, but youtube-dl hasn’t been doing great
|
||
recently. The tool is becoming slow, and it lacks quite a few features
|
||
it could really benefit from. While it is important to acknowledge its
|
||
historical importance, I think it is now time to move on, and its
|
||
successor shall be [[https://github.com/yt-dlp/yt-dlp][yt-dlp]]. In my experience, this youtube-dl fork is
|
||
much faster than youtube-dl itself on top of providing additional
|
||
features such as [[https://github.com/yt-dlp/yt-dlp#sponsorblock-options][SponsorBlock integration]].
|
||
|
||
How do you replace youtube-dl with yt-dlp then? If you use Arch Linux
|
||
or one of its derivates (I hope not Manjaro though), you can simply
|
||
install ~yt-dlp-drop-in~ from the AUR.
|
||
#+begin_src bash
|
||
paru -S yt-dlp-drop-in
|
||
# or if you prefer yay
|
||
yay -S yt-dlp-drop-in
|
||
# or whichever AUR helper you prefer, as long as it is NOT yaourt
|
||
#+end_src
|
||
|
||
If you are not an Arch Linux user, check out [[https://www.funkyspacemonkey.com/replace-youtube-dl-with-yt-dlp-how-to-make-mpv-work-with-yt-dlp][this article]], it will
|
||
help you.
|