Organising Series with denote-sequence¶
denote.el basics - revisted¶
Since I was not using it at the time, I neglected to mention a component of denote’s file naming scheme:
<timestamp>==<signature>--<title>__<tags>
where:
<timestamp>captures the date and time the file was created and acts as the file’s unique identifier,<signature>NEW!<title>a lowercase-separated string which captures your traditional file name,<tags>(called keywords by denote) is a_separated string of tag names.
To quote the documentation:
File names can include an arbitrary string of alphanumeric characters in the SIGNATURE field. Signatures have no clearly defined purpose and are up to the user to define.
The denote-sequence extension defines a number of schemes for use with this field allowing for hierarchies of related notes to be defined.
Personally I prefer the numeric scheme
init.el(use-package denote-sequence
:ensure t
:after denote
:custom
(denote-sequence-scheme 'numeric))
Which provides the following pattern:
20250102T120001==1--top-level-one.txt
20250102T120004==1=1--child-one.txt
20250102T120005==1=2--child-two.txt
20250102T120002==2--top-level-two.txt
20250102T120007==2=1--child-one.txt
20250102T120008==2=1=1--nested-child-one.txt
20250102T120003==3--top-level-three.txt
and so on.
reStructuredText and denote.el¶
As before I am using reStructuredText, which denote.el does not support by default.
To use denote-sequence we need to extend the rst definition from last time to include support for the signature field.
init.el(use-package denote
:ensure t
:hook ((dired-mode . denote-dired-mode))
:config
;; Add reStructuredText support to denote
(add-to-list 'denote-file-types `(rst
:extension ".rst"
:date-key-regexp "^:date:"
:date-value-function denote-date-iso-8601
:date-value-reverse-function denote-extract-date-from-front-matter
:front-matter ":title: %s\n:date: %s\n:tags: %s\n:identifier: %s\n:signature: %s\n\n"
:title-key-regexp "^:title:"
:title-value-function identity
:title-value-reverse-function denote-trim-whitespace
:signature-key-regexp ":signature:"
:signature-value-function identity
:signature-value-reverse-function denote-trim-whitespace
:keywords-key-regexp "^:tags:"
:keywords-value-function ,(lambda (ks) (string-join ks ", "))
:keywords-value-reverse-function denote-extract-keywords-from-front-matter
:identifier-key-regexp "^:identifier:"
:identifier-value-function identity
:identifier-value-reverse-function denote-trim-whitespace
:link ":denote:link:`%2$s <%1$s>`"
:link-in-context-regexp ,(concat ":denote:link:`.*?<\\(?1:" denote-id-regexp "\\)>`"))))
Sphinx and denote.el¶
Note
This section contains just the highlights, see the complete implementation for full details.
The previous post did all the heavy lifting when it comes to the Sphinx integration, so adding denote-sequence support was surprisingly straightforward, only requiring a few tweaks to make the information available to the relevant components.
Discovering Sequences¶
To detect which files belong to which sequence (if any), the code for parsing filenames needed to be updated to handle signatures
# denote/record.py
FILENAME_PATTERN = re.compile(
r"""
(?P<identifier>\d{8}T\d{6})
(==(?P<signature>[^-]+))?
--(?P<title>[^_]+)
__(?P<tags>[^.]+)
""",
re.VERBOSE,
)
@dataclasses.dataclass
class Record:
...
signature: str | None
sequence: tuple [int, ...] | None
@classmethod
def parse(cls, filename: str | None) -> Record | None:
if (match := FILENAME_PATTERN.match(filename)) is None:
return None
...
sequence = None
if (signature := match.group("signature")) is not None:
sequence = tuple(int(s) for s in signature.split("="))
...
return cls(
...
signature=signature,
sequence=sequence,
)
Indexing Sequences¶
To represent a sequence hierachy I introduced this Sequence class.
@dataclasses.dataclass
class Sequence:
"""Used to represent the hierarchy described by a sequence."""
identifier: str | None = dataclasses.field(default=None)
"""The identifier of the note at this node in the sequence, if known"""
children: dict[int, Sequence] = dataclasses.field(default_factory=dict)
"""The list of child nodes in the sequence, if any"""
Storing child nodes in a dictionary, rather than a list allows us to support sequences with gaps, or constructing sequences out of order without too much extra thought.
I don’t think I mentioned it last time, but to make it easy to select records by certain criteria all records are stored in a modified ditctionary that maintains numerous indexes of identifiers:
class RecordCollection(collections.UserDict[str, Record]):
"""A dictionary of records that also maintains a number of groupings"""
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self._by_tag = {}
self._by_year = {}
self._by_identifier: dict[str, str] = {}
self._by_sequence: Sequence = Sequence()
When building the sequence hierachy the main thing to consider is that sequences can be constructed out of order:
def __setitem__(self, key: str, item: Record) -> None:
super().__setitem__(key, item)
...
if (sequence := item.sequence) is not None:
node = self._by_sequence
for idx in sequence:
if idx not in node.children:
node.children[idx] = Sequence()
node = node.children[idx]
node.identifier = item.identifier
Rendering Sequences¶
Currently, the main way sequences are shown on this site is the “Related Pages” section in the sidebar which is only shown on pages that are part of a sequence:
{%- if record and record.sequence %}
<section class="post-related">
<h5>Related Pages</h5>
{%- set sequence = denote.records.sequence_hierarchy(record.sequence) %}
{{ render_sequence(sequence) }}
</section>
{%- endif %}
Using the index automatically maintained by the __setitem__ method in the previous section, the implementation of the sequence_hierarchy helper is easy enough.
By definition, all records in a sequence will start with the same number so we only need to consider the first element of a sequence’s tuple:
def sequence_hierarchy(self, key: tuple[int, ...]) -> Sequence:
"""Return the hierarchy of all posts in a sequence"""
return self._by_sequence.children[key[0]]
Going back to the template, the render_sequence macro is given the top-level Sequence node and recurses down the hierarchy rendering out a nested list as it goes:
{% macro render_sequence(node) -%}
{%- set item = denote.records.find(identifier=node.identifier) %}
{%- if node.children | length > 0 %}
{{ render_node(item) }}
<ul>
{%- for child in node.children.values() %}
<li>{{ render_sequence(child) }}</li>
{%- endfor %}
</ul>
{% else %}
{{ render_node(item) }}
{%- endif %}
{%- endmacro %}
Where the record at each node in the sequence is given to the render_node macro to generate the link to the corresponding page.
{% macro render_node(node) -%}
{% if node.identifier != record.identifier %}
<a class="reference internal" title="{{ node.title }}" href="{{ pathto(node.url) }}">
{{ node.title }}
</a>
{% else %}
<a class="current" title="{{ node.title }}" href="#">
{{ node.title }}
</a>
{% endif %}
{% endmacro -%}