dnfmt and the format library

dunnenote-format is an open-source Rust library for reading and writing DunneNote notebooks, and dnfmt is the command-line tool built on it. Both are released under the MIT licence. They exist so that your notes stay readable, and can be moved into other tools, whether or not DunneNote is installed.

The source, the specification and the test notebooks are on GitHub at aacloudguy/dunnenote-format.

Install#

dnfmt is built from source. You need Rust (install it from rustup.rs). The repository pins the Rust version it needs, and cargo downloads it the first time you build.

git clone https://github.com/aacloudguy/dunnenote-format.git
cd dunnenote-format
cargo install --locked --path crates/dunnenote-cli
dnfmt --version

cargo install puts dnfmt in ~/.cargo/bin, which rustup adds to your path. To use the library from your own Rust program, add it as a Git dependency:

[dependencies]
dunnenote-format = { git = "https://github.com/aacloudguy/dunnenote-format", tag = "v0.1.1" }

What it can do#

"Built" means the feature is finished and passes its tests against notebooks written by DunneNote itself (see How it is tested). This is version 0.1.0, the first release. Until 1.0, a new minor version may change the library's API; the repository's changelog lists every change.

Supported versions#

dnfmt reads notebooks at schema 18 (format 0.18.0), the format of DunneNote 0.9. A notebook one version newer opens read-only; anything newer is refused with a message to update dnfmt. An older notebook is refused with a message to open it once in DunneNote, which upgrades it. dnfmt never upgrades a notebook itself.

Reading is safe at any time#

The reading commands (inspect, ls, cat, verify, export) open notebooks read-only and never take DunneNote's lock, so you can run them while the notebook is open in DunneNote. They change nothing in the notebook. Like any SQLite reader, they may leave the database's standard notebook.db-wal and notebook.db-shm companion files, which DunneNote creates too.

Commands#

inspect#

What the notebook is and what it holds:

$ dnfmt inspect Research.dunnenote
Research
  notebook id     01a0e92f-2e10-7ac3-a87a-8a8e9279c25f
  schema          18 (created as format 0.18.0)
  created by      dunnenote-svc-file/0.9.0
  sections        3
  pages           3 (0 archived, 0 templates)
  canvases        7 (0 archived)
    Calendar        1
    Data Table      1
    Picture         1
    Rich Text       2
    Sketch          1
    Editable table  1
  tags            0
  blobs           6 (925 B)

Add --json for the same facts as JSON.

ls#

The sections and pages in sidebar order, with their ids:

$ dnfmt ls Research.dunnenote
▣ Research  01a0e92f-2e82-7db0-a05f-a5715c9ed9dc
  ▸ Research  01a0e92f-2e83-7052-a1cd-63e2614b7628
    · Overview  (6 canvases)  01a0e92f-2e83-7052-a1cd-63fdf20d80a0
    ▸ Background  01a0e92f-2e84-7872-bdfd-8abaadd6de01
      · Reading list  (0 canvases)  01a0e92f-2e85-7161-9925-73fbdb5e4b58
  ▸ Planning  01a0e92f-2e85-7161-9925-740054ac08fe
    · Schedule  (1 canvas)  01a0e92f-2e86-7712-91ce-2224b2f5759c

cat#

A page, or one canvas, as plain text. Give it an id from ls:

$ dnfmt cat Research.dunnenote 01a0e92f-2e86-7712-91ce-2224b2f5759c

Rich text, tables and calendar events print their content. Pictures and sketches print a one-line summary.

verify#

Checks the database, the page tree, stored files and their reference counts. --full also re-hashes every stored file, which catches a damaged picture of the same size. It exits with status 1 when it finds damage, and --strict also fails on warnings, which suits scripts.

$ dnfmt verify Research.dunnenote --full
ok — 7 checks (full)

export#

Readable copies of a notebook. The output folder must be new or empty, and dnfmt never overwrites a file.

$ dnfmt export --md   Research.dunnenote Research-md
$ dnfmt export --csv  Research.dunnenote Research-csv
$ dnfmt export --json Research.dunnenote Research.json

• Markdown: one .md file per page, in folders named after your sections, with an index.md linking them all. Headings, lists, bold, italic, strike and links carry over; pictures are copied into assets/; sketches are drawn as SVG files; tables become Markdown tables; calendars become lists of events. Each page starts with a small header giving its title, id and tags.
• CSV: one file per table and one per calendar, named after the section and page they are on.
• JSON: the whole notebook as one document, including settings, layout, groups, tags and metadata. Pictures and other files are referenced by their SHA-256 hash rather than embedded. The document carries an export_version, so scripts can rely on its shape.

Markdown and CSV leave out archived pages and canvases unless you add --include-archived. They also leave out the hidden canvases that hold a form's drawn answers; the answers table still names them. JSON always includes everything.

new, add-section, add-page, add-canvas#

Create a notebook and add to it. Each command prints the id of what it made, so a script can pass it to the next command. root stands for the notebook itself.

$ dnfmt new Trip.dunnenote --name="Summer Trip"
$ dnfmt add-section Trip.dunnenote root "Plans"
01a0e963-4bb7-7183-9dd8-7bdf89e96f7e
$ dnfmt add-page Trip.dunnenote 01a0e963-4bb7-7183-9dd8-7bdf89e96f7e "Day 1"
01a0e963-4bc2-7ad2-a7be-a231f3e3c9e4
$ dnfmt add-canvas Trip.dunnenote 01a0e963-4bc2-7ad2-a7be-a231f3e3c9e4 rich-text day1.md
$ dnfmt add-canvas Trip.dunnenote 01a0e963-4bc2-7ad2-a7be-a231f3e3c9e4 picture map.png --alt="Route map"

• add-section and add-page add at the end; --first adds at the top. A new page gets the same empty text box DunneNote gives one, unless you add --no-text.
• add-canvas rich-text takes Markdown (.md: headings, paragraphs, lists, bold, italic, strike and links), plain text (.txt), or a document in DunneNote's own JSON form (.json). Use - to read from standard input.
• add-canvas sketch takes a JSON list of strokes, with points given as fractions of the canvas.
• add-canvas picture takes a PNG, JPEG, GIF, WebP or AVIF file, shown at its own size (up to 720 pixels wide). --alt sets its alt text.
• add-canvas table with a .csv or .json file adds a Data Table, read exactly as DunneNote imports a file: the first CSV row names the columns, and numbers, true/false and dates are recognised. Without a file it adds an empty Editable table of three columns and three rows.
• add-canvas calendar takes an .ics file and stores its events and attendees as DunneNote does. Events it can't read, such as one with no start time, are left out and counted.
• New canvases go below what is already on the page. --at=x,y and --size=width,height place them yourself.

table#

Change an Editable table. Name a column by its heading or by its key (c0, c1, …). Typed numbers are stored as numbers, as they are when you type into the table in DunneNote. A Data Table can't be changed: it keeps what it was imported with.

$ dnfmt table Trip.dunnenote <table-id> add-column "Cost"
c3
$ dnfmt table Trip.dunnenote <table-id> add-row "Column 1=Tent" "Cost=120"
$ dnfmt table Trip.dunnenote <table-id> set <row-id> Cost 95

form#

Build a form and submit it, as DunneNote's Submit button does.

$ dnfmt form Trip.dunnenote new root --name="Check-in"
$ dnfmt form Trip.dunnenote field <canvas-id> --name=Guest --required
$ dnfmt form Trip.dunnenote submit <page-id> "Guest=Ada Lovelace" "Signature=@sig.json"

• new makes a form page with its text box and an empty answers table.
• field makes a canvas a question. --label sets the text people see, --required makes it compulsory, and --remove turns it back into an ordinary canvas.
• submit fills in the fields you name (text, or @file for a Markdown file or, for a sketch, a strokes file) and then submits. One row is added to the answers table, with a Submitted time. A drawing or picture is kept in a hidden copy that the row refers to. New columns are added for new questions; once the table has rows, add --confirm to allow that. --utc-offset=<minutes> sets the time zone of the Submitted time, which is UTC otherwise.
• Forms that send their answers to a document or a file are submitted in DunneNote.

caption#

dnfmt caption Trip.dunnenote <picture-id> "Figure 1" adds a caption across the bottom of a picture and groups the two, as Add caption does. --placement=top, corner-tl (and the other corners) or movie place it elsewhere.

template#

$ dnfmt template Trip.dunnenote make <page-id>
$ dnfmt template Trip.dunnenote new <template-id> <section-id>

make adds a template next to the page, cleared as DunneNote clears it: text is emptied and pictures become placeholders, except canvases marked to keep their content. new makes a new page from a template.

archive and retrieve#

dnfmt archive <notebook> <id> --reason=superseded archives a page, a section, the notebook or a canvas. The reason is superseded, wrong, irrelevant or other, which needs --note="…". Nothing under an archived page or section changes, and dnfmt retrieve <notebook> <id> brings it back.

tag and meta#

$ dnfmt tag Trip.dunnenote add <id> "Travel/Italy"
$ dnfmt tag Trip.dunnenote alias "Travel/Italy" "Italia"
$ dnfmt meta Trip.dunnenote set <id> "place=Florence"

Tags work on pages, sections, canvases, tables and table rows. Names are matched the way DunneNote matches them, ignoring case and accents, so cafe finds Café. tag also has rm, rename, merge and delete; meta has rm, and geo=<lat>,<lon> for a location.

Editing safely#

• The editing commands refuse a notebook that is open in DunneNote. Close it there first. While a command runs it holds the notebook's lock, so DunneNote cannot open the notebook at the same moment.
• Each command is all or nothing: if anything is wrong, such as a picture DunneNote can't show or text it can't display exactly, the command changes nothing and says why.
• They only write notebooks at exactly schema 18, and they check the notebook for damage before changing it.
• They leave DunneNote's search index empty, and DunneNote rebuilds it the next time it opens the notebook.
• Keep a backup, as with any tool that changes your files.

How it is tested#

The library is tested against golden notebooks written by DunneNote itself: a development tool in DunneNote builds notebooks through the same code the app's buttons use, and records what the app reads back. The library must read each one exactly as the app does, field for field. Together they cover every canvas kind, captions, picture markup, groups, forms and their answers, templates, archived content, tags and metadata. The notebooks are published with the library, under the same licence, so other tools can test against them too.

Writing is tested the other way round. Notebooks the library writes, covering tables, calendars, forms and their answers, captions, templates, archiving, tags and metadata, and one of DunneNote's own notebooks with pages and canvases added, are opened by DunneNote's own code. The app must open them without repairing anything, pass its full health check with no findings, rebuild its search index, and read back exactly what the library wrote. Every text and sketch the library writes is also loaded by DunneNote's editor, which must accept it unchanged.

Related#

• The open .dunnenote format
• Check a notebook with dnfmt
• Export a notebook to Markdown
• Create a notebook from a script
• Add a table from a CSV file