Description
This software has been designed to enhance the Fossil SCM documentation features:
- Create hierarchical documentation similar to the DocBook layout.
- Enhance the existing Markdown with features similar to those found in MultiMarkDown.
The tool should be integrated with fossil
seamlessly, so it may be used with any Fossil repository, including, but not limited to, the Fossil SCM itself.
Extensions
For syntax examples for Deployed Extensions, check:
https://github.com/commonmark/CommonMark/wiki/Deployed-Extensions
Examples:
Fences (Fenced Code Blocks)
With optional info string~~~markdown ↩︎ … ~~~
~~~(markdown) ↩︎ … ~~~
~~~.markdown ↩︎ … ~~~
~~~math ↩︎ … ~~~
~~~poem ↩︎ … ~~~
~~~art ↩︎ … ~~~
TOC (Table Of Contents)
{toc}
[toc]
[[toc]]
@[toc](heading)
<!-- md-toc -->
Transclusion:
<<[transclude](file.ext)
<<(file.ext)
{{file.ext}}
DocBook-stylish layout
The current documentation for the Fossil SCM system uses a Wiki Index page, without any hierarchy.
It is somewhat hard to get the overview.
This tool aims to get an more readable overview of the documentation by imitating the DocBook documentation HTML layout.
Note that this software is not intended to emulate all available DocBook features.
This software just aims to simulate some of the DocBook's layout by creating "on-the-fly":
- table-of-contents (ToC)
- single/split pages
- navigation bars (navbars) at top and bottom of pages (when page in split)
- page number (optional)
We just use DocBook's layout for table-of-contents (ToC) and navigation bars (navbars) to improve the user experience.
Creating the layout "on-the-fly" means that the Markdown file is not modified by any means, just the HTML output.
The DocBook-stylish layout is enabled by default, but can be disabled in the configuration.
Update: Another way to add the ToC could be to do as StackEdit:
[TOC]
The configuration option for ToC could thus be:
off/disabled/never/0
(as in vanilla fossil)manual
(the string[TOC]
has a special meaning, as in StackEdit)on/enabled/always/1
(default behaviour infossil-fossmd
andfossmd
)
where manual
would work the same as StackEdit.
UPDATE: Check man(1) tidy
for exact syntax for different on/off
options, and use that syntax.
Update: Support for Modular documentation, similar to DocBook's XInclude
statement.
As a document grows in size, it is desireable to split it up into chapters. In DocBook, this is done by:
<xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="intro.xml" />
<xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="chapter1.xml" />
<xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="chapter2.xml" />
...
In FossMD, this is done using Transclusion (see below for details):
{{intro.md}}
{{chapter1.md}}
{{chapter2.md}}
...
MultiMarkdown-stylish enhancement
There are newer MarkDown formats which offer more features than John Gruber's original MarkDown flavor does, such as MultiMarkdown, which offer feautures such as:
- Tables
- Fenced code blocks with an optional Info string
- File Transclusion
Most (but not all) Markdown table formats supported by MultiMarkdown are also parsed correctly by Fossil.
Fenced Code Blocks may be used as an alternative to code blocks. Fenced code blocks are not supported by Fossil.
Fenced code blocks, used together with an Info String, is an easy way to, for example, syntax highlight source code, using only Markdown.
(Syntax highlighting, though not supported out-of-the-box in Fossil, may be achieved, as described here).
The example below is an example from CommonMark of how to higlight code using Fence Code Blocks.
```ruby
def foo(x)
return 3
end
```
As mentioned before, File Transclusion permits for including other files into the Markdown document, and splitting one big document into several smaller:
{{intro.md}}
{{chapter1.md}}
{{chapter2.md}}
Each included file is parsed as any Markdown document. File Transclusion is not supported by Fossil.
File Transclusion is not limited to Markdown file types. Any file may be included.
This allows for including for example source code files into Markdown.
To avoid parsing the included source code as Markdown, is has to be placed inside a Fence Code Block.
Combining Fenced Code Blocks and File Transclusion is a very convenient way to highlight an entire source code file in a Markdown document:
```ruby
{{foo.rb}}
```
How and if the syntax highlighting is done depends entirely on the configuration.
No information about how to parse Info Strings or how to syntax highlighting is hardcoded into FossMD,
so if syntax highlighting is not configured (which basically means, including a JavaScript URL), the Fenced Code Block will be highlighted.
That is, there is no "fallback" or "by-default" highlighting included in the FossMD source code.
Syntax highlighting in a browser is normally made by JavaScript, detecting a CSS tag in the from-Markdown-generated HTML,
such as <code class="language-ruby">
, and parsing/modifying the text inside that tag.
As any file type may be included by File Transclusion, included files are not limited to source code.
Any JavaScript library for parsing/modifying text may be used to generate any kind of HTML/SVG output.
For example, a sequence diagram could be included in a Markdown document like this (as an SVG image):
```sequence
Alice->Bob: Hello Bob, how are you?
Note right of Bob: Bob thinks
Bob-->Alice: I am good thanks!
```
(This syntax is already supported by the Marxico online Markdown editor, for example.)
The idea is to support any Info String, as long as it is configured by the user.
The advantage is that the user may add virtually any kind of contents to Markdown documents.
The disadvantage is that a document with custom Info Strings, for example sequence, is less "portable", as it may be difficult to read offline,
if some of the content doesn't render properly.
That is, sending such a Markdown document to other users requires them to have a JavaScript library configured to view sequence diagrams,
or they will just see the Fenced Code Block instead.
Exporting the document to for example PDF before distribution to a bigger audience is a recommended work-around.
Influences
Ideas for layout and enhancements have been picked from several projects, not only from DocBook and MultiMarkdown.
Read more about them in the Related section.
Background
Everyone (well, almost) hates to document their technical work.
If we found something being tricky in the documentation process, we will easily find an excuse to not create the document at all.
That is why it is so important that documentation tools must be as easy as possible to use.
Otherwise the documentation soon becomes outdated.
One of many DocBook features is that it creates "easy-browsable" HTML documents with table-of-contents and navigation bars created automatically.
A drawback for DocBook is the input source being XML, which makes it less-than-optimal when it comes to typing.
Again, an excuse to avoid creating and/or updating documents.
Another DocBook issue is the big toolchain and the big dependency list. DocBook is not a lightweight system.
Markdown documents, on the other hand, is lightweight, easy to write, but has no such built-in mechanism as creating table-of-contents and navigation bars.
A group of Markdown documents tends to be just a bunch of stand-alone documents, much like a Wiki, lacking a clear overview of the documentation.
This is true for any collection of Markdown documents, but what specially comes in mind is the documentation for the excellent Fossil SCM software.
This is best shown having a look at the documentation index page Fossil documentation index page, which is missing a hierarchical overview.
As we can see, the mentioned index page contains a list of Wiki pages, most of them not linked to each other.
Formatting the Wiki Index like a technical manual, similar to a DocBook index page, would create a clearer overview.
Here are two DocBook layout samples:
The DocBook manual itself and the BookDown home page (a PHP-based software for creating DocBook layout from Markdown)
It is quite common to include source code snippets in software documentation.
Using standard Markdown, this has to be done by Copy & Paste, and can be quite tedious to format correctly.
As mentioned before, features such as File Transclusion combined with automatic source highlighting makes this much easier.
Features
Also see Tickets.
Prioritary features
- TICKET: Create single and split HTML page
- CLOSED: Add
<meta charset="UTF-8">
if missing, so áéíóúåäö and letter friends look nice. - TICKET: Optional: CSS external stylesheets
- TICKET: Optional: External web fonts
- TICKET: JavaScript syntax highlighting UPDATE: Do syntax highlighting as any other fenced codeblock:
<code class="language-XXX">
The syntax highlighter mentioned in the Fossil docs is shjs.
According to its documentation,shjs 0.6.0
supports 39 languages, and is GPLv3 licensed.
Last updated in 2008, it should be considered an abandoned project.
An alternative, highlightjs, is currently maintained.highlight 0.9.5
supports 162 languages.
highlight
is BSD licensed.
(The license would only be an issue if the JavaScript library is to be included in the Fossil source code, which will probable never happen,
but it is nice to have the choice.) - TICKET: Footer text for images, tables, and diagrams
- TICKET: Responsive layout for smartphones and tablets:
CSS is top priority, JavaScript based frameworks (such as Bootstrap) less important.
Choose a lightweight responsive system from http://codecondo.com/minimal-css-frameworks-grid-systems/
(also available incss/
subdir):
- http://purecss.io, depends on http://nicolasgallagher.com/about-normalize-css/
- http://mincss.com, really tiny
- http://bookdown.io/css/nav.css from BookDown
- http://getskeleton.com/
skeleton.css
, not usingnormalize.css
- https://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/
tmpl-freebsd-docbook.css
from the FreeBSD Developer's Handbook
- http://purecss.io, depends on http://nicolasgallagher.com/about-normalize-css/
- TICKET: Look for a DropDown Menu (Desktop) and "Navigating Menu" (Phablet) responsive layout using only CSS, no JavaScript.
Update: Under construction (some JavaScript included): http://kuu.se/fossil/navbar.cgi RELATED: CSS dropdown navbar. See http://orgmode.org/manual/ for an example. - TICKET: The "C-fossil" version should be able to deal with both
.wiki
and.md
pages. UPDATE: Disable CSS/JS on.wiki
pages, limit included HTML tags to known tags. - TICKET: Load external files: Program source code, images, CVS (for tables, charts), ABC, MIDI (for musical score) UPDATE: File Transclusion
Secondary features
- TICKET: JavaScript search function
- TICKET: Create PDF output
- TICKET: Server page with JS File API (Drag N Drop Markdown files or folders)
- TICKET: SQLite3 file to save custom config settings: optional in the Perl version, as it requires either the sqlite C sources (and thus a C compiler), or the sqlite3 binary.
- TICKET: Helpful to generate printer-friendly pages? CSS Paged Media Module: http://www.w3.org/TR/css3-page/
- TODO: JavaScript spellchecker: http://www.jqueryscript.net/text/Browser-Based-Spell-Checking-Plugin-For-jQuery-Spellchecker.html
Alternative: http://www.afterthedeadline.com/ - both spell and grammar checker, but trickier to setup. - TICKET: Top (and Back ???) links both after headers and as ToC buttons
- TICKET: HTML Tidy to serve "clean" HTML as output: http://www.html-tidy.org/
Design
Some characteristics are heavy influenced by the design of fossil
:
- Minimalistic
- Monolithic
- Embedded web server on demand
- Configuration completely optional
- fossil-like syntax:
command [OPTIONS] subcommand [subcommand2 [subcommand3]]
Four pieces of software: Perl & C (standalone executables), "C-fossil" (patched into
fossil
), and Emacs package- Perl and C standalone versions should use the same HTML templates.
- The "C-fossil" version may add template files (but different from the Perl/C HTML templates) to the
extra=files
section in Fossil'smakemake.tcl
- The Emacs package should be seen as a front end to either the C or the Perl the standalone version.
The packet should work as (yet to decide): - either as an extension to
markdown.el
mode, so HTML output can be created in a similar way as when usingC-c C-c v
- or as a fork of either
impatient-mode.el
orflymd.el
, so HTML can be generated on-the-fly.
- Perl and C standalone versions should use the same HTML templates.
The Perl version and the standalone C version should be functionally identical.
The HTML templates are software language independent, and can be used interchangeably between versions.
The "C-fossil" version should be able to deal with .wiki
pages, even if contents is more restricted (no CSS, no Javascript) than for .md
pages.
To render graphical content (in certain Fenced code blocks), the software should rely only on client-side (browser) JavaScript.
No graphic libraries should be included in any script/executable.
Perl version
To create the monolithic Perl script fossmd.pl
, it is preferred to replace the original Markdown.pl
with the "OO interfaced" CPAN module.
The following packages will all be included in the same monolithic script:
- Text::Markdown
- Text::MultiMarkdown (as a replacement for Text::Markdown)
- HTML::Template
- http_this
Alternative Perl HTTP servers: - The default HTML templates
- The default CSS style sheets
- JavaScript, optionally used for popup navbars, drag n drop, etc.
- OPTIONAL (requires C compiler): Save custom config in SQLite DB, requires DBI and DBD::SQLite
The Text::Markdown module converts Markdown to a HTML body, without any header.
fossmd.pl
uses HTML::Template internally to create the table-of-contents, navigation bars, and page numbering.
TODO: A tool which concatenates all the CPAN modules, the CSS, HTML templates, JavaScript, and the code into a complete fossmd.pl
.
This tool may then be used to "update" fossmd.pl
to use the latest version of each CPAN module.
Perl license issues
There are no license issues with the Perl version, as the it will not integrate with Fossil SCM.
C version
The C version fossmd
doesn't have the same "monolithic" issues as the Perl version, as we can create the monolithic executable fossmd
at compile time.
The following "components" will be considered (very similar to the list of Perl packages)
- Markdown library alternatives:
- Direct links to Fossil code:
https://www.fossil-scm.org/xfer/doc/tip/src/markdown.md (info)
https://www.fossil-scm.org/xfer/doc/tip/src/markdown.c (parser)
https://www.fossil-scm.org/xfer/doc/tip/src/markdown_html.c (XHTML renderer) - libdiscount: supports(?) tables:
http://www.pell.portland.or.us/~orc/Code/discount/ - libsoldout, libdiscount fork:
https://github.com/faelys/libsoldout - commonmark: makes an effort to establish a new standard, supports automatic links and image links, but not tables:
http://spec.commonmark.org (common link) https://github.com/jgm/cmark (C specific link) - MultiMarkdown: C library which seems to support parsing of almost all features we need
http://fletcherpenney.net/multimarkdown/
- Direct links to Fossil code:
One of the libs libdiscount
%%% libsoldout software replaces Text::Markdown.
The libctemplate software (GPL) could replace HTML::Template for a standalone C executable, but cannot be used if integrating with Fossil.
Note that the use of templates is minimal, so maybe the "objcopy technique" http://kuu.se/fossil/objcopy.cgi/doc/tip/README.objcopy.md could be used for a simple template system.
The web server of the Fossil software replaces HTTP::Server::Simple. (Check the entry point void cmd_webserver(void)
in src/main.c
.)
The default HTML templates, the default CSS style sheets, and the JavaScript are declared as variables. (Again, either check Fossil to see how this is done, or use the "objcopy technique".)
OPTIONAL (requires C compiler): Save custom config in SQLite DB. Check Fossil to see how this is done.
C license issues
- The C implementation of HTML::Template,
libctemplate
, is GPLv3, and cannot be integrated with the BSD licensed Fossil SCM.
A workaround is to create a minimal template library from scratch, supporting only a small subset of all HTML::Template features. - The tested CSS frameworks are either MIT or BSD licensed.
This is not a license issue:
http://lists.fossil-scm.org:8080/pipermail/fossil-users/2015-March/020203.html - CommonMark is BSD2 licensed, which should not be an issue if we want to merge it into Fossil.
- MultiMarkdown is MIT licensed, probably not a problem. http://fletcherpenney.net/multimarkdown/
"C-Fossil" version
TODO: Check how the Markdown parsing is done in Fossil.
Create small test program using:
src/markdown.c
src/markdown_html.c
TODO: Check how the regex functions are implemented in Fossil.
Create regex test programs similar to https://kuu.se/fossil/cregex.cgi/raw?ci=trunk&filename=regperl.c using:
src/regexp.c
Emacs version
The Emacs version should be seen as tool, using fossmd as a backend.
We could follow three different path as base to generate MarkDown from within Emacs:
markdown.el mode, using
C-c C-c v
preview, which calls the Emacs Lisp browse-url function.
This opens a new browser tab each time the HTML contents is regenerated, so the browser may end up with lots of tabs. Suboptimal.markdown.el
uses nothing else than John Gruber'sMarkdown.pl
script as a backend.impatient-mode.el, which generates HTML contents as-you-type in the browser.
Does not include autoscroll (which is almost a must for an as-you-type feature to make sense). Depends on Emacs packagessimple-httpd
andhtmlize
.
MarkDown can be used as input, but requires a filter configuration:(defun markdown-html (buffer) (princ (with-current-buffer buffer (format "<!DOCTYPE html><html><title>Impatient Markdown</title><xmp theme=\"united\" style=\"display:none;\"> %s </xmp><script src=\"http://strapdownjs.com/v/0.2/strapdown.js\"></script></html>" (buffer-substring-no-properties (point-min) (point-max)))) (current-buffer)))
As we see, to use Markdown, Strapdown.js is an additional requirement.
A preferred way to generate Markdown is to use a backend solution.
Note: The filter above uses the obsolete-in-HTML5<xmp>
tag.
Extremely slow on big documents???
TODO: Check performance.flymd.el, which also generates HTML as-you-type, and accepts Markdown as input out-of-the-box.
Includes autoscroll.
It would be nice use a similar design (a single backend script for HTML content generation) when generating HTML as-you-type.
Additional code is obviously required as-you-type, for example pushing contents contoniously to the browser,
and to auto-scroll the browser window, for example.
Anyhow, the existing solutions use JavaScript not only for these functions, but also to generate the HTML contents.
Generating HTML as-you-type may cause a performance problem.
TODO: Check if the C standalone version is require as a backend, or the Perl version is fast enough.
TODO: Check if it is viable to use the Follsil version and fossil ui
or fossil server
as a backend.
Read BookDown
source code
TODO: Read the PHP source code, Check how the DocBook table-of-contents, navigation bars, Automatic page numbering are created.
git clone https://github.com/bookdown/Bookdown.Bookdown.git
Especially navigation header and footer templates:
Bookdown.Bookdown/templates/navheader.php
Bookdown.Bookdown/templates/navfooter.php
Naming convention
The generated web pages are using the prefix 1.*
, 1.1.*
, 1.2.*
, 1.1.1.*
, 1.1.2.*
etc. by default to indicate each page's level.
Only when using a configuration file, the prefix may be omitted.
How headers are translated into ToC (Table of Contents)
A common example
- The first header is a level 1 header, the document title.
- The second header is a level 2 header, the first chapter.
The second header defines the "chapter level", which is level 2 in this case, the default. - The third header is a level 3 header, the first subschapter of the first chapter.
- The fourth header is a level 2 header, the second chapter.
- The fifth header is a level 2 header, the third chapter.
- And so on.
A less common example
- The first header is a level 4 header. As it is the first header, it is considered the document title.
- The second header is a level 3 header. As it is the second header, it is considered the first chapter.
This sets the "chapter level", to level 3 in this case. - The third header is a level 3 header, the second chapter.
- The fourth header is a level 2 header.
It is a higher level than the "chapter level", but it is not a level 1 header, so it's not considered a title, so it is also considered a chapter. - The fifth header is a level 1 header. It is a level 1 header, so it's considered a title.
As the title appears in the middle of a document, it is considered a new "book" or "article".
In this case, the generated ToC will be "a set of books" (sort of).
A common example, multiple files
- The files are treated as a list, dealt with in this order:
- list of files given from the command line
- file list in a configuration file
- search list from a search result in the current or a given directory
- The first header in the first file is a level 1 header, the document title.
- The second header in the first file is a level 2 header, the first chapter.
- The first header in the second file is a level 1 header, the document title.
As the title appears in the middle of a document, it is considered a new "book" or "article".
In this case, the generated ToC will be "a set of books" (sort of).
Assume that this second book/article is big, so it is split into several files. - The second header in the second file is a level 2 header, the first chapter in the second book/article.
- The third header in the second file is a level 3 header, the first subschapter of the first chapter of the second book/article.
- The first header in the third file is a level 2 header, the second chapter in the second book/article.
- The first header in the fourth file is a level 2 header, the third chapter in the second book/article.
- And so on.
Configuration
There are basically three configuration cases:
Don't use a configuration file. Details
- Split document by Markdown headers.
- Split by filenames.
- Split by directories.
Use a configuration file. Details
- Split using nested Markdown lists in the configuration file.
Use a configuration file, and customized using external HTML templates.
The HTML header template may include links to external CSS style sheets and web fonts. Details- Split using nested Markdown lists.
Use external HTML templates to create an alternative table-of-contents and navigation bars.
Use external CSS style sheets and web fonts to create a different look.
- Split using nested Markdown lists.
Case 1. No configuration
When using no configuration file, a document can be split by Markdown headers, by using separate Markdown files, and/or by using a directory tree.
These methods can be combined as described below.
Case 1.1. No configuration - Split by Markdown headers.
# 1 `#` - This header starts a new page by default
## 2 `##` - This header also starts a new page by default
### 3 `###` - This header is included in the same page by default, but shows up as a separate chapter on the table-of-contents page, by default
## 2 `######` - This header is included in the same page by default, but shows up as a separate chapter on the table-of-contents page, by default
### 3 `######` - This header is included in the same page by default, but shows up as a separate chapter on the table-of-contents page, by default
#### 4 `######` - This header is included in the same page by default, but shows up as a separate chapter on the table-of-contents page, by default
##### `######` - This header is included in the same page by default, but shows up as a separate chapter on the table-of-contents page, by default
###### 6 `######` - This header is included in the same page by default, but shows up as a separate chapter on the table-of-contents page, by default
This is probably the most "natural" way to split sections in a Markdown document into HTML pages.
By default, the two top levels will be split into HTML pages.
This means that #
and ##
Markdown headers (corresponding to <h1>
and <h2>
HTML headers, respectively) will go into separate pages.
Any Markdown headers, level 1 to 6 (#
to ######
) will be reflected in the table-of-contents page by default.
Using this configuration method, there is no way to produce more than 6 levels in the table-of-contents page.
With this method, it is possible to create an entire HTML output directory using one single Markdown file as input.
Case 1.2. No configuration - Split by filenames.
Multiple Markdown documents may be used as input, as a list of files.
The list of files is given from the command line, and will appear in the same order in the HTML output.
The sections will be split in the same way as when using a single Markdown file.
If a directory is given as an argument, all Markdown files in that directory will be included by default. the sort order is alphabetical by default, so in that case it may be necessary to rename the filenames.
(Renaming files is not needed when using a configuration file.)
Optionally, the files can be ordered by modification date (ascending/descending).
An example of a directory ordered alphabetically:
/my/markdown/dir
|-- 1-introduction.md
|-- 2-whatis.md
|-- 3-getting-started.md
|-- 4-prerequisites.md
`-- 5-index.md
Case 1.3. No configuration - Split by directories.
This is very similar
An example of nested directories ordered alphabetically:
/my/markdown/dir
|-- 1-introduction
| |-- section1.md
| `-- section2.md
|-- 2-whatis
| |-- section1.md
| `-- section2.md
|-- 3-getting-started
| |-- section1.md
| `-- section2.md
|-- 4-prerequisites
| |-- section1.md
| `-- section2.md
`-- 5-index
|-- section1.md
`-- section2.md
Configuration file
%%%%%%% TODO: Resolve how to refer to relative links in document when files are split into chapters and into sections. TODO: How to specify this in ToC %%%%%%%
fossmd
comes with an embedded, default configuration.
The configuration file syntax is standard Markdown.
To dump he configuration to STDOUT:
fossmd config dump
To save the configuration as a file:
fossmd [--global] config save [filename]
The configuration file name is .fossmd-config.md
by default, saved in the current directory, considered the "project configuration".
If the --global
option is used, the configuration file name is saved as .fossmd-config-global.md
in the user's home directory.
fossmd
gets its configuration like this:
- Read the embedded configuration.
- Read global configuration from
.fossmd-config-global.md
, which may override, partly or entirely, the embedded configuration. - Read "project configuration" from
.fossmd-config.md
, which may override, partly or entirely, the global configuration.
To use a custom configuration file, run fossmd
like this:
fossmd -c <config-file> [command]
Configuration summary
A summary of the configuration entries:
# include-files
# exclude-files
# include-subdirs
# exclude-subdirs
# sort-by
# sort-ascending
# toc
# icon-home
# icon-up
# icon-left
# icon-right
Configuration details
Lists (of files, for example), may be written on one single line separated by at one or more spaces, or may be written on separate lines.
The configuration file has the following entries, all optional:
# include-files
Description: List of Markdown filenames
Default value: *
(include all *.md
files)
Example:
file1.md
file2.md
file3.md
file4.md
# exclude-files
Description: List of Markdown filenames to be excluded
Default value: ''
(empty string, that is, do not exclude any *.md
files)
# include-subdirs
Description: List of subdirectory where to search for additional Markdown files
Default value: *
(include all files subdirectories)
Example:
1-introduction
2-whatis
3-getting-started
4-prerequisites
5-index
# exclude-subdirs
Description: List of subdirectory to be excluded from the search for additional Markdown files
Default value: ''
(empty string, that is, do not exclude any subdirectory)
# sort-by
Description: How to sort *md files
Options: date/name
Example: date
# sort-ascending
Description: Flag for ascending/descending sort
Options: 1/true/TRUE/0/false/FALSE
Example: 1
# toc
Example:
- Chapter 1
- Chapter 2
- Chapter 2, section 1
- Chapter 2, section 2
- Chapter 2, section 3
- Chapter 2, section 4
- Chapter 3
- Chapter 3, section 1
- Chapter 3, section 2
- Chapter 3, section 2, subsection 1
- Chapter 3, section 2, subsection 1
- Chapter 3, section 3
- Chapter 4
Output:
1. Chapter 1
2. Chapter 2
2.1 Chapter 2, section 1
2.2 Chapter 2, section 2
2.3 Chapter 2, section 3
2.4 Chapter 2, section 4
3. Chapter 3
3.1 Chapter 3, section 1
3.2 Chapter 3, section 2
3.2.1 Chapter 3, section 2, subsection 1
3.2.2 Chapter 3, section 2, subsection 1
3.3 Chapter 3, section 3
4. Chapter 4
# icon-home
Path to image or Base64 coded data for Home icon.
# icon-up
Path to image or Base64 coded data for Up arrow icon.
# icon-left
Path to image or Base64 coded data for Left arrow icon.
# icon-right
Path to image or Base64 coded data for Right arrow icon.
Configuration notes
The entries include-files/exclude-files
and include-subdirs/exclude-subdirs
may be misconfigured.
One example is this configuration:

BUG!
BUG!
BUG!
# include-files *
# exclude-files *
This configuration tries to include and exclude all Markdown documents, which is obviously impossible.
In this case, the Include
directives overrides the Exclude
directives, and a warning about bad configuration appears.
Customize layout and style
- Command to create HTML templates and CSS style sheets.
HTML Templates
Dump default embedded HTML header and footer.
CSS Style Sheets
Dump default embedded CSS Style Sheets.
Web fonts
Dump default embedded web fonts (none - or musical by default, but with instructions how to use).
Related projects
- Markdown
- DocBook
- BookDown
- Fossil
- PHP Markdown Extra
- CommonMark
- Pandoc
- MultiMarkdown
- MathJax
- KaTeX
- Markx
- abc notation
- abcjs
- D3
- CanvasJS Charts
Combining Markdown and DocBook lead to the existing BookDown software, which is similar to what we want to achieve with this software.
Anyway, BookDown is written in PHP, while we want to write this software in ANSI C.
The reason we want to use C is that one of the purposes with this software is to integrate it with Fossil, written in ANSI C.
Fossil comes with its own Markdown parser and renderer (for HTML) based on the original Markdown specification by John Gruber.
The original Markdown specification lacks support for tables, but that has been included in Fossil lately.
The syntax used for tables is the same as in the well specified PHP Markdown Extra project.
Even if Fossil's Markdown parser seems to fulfill our needs, we may have a look at the CommonMark project.
The project has recently made an effort to update the Markdown specification, and created parsers in different languages, among them, in C.
But even if we use Fossil's or CommonMark's C parser, there are some features that we have to look elsewhere for.
We want to be able to include external files into our documentation, such as program source code, and CSV files, to be listed as tables.
File inclusion is not part of the Markdown specification, but used as plugins in the Pandoc project.
We cannot use the Pandoc code directly (Pandoc is written in Haskell), but we will have a look at the syntax for how Pandoc handles included files.
Pandoc can be configured to parse only strict Markdown, or to accept additional features, such as tables, file inclusionm, math symbols, or musical scores.
This "feature level configuration" should be considered, even if in the end we may end up always enabling all features.
If we decide to be able to disable, for example, o block tables, math symbols or music scores, we also have to decide how to render such a block:
- Not at all (hidden)?
- As a
<code>
block? - A warning message: "Feature disabled, not rendered"?
Similar features as offered by PHP Markdown Extra can also be found in MultiMarkdown.
The advantage is that MultiMarkdown is written in C, so that code may be reused in our project.
In recent releases, the math symbols feature seems to been have disabled by default.
Instead, to support math symbols, MathJax or KaTeX will be used.
The same $LaTeX$ syntax will be used as with Markx. Also true for KaTeX???
To support music scores in Markdown, the abc notation is the natural choice due to its simple text format.
The musical scores themselves will be rendered using abcjs.
If heavily using musical scores in documentation, VexFlow should at least be evaluated.
http://www.vexflow.com/
http://www.vexflow.com/vextab/
http://www.vexflow.com/vextab/playground/?foo
Advantages:
- Straightforward syntax
- %%% Disdvantages:
Support 'include file MIDI/ABC' as a complement to typing ABC direcly in Markdown, using abcMIDI:
http://abc.sourceforge.net/abcMIDI/
There doesn't seem to exist any standard syntax for abc notation embedded in Markdown.
Anyhow, using a similar syntax to $LaTeX$ (surrounded by $
or $$
signs) seems to be reasonable, like this:
CHARTS FROM CSV: CanvasJS Charts and/or D3
- Syntax for program code blocks, Latex, and support for Flow charts and sequence diagrams https://marxi.co/#data-synchronization
%%%
Markdown
Advantages:
- Input source format is plain text, and very intuitive and easy to type.
- The toolchain is very lightweight, one single script, only depending on Perl, available by default on most modern desktop platforms.
- Output generation is fast.
- Documents are quickly created.
Disadvantages:
- There is no mechanism to create a structured hierarchy of documents using only Markdown.
A group of Markdown pages tend to suffer from the same dilemma as a group of Wiki pages: There is no easy way to structure them into chapters.- Pages are not auto-numbered.
- No automatically generated table-of-contents.
- Cannot include external files in Markdown documents, such as:
- Program code (to be rendered as code blocks, optionally syntax highlighted)
- CSV files (to be rendered as tables)
- Program code (to be rendered as code blocks, optionally syntax highlighted)
DocBook
Advantages:
- Easy to structure in a hierarchy. This is reflected in the easy-readable, automatically generated table-of-contents.
- The "DocBook standard navigation bar" is user-friendly.
- Pages are auto-numbered.
- Can produce many different output formats.
- Flexible output, layout, and styling.
Disadvantages:
- Toolchain is big, and dependency list is huge, so installation may be tricky, and output generation is slow.
- Input source format is XML, which is hard to type with a plain text editor.
- XML text editors can partly resolve this problem, but text is still hard to type.
- Mature WYSIWIG DocBook editors are still rare to find.
- Documents takes a long time to write.
BookDown
Advantages:
- Combines the best of Markdown and DocBook, so it almost makes exactly what we want to achieve.
- Nicely styled output.
Disadvantages:
- Written in PHP, which may require extra dependency installation steps on some platforms.
PHP itself is as good as any other language, but we have Fossil in mind when designing, and Fossil is written in C.
We want a 'monolithic installation', one single executable, without any external dependencies. - Configuration is written in JSON.
While using JSON is not necessarily a disadvantage, this software is aimed towards Markdown users.
So the naturally choice of configuration syntax is Markdown in our case. - Does BookDown require a configuration file to generate the table-of-contents?
We want the configuration file to be optional. - Lacks responsive layout for mobile devices.
Fossil (from a point a view as a Markdown parser/renderer, not as a SCM)
Advantages:
- Markdown C code is already included in the project (Fossil) we want to modify,
so little extra effort has to be done to integrate with source (this is quite obvious!) - Parses and renders Markdown files on the fly when using the
fossil ui
web interface. - Handles all kind of tables specified by PHP Markdown Extra, see section below for examples.
Disadvantages:
- The same disadvantages as with Markdown, except the support for PHP Markdown Extra tables.
- Lacks responsive layout for mobile devices (though this issue can be fixed by using responsive CSS/Javascript in customized skins).
PHP Markdown Extra
Advantages:
Based on John Gruber's Markdown, but with extra features:
- Inline HTML
- Markdown Inside HTML Blocks
- Special Attributes
- Fenced Code Blocks
- Tables
- Definition Lists
- Footnotes
- Abbreviations
- Ordered Lists
- Emphasis
- Backslash Escapes
Of these extra features, we want to use Tables (already implemented by Fossil) and Abbreviations (not implemented).
Disadvantages:
- Written in PHP, so the source code cannot be reused in our project.
PHP Markdown Extra: tables
Below follows the different allowed PHP Markdown Extra table syntaxes.
They all render correctly with Fossil (but not with John Gruber's original Markdown script):
Table 1: "Simple" table
syntax:
First Header | Second Header
------------- | -------------
Content Cell | Content Cell
Content Cell | Content Cell
rendered:
First Header | Second Header |
---|---|
Content Cell | Content Cell |
Content Cell | Content Cell |
Table 2: Table with added leading and tailing pipes to each line
syntax:
| First Header | Second Header |
| ------------- | ------------- |
| Content Cell | Content Cell |
| Content Cell | Content Cell |
rendered:
First Header | Second Header |
---|---|
Content Cell | Content Cell |
Content Cell | Content Cell |
Table 3: Table with mixed alignment
syntax:
| Item | Right-aligned value | Left-aligned value | Center-aligned value |
| --------- | -----:|:----- |:-----:|
| Computer | $1600 | $1600 | $1600 |
| Phone | $12 | $12 | $12 |
| Pipe | $1 | $1 | $1 |
rendered:
Item | Right-aligned value | Left-aligned value | Center-aligned value |
---|---|---|---|
Computer | $1600 | $1600 | $1600 |
Phone | $12 | $12 | $12 |
Pipe | $1 | $1 | $1 |
Table 4: Table with formatted cells
syntax:
| Function name | Description |
| ------------- | ------------------------------ |
| `help()` | Display the help window. |
| `destroy()` | **Destroy your computer!** |
rendered:
Function name | Description |
---|---|
help() |
Display the help window. |
destroy() |
Destroy your computer! |
Below follows an example MulltiMarkdown table which does not render correctly with Fossil:
Table 5: Table with caption and grouping
syntax:
[Table Caption]
| | Grouping ||
| Left align | Right align | Center align |
|:-----------|------------:|:------------:|
| This | This | This |
| column | column | column |
| will | will | will |
| be | be | be |
| left | right | center |
| aligned | aligned | aligned |
rendered:
[Table Caption] | | Grouping || | Left align | Right align | Center align | |:-----------|------------:|:------------:| | This | This | This | | column | column | column | | will | will | will | | be | be | be | | left | right | center | | aligned | aligned | aligned |
MultiMarkdown
Image 1: Image with caption
Below follows an example MulltiMarkdown image which adds a caption, but behaves differently when using Fossil.
No caption text is created.
Instead, using Fossil, the caption text goes into the "alt"
tag, and no "title"
tag is created:
syntax:

rendered:
Image 2: Image with b o l d e d caption
Below follows an example MulltiMarkdown image which adds a formatted (bold) caption, and image formatting,
but behaves differently when using Fossil.
No caption text is created.
Instead, using Fossil, the caption text goes into the "alt"
tag, and all text after the img URL goues into the "title"
tag:
syntax:
![This is a **bolded** caption][ABC]
[ABC]: img/abc.png "ABC title" width="3in" height="2.4in"
rendered:
PHP Markdown Extra: abbreviations
Markdown Extra adds supports for abbreviations (HTML tag <abbr>
).
Below follows the PHP Markdown Extra abbreviations syntax.
According to https://talk.commonmark.org/t/abbreviations-and-acronyms/890
Abbreviations are supported by:
- PHP Markdown Extra
- MultiMarkdown23
- Pandoc21 (turned off by default)
- Python Markdown13
- Kramdown17
- The latest release of Remarkable11 also uses this syntax.
Abbreviations are NOT supported by:
- Discount (supported, but uses different syntax)
- Lunamark
- GitHub (using raw HTML only)
- Redcarpet7
Written in MarkDown
syntax:
Abbreviation definitions:
*[HTML]: Hyper Text Markup Language
*[W3C]: World Wide Web Consortium
then, elsewhere in the document, write text such as:
The HTML specification
is maintained by the W3C.
rendered:
Abbreviation definitions:
*[HTML]: Hyper Text Markup Language
*[W3C]: World Wide Web Consortium
then, elsewhere in the document, write text such as:
The HTML specification is maintained by the W3C.
Note that this syntax does not render neither with Fossil (v. 2.1), nor with John Gruber's original Markdown script.
Written in HTML
The <abbr title="Hyper Text Markup Language">HTML</abbr> specification
is maintained by the <abbr title="World Wide Web Consortium">W3C</abbr>.
The HTML specification is maintained by the W3C.
CommonMark
Advantages:
- Well defined Markdown specification
- Several parsers and renderers, among them one written in C, so the source code may be reused in our project.
Disadvantages:
- Does not support neither PHP Markdown tables nor abbreviations
- Table support not defined:
https://github.com/jgm/CommonMark/wiki/Deployed-Extensions
Pandoc
Advantages:
- Supports a huge amount of input and output file formats
- Supports (via plugins) inclusion of external files in Markdown documents.
We should use the same syntax for file inclusion as Pandoc.
File types that specially come into mind:
- Program code (to be rendered as code blocks, optionally syntax highlighted)
- CSV files (to be rendered as tables)
- Program code (to be rendered as code blocks, optionally syntax highlighted)
Disadvantages:
- Written in Haskell, so the source code cannot be reused in our project.
TODO %%%
MultiMarkdown
https://en.wikipedia.org/wiki/MultiMarkdown We should probably stick to MultiMarkdown, which seems to provide all kind of stuff we need!
Advantages:
- Supports many extra features not found in the original Markdown implementation:
Disadvantages:
- License?
- Programming style?
cson_amalgamation.h
MultiMarkdown is released under both the GPL and MIT licenses. You may pick the license that best fits your needs.
Render graphics: Math symbols, Music scores, Charts, Flows and Sequences
Math symbols
Render Math symbols:
https://www.mathjax.org/
https://khan.github.io/KaTeX/
Markdown + Math:
http://markx.herokuapp.com/
Markx uses $LaTeX$ syntax in Markdown for Math symbols.
Just embed using $
s or \\(
and \\)
for inline equations and $$
s or \\[
and \\]
for display mode:
- Inline mode: $E=mc^2$
- Display mode: $$\frac{df(x)}{dx}=\lim_{h \to 0}{\frac{f(y+h)-f(y)}{h}}$$
Music scores
Suggestions for how to embed ABC notation in Markdown
While math symbols may be used with either Inline mode and Display mode, only Display mode makes sense for music scores.
There are at least two reasons:
- From a layout view, it is hard to mix text with a tune on the same line.
Using ABC notation, even the most minimal tune uses at least 3 lines of text:
X:1 K:C C2
Here is a list of formats from discussion about how to embed ABC notation in Markdown:
http://meta.music.stackexchange.com/questions/2507/how-could-abc-notation-be-embedded-in-our-posts-markdown
Note that these formats are only suggestions, not an established standard:
$ABC$
M: 4/4
L: 1/8
K: Dmaj
|:D3 EFGFC|D2 A2 AF2E|D4 B2C2:|
$ABC$
$$
M: 4/4
L: 1/8
K: Dmaj
|:D3 EFGFC|D2 A2 AF2E|D4 B2C2:|
$$
$$X:1
K:C
C2 C2 C > D E2 | E > D E > F G4 |]
w:Row, row, row your boat, gent-ly down the stream.$$
- Inline mode for ABC notation:
$ABC$ $ABC$
does not make sense. - Other suggestions for Display mode for ABC notation:
Either:
$$ABC$$
X:1
K:C
C2 C2 C > D E2 | E > D E > F G4 |]
w:Row, row, row your boat, gent-ly down the stream.
$$ABC$$
Or:
```abc
X:1
K:C
C2 C2 C > D E2 | E > D E > F G4 |]
w:Row, row, row your boat, gent-ly down the stream.
```
The Marxico editor uses the second syntax for all kind of extensions, using triple backticks followed the extension name: https://marxi.co
```python
PYTHON SOURCE CODE GOES HERE
```
```flow
FLOW CHART GOES HERE
```
```sequence
SEQUENCE DIAGRAM GOES HERE
```
So it is natural to use the same syntax for any other possible extension:
```abc
ABC NOTATION GOES HERE
```
```cvs
CVS DATA (TO BE DISPLAYED AS A HTML TABLE) GOES HERE
```
```chart
CANVASJS CHARTS OR D3 DATA GOES HERE
```
```include
FILE NAME GOES HERE
```
Note:
The include
statement uses the same syntax as described by the Pandoc "Include" package:
https://hackage.haskell.org/package/pandoc-include
Format
The Code Blocks like the following will include every file in a new line.
The reference paths should be either absolute or relative to the folder where the pandoc command will be executed.
```include
/absolute/file/path.md
relative/to/the/command/root.md
#do/not/include/this.md
```
Embed a single musical note in Markdown
Do't use ABC notation to just include "a musical note or two" in a text block, that would be overkill.
To include a note (without clef, lines or stave) in text:
- either use a custom web font
- or use the Unicode character directly in text (note: works with Fossil's Markdown parser, but not with John Gruber's)
Using a web font
Here is an example, using http://fontawesome.io/icon/music/:
Here goes some text before the note <i class="fa fa-music" aria-hidden="true"></i>, and some text after.
Here are a few spezialized web fonts for music symbols:
http://www.fontspace.com/dvm-publications/metdemo/5592/charmap
http://www.fontspace.com/robert-allgeyer/musiqwik/5595/charmap
http://www.fontspace.com/david-rakowski/lassus/3057/charmap
Using Unicode
Every editor has its way to insert Unicode.
Normally, copy paste should work fine, as with any other character.
In Emacs, put the cursor over the note:
♫ ♪
M-x describe-char
This gives us the info:
code point in charset: 0x266B
to input: type "C-x 8 RET HEX-CODEPOINT" or "C-x 8 RET NAME"
Example:
C-x 8<ENTER>
Insert character (Unicode name or hex): 266A
♪
The 2669 - 266F Unicode range renders fine in my browser, while the G clef sign (Unicode 1D11E) does not:
♩ ♪ ♫ ♬ ♭ ♮ ♯ 𝄞
https://en.wikibooks.org/wiki/Unicode/List_of_useful_symbols#Music http://xahlee.info/comp/unicode_music_symbols.html
Include MIDI files, convert to ABC, render with abcjs
TBD
JavaScript for rendering ABC music notation:
https://abcjs.net/
CanvasJS Charts
Advantages:
- Combines the best of Markdown and DocBook, so it almost makes exactly what we want to achieve.
Disadvantages:
- Written in PHP, which may require extra dependency installation steps on some platforms.
D3
Advantages:
- Combines the best of Markdown and DocBook, so it almost makes exactly what we want to achieve.
Disadvantages:
- Written in PHP, which may require extra dependency installation steps on some platforms.
var s = "JavaScript syntax highlighting";
alert(s);
Flows and Sequences
Alternative 1: Mermaid
Alternative 2: js-sequence-diagrams
Syntax examples: stackedit
Links
This page:
http://kuu.se/fossil/fossmd.cgi
Markdown
http://daringfireball.net/projects/markdown/syntax
http://daringfireball.net/projects/markdown/dingus
DocBook
http://docbook.org
BookDown
http://bookdown.io
Discount - C implementation of Markdown
http://www.pell.portland.or.us/~orc/Code/discount/
HTML::Template - Perl/CPAN implementation
http://search.cpan.org/~wonko/HTML-Template-2.95/
libctemplate - a C implementation of HTML::Template
http://libctemplate.sourceforge.net/
PHP Markdown Extra:
https://michelf.ca/projects/php-markdown/extra
CSS tree menu
http://www.thecssninja.com/css/css-tree-menu
http://cssdeck.com/labs/pure-css-tree-menu-framework
http://codepen.io/bisserof/pen/fdtBm/ (JavaScript)
http://www.bypeople.com/css-treeview/
JavaScript Syntax Highlight
JavaScript search
JSSINDEX
http://jssindex.sourceforge.net/
Responsive CSS
https://necolas.github.io/normalize.css
http://mincss.com/
http://getskeleton.com/
http://purecss.io/
http://getbootstrap.com/
libHaru - C library for creating PDF documents
https://github.com/libharu/libharu/wiki
PDF::Haru - PDF Creator depending on libHaru
http://search.cpan.org/~butilw/PDF-Haru-1.00/lib/PDF/Haru.pm
PDF::Create - Pure Perl PDF Creator
http://search.cpan.org/~manwar/PDF-Create-1.34/lib/PDF/Create.pm
PDF - JavaScript based
https://parall.ax/products/jspdf
http://pdfmake.org
http://pdfkit.org/
ToC generators
https://github.com/ardumont/markdown-toc (elisp)
https://github.com/thlorenz/doctoc (node.js)
PDF - 6 HASHES TEST
6 HASHES HERE
####### PDF - 7 HASHES TEST 7 HASHES HERE