Comments on C for Coding: Markdown and an Introduction to Parsing Expression...

In looking at what else is out there, you're also ...

2010-01-28T05:07:44.171+08:00

In looking at what else is out there, you're also looking at Pandoc, right? If nothing else the page http://code.google.com/p/pandoc/wiki/PandocVsMarkdownPl should be full of helpful corner cases to test.

Thank. You. So. Much. I was trying to write ...

2010-01-25T22:17:18.249+08:00

Thank. You. So. Much.

I was trying to write markdown in ANTLR and I was pulling my hair out because I felt like I was missing something essential, but this definitely clears things up.

Hi - I'm the author of peg-markdown, and also of t...

2010-01-25T00:55:39.849+08:00

Hi - I'm the author of peg-markdown, and also of two other PEG-based markdown parsers: lunamark, in lua (http://github.com/jgm/lunamark), and markdown-peg, in Haskell (http://github.com/jgm/markdown-peg). So I understand about itches you can't scratch. It wouldn't be hard at all to extend peg-markdown or lunamark to parse other wiki-like formats, but I haven't gotten around to that.

PEG isn't *quite* capable of parsing markdown, so I had to resort to some workarounds. For example, the markdown syntax for inline code is: for any n, a sequence of n backticks, then a string of characters (possibly including backticks) ended by a sequence of n backticks. (I'm simplifying; there are also some optional spaces.) It's easy enough in a PEG to specify this rule for any particular n, but there's no way to specify it for arbitrary n. I dealt with this in peg-markdown by limiting n to 1-5, which should be enough for any conceivable ordinary use. In lunamark, I was able to implement the full rule, because the luapeg library is more powerful than standard PEG.

I dealt with quotations, lists, and other constructs that embed lists of block elements by running the parser again on them, recursively. This isn't ideal, but I couldn't think of another way to do it with PEG.

As for memory usage, it's true that PEG parsers use a lot; peg-markdown uses about 4M of heap space while parsing a 179K file.

Whoops, forgot to add that PEG parsers aren't real...

2010-01-24T22:45:42.272+08:00

Whoops, forgot to add that PEG parsers aren't really suitable for use on webservers (where Markdown is most often found) because of their large memory footprints. A malicious user could try to DoS the server by submitting large slabs of text, rapidly exhausting the memory resources.

Hi. I'm the author of a wikitext-to-html translato...

2010-01-24T22:41:58.948+08:00

Hi. I'm the author of a wikitext-to-html translator (see http://wikitext.rubyforge.org/ for details) and while wiki markup is not Markdown it does share some characteristics.

I also looked at using ANTLR and in the end could only use it for the lexing part, not the parsing, because it really is best suited for well-formed computer languages where syntax errors are usually considered fatal and the translator is not expected to magically recover.

The problem with Markdown and wikitext, apart from its context-sensitivity, is that it isn't a programming language and input is often not well-formed.

So quite close to the beginning of development I came to the conclusion that I was going to need a hand-coded parser. Later on I dumped ANTLR for the lexing and replaced it with Ragel as it was much faster.

I do have an interest in PEG and in fact one of my projects includes a flexible PEG parser generator (see http://walrus.wincent.com/ for info), but for unreliable, error-prone input like this kind of markup I still think hand-coded parsers are the only way to go.

Heh woops. Fixed. :)

2010-01-24T17:42:40.274+08:00

Heh woops. Fixed. :)

"Scratch you can't itch"?

2010-01-24T17:25:32.971+08:00

"Scratch you can't itch"?