# cm6: Integrate yaml and markdown
**URL:** https://discuss.codemirror.net/t/cm6-integrate-yaml-and-markdown/4494
**Category:** Uncategorized
**Created:** [June 7, 2022, 4:04pm UTC](https://discuss.codemirror.net/t/cm6-integrate-yaml-and-markdown/4494 "2022-06-07T16:04:02Z")
**Posts on this page:** 12
**Page:** 1
### Author:  [@teucer](https://discuss.codemirror.net/u/teucer)
#### Post date: [June 7, 2022, 4:04pm UTC](https://discuss.codemirror.net/t/cm6-integrate-yaml-and-markdown/4494/1 "2022-06-07T16:04:02Z")
I have yaml documents where values are in markdown.
How can I Integrate both parsers (yaml legacy mode and markdown lezer)?
---
### Author:  [@pranav](https://discuss.codemirror.net/u/pranav)
#### Post date: [June 7, 2022, 5:09pm UTC](https://discuss.codemirror.net/t/cm6-integrate-yaml-and-markdown/4494/2 "2022-06-07T17:09:32Z")
You could write [an extension](https://github.com/lezer-parser/markdown#user-content-markdownextension) for the Markdown parser and [pass it](https://github.com/codemirror/lang-markdown#user-content-markdown%5Econfig.extensions) to the `@codemirror/lang-markdown` extension’s configuration. I did a similar effort for YAML frontmatter, you can [check it here](https://github.com/retronav/ixora/blob/main/src/plugins/frontmatter.ts) for reference.
---
### Author:  [@marijn](https://discuss.codemirror.net/u/marijn)
#### Post date: [June 7, 2022, 6:26pm UTC](https://discuss.codemirror.net/t/cm6-integrate-yaml-and-markdown/4494/3 "2022-06-07T18:26:18Z")
I think an extension for the Markdown parser is the wrong direction for this — you’ll want a YAML parser and then use `parseMixed` to parse some content as Markdown, not embed YAML in Markdown structure. But currently the only YAML parser is a simple legacy mode, which cannot be used as a basis for `parseMixed`, unfortunately.
---
### Author:  [@teucer](https://discuss.codemirror.net/u/teucer)
#### Post date: [June 7, 2022, 10:20pm UTC](https://discuss.codemirror.net/t/cm6-integrate-yaml-and-markdown/4494/4 "2022-06-07T22:20:15Z")
I tried to write a markdown extension by emulating how numbered lists are handled in the markdown parser. This seems to be challenging. Also it does not look right, as the document is ultimately a yaml file.
I am a little bit familiar with lark and thought I could translate this [grammar](https://github.com/ligurio/lark-grammars/blob/master/lark_grammars/grammars/yaml.lark) to lezer.
Assuming this possible (I can stick to simpler yaml files) how do I use `parseMixed`? Any examples?
---
### Author:  [@marijn](https://discuss.codemirror.net/u/marijn)
#### Post date: [June 8, 2022, 5:59am UTC](https://discuss.codemirror.net/t/cm6-integrate-yaml-and-markdown/4494/5 "2022-06-08T05:59:45Z")
That grammar looks suspiciously simple. The [tree sitter grammar](https://github.com/ikatyang/tree-sitter-yaml/blob/master/grammar.js) for YAML is a lot more messy, which aligns with that I’d expect for such a format.
If I understand correctly that you’d want to parse the content of some literals as Markdown syntax, the use of `parseMixed` here would look similar to the one the html parser uses to [nest parsers](https://github.com/lezer-parser/html/blob/main/src/content.js#L29) for tag content (though without the tag-recognizing logic).
---
### Author:  [@teucer](https://discuss.codemirror.net/u/teucer)
#### Post date: [June 10, 2022, 2:11pm UTC](https://discuss.codemirror.net/t/cm6-integrate-yaml-and-markdown/4494/6 "2022-06-10T14:11:07Z")
Below is my attempt to create a simplified grammar (I don’t need to support full yaml)
- Keys are snake case ascii letters and digits
- No handling of tags, anchors etc.
- No handling of nested properties
- No handling of block Sequence or Mapping
- Tried to mix json and python grammars
Getting `Inconsistent skip sets after newline Key ": " string`. I have an issue with new lines in `LongString`.
@marijn
Could you please have look and provide me some pointers?
```auto
@top YamlText { document }
@skip { space | newlineEmpty | Comment }
document { "---"? newline (property newline dedent)+ "..."? eof }
property { Key ": " value }
value { Scalar | LongString | Mapping | Sequence }
Scalar { True | False | Null | Number | SimpleString }
SimpleString { string }
LongString { quote? Multiline quote? | BlockString }
@skip {} {
Multiline { string (newline+ indent string)* newline (dedent | eof) }
BlockString {
Op space newline+
indent Multiline
(dedent | eof)
}
}
Mapping { "{" commaSep? "}" }
Sequence { "[" commaSep? "]" }
@context trackIndent from "./tokens.js"
@external tokens indentation from "./tokens" { indent, dedent }
@tokens {
True { "true" }
False { "false" }
Null { "null" | "~" }
Op { "|" | ">" | "|-" | ">-" | "|+" | ">+" }
Number { "-"? (int | int? frac?) exp? }
int { "0" | $[1-9] std.digit* }
frac { "." (std.digit+ | ".nan" | ".inf") }
exp { $[eE] $[+\-]? std.digit+ }
string { char* }
char { $[\u{20}\u{21}\u{23}-\u{5b}\u{5d}-\u{10ffff}] | "\\" esc }
esc { $["\\\/bfnrt] | "u" hex hex hex hex }
hex { $[0-9a-fA-F] }
Key { keyChar (std.digit | keyChar)* }
keyChar { std.asciiLetter | "_" }
quote { '"' }
Comment { "#" ![\n\r]* }
space { ($[\t\f] | "\\" $[\n\r])+ }
"{" "}" "[" "]" ":" "|" ">" "+" "-"
}
commaSep { expr ("," expr)* }
@external tokens newlines from "./tokens" { newline, newlineEmpty, eof }
@external propSource jsonHighlighting from "./highlight"
@detectDelim
```
---
### Author:  [@marijn](https://discuss.codemirror.net/u/marijn)
#### Post date: [June 13, 2022, 9:56am UTC](https://discuss.codemirror.net/t/cm6-integrate-yaml-and-markdown/4494/7 "2022-06-13T09:56:04Z")
It doesn’t know which set of skip rules to use after `": " string`, which may be either the end of a `SimpleString`, or inside a `Multiline` rule (which has local skip rules). This may be a language where putting the whitespace explicitly into the grammar, rather than using skip rules, is appropriate, since as far as I understand it the meaning of whitespace differs quite a bit depending on context in YAML.
---
### Author:  [@teucer](https://discuss.codemirror.net/u/teucer)
#### Post date: [June 13, 2022, 10:29pm UTC](https://discuss.codemirror.net/t/cm6-integrate-yaml-and-markdown/4494/8 "2022-06-13T22:29:06Z")
@marijn thank you for the reply.
I managed to get the basic version working [yaml.grammar](https://github.com/teucer/vite-repro/blob/main/src/parser/yaml.grammar).
I have a more advanced version that I will post shortly.
I have a couple questions on the grammar:
1. It would be helpful to have a repetition operator on the tokens, e.g. how do I succinctly create a token matching dates with format “2022-06-13”? My current solution is to manually repeat digits 4 times for the year. Is there another way?
2. How do I express that I am at the start of the line (^ in regex) ? I have implemented it with an external tokenizer. This is useful when I want to indicate that the current block mapping has ended.
3. How do I tell the parser to completely ignore tokens in certain contexts? E.g. I don’t want to parse comments in plain multiline strings. Currently I am using the highlighter to fix ex-post. Would adding a new cobstruct '@ignore" (like @skip) work?
4. There is an undocumented functionality: `( ![\n\r] | "\\" _ )`. My interpretation is that “it does not contain newline, but if it does it is escaped”. Is that correct?
5. Even though I have declared “~” as null, it is not parsed as such. Any pointers?
---
### Author:  [@marijn](https://discuss.codemirror.net/u/marijn)
#### Post date: [June 14, 2022, 6:57am UTC](https://discuss.codemirror.net/t/cm6-integrate-yaml-and-markdown/4494/9 "2022-06-14T06:57:54Z")
> [@teucer](#):
>
> It would be helpful to have a repetition operator on the tokens, e.g. how do I succinctly create a token matching dates with format “2022-06-13”? My current solution is to manually repeat digits 4 times for the year. Is there another way?
No, there’s no counted repetition operator. It just doesn’t come up often enough.
> [@teucer](#):
>
> How do I express that I am at the start of the line (^ in regex) ?
Set up your grammar so that the token only appears after a newline, I guess.
> [@teucer](#):
>
> - How do I tell the parser to completely ignore tokens in certain contexts?
Same answer: make sure the grammar only recognizes that token in that context.
> [@teucer](#):
>
> There is an undocumented functionality:
All of the syntax in that expression is documented. Do you mean the `![]` part? That’s a negated character set.
> [@teucer](#):
>
> Even though I have declared “~” as null, it is not parsed as such.
I don’t know what declaring something as null means here.
---
### Author:  [@teucer](https://discuss.codemirror.net/u/teucer)
#### Post date: [June 14, 2022, 11:33am UTC](https://discuss.codemirror.net/t/cm6-integrate-yaml-and-markdown/4494/10 "2022-06-14T11:33:24Z")
On 4., could you please explain the meaning of `(![\n\r] | "\\" _ )`? I don’t get what `_` refers to.
---
### Author:  [@marijn](https://discuss.codemirror.net/u/marijn)
#### Post date: [June 14, 2022, 12:12pm UTC](https://discuss.codemirror.net/t/cm6-integrate-yaml-and-markdown/4494/11 "2022-06-14T12:12:40Z")
`_` means “any character”
---
### Author:  [@teucer](https://discuss.codemirror.net/u/teucer)
#### Post date: [June 14, 2022, 2:27pm UTC](https://discuss.codemirror.net/t/cm6-integrate-yaml-and-markdown/4494/12 "2022-06-14T14:27:06Z")
Ok understood thx!
I have created `scalarProperty` as follows:
```auto
scalarProperty { Key ": " scalar newline }
scalar { Boolean | Null | Date | Number | Plain | String }
```
The issue is that `Plain` (unquoted string) overlaps with pretty much everything. I have tried to address it by setting token `@precedence`, e.g.
```auto
@precedence{Date, Number, Plain}
@precedence {Boolean, Plain}
...
```
Now when I enter `key: 2022-06-\n`, `2022` is interpreted as a `Number` and `-06-` as unknown.
I don’t understand why it is split: shouldn’t `2022-06-` be parsed as `Plain`?