qapi/parser: preserve indentation in QAPIDoc sections

Change get_doc_indented() to preserve indentation on all subsequent text
lines, and create a compatibility dedent() function for qapidoc.py that
removes indentation the same way get_doc_indented() did.

This is being done for the benefit of a new qapidoc generator which
requires that indentation in argument and features sections are
preserved.

Prior to this patch, a section like this:

```
@name: lorem ipsum
   dolor sit amet
     consectetur adipiscing elit
```

would have its body text be parsed into:

```
lorem ipsum
dolor sit amet
  consectetur adipiscing elit
```

We want to preserve the indentation for even the first body line so that
the entire block can be parsed directly as rST. This patch would now
parse that segment into:

```
lorem ipsum
   dolor sit amet
     consectetur adipiscing elit
```

This is helpful for formatting arguments and features as field lists in
rST, where the new generator will format this information as:

```
:arg type name: lorem ipsum
   dolor sit amet
     consectetur apidiscing elit
```

...and can be formed by the simple concatenation of the field list
construct and the body text. The indents help preserve the continuation
of a block-level element, and further allow the use of additional rST
block-level constructs such as code blocks, lists, and other such
markup.

This understandably breaks the existing qapidoc.py; so a new function is
added there to dedent the text for compatibility. Once the new generator
is merged, this function will not be needed any longer and can be
dropped.

I verified this patch changes absolutely nothing by comparing the
md5sums of the QMP ref html pages both before and after the change, so
it's certified inert. QAPI test output has been updated to reflect the
new strategy of preserving indents for rST.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240626222128.406106-6-jsnow@redhat.com>
[Lost commit message paragraph restored]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
This commit is contained in:
John Snow 2024-06-26 18:21:11 -04:00 committed by Markus Armbruster
parent 36c6dcc266
commit 939c639e1d
3 changed files with 40 additions and 23 deletions

View File

@ -26,6 +26,7 @@ https://www.sphinx-doc.org/en/master/development/index.html
import os
import re
import textwrap
from docutils import nodes
from docutils.parsers.rst import Directive, directives
@ -53,6 +54,19 @@ else:
__version__ = "1.0"
def dedent(text: str) -> str:
# Adjust indentation to make description text parse as paragraph.
lines = text.splitlines(True)
if re.match(r"\s+", lines[0]):
# First line is indented; description started on the line after
# the name. dedent the whole block.
return textwrap.dedent(text)
# Descr started on same line. Dedent line 2+.
return lines[0] + textwrap.dedent("".join(lines[1:]))
# Disable black auto-formatter until re-enabled:
# fmt: off
@ -164,7 +178,7 @@ class QAPISchemaGenRSTVisitor(QAPISchemaVisitor):
term = self._nodes_for_one_member(section.member)
# TODO drop fallbacks when undocumented members are outlawed
if section.text:
defn = section.text
defn = dedent(section.text)
else:
defn = [nodes.Text('Not documented')]
@ -202,7 +216,7 @@ class QAPISchemaGenRSTVisitor(QAPISchemaVisitor):
termtext.extend(self._nodes_for_ifcond(section.member.ifcond))
# TODO drop fallbacks when undocumented members are outlawed
if section.text:
defn = section.text
defn = dedent(section.text)
else:
defn = [nodes.Text('Not documented')]
@ -237,7 +251,7 @@ class QAPISchemaGenRSTVisitor(QAPISchemaVisitor):
dlnode = nodes.definition_list()
for section in doc.features.values():
dlnode += self._make_dlitem(
[nodes.literal('', section.member.name)], section.text)
[nodes.literal('', section.member.name)], dedent(section.text))
seen_item = True
if not seen_item:
@ -260,9 +274,12 @@ class QAPISchemaGenRSTVisitor(QAPISchemaVisitor):
continue
snode = self._make_section(section.tag)
if section.tag and section.tag.startswith('Example'):
snode += self._nodes_for_example(section.text)
snode += self._nodes_for_example(dedent(section.text))
else:
self._parse_text_into_node(section.text, snode)
self._parse_text_into_node(
dedent(section.text) if section.tag else section.text,
snode,
)
nodelist.append(snode)
return nodelist

View File

@ -448,7 +448,7 @@ class QAPISchemaParser:
indent = must_match(r'\s*', line).end()
if not indent:
return line
doc.append_line(line[indent:])
doc.append_line(line)
prev_line_blank = False
while True:
self.accept(False)
@ -465,7 +465,7 @@ class QAPISchemaParser:
self,
"unexpected de-indent (expected at least %d spaces)" %
indent)
doc.append_line(line[indent:])
doc.append_line(line)
prev_line_blank = True
def get_doc_paragraph(self, doc: 'QAPIDoc') -> Optional[str]:

View File

@ -117,8 +117,8 @@ doc symbol=Base
body=
arg=base1
description starts on a new line,
minimally indented
description starts on a new line,
minimally indented
doc symbol=Variant1
body=
A paragraph
@ -145,8 +145,8 @@ doc symbol=Alternate
arg=i
description starts on the same line
remainder indented the same
@b is undocumented
remainder indented the same
@b is undocumented
arg=b
feature=alt-feat
@ -158,11 +158,11 @@ doc symbol=cmd
body=
arg=arg1
description starts on a new line,
indented
description starts on a new line,
indented
arg=arg2
description starts on the same line
remainder indented differently
remainder indented differently
arg=arg3
feature=cmd-feat1
@ -178,16 +178,16 @@ some
section=TODO
frobnicate
section=Notes
- Lorem ipsum dolor sit amet
- Ut enim ad minim veniam
- Lorem ipsum dolor sit amet
- Ut enim ad minim veniam
Duis aute irure dolor
Duis aute irure dolor
section=Example
-> in
<- out
-> in
<- out
section=Examples
- *verbatim*
- {braces}
- *verbatim*
- {braces}
section=Since
2.10
doc symbol=cmd-boxed
@ -198,9 +198,9 @@ a feature
feature=cmd-feat2
another feature
section=Example
-> in
-> in
<- out
<- out
doc symbol=EVT_BOXED
body=