1  # Notes on automatic reference page generation
 2  
 3  Main takeaway point: ref page generation from the spec sources is ready to
 4  go. It can be tweaked in lots of small ways, but as it stands it's a huge
 5  improvement over the current ref pages and should be adopted ASAP.
 6  
 7  ## Approach
 8  
 9  This is done with some Python scripts that rely on the rigid structure
10  we've enforced on command and structure definitions in the spec. Everything
11  from the paragraph *preceding* the protos/structs include:: through the
12  validity:: include is pulled out and then split up into ref page sections.
13  In some cases, it isn't immediately obvious how to reorder to get all the
14  content relevant to a single command or struct blocked together in this
15  fashion, and some simplistic semantic tagging has been added for those cases
16  where the structural organization isn't appropriate.
17  
18  The changes to the spec are straightforward, though they appear large in
19  some places. What's happening is:
20  
21    * Validity includes are moved below all the relevant text.
22    * Structure definitions are moved after the validity includes for
23      commands, so they don't overlap or imbed. This has been done for
24      enumerant definitions in some cases, but those are harder to unwind.
25    * Blank lines are being removed from parameter / member definition bullet
26      lists, so they don't confuse the extraction script.
27    * Annotations in the form of asciidoc comments were added where
28      they were needed to assist the extraction scripts. These look
29      like
30      <br>// refBegin vkSomething short description of command for ref page title
31      <br>// refEnd vkSomething [list of related ref pages]
32    * A bunch of small consistency edits were done to match the style guide.
33      These (mostly) aren't directly related to the extraction process, but it
34      is an opportune time to do them.
35    * Pages that don't have extractable descriptive text in the specification,
36      such as Vk*Flags and some Vk*FlagBits types, get automatically generated
37      ref pages. These just define the interface and link to other pages using
38      that type.
39  
40  ## Things to be done
41  
42  These are (more or less) in decreasing priority order. Few of them are
43  difficult.
44  
45    * Something is messed up with the individual page generation targets in
46      the Makefile, so the genRef.py action isn't a dependency of
47      $(MANSOURCES) yet. See
48        https://www.cmcrossroads.com/article/rules-multiple-outputs-gnu-make?page=0%2C2
49    * Cross-page macro links are broken in the individual ref pages, due to
50      using the wrong macros, but are OK in the single-page apispec.html.
51      Should be easily fixed.
52    * manpdf (single-page PDF ref document) target is currently broken.
53    * Turn handle references from sname: to slink: so they link to the right
54      place. Consider using an hlink/hname macro instead. Similarly for
55      basetype: (which doesn't link, but could now with blink:/btext:). In
56      general, most of the ?name: macros can and should turn into ?link:,
57      aside from those directly in the section describing that name.
58    * Need to tweak asciidoc macros for man pages (manpages.conf).
59    * "See Also" links are automatically generated based on the type
60      dependencies between pages. This works well, but doesn't include logical
61      groupings (all clear commands, all drawing commands, etc.). These can be
62      added to the explicit crosslinks in // refEnd statements if desired.
63    * Remove autogenerated files from git and enhance the Makefiles to do all
64      the scripting work required to generate the ref page sources, vkapi.py,
65      etc. at build time.
66    * Some problems with a2x not finding NOTE icons from the asciidoc
67      distribution (worked around by copying into the man/images directory,
68      for now). It would sure be nice if the build were smart enough to use
69      the icons distributed with asciidoc, instead of needing separate copies.
70    * I'm not sure how to best deal with KHR extension ref pages. We could
71      generate them separately in their branches and combine with core pages,
72      or generate everything in the wsi_extensions branch - but that will
73      leave us with core ref pages including WSI extension material, without
74      any obvious indication of why or where. Changes for the KHR extension
75      branches will be small once the 1.0 changes are merged in, and I'll
76      undertake to do those once the core changes are accepted.
77    * Cross-page asciidoc links are broken. This is hard to fix. Probably the
78      best thing to do is strip \<\<\>\> links from the source, or perhaps
79      generate an HTML link to the live spec.
80    * VkResult page needs a better "See Also" section - but not the full list
81      of functions returning VkResult.
82    * Short descriptions are currently embedded in the spec source, and are
83      missing in some cases. Would it make more sense to add them to vk.xml?