1  Rebuilding coreboot image generation
  2  ====================================
  3  
  4  Current situation
  5  -----------------
  6  ChromeOS (CrOS) probably has the most complex image bundling process in the
  7  coreboot ecosystem. To make CrOS features more accessible to the wider
  8  coreboot community, we want to move these capabilities into upstream
  9  coreboot’s build system.
 10  
 11  Right now, the CrOS build system creates coreboot images, and various
 12  instances of the payload (with different configuration options), plus some
 13  more files (eg. EC firmware), then passes them to a CrOS-specific utility
 14  (`bundle_firmware.py`) to build the final image from that.
 15  
 16  `bundle_firmware` adds a flashmap (fmap) to the final image and creates
 17  additional CBFS filesystems in fmap regions. It then extracts some files from
 18  the original CBFS region (that was put in place carefully to later match to
 19  the default fmap region) and copies some of them into the others, as well as
 20  putting more data (eg. the bitmap data, keys) as raw data into other fmap
 21  regions.
 22  
 23  With the recent addition of more files to CBFS, both on the coreboot side
 24  (dsdt, FSP, and so on) and with ChromeOS specifics (eg. more files describing
 25  boot screens) we either need to expand the scope of bundle\_firmware or move
 26  the capability to build complex images to upstream coreboot’s build system.
 27  This document proposes to do the latter and outlines how this could be
 28  achieved.
 29  
 30  Problems with the current build system parts
 31  --------------------------------------------
 32  One common sentiment is that it should be possible to reuse some of the
 33  existing mechanisms that are supposed to be supplanted by this.
 34  The main concern during this design that precluded their use was that none of
 35  them provides a comprehensive solution to building complex coreboot based
 36  images:
 37  * fmap.dts and fmd provide a flash layout, but no assignment of files of regions
 38  * cbfs-files-y ends up as an internal make variable using
 39    `weird|formatting|to|deal|with|make’s|limitations`
 40  * make isn’t powerful enough to deal with ordering these entries in said
 41    variable to guarantee success if there’s enough room for the files. While that
 42    could be added, that becomes more make macro work indistinguishable from magic
 43    that people fail to understand, break and with good reason complain about
 44    to work around such issues, ChromeOS firmware uses a custom tool with even
 45    more special cases to finally build the image it needs. If coreboot upstream
 46    is to support vboot, it should also be powerful enough not to need magic tools
 47    that only live within downstream projects.
 48  
 49  Requirements
 50  ------------
 51  A complete ChromeOS coreboot image consists of (depending on the device)
 52  * platform specific data in raw fmap regions (eg IFD, ME firmware),
 53  * the bootblock (coming from the bootblock),
 54  * three copies of coreboot, consisting of the stages (verstage, romstage,
 55    ramstage) plus data,
 56  * depthcharge plus data (with each of the coreboot copies),
 57  * EC firmware files (with each of the coreboot copies),
 58  * signatures over several parts of the image and
 59  * some final checksumming over parts of the image to satisfy boot ROM
 60    tests on ARM
 61  
 62  A complete upstream coreboot image (with fallback/normal switch configuration,
 63  using a yet to be implemented switching scheme based on fmaps) consists of
 64  * platform specific data in raw fmap regions (eg IFD, ME firmware),
 65  * two copies of coreboot, consisting of
 66   * the bootblock and
 67   * the stages (romstage, ramstage) plus data,
 68  * payload plus data (with each of the coreboot copies),
 69  
 70  Since a single platform is potentially built with different payload
 71  configurations (eg. modding a Chromebook to not use the verified ChromeOS
 72  boot scheme), some concerns need to be kept separate:
 73  * Platform requirements that have nothing to do with the payload or boot schemes
 74   * IFD, ME, … need to copied to the right place
 75   * boot ROM requirements such as checksums must be honored
 76  * Payload/boot scheme requirements
 77   * Having one to three regions with certain files copied into them
 78  
 79  Proposal
 80  --------
 81  The proposal is based on manifest files that describe certain aspects of the
 82  final image.
 83  The number of manifest files may change over time, but this seems to be a
 84  reasonable approach for now. As long as coreboot uses fmap and cbfs, there
 85  should be few need to change the language, since composition is done through
 86  files.
 87  
 88  The final image is generated by a utility that is handed a number of manifests
 89  and the size of the flash (derived from `CONFIG_ROM_SIZE`). These manifest files
 90  deal with different concerns, with the following an example that should match
 91  current use cases:
 92  
 93  Chipset manifest
 94  ----------------
 95  The chipset details if there are any non-coreboot regions, and assigns them
 96  names, locations, sizes and file contents and prepares a region for what is
 97  “platform visible” (eg. IFD’s BIOS region) that may be of flexible size
 98  (depending on the flash chip’s size). For the purpose of this document, that
 99  region is called “BIOS”.
100  It can also specify if there’s a post processing requirement on the final
101  image.
102  
103  coreboot manifest
104  -----------------
105  coreboot provides lists of the files it generates for each category it’s
106  building (eg. bootblock, verstage, romstage, ramstage). They not only contain
107  the stages themselves, but also additional files (eg. dsdt belongs to ramstage
108  since that’s where it is used)
109  
110  Boot method manifest
111  --------------------
112  The boot method manifest can subdivide the BIOS region, eg. using it directly
113  (for coreboot’s “simple” bootblock), splitting it in two (for coreboot’s
114  fallback/normal) or in many parts (for ChromeOS, which requires two CBFS
115  regions, one for GBB, several for VPD, …).
116  It also specifies which of the file lists specified earlier belong in which
117  region (eg. with verstage verifying romstage, verstage needs to be only in
118  ChromeOS’ RO region, while romstage belongs in RO and both RW regions).
119  It can also specify a post processing step that is executed before the
120  chipset’s.
121  
122  Payload and additional manifests
123  --------------------------------
124  External components should also provide manifests to add files to categories.
125  This way the payload and other components (eg. EC firmware) can be developed
126  without needing to touch the central boot method manifest (that likely resides
127  in the coreboot tree, given that coreboot needs to deal with choosing fmap
128  regions already).
129  
130  coreboot build system
131  ---------------------
132  The coreboot build system will be split more distinctly in two phases: The
133  first is about building the files (with results like romstage.elf), while the
134  second phase covers the assembly of the final image.
135  
136  By having a global picture of the final image’s requirements, we can also
137  avoid issues where files added earlier may prevent later additions that have
138  stricter constraints - without resorting to hacks like
139  <https://chromium-review.googlesource.com/289491> that reorder the file
140  addition manually.
141  
142  Example
143  -------
144  As an example, we’ll define an Intel-based board with a postprocessing tool
145  (something that doesn’t exist, but isn’t hard to imagine):
146  
147  It specifies an IFD region, an ME, and the BIOS region. After the image is
148  built, the entire image needs to be processed (although the tool likely works
149  only on a small part of it)
150  
151  It’s built in a ChromeOS-like configuration (simplified at places to avoid
152  distracting from the important parts), so it has three CBFS regions, and
153  several data regions for its own purpose (similar to GBB, FWID, VPD, …). After
154  the regions are filled, one data region must be post-processed to contain
155  signatures to enable verifying other regions.
156  
157  ```
158  Chipset manifest
159  ================
160  # A region called IFD, starting at 0, ending at 4K
161  region IFD: 0 4K
162  # Add the specified file “raw” into the region.
163  # If the file is smaller than the region, put it at the bottom and fill up
164  # with 0xff
165  raw IFD: build/ifd.bin align=bottom empty=0xff
166  # Call the postprocessor on the data that ends up in IFD (in this example it
167  # might lock the IFD)
168  postprocess IFD: util/ifdprocess -l
169  
170  # a region called ME, starting at 4K, ending at 2M
171  region ME: 4K 2M
172  raw ME: 3rdparty/blobs/soc/intel/xanadu/me.bin align=bottom empty=0x00
173  
174  # a region called BIOS, starting at 2M, filling up the free space
175  # filling up fails (build error) if two regions are requested to fill up
176  # against each other
177  region BIOS: 2M *
178  
179  # This would define a region that covers the last 4K of flash.
180  # The BIOS region specified above will end right before it instead of
181  # expanding to end of flash
182  # region AUX: -4K -0
183  
184  # specify the tool that post-processes the entire image.
185  postprocess image: util/intelchksum/intelchksum.sh
186  
187  coreboot manifest
188  =================
189  # declare that build/verstage.elf belongs into the group ‘verstage’
190  # these groups are later referred to by the “cbfs” command.
191  group verstage: build/verstage.elf stage xip name=fallback/verstage
192  group romstage: build/romstage.elf stage xip name=fallback/romstage
193  group ramstage: build/ramstage.elf stage name=fallback/ramstage
194  compression=lzma
195  group ramstage: build/dsdt.aml compression=lzma
196  
197  boot method manifest
198  ====================
199  # Define RO as region inside BIOS, covering the upper half of the image.
200  # It’s a build error if the result crosses outside BIOS.
201  # math expressions are wrapped with ( ),
202  # and mentions of regions therein always refer to their size
203  subregion BIOS RO: ( image / 2 ) -0
204  
205  # Define RW to cover the rest of BIOS.
206  # The order of RW and RO doesn’t matter except to keep comments clearer.
207  # Dynamic items like RW (“*”) will be sized to fill unused space after
208  # everything else is placed.
209  subregion BIOS RW: 0 *
210  
211  # It may be necessary to separate the RO/RW definition into another manifest
212  # file
213  # that defines the RO configuration of the flash
214  
215  # Some more subregions, with dynamically calculated sizes
216  subregion RW RW_A: 0 ( RW / 2 )
217  subregion RW RW_B: * -0
218  subregion RW_A FW_MAIN_A: RW_A * -0
219  subregion RW_A VBLOCK_A: 0 64K
220  # foo +bar specifies start + size, not (start, end)
221  # also, start is given as “the end of VBLOCK_A”
222  # (while using a region in the “end” field means “start of region”)
223  subregion RW_A FWID_A: VBLOCK_A +64
224  
225  # To make the example not too verbose, RO only has the CBFS region
226  subregion RO BOOTSTUB: 0 *
227  
228  # Postprocess the data that ends up in VBLOCK_A,
229  # passing the listed regions as additional arguments.
230  # Circular dependencies are build errors.
231  postprocess VBLOCK_A(FW_MAIN_A): signtool
232  
233  # binding files to regions indirectly through groups
234  cbfs BOOTSTUB: verstage, romstage, ramstage, payload
235  cbfs FW_MAIN_A: romstage, ramstage, payload
236  
237  # defining defaults: unless overridden, in all regions that use CBFS (“*”),
238  # we want all files to come with SHA256 hashes.
239  # Wildcard defaults have lower priority than specific defaults.
240  # Other conflicts lead to a build error.
241  cbfsdefaults *: hash=sha3
242  
243  payload manifest
244  ================
245  group payload: payload.elf payload
246  group payload: bootscreen.jpg name=splashscreen.jpg type=splashscreen
247  
248  EC firmware manifest
249  ====================
250  # overrides the cbfsdefault above
251  group payload: ecrw.bin name=ecrw hash=sha256
252  group payload: pdrw.bin name=pdrw hash=sha256
253  ```
254  
255  manifest parsing
256  ----------------
257  The exact BNF is work in progress.
258  
259  Some parser rules are
260  * one line per statement
261  * '#' introduces a command until the end of line
262  
263  Some processing rules
264  * When there’s a conflict (eg. two statements on what to do to a region,
265    overlap, anything that can’t be determined), that is a build error.
266  * the order of statements doesn’t matter, enabling simple addition of more
267    manifests where the need arises.
268