1  Linker Script Generation
  2  ========================
  3  :link_to_translation:`zh_CN:[中文]`
  4  
  5  Overview
  6  --------
  7  
  8  There are several :ref:`memory regions<memory-layout>` where code and data can be placed. Code and read-only data are placed by default in flash,
  9  writable data in RAM, etc. However, it is sometimes necessary to change these default placements. For example, it may
 10  be necessary to place critical code in RAM for performance reasons or to place code in RTC memory for use in a wake stub or the ULP coprocessor.
 11  
 12  With the linker script generation mechanism, it is possible to specify these placements at the component level within ESP-IDF. The component presents
 13  information on how it would like to place its symbols, objects or the entire archive. During build the information presented by the components are collected,
 14  parsed and processed; and the placement rules generated is used to link the app.
 15  
 16  Quick Start
 17  ------------
 18  
 19  This section presents a guide for quickly placing code/data to RAM and RTC memory - placements ESP-IDF provides out-of-the-box.
 20  
 21  For this guide, suppose we have the following::
 22  
 23      - components/
 24                      - my_component/
 25                                      - CMakeLists.txt
 26                                      - component.mk
 27                                      - Kconfig
 28                                      - src/
 29                                            - my_src1.c
 30                                            - my_src2.c
 31                                            - my_src3.c
 32                                      - my_linker_fragment_file.lf
 33  
 34  
 35  - a component named ``my_component`` that is archived as library ``libmy_component.a`` during build
 36  - three source files archived under the library, ``my_src1.c``, ``my_src2.c`` and ``my_src3.c`` which are compiled as ``my_src1.o``, ``my_src2.o`` and ``my_src3.o``, respectively
 37  - under ``my_src1.o``, the function ``my_function1`` is defined; under ``my_src2.o``, the function ``my_function2`` is defined
 38  - there exist bool-type config ``PERFORMANCE_MODE`` (y/n) and int type config ``PERFORMANCE_LEVEL`` (with range 0-3) in ``my_component``'s Kconfig
 39  
 40  
 41  Creating and Specifying a Linker Fragment File
 42  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 43  
 44  Before anything else, a linker fragment file needs to be created. A linker fragment file
 45  is simply a text file with a ``.lf`` extension upon which the desired placements will be written.
 46  After creating the file, it is then necessary to present it to the build system. The instructions for the build systems
 47  supported by ESP-IDF are as follows:
 48  
 49  Make
 50  """"
 51  
 52  In the component's ``component.mk`` file, set the variable ``COMPONENT_ADD_LDFRAGMENTS`` to the path of the created linker
 53  fragment file. The path can either be an absolute path or a relative path from the component directory.
 54  
 55  .. code-block:: make
 56  
 57      COMPONENT_ADD_LDFRAGMENTS += my_linker_fragment_file.lf
 58  
 59  CMake
 60  """""
 61  
 62  In the component's ``CMakeLists.txt`` file, specify argument ``LDFRAGMENTS`` in the ``idf_component_register`` call.
 63  The value of ``LDFRAGMENTS`` can either be an absolute path or a relative path from the component directory to the
 64  created linker fragment file.
 65  
 66  .. code-block:: cmake
 67  
 68      # file paths relative to CMakeLists.txt
 69      idf_component_register(...
 70                             LDFRAGMENTS "path/to/linker_fragment_file.lf" "path/to/another_linker_fragment_file.lf"
 71                             ...
 72                             )
 73  
 74  
 75  Specifying placements
 76  ^^^^^^^^^^^^^^^^^^^^^
 77  
 78  It is possible to specify placements at the following levels of granularity:
 79  
 80      - object file (``.obj`` or ``.o`` files)
 81      - symbol (function/variable)
 82      - archive (``.a`` files)
 83  
 84  .. _ldgen-placing-object-files :
 85  
 86  Placing object files
 87  """"""""""""""""""""
 88  
 89  Suppose the entirety of ``my_src1.o`` is performance-critical, so it is desirable to place it in RAM.
 90  On the other hand, the entirety of ``my_src2.o`` contains symbols needed coming out of deep sleep, so it needs to be put under RTC memory.
 91  In the the linker fragment file, we can write:
 92  
 93  .. code-block:: none
 94  
 95      [mapping:my_component]
 96      archive: libmy_component.a
 97      entries:
 98          my_src1 (noflash)     # places all my_src1 code/read-only data under IRAM/DRAM
 99          my_src2 (rtc)         # places all my_src2 code/ data and read-only data under RTC fast memory/RTC slow memory
100  
101  What happens to ``my_src3.o``? Since it is not specified, default placements are used for ``my_src3.o``. More on default placements
102  :ref:`here<ldgen-default-placements>`.
103  
104  Placing symbols
105  """"""""""""""""
106  
107  Continuing our example, suppose that among functions defined under ``object1.o``, only ``my_function1`` is performance-critical; and under ``object2.o``,
108  only ``my_function2`` needs to execute after the chip comes out of deep sleep. This could be accomplished by writing:
109  
110  .. code-block:: none
111  
112      [mapping:my_component]
113      archive: libmy_component.a
114      entries:
115          my_src1:my_function1 (noflash)
116          my_src2:my_function2 (rtc)
117  
118  The default placements are used for the rest of the functions in ``my_src1.o`` and ``my_src2.o`` and the entire ``object3.o``. Something similar
119  can be achieved for placing data by writing the variable name instead of the function name, like so::
120  
121         my_src1:my_variable (noflash)
122  
123  .. warning::
124  
125      There are :ref:`limitations<ldgen-symbol-granularity-placements>` in placing code/data at symbol granularity. In order to ensure proper placements, an alternative would be to group
126      relevant code and data into source files, and :ref:`use object-granularity placements<ldgen-placing-object-files>`.
127  
128  Placing entire archive
129  """""""""""""""""""""""
130  
131  In this example, suppose that the entire component archive needs to be placed in RAM. This can be written as:
132  
133  .. code-block:: none
134  
135      [mapping:my_component]
136      archive: libmy_component.a
137      entries:
138          * (noflash)
139  
140  Similarly, this places the entire component in RTC memory:
141  
142  .. code-block:: none
143  
144      [mapping:my_component]
145      archive: libmy_component.a
146      entries:
147          * (rtc)
148  
149  Configuration-dependent placements
150  """"""""""""""""""""""""""""""""""
151  
152  Suppose that the entire component library should only have special placement when a certain condition is true; for example, when ``CONFIG_PERFORMANCE_MODE == y``.
153  This could be written as:
154  
155  .. code-block:: none
156  
157      [mapping:my_component]
158      archive: libmy_component.a
159      entries:
160          if PERFORMANCE_MODE = y:
161              * (noflash)
162          else:
163              * (default)
164  
165  For a more complex config-dependent placement, suppose the following requirements: when ``CONFIG_PERFORMANCE_LEVEL == 1``, only ``object1.o`` is put in RAM;
166  when ``CONFIG_PERFORMANCE_LEVEL == 2``, ``object1.o`` and ``object2.o``; and when ``CONFIG_PERFORMANCE_LEVEL == 3`` all object files under the archive
167  are to be put into RAM. When these three are false however, put entire library in RTC memory. This scenario is a bit contrived, but,
168  it can be written as:
169  
170  .. code-block:: none
171  
172      [mapping:my_component]
173      archive: libmy_component.a
174      entries:
175          if PERFORMANCE_LEVEL = 1:
176              my_src1 (noflash)
177          elif PERFORMANCE_LEVEL = 2:
178              my_src1 (noflash)
179              my_src2 (noflash)
180          elif PERFORMANCE_LEVEL = 3:
181              my_src1 (noflash)
182              my_src2 (noflash)
183              my_src3 (noflash)
184          else:
185              * (rtc)
186  
187  Nesting condition-checking is also possible. The following is equivalent to the snippet above:
188  
189  .. code-block:: none
190  
191      [mapping:my_component]
192      archive: libmy_component.a
193      entries:
194          if PERFORMANCE_LEVEL <= 3 && PERFORMANCE_LEVEL > 0:
195              if PERFORMANCE_LEVEL >= 1:
196                  object1 (noflash)
197                  if PERFORMANCE_LEVEL >= 2:
198                      object2 (noflash)
199                      if PERFORMANCE_LEVEL >= 3:
200                          object2 (noflash)
201          else:
202              * (rtc)
203  
204  .. _ldgen-default-placements:
205  
206  The 'default' placements
207  ^^^^^^^^^^^^^^^^^^^^^^^^
208  
209  Up until this point, the term  'default placements' has been mentioned as fallback placements when the
210  placement rules ``rtc`` and ``noflash`` are not specified. It is important to note that the tokens ``noflash`` or ``rtc`` are not merely keywords, but are actually
211  entities called fragments, specifically :ref:`schemes<ldgen-scheme-fragment>`.
212  
213  In the same manner as ``rtc`` and ``noflash`` are schemes, there exists a ``default`` scheme which defines what the default placement rules should be.
214  As the name suggests, it is where code and data are usually placed, i.e. code/constants is placed in flash, variables
215  placed in RAM, etc.  More on the default scheme :ref:`here<ldgen-default-scheme>`.
216  
217  .. note::
218      For an example of an ESP-IDF component using the linker script generation mechanism, see :component_file:`freertos/CMakeLists.txt`.
219      ``freertos`` uses this to place its object files to the instruction RAM for performance reasons.
220  
221  This marks the end of the quick start guide. The following text discusses the internals of the mechanism in a little bit more detail.
222  The following sections should be helpful in creating custom placements or modifying default behavior.
223  
224  Linker Script Generation Internals
225  ----------------------------------
226  
227  Linking is the last step in the process of turning C/C++ source files into an executable. It is performed by the toolchain's linker, and accepts
228  linker scripts which specify code/data placements, among other things. With the linker script generation mechanism, this process is no different, except
229  that the linker script passed to the linker is dynamically generated from: (1) the collected :ref:`linker fragment files<ldgen-linker-fragment-files>` and
230  (2) :ref:`linker script template<ldgen-linker-script-template>`.
231  
232  .. note::
233  
234      The tool that implements the linker script generation mechanism lives under :idf:`tools/ldgen`.
235  
236  .. _ldgen-linker-fragment-files :
237  
238  Linker Fragment Files
239  ^^^^^^^^^^^^^^^^^^^^^
240  
241  As mentioned in the quick start guide, fragment files are simple text files with the ``.lf`` extension containing the desired placements. This is a simplified
242  description of what fragment files contain, however. What fragment files actually contain are 'fragments'. Fragments are entities which contain pieces of information which, when put together, form
243  placement rules that tell where to place sections of object files in the output binary. There are three types of fragments: :ref:`sections<ldgen-sections-fragment>`,
244  :ref:`scheme<ldgen-scheme-fragment>` and :ref:`mapping<ldgen-mapping-fragment>`.
245  
246  Grammar
247  """""""
248  
249  The three fragment types share a common grammar:
250  
251  .. code-block:: none
252  
253      [type:name]
254      key: value
255      key:
256          value
257          value
258          value
259          ...
260  
261  - type: Corresponds to the fragment type, can either be ``sections``, ``scheme`` or ``mapping``.
262  - name: The name of the fragment, should be unique for the specified fragment type.
263  - key, value: Contents of the fragment; each fragment type may support different keys and different grammars for the key values.
264  
265  .. note::
266  
267      In cases where multiple fragments of the same type and name are encountered, an exception is thrown.
268  
269  .. note::
270  
271      The only valid characters for fragment names and keys are alphanumeric characters and underscore.
272  
273  
274  .. _ldgen-condition-checking :
275  
276  **Condition Checking**
277  
278  Condition checking enable the linker script generation to be configuration-aware. Depending on whether expressions involving configuration values
279  are true or not, a particular set of values for a key can be used. The evaluation uses ``eval_string`` from kconfiglib package
280  and adheres to its required syntax and limitations. Supported operators are as follows:
281  
282      - comparison
283          - LessThan ``<``
284          - LessThanOrEqualTo ``<=``
285          - MoreThan ``>``
286          - MoreThanOrEqualTo ``>=``
287          - Equal ``=``
288          - NotEqual ``!=``
289      - logical
290          - Or ``||``
291          - And ``&&``
292          - Negation ``!``
293      - grouping
294          - Parenthesis ``()``
295  
296  Condition checking behaves as you would expect an ``if...elseif/elif...else`` block in other languages. Condition-checking is possible
297  for both key values and entire fragments. The two sample fragments below are equivalent:
298  
299  .. code-block:: none
300  
301      # Value for keys is dependent on config
302      [type:name]
303      key_1:
304          if CONDITION = y:
305              value_1
306          else:
307              value_2
308      key_2:
309          if CONDITION = y:
310              value_a
311          else:
312              value_b
313  
314  .. code-block:: none
315  
316      # Entire fragment definition is dependent on config
317      if CONDITION = y:
318          [type:name]
319          key_1:
320              value_1
321          key_2:
322              value_b
323      else:
324          [type:name]
325          key_1:
326              value_2
327          key_2:
328              value_b
329  
330  
331  **Comments**
332  
333  Comment in linker fragment files begin with ``#``. Like in other languages, comment are used to provide helpful descriptions and documentation
334  and are ignored during processing.
335  
336  Compatibility with ESP-IDF v3.x Linker Script Fragment Files
337  """"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
338  
339  ESP-IDF v4.0 brings some changes to the linker script fragment file grammar:
340  
341  - indentation is enforced and improperly indented fragment files generate a parse exception; this was not enforced in the old version but previous documentation and examples demonstrates properly indented grammar
342  - move to ``if...elif...else`` structure for conditionals, with the ability to nest checks and place entire fragments themselves inside conditionals
343  - mapping fragments now requires a name like other fragment types
344  
345  Linker script generator should be able to parse ESP-IDF v3.x linker fragment files that are indented properly (as demonstrated by
346  the ESP-IDF v3.x version of this document). Backward compatibility with the previous mapping fragment grammar (optional
347  name and the old grammar for conditionals) has also been retained but with a deprecation warning. Users should switch to the newer grammar discussed
348  in this document as support for the old grammar is planned to be removed in the future.
349  
350  Note that linker fragment files using the new ESP-IDF v4.0 grammar is not supported on ESP-IDF v3.x, however.
351  
352  Types
353  """""
354  
355  .. _ldgen-sections-fragment :
356  
357  **Sections**
358  
359  Sections fragments defines a list of object file sections that the GCC compiler emits. It may be a default section (e.g. ``.text``, ``.data``) or
360  it may be user defined section through the ``__attribute__`` keyword.
361  
362  The use of an optional '+' indicates the inclusion of the section in the list, as well as sections that start with it. This is the preferred method over listing both explicitly.
363  
364  .. code-block:: none
365  
366      [sections:name]
367      entries:
368          .section+
369          .section
370          ...
371  
372  Example:
373  
374  .. code-block:: none
375  
376      # Non-preferred
377      [sections:text]
378      entries:
379          .text
380          .text.*
381          .literal
382          .literal.*
383  
384      # Preferred, equivalent to the one above
385      [sections:text]
386      entries:
387          .text+              # means .text and .text.*
388          .literal+           # means .literal and .literal.*
389  
390  .. _ldgen-scheme-fragment :
391  
392  **Scheme**
393  
394  Scheme fragments define what ``target`` a sections fragment is assigned to.
395  
396  .. code-block:: none
397  
398      [scheme:name]
399      entries:
400          sections -> target
401          sections -> target
402          ...
403  
404  Example:
405  
406  .. code-block:: none
407  
408      [scheme:noflash]
409      entries:
410          text -> iram0_text          # the entries under the sections fragment named text will go to iram0_text
411          rodata -> dram0_data        # the entries under the sections fragment named rodata will go to dram0_data
412  
413  .. _ldgen-default-scheme:
414  
415  The ``default`` scheme
416  
417  There exists a special scheme with the name ``default``. This scheme is special because catch-all placement rules are generated from
418  its entries. This means that, if one of its entries is ``text -> flash_text``, the placement rule
419  
420  .. code-block:: none
421  
422      *(.literal .literal.* .text .text.*)
423  
424  will be generated for the target ``flash_text``.
425  
426  These catch-all rules then effectively serve as fallback rules for those whose mappings were not specified.
427  
428  
429  The ``default scheme`` is defined in :component_file:`{IDF_TARGET_PATH_NAME}/ld/{IDF_TARGET_PATH_NAME}_fragments.lf`. The ``noflash`` and ``rtc`` scheme fragments which are
430  built-in schemes referenced in the quick start guide are also defined in this file.
431  
432  
433  .. _ldgen-mapping-fragment :
434  
435  **Mapping**
436  
437  Mapping fragments define what scheme fragment to use for mappable entities, i.e. object files, function names, variable names, archives.
438  
439  .. code-block:: none
440  
441      [mapping:name]
442      archive: archive                # output archive file name, as built (i.e. libxxx.a)
443      entries:
444          object:symbol (scheme)      # symbol granularity
445          object (scheme)             # object granularity
446          * (scheme)                  # archive granularity
447  
448  There are three levels of placement granularity:
449  
450      - symbol: The object file name and symbol name are specified. The symbol name can be a function name or a variable name.
451      - object: Only the object file name is specified.
452      - archive: ``*`` is specified, which is a short-hand for all the object files under the archive.
453  
454  To know what an entry means, let us expand a sample object-granularity placement:
455  
456  .. code-block:: none
457  
458      object (scheme)
459  
460  Then expanding the scheme fragment from its entries definitions, we have:
461  
462  .. code-block:: none
463  
464      object (sections -> target,
465              sections -> target,
466              ...)
467  
468  Expanding the sections fragment with its entries definition:
469  
470  .. code-block:: none
471  
472      object (.section,      # given this object file
473              .section,      # put its sections listed here at this
474              ... -> target, # target
475  
476              .section,
477              .section,      # same should be done for these sections
478              ... -> target,
479  
480              ...)           # and so on
481  
482  Example:
483  
484  .. code-block:: none
485  
486      [mapping:map]
487      archive: libfreertos.a
488      entries:
489          * (noflash)
490  
491  .. _ldgen-symbol-granularity-placements :
492  
493  On Symbol-Granularity Placements
494  """"""""""""""""""""""""""""""""
495  
496  Symbol granularity placements is possible due to compiler flags ``-ffunction-sections`` and ``-ffdata-sections``. ESP-IDF compiles with these flags by default.
497  If the user opts to remove these flags, then the symbol-granularity placements will not work. Furthermore, even with the presence of these flags, there are still other limitations to keep in mind
498  due to the dependence on the compiler's emitted output sections.
499  
500  For example, with ``-ffunction-sections``, separate sections are emitted for each function; with section names predictably constructed i.e. ``.text.{func_name}``
501  and ``.literal.{func_name}``. This is not the case for string literals within the function, as they go to pooled or generated section names.
502  
503  With ``-fdata-sections``, for global scope data the compiler predictably emits either ``.data.{var_name}``, ``.rodata.{var_name}`` or ``.bss.{var_name}``; and so ``Type I`` mapping entry works for these.
504  However, this is not the case for static data declared in function scope, as the generated section name is a result of mangling the variable name with some other information.
505  
506  .. _ldgen-linker-script-template :
507  
508  Linker Script Template
509  ^^^^^^^^^^^^^^^^^^^^^^
510  
511  The linker script template is the skeleton in which the generated placement rules are put into. It is an otherwise ordinary linker script, with a specific marker syntax
512  that indicates where the generated placement rules are placed.
513  
514  To reference the placement rules collected under a ``target`` token, the following syntax is used:
515  
516  .. code-block:: none
517  
518      mapping[target]
519  
520  Example:
521  
522  The example below is an excerpt from a possible linker script template. It defines an output section ``.iram0.text``, and inside is a marker referencing
523  the target ``iram0_text``.
524  
525  .. code-block:: none
526  
527      .iram0.text :
528      {
529          /* Code marked as runnning out of IRAM */
530          _iram_text_start = ABSOLUTE(.);
531  
532          /* Marker referencing iram0_text */
533          mapping[iram0_text]
534  
535          _iram_text_end = ABSOLUTE(.);
536      } > iram0_0_seg
537  
538  Suppose the generator collected the fragment definitions below:
539  
540  .. code-block:: none
541  
542      [sections:text]
543          .text+
544          .literal+
545  
546      [sections:iram]
547          .iram1+
548  
549      [scheme:default]
550      entries:
551          text -> flash_text
552          iram -> iram0_text
553  
554      [scheme:noflash]
555      entries:
556          text -> iram0_text
557  
558      [mapping:freertos]
559      archive: libfreertos.a
560      entries:
561          * (noflash)
562  
563  Then the corresponding excerpt from the generated linker script will be as follows:
564  
565  .. code-block:: c
566  
567      .iram0.text :
568      {
569          /* Code marked as runnning out of IRAM */
570          _iram_text_start = ABSOLUTE(.);
571  
572          /* Placement rules generated from the processed fragments, placed where the marker was in the template */
573          *(.iram1 .iram1.*)
574          *libfreertos.a:(.literal .text .literal.* .text.*)
575  
576          _iram_text_end = ABSOLUTE(.);
577      } > iram0_0_seg
578  
579  ``*libfreertos.a:(.literal .text .literal.* .text.*)``
580  
581      Rule generated from the entry ``* (noflash)`` of the ``freertos`` mapping fragment. All ``text`` sections of all
582      object files under the archive ``libfreertos.a`` will be collected under the target ``iram0_text`` (as per the ``noflash`` scheme)
583      and placed wherever in the template ``iram0_text`` is referenced by a marker.
584  
585  ``*(.iram1 .iram1.*)``
586  
587      Rule generated from the default scheme entry 	``iram -> iram0_text``. Since the default scheme specifies an ``iram -> iram0_text`` entry,
588      it too is placed wherever ``iram0_text`` is referenced by a marker. Since it is a rule generated from the default scheme, it comes first
589      among all other rules collected under the same target name.
590  
591      The linker script template currently used is :component_file:`{IDF_TARGET_PATH_NAME}/ld/{IDF_TARGET_PATH_NAME}.project.ld.in`, specified by the ``{IDF_TARGET_PATH_NAME}`` component; the
592      generated output script is put under its build directory.