Generated platform descriptions (.yaml and .json) do not describe final device regions

[Ivan-Velickovic]

Files generated from the kernel build system, such as platform_gen.json and platform_gen.yaml currently list the device regions used by the kernel, e.g for the ZCU102:

devices:
- end: 0xf9010000
  start: 0x80000000
- end: 0xf9020000
  start: 0xf9011000
- end: 0xf9040000
  start: 0xf9021000
- end: 0xfd800000
  start: 0xf9041000
- end: 0x800000000
  start: 0xfd801000
- end: 0x10000000000
  start: 0x880000000
memory:
- end: 0x80000000
  start: 0x0
- end: 0x880000000
  start: 0x800000000

The problem that I am finding is that these device regions are not necessarily used by the kernel as some of them depend on the configuration of the kernel. E.g the GIC VCPU control region is listed, but that is only used (and hence reserved) by the kernel in hypervisor mode.

The C header export works by outputting an #ifdef macro based on a hardware description file, e.g

seL4/tools/hardware.yml

Line 32 in a8eebab

macro: CONFIG_ARM_HYPERVISOR_SUPPORT

Not sure what the best way around it, would appreciate comments before I make a PR with a particular approach.

This currently blocks seL4/microkit#35 as Microkit needs a way of knowing the exact reserved kernel regions as build-time.

[Ivan-Velickovic]

The only use case of these files (that I can find) is in https://github.com/seL4/seL4_tools/blob/master/cmake-tool/helpers/shoehorn.py but it only looks at the memory field and so that's probably why this has not come up before.

[axel-h]

Can't we just add another field condition here that holds an array with the conditions?

[kent-mcleod]

I ran into this in an experiment with generating the initial untyped list in the hardware_gen.py list.
My solution was to pass in the config options to the invocation of the script:

diff --git a/config.cmake b/config.cmake
index 53deca991..63c9dc6bc 100644
--- a/config.cmake
+++ b/config.cmake
@@ -571,6 +571,11 @@ if(DEFINED KernelDTSList AND (NOT "${KernelDTSList}" STREQUAL ""))
                 "${config_schema}" --yaml --yaml-out "${platform_yaml}" --sel4arch
                 "${KernelSel4Arch}" --addrspace-max "${KernelPaddrUserTop}" --json --json-out
                 "${platform_json}"
+                --kernel-config-flags
+                    "CONFIG_PRINTING=${KernelPrinting}"
+                    "CONFIG_ARM_HYPERVISOR_SUPPORT=${KernelARMHypervisorSupport}"
+                    "CONFIG_ARM_SMMU=${KernelArmSMMU}"
+                    "CONFIG_TK1_SMMU=${KernelTk1SMMU}"
             RESULT_VARIABLE error
         )

(The invocation itself had to be moved to the bottom of config.cmake)

Then in the script itself:

index 6eb91f384..260274aad 100644
--- a/tools/hardware_gen.py
+++ b/tools/hardware_gen.py
@@ -74,6 +74,7 @@ if __name__ == '__main__':
                         required=True, type=argparse.FileType('r'))
     parser.add_argument('--hardware-schema', help='YAML file containing schema for hardware config',
                         required=True, type=argparse.FileType('r'))
+    parser.add_argument('--kernel-config-flags', help='list of config params', action='append', nargs='+')
     parser.add_argument('--sel4arch', help='seL4 architecture to generate for',
                         required=True)
     parser.add_argument('--addrspace-max',

diff --git a/tools/hardware/outputs/c_header.py b/tools/hardware/outputs/c_header.py
index 9cab57ef0..96fb3fb4c 100644
--- a/tools/hardware/outputs/c_header.py
+++ b/tools/hardware/outputs/c_header.py

@@ -131,7 +165,7 @@ static const p_region_t BOOT_RODATA avail_p_regs[] = {
 '''


-def get_kernel_devices(tree: FdtParser, hw_yaml: HardwareYaml) -> (List, Dict):
+def get_kernel_devices(tree: FdtParser, hw_yaml: HardwareYaml, kernel_config_dict) -> (List, Dict):
     '''
     Given a device tree and a set of rules, returns a tuple (groups, offsets).

@@ -149,6 +183,9 @@ def get_kernel_devices(tree: FdtParser, hw_yaml: HardwareYaml) -> (List, Dict):
         dev_rule = hw_yaml.get_rule(dev)
         new_regions = dev_rule.get_regions(dev)
         for reg in new_regions:
+            if reg.macro in kernel_config_dict:
+                if kernel_config_dict[reg.macro] != "ON":
+                    continue

@@ -212,8 +352,18 @@ def run(tree: FdtParser, hw_yaml: HardwareYaml, config: Config, args: argparse.N
     if not args.header_out:
         raise ValueError('You need to specify a header-out to use c header output')

+    kernel_config_dict = dict()
+    for option in sum(args.kernel_config_flags, []):
+        name, val = option.split('=')
+        kernel_config_dict[name] = val
+
+

[Ivan-Velickovic]

My solution was to pass in the config options to the invocation of the script:

Thanks. Main problem I see is it breaking once we add something new to hardware.yml or if there is a more complicated macro like CONFIG_A && CONFIG_B. But we can just have assertions in the script so it's obvious when someone extends hardware.yml, I guess there's we shouldn't prematurely generalise...

I'll make a PR with your patch.