If kernel_images.boot_image_outs is set, it assumes a list of defaults.
Clarify that vendor-bootconfig.img is always added on build_vendor_boot,
because it doesn't know if VENDOR_BOOTCONFIG is actually set or not.

Bug: 361735034
Change-Id: I51588b63bec52c8da7f62ee44d203ce6e7eded49

It was previously incorrectly missed. If the build config
does NOT specify RAMDISK_COMPRESS/RAMDISK_EXT, building boot images
may break.

This is hit when
- kernel_images(build_initramfs = True)
- INITRAMFS_VENDOR_RAMDISK_FRAGMENT_NAME is empty
in which case build_boot_images will try to build
ramdisk.{ramdisk_compress}; but it may be unable to do so when
RAMDISK_COMPRESS/RAMDISK_DECOMPRESS/RAMDISK_EXT is not set in the build
config but only in BUILD files.

Bug: 361733833
Change-Id: If1fd54a9197404b2e4112e0b2d8a169caf2742af

Commit d08605f4 ("kleaf: Add support for specifying modules for
recovery/charger") accidentally made it so that kleaf always tries
to generate the vendor[_kernel]_boot.modules.* lists, even when
neither vendor_boot.img or vendor_kernel_boot.img need to be built.

Fix this by only attempting to generate those artifacts if either
vendor_boot.img or vendor_kernel_boot.img need to be built.

Bug: 324304750
Fixes: d08605f4 ("kleaf: Add support for specifying modules for recovery/charger")
Signed-off-by: Isaac J. Manjarres <isaacmanjarres@google.com>
(cherry picked from https://android-review.googlesource.com/q/commit:100076d823899780662a6fdec71f92be35f6d9f7)
Merged-In: I31ec16fc543e55f33319e017c7d5976e825f581d
Change-Id: I31ec16fc543e55f33319e017c7d5976e825f581d

* Additionally include ddk_module as suggestion for build external
  kernel modules.

Bug: 306122000
Change-Id: I15eaefc4a205c23c22b3db26d9d1e31654507e56
Signed-off-by: Ulises Mendez Martinez <umendez@google.com>
(cherry picked from commit d6e4d254)

* This is done by keeping a local copy of the make generated one, for
  each ddk_module or kernel_module. For kernel_module_group or
  kernel_modules_install this becomes a depset which is traversed in
  postorder (dependencies first) when needed.

* With this change kernel_images (initiramfs, (system|vendor)_dlkm) can
  control whether to use this new approach or keep the old behavior with
  the variable `create_modules_order` (set to True by default).

Bug: 306122000
Change-Id: Ic636c2da8d77e6c25e7eb9c8f6de68904828a9ad
Signed-off-by: Ulises Mendez Martinez <umendez@google.com>
(cherry picked from commit 1d3379dd)

Currently, the same modules.load file is used for normal boot,
recovery/fastbootd and charger modes during first-stage init. This
means that during normal boot, the modules that are needed for
recovery and charger modes are loaded during first-stage init, which
is not required.

Add support for specifying a list of modules that are required for
booting into recovery or charger mode. This introduces two new
attributes to the kernel_images() macro called: modules_recovery_list
and modules_charger_list. Each attribute points to a list of modules
that are needed to boot into each mode respectively. This list can
be generated dynamically by concatenating the list of vendor_boot or
vendor_kernel_boot modules required for first stage init, along with
the list of modules needed to boot into each mode.

Defining these attributes also produces a file named
modules.load.[recovery/charger] in the initramfs, which init uses
instead of modules.load when booting into recovery or charger mode
respectively to load the first-stage init and recovery and charger
modules. In case of normal boot, init will use the modules.load
file, which will only contain the modules needed for first
stage init, and the recovery and charger modules will be loaded
from the vendor_dlkm partition during second-stage init.

Bug: 266752750
Change-Id: I11503b11683ef64cf0933b8641959ea5acd63ab1
Signed-off-by: Isaac J. Manjarres <isaacmanjarres@google.com>
(cherry picked from commit b9ba4b67)
[isaacmanjarres: Re-wrote some of the commit message/implemented it
as part of kleaf. Also added tests for the new attributes.]
(cherry picked from commit d08605f4)
[isaacmanjarres: Resolved trivial merge conflicts when backporting
from main.]

Generate flatten images if kernel_images have
build_system_dlkm_flatten is set True. These
images don't have the `uname -r` in the directory
structure and all modules are available as
lib/modules/*.ko with modules.load and modules.dep
depmod artifacts.

Generates images as follow:

If system_dlkm_fs_type is set:
  system_dlkm.flatten.<system_dlkm_fs_type>.img

If system_dlkm_fs_types is set:
  system_dlkm.flatten.<fs_type>.img for each fs types

By default when no fs type is specified:
  system_dlkm.flatten.ext4.img

Bug: 301178664
Test: Manual image verification with different options
Test: TH
Change-Id: I4b7815d131d3d26adaa35c4550e76db71aa33936
Signed-off-by: Ramji Jiyani <ramjiyani@google.com>

Introduces system_dlkm_fs_types list attribute for the
kernel_images rule to build the system_dlkm partition
images for all filesystem provided in the list.

Deprecates the system_dlkm_fs_type but gives it priority
for backward compitability resons.

Behavior based on values of system_dlkm_fs_type &
system_dlkm_fs_types:

    Both specified: Fail with deprecation message
    Both not specified: system_dlkm.img with ext4
	This is same as current default behavior when
	system_dlkm_fs_type is not specified.
    system_dlkm_fs_type="fs": system_dlkm.img with given fs
	This is same as current behavior.
    system_dlkm_fs_types=[list_of_filesystems]: system_dlkm.fs.img
	for each file system type in list_of_filesystems

Bug: 297407875
Test: TH, Manual with all permutations & combinations
Change-Id: Ie8ee880dbbc41576603a1ab1b56f0307a6ab2e90
Signed-off-by: Ramji Jiyani <ramjiyani@google.com>
(cherry picked from commit b0fedfdc)

This ensures that these extensions, which are implementation
details of Kleaf, doesn't get loaded outside of kleaf.

The public symbols are re-exported in kernel.bzl.

If this breaks a build, one may use
--nocheck_bzl_visibility to temporarily disable
the check.

Also fix buildifier warnings.

gki_artifacts.bzl are not included in this change due to an internal usage.

Test: TH
Bug: 283857454
Change-Id: If8139982e0e55a72729fbc058ec99965d069195e

William McVicker authored 1 year ago and William McVicker committed 1 year ago

If `vendor_dlkm_archive = True` is set, then the tarball
`vendor_dlkm_staging_archive.tar.gz` is created. This can be used to
easily get the vendor_dlkm modules without have to mount the vendor_dlkm
ext4 image.

Bug: 281621377
Change-Id: I28023de9c42243f3859078f9e424c84cf7400769

After this change, if dedup_dlkm_modules is set:

- When building vendor_dlkm, the staging directory from
  system_dlkm is extracted to MODULES_STAGING_DIR beforehand,
  so that depmod can find them. This allows modules from
  vendor_dlkm to link to modules from system_dlkm.
- system_dlkm.modules.load is concatenated to $DIST_DIR/modules.load
  (after modules.load from vendor_boot) so that the list of modules
  from system_dlkm are excluded from vendor_dlkm.modules.load list.

This means that after this change, if dedup_dlkm_modules is set
for a build, a device effectively enables the feature that allows
vendor_dlkm modules to depend on system_dlkm modules.

If dedup_dlkm_modules is set, additional changes are required
on the device so that userspace loads system_dlkm modules before
loading vendor_dlkm modules.

Note that even with this change, vendor_boot modules and
system_dlkm modules themselves (the *.ko files) aren't excluded
from vendor_dlkm.img built from the kernel source tree if
vendor_dlkm_modules_list specifies .* (all modules). They are only
excluded from modules.load. However, the images from the kernel
source tree are for development purposes. It is expected that
the system_dlkm / vendor_dlkm images are built from the platform
source tree, excluding modules not in the corresponding modules.load
file.

A subsequent optimization would be to exclude vendor_boot / system_dlkm
modules from this "debug" vendor_dlkm image built from the kernel
source tree. This optimization is not covered by this change.

Bug: 272523944
Bug: 272525097

Change-Id: I86197ed03bc32563399f69e8f93eaefc585a9464

Enable users to pass in etc files that need to be installed to
vendor_dlkm.img /etc folder.

Bug: 270603150
Bug: 271443724
Test: build kernel and check vendor_dlkm.img and dist folder

Change-Id: I63a2f348ced7681180e26647bfdb3d4c09de735b

system_dlkm and vendor_dlkm can be ext4 or erofs images
as these are supported read only filesystem in platform
builds.

Add an option to choose between ext4/erofs for these
partitions for the kernel's development only images to
match patform choices with ext4 as the default if choice
is not provided.

Choice will be passed as environemental variables to build_utils.sh:
SYSTEM_DLKM_FS_TYPE=["ext4" | "erofs"] for build_system_dlkm_image
VENDOR_DLKM_FS_TYPE=["ext4" | "erofs"] for build_vendor_dlkm_image

Bug: 261617174
Bug: 232430739
Test: TH
Change-Id: I4a37e9d0116bb4b2f38fc119011a441d7c73e9f7
Signed-off-by: Ramji Jiyani <ramjiyani@google.com>

* This is in favor of declaring the setup using Kleaf.
* This should have a lower impact as in both cases the
 default is always `//tools/mkbootimg:mkbootimg.py`

Bug: 260776236
Change-Id: I462894827e8b3fef8bbb1f9a23b35eb43e475cdf
Signed-off-by: Ulises Mendez Martinez <umendez@google.com>

Building the boot image does not necessary require
the existence of initramfs.

Test: TH
Bug: 260031024
Change-Id: I06f3cab03066009c5d4ccad30392e5b2dfb8d858

Bug: 260012551
Change-Id: Ie6ebe4f3c239eb43b8565973979d07c43e1a647a
Signed-off-by: Wang Chen <wangchen20@iscas.ac.cn>
Signed-off-by: Alistair Delva <adelva@google.com>

* For backwards compatibility, options provided trough
  `kernel_image` rule will take precedence over the
  ones specified in build.config.

Bug: 236012223
Signed-off-by: Ulises Mendez Martinez <umendez@google.com>
Change-Id: I6f26ee12ffd1e9198fdeef4f794780de2188cc64

* This is so kleaf support natively the following configs:
  * LZ4_RAMDISK
  * LZ4_RAMDISK_COMPRESS_ARGS

* The options are still falling back to the config values.

Bug: 236012223
Signed-off-by: Ulises Mendez Martinez <umendez@google.com>
Change-Id: Iac60252b368da1f06ef3b49b3fb372dd729e9aea

* Buildifier provides linter recommendations when
 `--lint=warn` is used, so adding the ones for
 these  files before working with them so the
 cleanup changes are separated from the actual
 changes.

Bug: 236012223
Bug: 252888476
Signed-off-by: Ulises Mendez Martinez <umendez@google.com>
Change-Id: I5d1c5a8bfd99d2efbe8f32420225507ac8296d31

This passes things like visibility and tags to internal targets.

Test: TH
Bug: 244786402

Change-Id: I28a33f7d1351d031fdc04336f6077a9bf3023529

vendor_kernel_boot.img would leave empty in the field of bootconfig.
Only copy vendor-bootconfig.img when building vendor_boot.img to avoid
building error.

Bug: 236355852
Signed-off-by: Lucas Wei <lucaswei@google.com>
Change-Id: I3569568801e953d52285640109a18e61714faaaa

system_dlkm_image produces staging_archive and modules.load
files. Also download them and put them as part of distribution
when building against downloaded prebuilts.

With system_dlkm_staging_archive.tar.gz from GKI
piped to the device's system_dlkm rule, instead
of creating the staging directory, unarchive the archive
and build system_dlkm from there.

As part of this change, kernel_images now also
accept these arguments:
- system_dlkm_modules_list
- system_dlkm_modules_blocklist
- system_dlkm_props

... which corresponds to the upper-case version of the build configs,
respectively. Their semantics are the same as the build configs.
Unlike vendor_dlkm_*:
- system_dlkm_modules_list falls back to modules_list if not set
- system_dlkm_modules_blocklist falls back to modules_blocklist if not set

Test: build slider_images_system_dlkm_image with build_system_dlkm=True
Bug: 233401974
Bug: 238786592

Change-Id: Ie9a2073d571b23eee58764ef9ced0976e5fa9622
Signed-off-by: Yifan Hong <elsk@google.com>

Unlike MODULES_OPTIONS in build.sh, modules_options in
Kleaf points to a file that contains the lines.

Fix comments and build rules that previously assume
it contains strings.

Test: bazel test //common:kernel_aarch64_tests
Fixes: 241162984
Change-Id: Ie3ab2544f91cbcec55e5af390fc06eba400add14

The filesystem format is an implementation detail.

Test: none
Change-Id: I241b038760b6d5bdb9104f0282cadb601a500110

After we supporting build vendor_kernel_boot image for kernel
artifacts, we should also update file name of *.modules.load
with its related image name.

Bug: 234554094
Test: TH
Change-Id: Ia78f318bfe7a203046ca479c92d4f26947b09402
Signed-off-by: Yifan Hong <elsk@google.com>

This makes kernel.bzl smaller.

Test: TH
Test: bazel build //common:kernel_aarch64_dist //gs/google-modules/soc-modules:slider_dist --nobuild
Test: bazel build //build/kernel/kleaf:docs
Bug: 233156052
Change-Id: I28ba7ca430070b57267079e24ebe0e1477f0d9fc
Signed-off-by: Yifan Hong <elsk@google.com>