os-collect-config needs a local data source collector for configuration data. This will allow individual elements to drop files into a well-known location to set the initial configuration data of an instance.
There is already a heat_local collector, but that uses a single hard coded path of /var/lib/heat-cfntools/cfn-init-data.
A new collector class will be added to os-collect-config that collects configuration data from JSON files in a configurable list of directories with a well known default of /var/lib/os-collect-config/local-data.
The collector will return a list of pairs of JSON files and their content, sorted by the JSON filename in traditional C collation. For example, if /var/lib/os-collect-config/local-data contains bar.json and foo.json
- [ (‘bar.json’, bar_content),
- (‘foo.json’, foo_content) ]
This new collector will be configured first in DEFAULT_COLLECTORS in os-collect-config. This means all later configured collectors will override any shared configuration keys from the local datasource collector.
Elements making use of this feature can install a json file into the /var/lib/os-collect-config/local-data directory. The os-collect-config element will be responsible for creating the /var/lib/os-collect-config/local-data directory at build time and will create it with 0755 permissions.
There is already a mechanism in os-apply-config to specify arbitrary files to look at for configuration data via setting the OS_CONFIG_FILES environment variable. However, this is not ideal because each call to os-apply-config would have to be prefaced with setting OS_CONFIG_FILES, or it would need to be set globally in the environment (via an environment.d script for instance). As an element developer, this is not clear. Having a robust and clear documented location to drop in configuration data will be simpler.
There is already a collector that reads from local data, but it must be configured to read explicit file paths. This does not scale well if several elements want to each provide local configuration data, in that you’d have to reconfigure os-collect-config itself. We could modify the heat_local collector to read from directories instead, while maintaining backwards compatibility as well, instead of writing a whole new collector. However, given that collectors are pretty simple implementations, I’m proposing just writing a new one, so that they remain generally single purpose with clear goals.
An additional collector will be running as part of os-collect-config, but its execution time should be minimal.
We will need to make clear in documentation when to use this feature versus what to expose in a template or specify via passthrough configuration. Configuration needed at image build time where you need access to those values at instance run time as well are good candidates for using this feature.
Unit tests will be written for the new collector. The new collector will also eventually be tested in CI because there will be an existing element that will configure the persistent data directory to /mnt/state that will make use of this implementation.
The ability of elements to drop configuration data into a well known location should be documented in tripleo-image-elements itself so folks can be made better aware of the functionality.