Machine Readable Sample Config¶
Currently the sample configs generated by oslo.config are targeted for consumption by humans. However, many deployment tools would benefit from a sample config that is machine readable.
All deployment tools have to solve a similar problem: how to generate the config files for each service at deployment time. There are various approaches to this today, including manually updated templates, semi-automatically generated templates, and using an ini file tool to update the existing sample config for each project.
These approaches all have various drawbacks. Template approaches generally require some level of tedious human editing to template all of the values the deployment tool needs to set. Ini file tools are not aware of the sample config format and thus are only able to set values in the correct section, which usually separates them from the sample config documentation of the option. None of these approaches make it easy for a deployment tool to provide a user-friendly interface to the config options.
The problem with the current ini format sample config in this case is that it is not easily machine-readable. The content is all written in commented blocks which would have to be parsed by a deployment tool. Since that comment format is not guaranteed to remain the same, it would require ongoing maintenance by each deployment tool using this approach.
The proposal is to write machine-readable sample config files that output the same data as the existing ini files, but in a YAML or JSON format that can be more easily consumed by deployment tools.
Example of proposed output:
generator_options: args: --config-file=etc/nova/nova-config-generator.conf output_file: etc/nova/nova.conf.yaml wrap_width: 80 namespace: - nova.conf - oslo.log ... summarize: False # minimal intentionally omitted as I don't think it makes sense in # this context [more generator details can be added as desired] options: DEFAULT: image_service: # The list of parameters is for example only. The actual output # can include anything deployment tools would find useful. default: nova.image.glance.GlanceImageService documentation: Service used to search for and retrieve images. type: string required: True deprecated: False deprecated_reason: blabla deprecated_for_removal: False deprecated_since: XXX deprecated_name: old_parameter [all other parameters passed to the opt constructor] new_opt: deprecated_name: old_opt ... ... conductor: workers: (...) ... ... deprecated_options: DEFAULT: old_opt: default: foo documentation: Example deprecated opt type: string replacement_name: new_opt replacement_group: DEFAULT ...
This structure would contain all of the details needed to write a well-formatted config file, and it would be easier to inject values in the appropriate place because we would be starting from the actual opt data instead of an ini file full of comments. It should require little to no manual maintenance because it would be generated from sample config data already provided by each project.
Another suggested change in this format from the existing generated sample config is to write deprecated opts as first-class citizens rather than just as a deprecated_name/group on the new opt. This way the structure will still align with the previous config, which should make it easier to transition from release to release. These opts would, of course, be marked deprecated so it is clear that they should no longer be used. An example of this is included above.
Note on the implementation: In order to easily support both YAML and JSON formats, I think the generator should work with nested dicts that can simply be output using the appropriate module. Since YAML and JSON formatters already exist we shouldn’t need to reinvent those wheels.
Some of the existing methods for doing this and their drawbacks are discussed above in the problems section. These methods work, but there is a lot of duplication of effort across all deployment projects so it is not desirable to continue this way.
Each project could automate their config file generation, similar to the OpenStack Helm tool in the References section, but there again we have a duplication problem that oslo.config can solve.
Impact on Existing APIs¶
We would need to add a –format parameter to the oslo-config-generator so users can generate sample configs with the new format(s). This would default to the existing ini-style format for backwards compatibility.
Should be none. For the most part this is just a new format for data that was already being generated. One possible exception would be capturing the parameters to the config generator, but I don’t see anything in there that should be sensitive.
Generating the new file formats will take additional time, but by default only the existing format will be created so there should be no change.
Outside of the –format CLI param, no new config opts are proposed.
None. Developers are already exposing config opts to the config generator, and this is simply adding another use for that.
This functionality would be unit tested in the same way as the existing config generator. We would need to add YAML/JSON scenarios to the unit tests.
- Primary assignee:
- Other contributors:
Target Milestone for completion: We would like to complete this work in Pike, probably for consumption by deployment tools in the Queens cycle.
Implement YAML output
Implement JSON output
Add test scenarios for new formats
Not being incubated.
Anticipated API Stabilization¶
This work is targeted at deployment tools, so there should be little to no user-facing documentation changes required. We would want to reference the existence of these new sample formats in the oslo.config documentation.
The format and structure of the output files will need to be documented for deployment tool consumption.
Cross-Deployment Tool PTG Session, specifically the Problem #1 section.
This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode