Now users manually should configure most important hadoop configurations. It would be friendly to provide advices about cluster configurations for users.
Now users manually should configure most important hadoop configs, but it’s required to have advanced knowledge in Hadoop. Most configs are complicated and not all users know them. We can provide advices about cluster configuration and automatically configure few basic configs, that will improve user experience. Created workaround can extended in future with new confiuguration and advices.
It’s proposed to add calculator, which would automatically configure most important configurations in dependency cluster specification: available disk space, ram, cpu, and so on. Such calculator already implemented in Ambari (see  and ), and we can use it as well. We should have ability to switch off autoconfiguration and if user also manually configured some hadoop config, autoconfiguration also will not be applied.
The following list of configs will be configured, using formulas from  and :
Also as a simple example we can autoconfigure before cluster validation dfs.replication if amout of datanodes less than default value.
Also it’s required to add new plugin SPI method recommend_configs which will autoconfigure cluster configs.
It’s required to add new column use_autoconfig to cluster, cluster_template, node_group, node_group_template, templates_relations objects in DB. By default use_autoconfig will be True. If use_autoconfig is False, then we will not use autoconfiguration during cluster creation. If none of the configs from the list above are configured manually and use_autoconfig is True, then we will autoconfigure configs from list above. Same behaviour will be used for node_groups configs autoconfiguration.
Need to support of switch off autoconfiguration.
Need to support of switch off autoconfiguration via python-saharaclient.
Need to add new checkbox which will allow to swith off autoconfiguration from Horizon during cluster creation/scaling. If plugin doesn’t support autoconfig this checkbox will not be displayed. We can use _info field at  for field.
Proposed change will consists with following steps:
Depends on Openstack requirements
Unit tests will be implemented for this feature. Sahara CI also can start use autoconfiguration as well.
Need to document feature and all rules, which will be used for autoconfiguration.
 https://apache.googlesource.com/ambari/+/a940986517cbfeb2ef889f0d8a45579b27adad1c/ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py  https://apache.googlesource.com/ambari/+/a940986517cbfeb2ef889f0d8a45579b27adad1c/ambari-server/src/main/resources/stacks/HDP/2.1/services/stack_advisor.py  https://github.com/openstack/sahara/blob/master/sahara/service/api.py#L188