Add CM API Library into Sahara

This specification proposes to add CM API library into Sahara so that the Cloudera plugin will not have to depend on a third party library support.

Problem description

Now the Cloudera plugin is depending on a third party library cm_api (by Cloudera). This library is not python3 compatible, and Cloudera has no resource to fix it. Therefore, it is hard to put this library into requirements.txt file. To fix this issue, we plan to implement a subset of CM APIs and put them into Sahara project, so that the Cloudera plugin will not depend on a third-party library, and we can enable this plugin by default.

Proposed change

Now the CM APIs used in Cloudera plugin include (maybe not all included):

  • ApiResource.create_cluster

  • ApiResource.get_cluster

  • ApiResource.get_all_hosts

  • ApiResource.delete_host

  • ApiResource.get_cloudera_manager

  • ClouderaManager.create_mgmt_service

  • ClouderaManager.hosts_start_roles

  • ClouderaManager.get_service

  • ApiCluster.start

  • ApiCluster.remove_host

  • ApiCluster.create_service

  • ApiCluster.get_service

  • ApiCluster.deploy_client_config

  • ApiCluster.first_run

  • ApiService.create_role

  • ApiService.delete_role

  • ApiService.refresh

  • ApiService.deploy_client_config

  • ApiService.start

  • ApiService.restart

  • ApiService.start_roles

  • ApiService.format_hdfs

  • ApiService.create_hdfs_tmp

  • ApiService.create_yarn_job_history_dir

  • ApiService.create_oozie_db

  • ApiService.install_oozie_sharelib

  • ApiService.create_hive_metastore_tables

  • ApiService.create_hive_userdir

  • ApiService.create_hive_warehouse

  • ApiService.create_hbase_root

  • ApiService.update_config

  • ApiServiceSetupInfo.add_role_info

  • ApiRole.update_config

Those APIs are what we need to implement in our CM APIs. We can create a directory client in plugin cdh directory, and put the lib files in this directory.



Data model impact


REST API impact


Other end user impact


Deployer impact

Deployers will no longer need to install cm_api package anymore.

Developer impact

When new version of CDH is released, if it is incompatible with current used client, or use some new APIs, developer may need to update the client when adding support for new CDH release.

Sahara-image-elements impact


Sahara-dashboard / Horizon impact




Primary assignee:

ken chen

Other contributors:

ken chen

Work Items

The work items will include:

  • Add a directory client in cdh plugin directory, and put lib files under this directory.

  • Change all current cm_api imports into using the new client.




Sahara Integration test for CDH plugin is enough.

Documentation Impact

Documents about CDH plugin prerequisites and enabling should be modified, for cm_api is not required any more.