Audit parameters should be persistent in Db¶
https://blueprints.launchpad.net/watcher/+spec/persistent-audit-parameters
Problem description¶
When you start an audit, a new Audit object is created in Db with a reference to an Audit Template object.
When processing an audit request, parameters taken into account by the Watcher decision engine come from Audit object itself and also from Audit Template ones (goal name, …).
Problems:
If we update the Audit Template parameters or if we remove the Audit Template, we lose important information about the Audit itself.
What about data integrity for soft-deleted audit objects (useful for history) ?
If we introduce parameters for goals (as thresholds, …) , we should be able to update them. So how to keep persistent data for all audits done to satisfy such configurable goals ?
How do we handle periodic audits when an audit template is updated ?
If we create an Audit Template with no strategy but with a Goal there is a strategy selector that will automatically pick the most suited strategy for the goal we specified (watcher-strategy-selector) and since the Audit has no record of which strategy was picked, we lose the information stating the actual name of the strategy and the global efficacy is calculated upon. Also, there can be a different strategy for an Audit simply because the selection criteria changed over time (Ref. blueprint efficacy indicator and watcher-strategy-selector). How do we persist strategy for an audit for these cases ?
Other Audit impacting scenarios handled in different blueprints -
1. Impact on Audit Template and Audit when a Goal is deleted - Addressed in blueprint soft-delete-goals
2. Impact on Audit Template and Audit when a Strategy is deleted - Addressed in blueprint get-goal-from-strategy
Use Cases¶
As an Administrator I need to ensure data integrity and consistency of an Audit when an Audit Template is updated or removed.
As an Administrator I should be able to purge an`Audit Template`_ without impacting Audit
Project Priority¶
Essential for Newton-2
Proposed change¶
Modify audit schema to include strategy_id, goal_id and host_aggregate
Alternatives¶
Audit Template to be immutable if there is any active Audit referencing it. Any modifications to Audit Template should create a new record, so the existing audits are not impacted.
Data model impact¶
When an audit template is updated, an audit object should not lose related data. We need to store goal_id and strategy_id in audit object. This ensures data integrity for optimization-threshold parameters and strategy selections (Ref. blueprint efficacy indicator and watcher-strategy-selector)
Current audit structure:
+-------------------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------------+-------------+------+-----+---------+----------------+
| created_at | datetime | YES | | NULL | |
| updated_at | datetime | YES | | NULL | |
| deleted_at | datetime | YES | | NULL | |
| deleted | int(11) | YES | | NULL | |
| id | int(11) | NO | PRI | NULL | auto_increment |
| uuid | varchar(36) | YES | UNI | NULL | |
| type | varchar(20) | YES | | NULL | |
| state | varchar(20) | YES | | NULL | |
| deadline | datetime | YES | | NULL | |
| audit_template_id | int(11) | NO | MUL | NULL | |
+-------------------+-------------+------+-----+---------+----------------+
+---------------------+--------------------------------------+
| Property | Value |
+---------------------+--------------------------------------+
| deadline | None |
| type | ONESHOT |
| uuid | ab49ee15-fa2b-4d3d-962f-01fe28b7fd92 |
| audit_template_uuid | 7f1e48ba-2639-498e-8fff-9508cf706665 |
+---------------------+--------------------------------------+
Proposed audit structure:
+-------------------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------------+-------------+------+-----+---------+----------------+
| created_at | datetime | YES | | NULL | |
| updated_at | datetime | YES | | NULL | |
| deleted_at | datetime | YES | | NULL | |
| deleted | int(11) | YES | | NULL | |
| id | int(11) | NO | PRI | NULL | auto_increment |
| uuid | varchar(36) | YES | UNI | NULL | |
| type | varchar(20) | YES | | NULL | |
| state | varchar(20) | YES | | NULL | |
| deadline | datetime | YES | | NULL | |
| goal_id | int(11) | NO | MUL | NULL | |
| strategy_id | int(11) | YES | MUL | NULL | |
| host_aggregate | int(11) | YES | | NULL | |
+-------------------+-------------+------+-----+---------+----------------+
+---------------------+--------------------------------------+
| Property | Value |
+---------------------+--------------------------------------+
| deadline | None |
| type | ONESHOT |
| uuid | ab49ee15-fa2b-4d3d-962f-01fe28b7fd92 |
| goal_uuid | None |
| strategy_uuid | None |
+---------------------+--------------------------------------+
REST API impact¶
Impacts following audit REST -
GET /v1/audits
GET /v1/audits/(audit_uuid)
POST /v1/audits
PATCH /v1/audits
GET /v1/audits/detail
Security impact¶
None
Notifications impact¶
None
Other end user impact¶
Impacts openstack watcher command line:
The python-watcherclient audit-create should have options for strategy-uuid, goal-uuid and host_aggregate while creating an audit
The python-watcherclient audit-update should include options for updating strategy-uuid, goal-uuid and host_aggregate for an audit
Performance Impact¶
None
Other deployer impact¶
Developer impact¶
None
Implementation¶
Assignee(s)¶
Primary assignee:
Other Contributors: hvprash michaelgugino
Work Items¶
Update Audit object in /watcher/db/sqlalchemy/models.py to include strategy_id, goal_id and host_aggregate
Synchronize audit_template and audit. Refactor the code to update strategy_id, goal_id and host_aggregate in both audit template and audit during CRUD operations. Following python files will be impacted: -
/watcher/objects/audit_template.py-/watcher/objects/audit.py-/watcher/tests/api/v1/test_audit_templates.py-/watcher/tests/api/v1/test_audits.py-/watcher/tests/db/test_audit.py-/watcher/tests/db/test_audit_template.pyAdditional validation while soft deleting an audit template and purging. De-reference any active audits refering to an audit template: -
/watcher/objects/audit_template.py-/watcher/objects/audit.py-/watcher/db/purge.pyUpdate unit tests and integration tests (Tempest scenarios)
Changes to database migration script
Dependencies¶
None
Testing¶
Documentation Impact¶
References¶
IRC discussions:
History¶
None
