Support for using Aggregates in Datalog¶
https://blueprints.launchpad.net/congress/+spec/datalog-aggregates
Aggregates in datalog are required for writing policies involving multiple records in a single ATOM.
Problem description¶
Consider a policy statement “The maximum number of ports on the Subnet X belonging to tenant Y should not exceed N” To realize this policy, we need to use aggregations like COUNT, MAX, SUM etc. which span across multiple records of the table.
The current Datalog support in Congress does not allow writing policies with such aggregates.
Also there is no way to simulate aggregations using existing Datalog support.
When a policy engine subscribes to an aggregate on a data source table, the aggregates must be updated every time there is a change in the corresponding table.
Proposed change¶
The aggregate support for Datalog allows writing policy atoms of the following form:
Example policies involving aggregates:
1. Restriction on number of ports on a subnet error(subnet) :- neutron:networks.subnets(subnet, port), neutron:ports(port, ip), count(num_port, port), not gt(num_port, 10)
2. Restriction on average CPU Load on a host error(host) :- ceilometer:meters(host, cpu_util), nova:servers(host, vm_host), average(avg_cpu, vm_host, cpu_util), not gt(avg_cpu, 80) average(x, y, z) :- sum(z, cpu_util), count(y, vm_host), div(x, z, y)
(Note: These are tentative policy statements and may be subject to change as the design/implementation progresses)
The aggregates like “SUM”, “COUNT” must be defined as ANTLR Terms and Congress must be able to parse and compile these terms.
The initial implementation will use SUM and COUNT as prototype aggregations. Other similar aggregates can be considered as enhancements to be taken up at a later date.
The runtime execution should be able to update tables corresponding to the aggregate terms with computed values
Alternatives¶
N/A
Data model impact¶
No data model changes are expected.
REST API impact¶
This change does not impact REST APIs as the changes involved will only extend the policy syntax which is fed as input to the policy APIs
Security impact¶
None.
Notifications impact¶
None
Other end user impact¶
End user will be able to write policies involving aggregates.
Performance impact¶
Evaluation of policies involving aggregates may involve additional processing and thereby impacting performance.
Other deployer impact¶
None.
Developer impact¶
None.
Implementation¶
Assignee(s)¶
- Primary assignee:
Madhu Mohan (mmohan@mvista.com)
Work items¶
Add new terms in Congress.g to allow ANTLR to recognize aggregate keywords
Extend parser/compiler to accept aggregate keywords as part of the rule syntax and validate
Extend runtime to compute aggregate values and update the corresponding tables
Add sufficient tests to validate support for aggregates
Dependencies¶
None.
Testing¶
Add test cases involving aggregates
Add Tempest tests
Create sample policies to test some real scenarios
Documentation impact¶
Docs for external consumption would need to be updated with explanation on writing aggregate-based policies
