Add policy engine class to API¶
This blueprint makes it possible to access policy-engines (other than the domain-agnostic policy engine) via the API.
Problem description¶
Currently there is no way to access policy engines other than the current domain-agnostic policy engine through the API. Now that we are adding other policy engines, we need to give the user a way to interact with them.
In addition to being useful for domain-specific policy engines, enabling multiple policy engines should make it easier to handle upgrades for the domain-agnostic policy engine: bring up the new one alongside the old one, then swap the old one with the new one atomically.
Proposed change¶
Here we discuss two proposals. We list tradeoffs in the next section.
Proposal 1: Type-based¶
Here we simply add a top-level ‘policy-engines’ API endpoint, giving us …
/v1/policy-engines/<engine-id>/… /v1/data-sources/<datasource-id>/… /v1/system/…
For example: /data-sources/nova/… /data-sources/neutron/… /policy-engines/agnostic/…
Data-sources would support: * schema: available tables * actions: available actions * status: status
Policy-engines would support:
schema: available tables (e.g. classification:connected_to_internet)
actions: available actions (e.g. scripts built into a policy-engine for carrying out some task)
status: status
policies: available policies
That is, the only difference between a policy-engine and a data-source is that the datasource doesn’t support ‘policies’. If we were to think of ‘policies’ as simply ‘modules’ then we could imagine a datasource-driver exposing hierarchical tables, just like a policy-engine does. And in that case, the data-source would have something analogous to the policy-engine ‘policies’.
Proposal 2: Service-based¶
Here we do away with the types policy-engine and data-sources and give clients the ability to access both policy-engines and datasources from the same endpoint. This is possible because you cannot give a policy-engine and a datasource the same name (a restriction in place to ensure references to other services in policy are unambiguous).
/v1/services/<service-id>/… /v1/system/…
For example: /v1/services/nova/… /v1/services/neutron/… /v1/services/agnostic/…
All services would support:
schema: available tables (e.g. classification:connected_to_internet)
actions: available actions (e.g. scripts built into a policy-engine for carrying out some task)
status: status
policies: available policies
In short, this proposal treats everything as if it is a policy-engine. The datasource policy engines prohibit users from making changes directly (and even if they did, they would only accept a very restricted form of policy statements: ground facts).
This approach can be backwards compatible as well. We can still support
/v1/datasources/<datasource-id>/…
which gets routed to
/v1/services/<datasource-id>.
And we can support
/v1/policies /v1/actions /v1/tables /v1/status
as syntactic sugar for
/v1/services/<default-service>/policies /v1/services/<default-service>/actions /v1/services/<default-service>/tables /v1/services/<default-service>/status
This would enable the user to choose a single touchpoint for managing policy in the datacenter, while at the same time enabling them direct access to all the services in the datacenter. (The only worry here is that )
To set the <default-service>, we’d want a parameter the user can set dynamically (to help with upgrade). Right now we would set that to /services/agnostic. And maybe we can have arbitrary aliases for services as well, so that we can upgrade any service without changing policy.
One worry with providing the /v1/policies, etc. endpoints is that it may seem to mask Congress’s overall status, policies, actions, and tables. That is, people might expect those endpoints to aggregate all the potential policies, actions, tables, and statuses. But if such functionality ever becomes necessary, we can attach those endpoints to /v1/services, giving us the following end points.
/services/policies /services/actions /services/tables /services/status
Here is an example of the entry points if we had Nova, GBP, and our policy engine. The name of the service is whatever name DSE expects.
/policies /actions /tables /services/policies /services/actions /services/tables
/services/nova/policies -> empty /services/nova/actions -> createVM, deleteVM, migrateVM, etc. /services/nova/tables -> servers, hosts, etc. /services/gbp/policies /services/gbp/actions /services/gbp/tables /services/engine/policies /services/engine/actions /services/engine/tables
/system/drivers/
/system/engine-drivers/nova-uber-driver /system/datasource-drivers/nova-uber-driver /system/action-drivers/nova-uber-driver
/users /stats
One benefit to enabling people to modify domain-specific policy engines through the Congress API is that we provide a single policy language for managing all the policy engines running in the datacenter. For delegation, we already need adapters that translate Datalog into the native language of each policy engine, so here we expose that functionality directly to the user as well.
Alternatives¶
N/A
Tradeoffs¶
Pros for the type-based approach: * Easy for users to understand * Simple extension of the current API
Cons for the type-based approach:
Awkward that data-sources and policy-engines implement almost exactly the same interface and have separate namespaces, but are represented as distinct classes in the API.
Enables us to build datasources with significantly different programmatic interface than policy-engines. If at the API-layer the two classes of objects were almost indistinguishable, it would lead to better abstraction and interfaces in the underlying implementation.
Pros for the the service-based approach:
All services running on the DSE are accessed identically from the API. This is a more natural reflection of the reality of the nature of those services.
Cons for the service-based approach:
Bigger change
May be more difficult for users to understand initially.
Eventually the policy-engine class will include functionality that the datasource class does not. Executing that functionality on a datasource will cause a 404, and we cannot predict which will occur based on just the URL.
Doing something like listing all the datasources will require an API like /v1/services?action=list&type=datasource instead of the more obvious /v1/data-sources/.
Overall, the types (datasource vs. policyengine) will be present in both proposals, but they will be emphasized much less in the service-based approach. The service-based approach is closer to Python in that the system isn’t able to look at the code you’ve written (the URL) and check if the method you asked for exists. The type-based approach is closer to C/Java in that the system IS able to tell you if the method exists by just looking at the code (URL).
Typically policy systems are quite dynamic in nature (you can change the policy/code at runtime), and hence are closer to dynamic programming languages like Python than to static languages like C/Java. We therefore typically bias our decisions toward dynamism, which in this case would mean the service-based approach.
Policy¶
N/A
Policy actions¶
N/A
Data sources¶
N/A
Data model impact¶
N/A
REST API impact¶
See above. No changes to API results–just the paths for invoking them.
Security impact¶
N/A
Notifications impact¶
N/A
Other end user impact¶
N/A
Performance impact¶
N/A
Other deployer impact¶
See above.
Developer impact¶
N/A
Implementation¶
Assignee(s)¶
- Primary assignee:
<launchpad-id or None>
- Other contributors:
<launchpad-id or None>
Work items¶
Once we decide on the approach, we will figure out the necessary work items. But here’s a rough cut.
type-based approach: add routes, create congress/api/engine_model.py, modify congress/api/*_model to enable tables/actions/policies/etc. for engines.
service-based approach: add routes, create congress/api/service_model.py, (including an API to list different types of objects), modify the congress/api/*_model to eliminate distinction between datasources and policies
Dependencies¶
Assumes that we have added an API call for ‘actions’, though this work could be done without that: add-action-listing.
Testing¶
Change unit tests in congress/tests/test_congress.py
Change congress_pythonclient (which will handle tempest tests)
Documentation impact¶
Many URLs may change.
References¶
N/A
