Include the URL of your launchpad blueprint:
As we traverse the dependency graph during an update, we need to keep a record of our progress so that we can resume the traversal later in case it is interrupted.
Add a new table, SyncPoint, to the database with the following rows:
The first three fields should form a composite primary key. That should allow us to do a quick get of a SyncPoint given a graph key (resource key + is_update direction) and traversal ID (i.e. without doing a query). The stack key together with the traversal ID allows us to query for all of the SyncPoints associated with a particular traversal (e.g. to delete them if the traversal is cancelled.)
The input data will contain a map of graph keys (resource key of the Resource that was current at the beginning of the update + is_update direction) to resource key (may be different if the resource was replaced), RefID and attribute data. Thus the input data pushed from previously-updated resources is cached until such time as the current resource is ready for it. This data will likely be serialised in JSON format, and could be quite large.
A prototype for this is https://github.com/zaneb/heat-convergence-prototype/blob/resumable/converge/sync_point.py
Updates to the input data must be atomic, and must use the “UPDATE ... WHERE ...” form discussed in http://www.joinfu.com/2015/01/understanding-reservations-concurrency-locking-in-nova/ - which probably means adding an extra integer field that is incremented on every write (since we can’t really query on a text field).