Recap of the previous endpoint bug, now fixed:
- The DUT1 endpoint indicated updating the updatable_config for both the cluster and cluster-keep-warm
PersistentStates
- But the kotekan config for keep-warm didn't include the updatable_config (an oversight on our part)
With this bad coco endpoint, the program flow in coco was something like:
- Receive update from DUT1, which indicates update to both cluster and cluster-keep-warm config
- coco updates the value in the cluster
PersistentState
- coco fails to update the value in the cluster-keep-warm
PersistentState
- Endpoint processing fails (with result 500 Internal Server Error)
This is a problem, because coco updated the cluster state but never sent it! So, later:
- coco runs the schedule cluster config check
- cluster config check fails because coco's internal copy of the cluster config (with the new DUT1 value) differs from what is on the cluster (no DUT1 update)
So one or more of these should be done:
- coco needs to be able to guarantee that changes to its internal state get pushed to downstream endpoints
- coco separately tracks states it wants to send vs. configs it has sent
- a failure in an endpoint causes a rollback of whatever was updated by the endpoint
Not sure which of these is best.
Recap of the previous endpoint bug, now fixed:
PersistentStatesWith this bad coco endpoint, the program flow in coco was something like:
PersistentStatePersistentStateThis is a problem, because coco updated the
clusterstate but never sent it! So, later:So one or more of these should be done:
Not sure which of these is best.