Versioning Application Deployments (k8s v2 provider)


#1

What is Spinnakers take on versioning of an application?

The build of our services creates a manifest and a docker image, that are then run through a pipeline to deploy. I would like to have a versioned catalog, of each execution of the pipeline with a snapshot of the artifacts used.
I have seen that there is a record of each pipeline execution, but it doesn’t appear to capture the artifacts specified in the trigger.

The use case here is to deploy an application version to a different account (outside production) to do more invasive debugging, that is not allowed on the live production system.


#2

The history of artifacts supplied to the execution are stored with the execution – but we don’t encourage users to treat that as the absolute source of truth because we don’t make any guarantees about how durable the storage is (right now it depends on redis).

The general recommendation is to capture and store the artifacts sent to a pipeline elsewhere outside of Spinnaker at least for now, while we harden the execution repository. This work is in progress and should hopefully be completed late 2018.

FYI @ezimanyi for any thoughts around exposing a catalog of artifacts alone associated with pipeline executions.


#3

As @lwander says, right now there’s not really a way in Spinnaker to keep track of the artifacts produced by various executions, but it’s on the roadmap and something I’ll be working on over the next couple of months.

Effectively the idea is that you’d be able to (in some form) see the artifacts associated with each execution, as well as any relevant provenance information (what build it came from, etc.). I’m happy to chat more about it this so I can try to account for your use cases as I work on it.


#4

@lwanda The problem I ran into, is that I was supplying the initial artifacts that kicked off the pipeline in the trigger, by using a webhook trigger and adding the artifact definitions to the payload. It seems that the trigger artifacts are not reused when re-running the execution. I have found a way to work around this, by using the Jenkins integration and an expression to extract the relevant artifact information from the build properties.

@ezimanyi Thanks! I’d be interested in knowing the details of the design once its available. The main use case I have is that we have artifacts that are generated outside Spinnaker (k8s manifest template + docker image in our case) that are then fed into the pipeline to do the deployment. The manifest template and docker image essentially represent a version of an application, so I would like to be able to select that application version (i.e. the manifest template + docker image) and feed it into a pipeline to allow deploying historical versions, perhaps with different parameters (e.g. the account being targeted)

Not sure how artifacts that are generated during a pipeline execution would fit in that model though.


#5

@Eli_Jordan: Your use case sounds like it would fit in with the work I’m doing (though perhaps a bit down the road). Once Spinnaker has a registry of the artifacts it has encountered (and provenance information) my goal would be to be able to re-use an artifact to start a pipeline. For artifacts generated in a pipeline, we’d have to think a bit more as you mention—it would probably be confusing to override artifacts that are produced somewhere in the middle of a pipeline. But in that case maybe even parent-child pipelines would be a good solution: for the normal flow, a parent pipeline produces an artifact and passes it to a child pipeline. If you wanted later, you could just start the child pipeline alone with a specified artifact.