Working with dynamic attributes across the pipeline

#1

I did define a Spinnaker pipeline stage Configuration as follows:

Considering the following stream coming from Pub/Sub:

{ 
    "action" : "INSERT", 
    "digest" : "gcr.io/my-project/catalog@sha256:6ec128e26cd5...", 
    "tag" : "gcr.io/my-project/catalog:1.1"
}
  1. Do I need to explicitly specify a value for PROJECT or does Spinnaker automatically find it out?
  2. How can I extract from tag just the image version?
  3. How can I then pass tag and a new defined version attribute to the next pipeline stage?

Supposing I get a Kubernetes manifest as follows:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: catalog
spec:
  replicas: 1
  selector:
    matchLabels:
      app: catalog
  template:
    metadata:
      labels:
        app: catalog
    spec:
      containers:
      - name: catalog
        image: gcr.io/GCP_PROJECT/catalog:VERSION
        imagePullPolicy: Always

How can I replace GCP_PROJECT with PROJECT and VERSION with version within the manifest above?

Thanks, SZ.

0 Likes

#2

@subzeno: As long as you have configured messageFormat=GCR on the pubsub subscription, Spinnaker will automatically create an artifact representing the pushed image. So you would not directly pass the tag and version to the next stage, but would just pass along the artifact that was generated

When you deploy a manifest, you specify just the image (without tag), then select the artifact you want to deploy in the “Req. artifacts to bind” field below. Spinnaker will then look in your manifest and replace any references to that image with the fully-qualified (including) tag version of the image before deploying.

I’m not sure I fully understand the question around what project. In general you’d want to specify the full image (without tag) as artifact constraint: gcr.io/GCP_PROJECT/catalog. Are you expecting to deploy images with the same name, but from different projects, using the same Spinnaker pipeline?

0 Likes

#3

Hi @ezimanyi and thanks for your answer.
As a far as I understood, I do not need to change a line inside my manifest.
By binding the required artifact to the k8s manifest, it will just replace:

image: gcr.io/GCP_PROJECT/catalog:VERSION

with:

gcr.io/gcp-demoproject/catalog:v.1.0

Correct?

I do not plan to use the same pipeline within another project, but just trying to understand how attributes are dynamically resolved by Spinnaker. As well, I would like to know if I can pass the attributes across different stages (for instance, to use tag, version etc. inside a script).

0 Likes

#4

@subzeno: Yes, that’s correct—though to be clear you don’t even need the placeholder VERSION in the manifest.

So you’d probably want the following:

  • Your artifact defines the docker image as gcr.io/gcp-demoproject/catalog
  • Your manifest deploys image: gcr.io/gcp-demoproject/catalog

When the pipeline starts, the trigger will bind the incoming artifact to the fully-qualified image (with tag), and the deploy that specific image.

You do need to explicitly state the project in both places as there’s no project-specific automatic replacement…the replacement logic assumes that the manifest has a fully qualified image name that is only missing the tag.

In general, the recommended way to pass attributes of the image between stages is to pass the artifact itself. If you need to access specific properties of the artifact (ex: version, etc) you can probably use SpEL expressions to do so, though I’m not overly familiar with SpEL.

0 Likes

#5

Thanks a lot, a very clear explanation.

Just a last question about the matter: is there any way to perform a string replacement through the pipeline itself, for instance using sed?

0 Likes

#6

There’s not a way that I’m aware of to do that sort of arbitrary replacement during a pipeline. Depending on what you’re trying to do, pipeline templates might be a reasonable solution. They would allow you to create a template that has certain fields as variables, and to then create instances of pipelines from that template by specifying the variables.

0 Likes