Unable to add multiple GKE cluster to Spinnaker


#1

Hi Team,

We have deployed Spinnaker in a GKE cluster and able to see the applications deployed in the same GKE cluster. Now we are trying to add new GKE cluster from the same GCP account to Spinnaker but we are not able to do so. We are using the following link for the same: https://www.spinnaker.io/setup/quickstart/halyard-gke-deploy-rbac/

We are getting the following error:
hal config provider kubernetes account add my-account --context $(kubectl config current-context)

  • Get current deployment
    Success
  • Add the my-account account
    Success
    Problems in halconfig:
  • WARNING There is a newer version of Halyard available (1.14.0),
    please update when possible
    ? Run ‘sudo apt-get update && sudo apt-get install
    spinnaker-halyard -y’ to upgrade
  • Successfully added account my-account for provider
    kubernetes.
    spinnaker@cd-spinnaker-halyard-0:/workdir$
    spinnaker@cd-spinnaker-halyard-0:/workdir$ hal deploy apply
  • Get current deployment
    Success
  • Prep deployment
    Failure
    Problems in default.provider.kubernetes.my-test-account:
    ! ERROR The chosen registry “my-gcr-account” has not been
    configured in your halconfig.
    ? Either add “my-gcr-account” as a new Docker Registry account, or
    pick a different one.
    ? Options include:
    • gcr

Problems in default.provider.kubernetes.my-account:
! ERROR You have not specified any docker registries to deploy
to.
? Add a docker registry that can be found in this deployment’s
dockerRegistries provider.
? Options include:

  • gcr

Problems in default.provider.kubernetes.jenkins-test:
! ERROR You have not specified any docker registries to deploy
to.
? Add a docker registry that can be found in this deployment’s
dockerRegistries provider.
? Options include:

  • gcr

Problems in halconfig:

  • WARNING There is a newer version of Halyard available (1.14.0),
    please update when possible
    ? Run ‘sudo apt-get update && sudo apt-get install
    spinnaker-halyard -y’ to upgrade

Problems in default.security:

  • WARNING Your UI or API domain does not have override base URLs
    set even though your Spinnaker deployment is a Distributed deployment on a
    remote cloud provider. As a result, you will need to open SSH tunnels against
    that deployment to access Spinnaker.
    ? We recommend that you instead configure an authentication
    mechanism (OAuth2, SAML2, or x509) to make it easier to access Spinnaker
    securely, and then register the intended Domain and IP addresses that your
    publicly facing services will be using.

Problems in default.provider.kubernetes.test-account:
! ERROR You have not specified any docker registries to deploy
to.
? Add a docker registry that can be found in this deployment’s
dockerRegistries provider.
? Options include:

  • gcr

  • Failed to prep Spinnaker deployment

Please suggest any links or approach for the same.

Thanks,
Sagar Patil.


#2

It looks like you’ve set up the new account to point to the my-gcr-account docker registry, but that is not a registry that is configured in your hal config. (Maybe that was part of a copy-pasted command from the docs?) From the error message, it looks like you do have a docker registry account called gcr so you could change the new account to use that registry.


#3

Hi Eric,

Thanks for the reply and pointing out the typo.

We have completed all the steps as per the doc. Also, we are able to see the deployments on the newly added cluster in spinnaker, but the pod running in the applications are showing as virtual machines, not as kubernetes pods. When we have deployed Spinnaker on a single cluster and deployed the application using CI-CD on the same k8’s cluster the application was seen as pods not an virtual machine. So is this some kind of issue or we need to change settings in Spinnaker.

Also is there any documentation on how to configure Spinnaker to deploy the application to different k8’s cluster.

Please do reply.

Thanks,
Sagar Patil.


#4

Hi Sagar,

I use Spinnaker to deploy to multiple GKE cluster on multiple projects on GCP.

You have also have to specify the docker registry while adding the GKE cluster. Sample command
hal config provider kubernetes account add (Account Name)
–docker-registries (GCR Account Name)
–context $(kubectl config current-context)


#5

Hi Amol,

I have added multiple clusters from the same GCP project but I have done using the same command which you have specified but as I have just a single GCP account I still need to use the “account add” command to do the same.

Now this is done and my Spinnaker which is deployed on GKE is able to discover the deployments on the other GKE cluster. But the application details show the pod as a Virtual machine. Also I want to know how to manage CI-CD on the other cluster(Any reference link would be helpful).

Thanks,
Sagar Patil


#6

Hi Sagar,

As the topic which you have mentioned in the original thread is now resolved you can ping me on Spinnaker Slack and we can discuss your other issues there. My id is aamol over there.