Kubernetes V2 Provider Service Account Issue


#1

Hi,

I have successfully deployed Spinnaker to its own namespace, and it’s up and running.

I’ve created a ServiceAccount called ‘spinnaker-service-account-kv2’ for Spinnaker to use.

I’ve configured kubeconfig with the token for this account.

I have defined a V2 provider account called ‘v2-dev-website’ to allow Spinnaker to deploy to the ‘dev-website’ namespace, which involved configuring a context in kubeconfig. The context sets ‘user: spinnaker-service-account-kv2’ and ‘namespace: dev-website’.

The provider account was created like this:

hal config provider kubernetes account add v2-dev-website \
    --provider-version v2 \
    --namespaces dev-website \
    --context v2-dev-website \
    --service-account true

And in my pipeline, I’ve selected an account to use for the manifest deployment.

So I think that’s all I need to do to make it work. But it doesn’t.

When it tries to deploy, I get the error:

Exception ( Deploy Manifest )
deployKubernetesManifest.deployKubernetesManifest.deployment.notValidKind

There are no errors in the clouddriver logs when this happens, so I thought it may have been caused by an initialisation error. So I restarted the clouddriver pod and watched the logs. There are lots of errors like this:

Kind 'service' will not be cached in account 'v2-dev-website' for reason: 'Failed to read [service] from dev-website: Error from server (Forbidden): services is forbidden: User "system:serviceaccount:spinnaker:default" cannot list services in the namespace "dev-website"

The logs don’t mention ‘spinnaker-service-account-kv2’ at all. So the v2-dev-website account appears to be using the ‘default’ Service Account instead of the one specified in the kubeconfig and ‘account add’ command.

I’ve trawled the documentation and can’t find anything I’ve obviously done wrong.

We’re hoping to start deploying some real services in the next couple of days, so any ideas would be appreciated!

Simon.


#2

I suppose the crux of my question is… How does clouddriver know which service account to use?

If we can answer that, I should be able to fix this. Unless there’s a bug.


#3

In case anyone else comes across the same issue, this is what I had to do to resolve:

  • Set serviceAccount to false
  • Specify the location of the kubeconfig (even though it’s in the default location)

E.g.

hal config provider kubernetes account add v2-dev-website \
    --provider-version v2 \
    --namespaces dev-website \
    --context v2-dev-website \
    --service-account false \
    --kubeconfig-file /home/spinnaker/.kube/config