Spinnaker: AKS account not showing on UI

I’ve configured spinnaker cloud provider as kubernetes with below commands

hal config provider kubernetes enable
kubectl config current-context
CONTEXT=$(kubectl config current-context)
hal config provider kubernetes account add my-k8s-v2-account     --provider-version v2     --context $CONTEXT
hal config features edit --artifacts true

but this account is not visible on spinnaker UI

and in logs its shows error as below

Nov 29 12:07:43 47184UW2DDevLVM2 gate[34594]: 2019-11-29 12:07:43.860 ERROR 34594 --- [TaskScheduler-5] c.n.s.g.s.DefaultProviderLookupService : Unable to refresh account details cache, reason: timeout

please advise… thanks…

here’s my hal deploy diff command output

+ Get current deployment
  Success
+ Determine config diff
  Success
~ EDITED
default.persistentStorage.redis
- port 6379 -> null
- host localhost -> null
~ EDITED
telemetry

I’ve provisioned new VM and did all installation process from scratch but still same issue :frowning:

here is ~/.kube/config file

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: xxx
    server: https://xxx:443
  name:xxx
contexts:
- context:
    cluster: xxx
    user: xxx
  name: xxx
current-context: xxx
kind: Config
preferences: {}
users:
- name: xxx
  user:
    client-certificate-data: xxx
    client-key-data: xxx
    token: xxx

and here is ~/.hal/config file

currentDeployment: default
deploymentConfigurations:
- name: default
  version: 1.17.2
  providers:
    appengine:
      enabled: false
      accounts: []
    aws:
      enabled: false
      accounts: []
      bakeryDefaults:
        baseImages: []
      defaultKeyPairTemplate: '{{name}}-keypair'
      defaultRegions:
      - name: xxx
      defaults:
        iamRole: BaseIAMRole
    ecs:
      enabled: false
      accounts: []
    azure:
      enabled: false
      accounts: []
      bakeryDefaults:
        templateFile: azure-linux.json
        baseImages: []
    dcos:
      enabled: false
      accounts: []
      clusters: []
    dockerRegistry:
      enabled: false
      accounts: []
    google:
      enabled: false
      accounts: []
      bakeryDefaults:
        templateFile: gce.json
        baseImages: []
        zone: us-central1-f
        network: default
        useInternalIp: false
    kubernetes:
      enabled: true
      accounts:
      - name: xxx
        requiredGroupMembership: []
        providerVersion: V2
        permissions: {}
        dockerRegistries: []
        context: xxx
        configureImagePullSecrets: true
        cacheThreads: 1
        namespaces: []
        omitNamespaces: []
        kinds: []
        omitKinds: []
        customResources: []
        cachingPolicies: []
        kubeconfigFile: /home/xxx/.kube/config
        oAuthScopes: []
        onlySpinnakerManaged: false
      primaryAccount: xxx
    oracle:
      enabled: false
      accounts: []
      bakeryDefaults:
        templateFile: oci.json
        baseImages: []
    cloudfoundry:
      enabled: false
      accounts: []
  deploymentEnvironment:
    size: SMALL
    type: LocalDebian
    imageVariant: SLIM
    updateVersions: true
    consul:
      enabled: false
    vault:
      enabled: false
    customSizing: {}
    sidecars: {}
    initContainers: {}
    hostAliases: {}
    affinity: {}
    tolerations: {}
    nodeSelectors: {}
    gitConfig:
      upstreamUser: spinnaker
    livenessProbeConfig:
      enabled: false
    haServices:
      clouddriver:
        enabled: false
        disableClouddriverRoDeck: false
      echo:
        enabled: false
  persistentStorage:
    persistentStoreType: azs
    azs:
      storageAccountName: xxx
      storageAccountKey: xxx
      storageContainerName: xxx
    gcs:
      rootFolder: front50
    redis: {}
    s3:
      rootFolder: front50
    oracle: {}
  features:
    auth: false
    fiat: false
    chaos: false
    entityTags: false
    artifacts: true
  metricStores:
    datadog:
      enabled: false
      tags: []
    prometheus:
      enabled: false
      add_source_metalabels: true
    stackdriver:
      enabled: false
    newrelic:
      enabled: false
      tags: []
    period: 30
    enabled: false
  notifications:
    slack:
      enabled: false
    twilio:
      enabled: false
      baseUrl: https://api.twilio.com/
    github-status:
      enabled: false
  timezone: America/Los_Angeles
  ci:
    jenkins:
      enabled: false
      masters: []
    travis:
      enabled: false
      masters: []
    wercker:
      enabled: false
      masters: []
    concourse:
      enabled: false
      masters: []
    gcb:
      enabled: false
      accounts: []
  repository:
    artifactory:
      enabled: false
      searches: []
  security:
    apiSecurity:
      ssl:
        enabled: false
      overrideBaseUrl: http://xxx:8084/
    uiSecurity:
      ssl:
        enabled: false
      overrideBaseUrl: http://xxx:9000/
    authn:
      oauth2:
        enabled: false
        client: {}
        resource: {}
        userInfoMapping: {}
      saml:
        enabled: false
        userAttributeMapping: {}
      ldap:
        enabled: false
      x509:
        enabled: false
      iap:
        enabled: false
      enabled: false
    authz:
      groupMembership:
        service: EXTERNAL
        google:
          roleProviderType: GOOGLE
        github:
          roleProviderType: GITHUB
        file:
          roleProviderType: FILE
        ldap:
          roleProviderType: LDAP
      enabled: false
  artifacts:
    bitbucket:
      enabled: false
      accounts: []
    gcs:
      enabled: false
      accounts: []
    oracle:
      enabled: false
      accounts: []
    github:
      enabled: false
      accounts: []
    gitlab:
      enabled: false
      accounts: []
    gitrepo:
      enabled: false
      accounts: []
    http:
      enabled: false
      accounts: []
    helm:
      enabled: false
      accounts: []
    s3:
      enabled: false
      accounts: []
    maven:
      enabled: false
      accounts: []
    templates: []
  pubsub:
    enabled: false
    google:
      enabled: false
      pubsubType: GOOGLE
      subscriptions: []
      publishers: []
  canary:
    enabled: false
    serviceIntegrations:
    - name: google
      enabled: false
      accounts: []
      gcsEnabled: false
      stackdriverEnabled: false
    - name: prometheus
      enabled: false
      accounts: []
    - name: datadog
      enabled: false
      accounts: []
    - name: signalfx
      enabled: false
      accounts: []
    - name: aws
      enabled: false
      accounts: []
      s3Enabled: false
    - name: newrelic
      enabled: false
      accounts: []
    reduxLoggerEnabled: true
    defaultJudge: NetflixACAJudge-v1.0
    stagesEnabled: true
    templatesEnabled: true
    showAllConfigsEnabled: true
  plugins:
    plugins: []
    enabled: false
    downloadingEnabled: false
    pluginConfigurations:
      plugins: {}
  webhook:
    trust:
      enabled: false
  telemetry:
    enabled: false
    endpoint: https://stats.spinnaker.io
    instanceId: xxx
    connectionTimeoutMillis: 3000
    readTimeoutMillis: 5000

Here are the commands used to install spinnaker

az login
az aks get-credentials --resource-group xxx --name xxx
curl -O https://raw.githubusercontent.com/spinnaker/halyard/master/install/debian/InstallHalyard.sh
sudo bash InstallHalyard.sh --user xxx
hal config provider kubernetes enable
CONTEXT=$(kubectl config current-context)
hal config provider kubernetes account add xxx \
    --provider-version v2 \
    --context $CONTEXT
hal config features edit --artifacts true  
hal config deploy edit --type localdebian
hal config storage azs edit --storage-account-name xxx --storage-account-key xxx
hal config storage edit --type azs
hal version list
hal config version edit --version 1.17.2
sudo hal deploy apply
echo "host: 0.0.0.0" | tee \
    ~/.hal/default/service-settings/gate.yml \
    ~/.hal/default/service-settings/deck.yml
hal config security ui edit \
    --override-base-url http://xxx:9000/
hal config security api edit \
    --override-base-url http://xxx:8084/
sudo hal deploy apply

Found below exceptions logs

Dec  2 11:12:07 47184UW2DDevLVM2 clouddriver[23908]: 2019-12-02 11:12:07.424 ERROR 23908 --- [1-7002-exec-105] c.n.s.k.w.e.GenericExceptionHandlers     : Internal Server Error
Dec  2 11:12:07 47184UW2DDevLVM2 clouddriver[23908]: java.lang.NullPointerException: null
Dec  2 11:12:07 47184UW2DDevLVM2 clouddriver[23908]: #011at com.netflix.spinnaker.clouddriver.kubernetes.health.KubernetesHealthIndicator.health(KubernetesHealthIndicator.java:48) ~[clouddriver-kubernetes-6.4.1-20191111102213.jar:6.4.1-20191111102213]
Dec  2 11:12:07 47184UW2DDevLVM2 clouddriver[23908]: #011at org.springframework.boot.actuate.health.CompositeHealthIndicator.health(CompositeHealthIndicator.java:95) ~[spring-boot-actuator-2.1.7.RELEASE.jar:2.1.7.RELEASE]
Dec  2 11:12:07 47184UW2DDevLVM2 clouddriver[23908]: #011at org.springframework.boot.actuate.health.HealthEndpoint.health(HealthEndpoint.java:50) ~[spring-boot-actuator-2.1.7.RELEASE.jar:2.1.7.RELEASE]
Dec  2 11:12:07 47184UW2DDevLVM2 clouddriver[23908]: #011at org.springframework.boot.actuate.health.HealthEndpointWebExtension.health(HealthEndpointWebExtension.java:53) ~[spring-boot-actuator-2.1.7.RELEASE.jar:2.1.7.RELEASE]

plus localhost 7002 is not responding

hexunix@47184UW2DDevLVM2:~$ curl -v http://localhost:7002/credentials
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 7002 (#0)
> GET /credentials HTTP/1.1
> Host: localhost:7002
> User-Agent: curl/7.58.0
> Accept: */*
>

just by looking at the configurations i see some mixing of concepts.

hal config deploy edit --type localdebian

Why did you use local debian in an AKS cluster?

I recommend you watch the lynda.com Spinnaker course.
Otherwise easiest way to install spinnaker:

  1. Minnaker ( Spinnaker runnin in K3S, K3S runs inside any compute instance) https://github.com/armory/minnaker
  2. Deploy an AKS cluster. Install VScode, the Kubernetes plugin and Helm. Select the kubernetes plugin from VS code and navigate in Helm Repos section until you find “stable/spinnaker” option and install to your AKS cluster

Once you have your AKS cluster with spinnaker running please acess the Halyard pod and begin configuring stuff there.

Always apply inside the Halyard Pod:

hal deploy apply

Why did you use local debian in an AKS cluster?

It’s like Spinnaker that I’ve installed in Local Debian server as per –

https://www.spinnaker.io/setup/install/environment/

The recommended path is a distributed installation onto a Kubernetes cluster, but all of these methods are supported:

* [Distributed installation](https://www.spinnaker.io/setup/install/environment/#distributed-installation) on KubernetesHalyard deploys each of Spinnaker’s [microservices](https://www.spinnaker.io/reference/architecture) separately.  **This is highly recommended for use in production.**
* [Local installations](https://www.spinnaker.io/setup/install/environment/#local-debian) of Debian packagesSpinnaker is deployed on a single machine. This is ok for smaller Spinnaker deployments, but Spinnaker will be unavailable when it’s being updated.
* [Local git installations](https://www.spinnaker.io/setup/install/environment/#local-git) from githubThis is for developers contributing to the Spinnaker project. If you’re a contributor, you’ll probably have two separate installations—a distributed one for using Spinnaker in production, and this local Git one for developing Spinnaker contributions.

In this same server, I’ve connected AKS using kubectl, and I could able to perform actions on AKS from this server, hence spinnaker will use this underlying AKS k8s platform to deploy applications in AKS, where Spinnaker is installed locally on the ubuntu server.

hope this clarifies the setup.