"Foolproof" install instructions?


#21

@Amit_Mujawar You might try this as well:


#22

Guys, I’m trying this halyard one click approach as an alternative to helm chart install which worked mostly without issues. The problem I’m with this installer is as follows: deck and fron50 get stuck in crashloopback while get never reaches ready state. Here are my versions:kubectl version
Client Version: version.Info{Major:“1”, Minor:“9”, GitVersion:“v1.9.4”, GitCommit:“bee2d1505c4fe820744d26d41ecd3fdd4a3d6546”, GitTreeState:“clean”, BuildDate:“2018-03-12T16:29:47Z”, GoVersion:“go1.9.3”, Compiler:“gc”, Platform:“linux/amd64”}
Server Version: version.Info{Major:"", Minor:"", GitVersion:“v1.9.0”, GitCommit:“925c127ec6b946659ad0fd596fa959be43f0cc05”, GitTreeState:“clean”, BuildDate:“2018-01-26T19:04:38Z”, GoVersion:“go1.9.1”, Compiler:“gc”, Platform:“linux/amd64”}

Error:
NAME READY STATUS RESTARTS AGE
minio-deployment-5d84f45dd5-97mqv 1/1 Running 0 28m
spin-clouddriver-6b64f77df4-rmpwm 1/1 Running 0 26m
spin-deck-6cc8c8756c-p24tv 0/1 Error 10 26m
spin-echo-77466ddd9-7mml8 1/1 Running 0 26m
spin-front50-68c5f5cdcc-mwq9k 0/1 CrashLoopBackOff 9 26m
spin-gate-57cc9494f7-gbs9b 0/1 Running 0 26m
spin-halyard-69878d4bb4-gcdp9 1/1 Running 0 28m
spin-orca-cc4b685d6-c262t 1/1 Running 0 26m
spin-redis-8677d6df6f-bkw4j 1/1 Running 0 26m


#23

What do kubectl logs and kubectl describe say about the spin-deck and spin-front50 pods?

What kind of kubernetes environment are you deploying in? Managed through GKE / AKS / ECS? Or on-prem / baremetal? How many nodes? How much memory do the nodes have?


#24

Are these instructions specific to deploying to a k8s cluster on AWS?

I deployed the resources depicted in https://spinnaker.io/downloads/kubernetes/quick-install.yml to a GKE k8s cluster, and everything seems to be running other than front50 and gate.

For front50, I’m seeing the following error in the logs:

Caused by: com.amazonaws.SdkClientException: Unable to load AWS credentials from any provider in the chain at com.amazonaws.auth.AWSCredentialsProviderChain.getCredentials(AWSCredentialsProviderChain.java:131) ~[aws-java-sdk-core-1.11.173.jar:na] at com.amazonaws.http.AmazonHttpClient$RequestExecutor.getCredentialsFromContext(AmazonHttpClient.java:1118) ~[aws-java-sdk-core-1.11.173.jar:na] at com.amazonaws.http.AmazonHttpClient$RequestExecutor.runBeforeRequestHandlers(AmazonHttpClient.java:758) ~[aws-java-sdk-core-1.11.173.jar:na] at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:722) ~[aws-java-sdk-core-1.11.173.jar:na] at 

#25

The quick-install doesn’t assume you’re on AWS but does rely on minio, an object storage server that is API-compatible with S3 and that Spinnaker interacts with via the AWS SDK.

I’ll take a deeper look at this tomorrow and try to figure out what’s going wrong.


#26

the issue with the quick-install should now be resolved - the manifest was relying on an older version of halyard.


#27

SBSW,

I will get you logs a little later. I’m using minikube setup for my Kube cluster. 4 vcpu and 16gb, the same cluster allows for spinnaker to be installed with helm charts.

Thanks.


#28

Deck Logs:
kubectl logs -n spinnaker spin-deck-6cc8c8756c-p24tv
cp: cannot create regular file ‘spinnaker.conf’: Permission denied
sed: can’t read spinnaker.conf: No such file or directory
sed: can’t read spinnaker.conf: No such file or directory
sed: can’t read spinnaker.conf: No such file or directory
mv: cannot stat ‘spinnaker.conf’: No such file or directory
ERROR: Site spinnaker does not exist!
cp: cannot create regular file ‘ports.conf’: Permission denied
sed: can’t read ports.conf: No such file or directory
sed: can’t read ports.conf: No such file or directory
mv: cannot stat ‘ports.conf’: No such file or directory
cp: cannot create regular file ‘passphrase’: Permission denied
sed: can’t read passphrase: No such file or directory
chmod: cannot access ‘passphrase’: No such file or directory
mv: cannot stat ‘passphrase’: No such file or directory
cp: cannot create regular file ‘/opt/deck/html/settings.js’: Permission denied
mkdir: cannot create directory ‘/var/run/apache2’: Permission denied
apache2: Syntax error on line 80 of /etc/apache2/apache2.conf: DefaultRuntimeDir must be a valid directory, absolute or relative to ServerRoot
Action ‘-D FOREGROUND’ failed.
The Apache error log may have more information.

Front50 logs:
2018-04-11 22:31:19.848 INFO 1 — [ main] trationDelegate$BeanPostProcessorChecker : Bean ‘methodSecurityMetadataSource’ of type [org.springframework.security.access.method.DelegatingMethodSecurityMetadataSource] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2018-04-11 22:31:20.565 INFO 1 — [ main] s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat initialized with port(s): 8080 (http)
2018-04-11 22:31:20.583 INFO 1 — [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
2018-04-11 22:31:20.584 INFO 1 — [ main] org.apache.catalina.core.StandardEngine : Starting Servlet Engine: Apache Tomcat/8.5.20
2018-04-11 22:31:20.773 INFO 1 — [ost-startStop-1] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
2018-04-11 22:31:20.773 INFO 1 — [ost-startStop-1] o.s.web.context.ContextLoader : Root WebApplicationContext: initialization completed in 3236 ms
2018-04-11 22:31:21.524 INFO 1 — [ost-startStop-1] o.s.b.w.servlet.FilterRegistrationBean : Mapping filter: ‘authenticatedRequestFilter’ to: [/]
2018-04-11 22:31:21.525 INFO 1 — [ost-startStop-1] o.s.b.w.servlet.FilterRegistrationBean : Mapping filter: ‘simpleCORSFilter’ to: [/
]
2018-04-11 22:31:21.525 INFO 1 — [ost-startStop-1] o.s.b.w.servlet.FilterRegistrationBean : Mapping filter: ‘metricsFilter’ to: [/]
2018-04-11 22:31:21.525 INFO 1 — [ost-startStop-1] o.s.b.w.servlet.FilterRegistrationBean : Mapping filter: ‘characterEncodingFilter’ to: [/
]
2018-04-11 22:31:21.525 INFO 1 — [ost-startStop-1] o.s.b.w.servlet.FilterRegistrationBean : Mapping filter: ‘fiatAuthenticationFilter’ to urls: [/]
2018-04-11 22:31:21.526 INFO 1 — [ost-startStop-1] o.s.b.w.servlet.FilterRegistrationBean : Mapping filter: ‘hiddenHttpMethodFilter’ to: [/
]
2018-04-11 22:31:21.526 INFO 1 — [ost-startStop-1] o.s.b.w.servlet.FilterRegistrationBean : Mapping filter: ‘httpPutFormContentFilter’ to: [/]
2018-04-11 22:31:21.526 INFO 1 — [ost-startStop-1] o.s.b.w.servlet.FilterRegistrationBean : Mapping filter: ‘requestContextFilter’ to: [/
]
2018-04-11 22:31:21.527 INFO 1 — [ost-startStop-1] .s.DelegatingFilterProxyRegistrationBean : Mapping filter: ‘springSecurityFilterChain’ to: [/]
2018-04-11 22:31:21.529 INFO 1 — [ost-startStop-1] o.s.b.w.servlet.FilterRegistrationBean : Mapping filter: ‘webRequestLoggingFilter’ to: [/
]
2018-04-11 22:31:21.529 INFO 1 — [ost-startStop-1] o.s.b.w.servlet.FilterRegistrationBean : Mapping filter: ‘applicationContextIdFilter’ to: [/*]
2018-04-11 22:31:21.529 INFO 1 — [ost-startStop-1] o.s.b.w.servlet.ServletRegistrationBean : Mapping servlet: ‘dispatcherServlet’ to [/]
2018-04-11 22:31:25.060 WARN 1 — [ main] ationConfigEmbeddedWebApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name ‘permissionsController’: Unsatisfied dependency expressed through field ‘applicationPermissionDAO’; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name ‘applicationPermissionDAO’ defined in class path resource [com/netflix/spinnaker/front50/config/S3Config.class]: Unsatisfied dependency expressed through method ‘applicationPermissionDAO’ parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name ‘s3StorageService’ defined in class path resource [com/netflix/spinnaker/front50/config/S3Config.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.netflix.spinnaker.front50.model.S3StorageService]: Factory method ‘s3StorageService’ threw exception; nested exception is com.amazonaws.SdkClientException: Unable to load AWS credentials from any provider in the chain
2018-04-11 22:31:25.064 INFO 1 — [ main] o.apache.catalina.core.StandardService : Stopping service [Tomcat]
2018-04-11 22:31:25.069 WARN 1 — [ost-startStop-1] o.a.c.loader.WebappClassLoaderBase : The web application [ROOT] appears to have started a thread named [spectator-gauge-polling-0] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)
2018-04-11 22:31:25.085 INFO 1 — [ main] utoConfigurationReportLoggingInitializer :

Error starting ApplicationContext. To display the auto-configuration report re-run your application with ‘debug’ enabled.
2018-04-11 22:31:25.095 ERROR 1 — [ main] o.s.boot.SpringApplication : Application startup failed


#29

Hey, thanks for logs and description of your k8s environment.

I haven’t tested the quick-install against minikube yet but the manifest relies on a PersistentVolumeClaim with StorageClass of “standard” and expects the cluster to provision that dynamically. I don’t know how minikube handles that. I’ll try to find some time next week to get minikube setup and to test the quick-install against it.


#30

Scott,

Thanks so much, I’d love to get this install working so i can enable ACA and try out some Kayenta features.


#31

@lwander @NickChase These instructions worked for me on a Azure Kubernetes cluster. I had to change the PVC type to ‘default’ in the yaml file.

How do I go about adding additional Kubernetes clusters to the hal config? What is the best way to copy the data from my kubeconfig to the halyard pod since the pod does not have nano or vim? It does not let me install any editors.


#32

I’d like to hear the answer to the “editors” issue; I’ve resorted to appending to files from the command line, but there’s got to be a better way. (For reference: https://askubuntu.com/questions/21555/command-to-append-line-to-a-text-file-without-opening-an-editor)

---- Nick


#33

I would start by making a copy of the manifests located here.

Then, create a Secret in the cluster running halyard containing your kubeconfig with credentials, and mount it on the halyard deployment.

Finally, edit the mounted halconfig to include a new account that points to the kubeconfig you mounted similar to here.

I recommend creating a service account like shown here for the kubeconfig you mount.


#34

when I deploy that to a fresh Azure AKS cluster (tried both with and without RBAC enabled), I get unbound persistent volume claims


#35

which line did you change exactly?


#36

OK found it out. Needed to change all the storageClassName: standard lines to storageClassName: default