Error installing Istio 0.8.0 on Kubernetes using Bake (manifest)


#1

Trying to install Istio 0.8.0 on GKE using Baking from istio helm chart. The bake stage completes successfully and produces an artifact. Deploy from artifact fails with the following error:
textPayload: "retrofit.RetrofitError: 500
at retrofit.RetrofitError.httpError(RetrofitError.java:40)
at retrofit.RestAdapter$RestHandler.invokeRequest(RestAdapter.java:388)
at retrofit.RestAdapter$RestHandler.invoke(RestAdapter.java:240)
at com.sun.proxy.$Proxy96.fetchArtifact(Unknown Source)
at com.netflix.spinnaker.orca.clouddriver.DelegatingOortService.fetchArtifact(DelegatingOortService.java:99)
at com.netflix.spinnaker.orca.clouddriver.tasks.manifest.DeployManifestTask.lambda$execute$1(DeployManifestTask.java:104)
at com.netflix.spinnaker.kork.core.RetrySupport.retry(RetrySupport.java:26)
at com.netflix.spinnaker.orca.clouddriver.tasks.manifest.DeployManifestTask.execute(DeployManifestTask.java:102)
at com.netflix.spinnaker.orca.q.handler.RunTaskHandler$handle$1$1.invoke(RunTaskHandler.kt:88)
at com.netflix.spinnaker.orca.q.handler.RunTaskHandler$handle$1$1.invoke(RunTaskHandler.kt:52)
at com.netflix.spinnaker.orca.q.handler.AuthenticationAware$sam$java_util_concurrent_Callable$0.call(AuthenticationAware.kt)
at com.netflix.spinnaker.security.AuthenticatedRequest.lambda$propagate$1(AuthenticatedRequest.java:79)
at com.netflix.spinnaker.orca.q.handler.AuthenticationAware$DefaultImpls.withAuth(AuthenticationAware.kt:49)
at com.netflix.spinnaker.orca.q.handler.RunTaskHandler.withAuth(RunTaskHandler.kt:52)
at com.netflix.spinnaker.orca.q.handler.RunTaskHandler$handle$1.invoke(RunTaskHandler.kt:87)
at com.netflix.spinnaker.orca.q.handler.RunTaskHandler$handle$1.invoke(RunTaskHandler.kt:52)
at com.netflix.spinnaker.orca.q.handler.RunTaskHandler$withTask$1.invoke(RunTaskHandler.kt:161)
at com.netflix.spinnaker.orca.q.handler.RunTaskHandler$withTask$1.invoke(RunTaskHandler.kt:52)
at com.netflix.spinnaker.orca.q.handler.OrcaMessageHandler$withTask$1.invoke(OrcaMessageHandler.kt:48)
at com.netflix.spinnaker.orca.q.handler.OrcaMessageHandler$withTask$1.invoke(OrcaMessageHandler.kt:32)
at com.netflix.spinnaker.orca.q.handler.OrcaMessageHandler$withStage$1.invoke(OrcaMessageHandler.kt:58)
at com.netflix.spinnaker.orca.q.handler.OrcaMessageHandler$withStage$1.invoke(OrcaMessageHandler.kt:32)
at com.netflix.spinnaker.orca.q.handler.OrcaMessageHandler$DefaultImpls.withExecution(OrcaMessageHandler.kt:67)
at com.netflix.spinnaker.orca.q.handler.RunTaskHandler.withExecution(RunTaskHandler.kt:52)
at com.netflix.spinnaker.orca.q.handler.OrcaMessageHandler$DefaultImpls.withStage(OrcaMessageHandler.kt:54)
at com.netflix.spinnaker.orca.q.handler.RunTaskHandler.withStage(RunTaskHandler.kt:52)
at com.netflix.spinnaker.orca.q.handler.OrcaMessageHandler$DefaultImpls.withTask(OrcaMessageHandler.kt:41)
at com.netflix.spinnaker.orca.q.handler.RunTaskHandler.withTask(RunTaskHandler.kt:52)
at com.netflix.spinnaker.orca.q.handler.RunTaskHandler.withTask(RunTaskHandler.kt:154)
at com.netflix.spinnaker.orca.q.handler.RunTaskHandler.handle(RunTaskHandler.kt:64)
at com.netflix.spinnaker.orca.q.handler.RunTaskHandler.handle(RunTaskHandler.kt:52)
at com.netflix.spinnaker.q.MessageHandler$DefaultImpls.invoke(MessageHandler.kt:36)
at com.netflix.spinnaker.orca.q.handler.OrcaMessageHandler$DefaultImpls.invoke(OrcaMessageHandler.kt)
at com.netflix.spinnaker.orca.q.handler.RunTaskHandler.invoke(RunTaskHandler.kt:52)
at com.netflix.spinnaker.orca.q.audit.ExecutionTrackingMessageHandlerPostProcessor$ExecutionTrackingMessageHandlerProxy.invoke(ExecutionTrackingMessageHandlerPostProcessor.kt:47)
at com.netflix.spinnaker.q.QueueProcessor$pollOnce$1$1.run(QueueProcessor.kt:83)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
"

Spinnaker version is 1.8, GKE/k8s version is 1.10.5-gke.0
I have successfully installed Nginx using the same helm > bake > deploy method on the same spinnaker/cluster.

Any thoughts?


#2

Can you provide a link to the helm package you were trying to install?

Also, can you try and grab logs from the clouddriver service during this failed deployment?

So far we haven’t taken steps to make deploying Istio or, more generally, CRD specs any easier with Spinnaker. Right now Spinnaker expects that this sort of 1-time setup is handled elsewhere, or at the very least that the deployment of the CRD spec and an instance of the CRD is done separately. My guess is that Spinnaker sees a resource like a RouteRule before encountering the CRD spec and fails to validate it.


#3

I downloaded the latest Istio artifacts using
curl -L https://git.io/getLatestIstio | sh -
Which is how I typically install Istio using Helm charts
I created a tar using the helm package command, uploaded to GCS bucket for spinnaker as documented
Clouddriver is only showing one error, which is
receiveTimestamp: “2018-07-06T20:12:33.563200888Z”
resource: {…}
severity: “INFO”
textPayload: "2018-07-06 20:12:28.566 ERROR 1 — [0.0-7002-exec-5] c.n.s.k.w.e.GenericExceptionHandlers : Internal Server Error
"
I can upload entire log as well.
Thanks for looking into this