Spinnaker storage with Ceph


#1

I’m trying to use a Ceph storage backend for Spinnaker and I’ve created a bucket and verified I can access it with s3cmd and minio client. However after configuring storage through Halyard for S3 and then doing a hal deploy apply, Spinnaker comes back with the following error:
- Apply deployment
Failure
Problems in Global:
! ERROR Unable to retrieve profile “clouddriver-caching.yml”: 401
Unauthorized
Anonymous caller does not have storage.objects.get access to
halconfig/clouddriver/3.4.2-20180828182842/clouddriver-caching.yml.

- Failed to deploy Spinnaker.
com.netflix.spinnaker.halyard.cli.services.v1.ExpectedDaemonFailureException: Failed to deploy Spinnaker.
	at com.netflix.spinnaker.halyard.cli.services.v1.OperationHandler.get(OperationHandler.java:45)
	at com.netflix.spinnaker.halyard.cli.command.v1.AbstractRemoteActionCommand.runRemoteAction(AbstractRemoteActionCommand.java:50)
	at com.netflix.spinnaker.halyard.cli.command.v1.AbstractRemoteActionCommand.executeThis(AbstractRemoteActionCommand.java:103)
	at com.netflix.spinnaker.halyard.cli.command.v1.NestableCommand.safeExecuteThis(NestableCommand.java:201)
	at com.netflix.spinnaker.halyard.cli.command.v1.NestableCommand.execute(NestableCommand.java:149)
	at com.netflix.spinnaker.halyard.cli.command.v1.NestableCommand.execute(NestableCommand.java:152)
	at com.netflix.spinnaker.halyard.cli.command.v1.NestableCommand.execute(NestableCommand.java:152)
	at com.netflix.spinnaker.halyard.cli.Main.main(Main.java:46)
Caused by: java.lang.Exception: Unable to retrieve profile "clouddriver-caching.yml": 401 Unauthorized
Anonymous caller does not have storage.objects.get access to halconfig/clouddriver/3.4.2-20180828182842/clouddriver-caching.yml.
	at com.netflix.spinnaker.halyard.deploy.spinnaker.v1.profile.RegistryBackedProfileFactory.getBaseProfile(RegistryBackedProfileFactory.java:47)
	at com.netflix.spinnaker.halyard.deploy.spinnaker.v1.profile.ProfileFactory.getProfile(ProfileFactory.java:57)
	at com.netflix.spinnaker.halyard.deploy.spinnaker.v1.service.distributed.kubernetes.v2.KubernetesV2ClouddriverCachingService.getProfiles(KubernetesV2ClouddriverCachingService.java:53)
	at com.netflix.spinnaker.halyard.deploy.services.v1.GenerateService.generateConfig(GenerateService.java:141)
	at com.netflix.spinnaker.halyard.deploy.services.v1.DeployService.deploy(DeployService.java:268)
	at com.netflix.spinnaker.halyard.controllers.v1.DeploymentController.lambda$deploy$20(DeploymentController.java:262)
	at com.netflix.spinnaker.halyard.core.DaemonResponse$StaticRequestBuilder.build(DaemonResponse.java:127)
	at com.netflix.spinnaker.halyard.core.tasks.v1.TaskRepository.lambda$submitTask$1(TaskRepository.java:48)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.Throwable: 401 Unauthorized
Anonymous caller does not have storage.objects.get access to halconfig/clouddriver/3.4.2-20180828182842/clouddriver-caching.yml.
	at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1070)
	at com.google.api.client.googleapis.media.MediaHttpDownloader.executeCurrentRequest(MediaHttpDownloader.java:245)
	at com.google.api.client.googleapis.media.MediaHttpDownloader.download(MediaHttpDownloader.java:199)
	at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeMediaAndDownloadTo(AbstractGoogleClientRequest.java:562)
	at com.google.api.services.storage.Storage$Objects$Get.executeMediaAndDownloadTo(Storage.java:6131)
	at com.netflix.spinnaker.halyard.core.registry.v1.GoogleProfileReader.getContents(GoogleProfileReader.java:113)
	at com.netflix.spinnaker.halyard.core.registry.v1.GoogleProfileReader.readProfile(GoogleProfileReader.java:73)
	at com.netflix.spinnaker.halyard.core.registry.v1.ProfileRegistry.readProfile(ProfileRegistry.java:50)
	at com.netflix.spinnaker.halyard.deploy.spinnaker.v1.profile.RegistryBackedProfileFactory.getBaseProfile(RegistryBackedProfileFactory.java:41)
	... 8 more

#2

Doing a hal config storage s3 returns
S3PersistentStore(bucket=spin-stage, rootFolder=front50, region=null, endpoint=https://myceph.domain.com, accessKeyId=redacted, secretAccessKey=redacted)