Fetching manifests from a git repo


#1

Hi there!

I recently got started with Spinnker 1.6.0 and am particularly interested in using it to drive deployments to Kubernetes clusters. At our company we use a stash (aka bitbucket server) repo that holds our Kubernetes manifest files. I’m having a hard time figuring out how to fetch manifest files from a repo that isn’t GitHub based. Does enabling a Stash account on Igor somehow enable this workflow?

Any help would be much appreciated.

Thanks!
// Jon


#2

Hey Jon, there’s no support for fetching stash artifacts yet, but adding support is pretty straight-forward. Given that we can already trigger off of stash changes in Igor, all that’s needed is support for stash credentials in Clouddriver. See the commit that added GitHub support here.

If you (or someone else) is inclined I’d be happy to help you get support in place.


#3

Thanks for getting back to me and providing that context. I’ll let you know if we’re going to start implementing it. If we do, it probably won’t be for a while since we’re still evaluating our options in the CD-o-sphere.


#4

@lwander I might be interested to give it a try on adding the stash support, what would be the next step or process to get that in place?


#5

Awesome! I would take a look at the commit I referenced in this comment. Adding support should be as simple as implementing the same functionality.


#6

I found this one:


and seems it will work for us for BB,
thing is when I enable it I don’t see it in dropdown in spinnaker .
@lwander, can you confirm it should be there?


#7

Which steps did you take to enable it @ckorobov?


#8

Here they are, @lwander :

hal config features edit --artifacts true
hal config artifact http account add my-http-account 

and I see this from hal:

root@cklaptop:/workdir# hal config artifact http account get test-http
+ Get current deployment
  Success
+ Get test-http artifact account
  Success
+ Artifact account test-http: 
 HttpArtifactAccount(name=test-http, username=null, password=null, usernamePasswordFile=null)
root@cklaptop:/workdir# 

and hal config:

  artifacts:
gcs:
  enabled: false
  accounts: []
github:
  enabled: true
  accounts:
  - name: my-github-artifact-account
    tokenFile: /root/.hal/github_token
http:
  enabled: true
  accounts:
  - name: test-http

Still in GUI it’s only github, GCS and docker options.


#9

Awesome, thanks for all the details – what are you seeing in the UI? No accounts, or just not the ones you’ve added? Can you hit the API endpoint (gate, usually at localhost:8084) with the path /artifacts/credentials and see if they show up there?


#10

Only Github:

root@cklaptop:/workdir# curl http://spinnaker-api/artifacts/credentials
[{“name”:“my-github-artifact-account”}]

UI looks like this:


#11

@lwander, do you need any additional input?


#12

@ckorobov, were you able to sort this out? I have the same issue as this.

Thanks.


#13

Yes, just go with custom, use URL from BB, and use credentials as described here (I created separate read-only user):


#14

Thanks for your quick reply. Will try it out.

Cheers.


#15

It’s not clear how to specify the http credentials:

http:
  enabled: true
  accounts:
  - name: bitbucket.steveszabo
    usernamePasswordFile: /home/spinnaker/.hal/password.bitbucket


#16

Hi all, I’m having trouble checking out artifacts from an on prem Bitbucket instance.
I have configured the credentials correctly. When I query gate for the credentials it returns me correct response.
[{“name”:“embedded-artifact”},{“name”:“eg-bitbucket-account”}]

The error I keep getting is :

##### Exception ( Deploy Manifest )
Missing required field 'metadata' on manifest: {errors=[{exceptionName=com.atlassian.bitbucket.AuthorisationException, message=You are not permitted to access this resource}]}

I tried hitting the Bitbucket resource using curl and I get the correct response. If change the password to an invalid value and I get the same error as above using curl.

I know I ran hal deploy apply after setting up bitbucket artifact account and I see my clouddriver and igo service redeployed.

I’m not quite sure what could possibly be wrong ? It seems like Spinnaker isn’t using the correct account ?
Any suggestions.

Here are the logs:
2018-08-10 18:35:11.506 ERROR 1 — [0.0-7002-exec-4] c.n.s.k.w.e.GenericExceptionHandlers : Internal Server Error

com.netflix.spinnaker.clouddriver.kubernetes.v2.description.manifest.MalformedManifestException: Missing required field 'metadata' on manifest:
{errors=[{exceptionName=com.atlassian.bitbucket.AuthorisationException, message=You are not permitted to access this resource}]}
	at com.netflix.spinnaker.clouddriver.kubernetes.v2.description.manifest.MalformedManifestException.missingField(MalformedManifestException.java:27) ~[clouddriver-kubernetes-2.54.0-SNAPSHOT.jar:2.54.0-SNAPSHOT]
	at com.netflix.spinnaker.clouddriver.kubernetes.v2.description.manifest.KubernetesManifest.getRequiredField(KubernetesManifest.java:45) ~[clouddriver-kubernetes-2.54.0-SNAPSHOT.jar:2.54.0-SNAPSHOT]
	at com.netflix.spinnaker.clouddriver.kubernetes.v2.description.manifest.KubernetesManifest.getMetadata(KubernetesManifest.java:73) ~[clouddriver-kubernetes-2.54.0-SNAPSHOT.jar:2.54.0-SNAPSHOT]
	at com.netflix.spinnaker.clouddriver.kubernetes.v2.description.manifest.KubernetesManifest.getNamespace(KubernetesManifest.java:88) ~[clouddriver-kubernetes-2.54.0-SNAPSHOT.jar:2.54.0-SNAPSHOT]
	at com.netflix.spinnaker.clouddriver.kubernetes.v2.validator.manifest.KubernetesDeployManifestValidator.validate(KubernetesDeployManifestValidator.java:55) ~[clouddriver-kubernetes-2.54.0-SNAPSHOT.jar:2.54.0-SNAPSHOT]
	at com.netflix.spinnaker.clouddriver.kubernetes.v2.validator.manifest.KubernetesDeployManifestValidator$validate.call(Unknown Source) ~[na:na]
	at com.netflix.spinnaker.clouddriver.controllers.OperationsController$_convert_closure1$_closure5.doCall(OperationsController.groovy:163) ~[clouddriver-web-2.54.0-SNAPSHOT.jar:2.54.0-SNAPSHOT]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_171]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_171]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_171]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_171]
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) ~[groovy-all-2.4.13.jar:2.4.13]
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) ~[groovy-all-2.4.13.jar:2.4.13]
	at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294) ~[groovy-all-2.4.13.jar:2.4.13]
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) ~[groovy-all-2.4.13.jar:2.4.13]
	at groovy.lang.Closure.call(Closure.java:414) ~[groovy-all-2.4.13.jar:2.4.13]
	at org.codehaus.groovy.runtime.DefaultGroovyMethods.callClosureForMapEntry(DefaultGroovyMethods.java:5276) ~[groovy-all-2.4.13.jar:2.4.13]
	at org.codehaus.groovy.runtime.DefaultGroovyMethods.collect(DefaultGroovyMethods.java:3478) ~[groovy-all-2.4.13.jar:2.4.13]
	at org.codehaus.groovy.runtime.DefaultGroovyMethods.collect(DefaultGroovyMethods.java:3495) ~[groovy-all-2.4.13.jar:2.4.13]
	at org.codehaus.groovy.runtime.dgm$68.invoke(Unknown Source) ~[groovy-all-2.4.13.jar:2.4.13]
	at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:274) ~[groovy-all-2.4.13.jar:2.4.13]
	at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56) ~[groovy-all-2.4.13.jar:2.4.13]
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125) ~[groovy-all-2.4.13.jar:2.4.13]
	at com.netflix.spinnaker.clouddriver.controllers.OperationsController$_convert_closure1.doCall(OperationsController.groovy:149) ~[clouddriver-web-2.54.0-SNAPSHOT.jar:2.54.0-SNAPSHOT]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_171]