Managing pipelines through pipeline APIs


#1

Without having had the ability to dig into the pipeline templates in Spinnaker (mainly because they seem very far from stable and lack documentation). I have created a script that can take a template, generate our pipeline json files and then uploads them all via the pipeline API. I have been doing this manually up until now and recently just created a Jenkins job that I wanted to use to do this for me so others on my team can modify these.

To solve the problem locally when Auth is enabled, I found out that I could do something like this:

  curl -c cookies.jar -X POST -H "Authorization: Bearer ${SPINNAKER_GATE_TOKEN}" -Lv ${SPINNAKER_GATE_HOST}/login
  SPINNAKER_API_SESSION=$(awk '/SESSION/ { print $NF }' cookies.jar)
  rm cookies.jar

where SPINNAKER_GATE_TOKEN is retrieved through something like gcloud auth print-access-token because we are using Google OAuth. Unfortunately, we currently require @company.com domains to Authenticate which is a problem for our Jenkins Service account. I know that Authz is different than Authn in Spinnaker but are there any suggested workarounds such that my Google Jenkins Service account could be made to work… Alternatively we do also have a Spinnaker Service account. Any reason why the Spinnaker service account couldn’t be made to work with connecting to Spinnaker APIs?

Would love to hear how best to provide some sort of automated solution here. Thanks!


#2

Hi

We’ve created a CLI to help automating use cases like this: https://github.com/spinnaker/spin. The docs covering its supported use cases are here: https://www.spinnaker.io/guides/spin/cli/. We just added OAuth2 authentication against Spinnaker recently, and X.509 is also supported.

spin is still alpha but sounds like it suits your use case fairly well. Check it out and let us know if you have any questions.

Unfortunately, we currently require @company.com domains to Authenticate which is a problem for our Jenkins Service account

Can you go into more detail about the issue here?

Any reason why the Spinnaker service account couldn’t be made to work with connecting to Spinnaker APIs?

The Spinnaker service accounts were created to solve a particular issue with automated pipeline execution permissions. We’ve given some thought into expanding the scope for more/different operations, but we ultimately decided to steer away from inventing our own authentication/authorization scheme and implementing a secret store within Spinnaker itself.


#3

I was aware of this project although I hadn’t seen any guides posted about it. It seems like a bit much to take on right now when we already have a process in place that doesn’t require re-learning a whole new toolset. I will mess around with this in my spare time but so far even getting it installed has been a little bit of a headache.

Sad to hear about the service account not being able to make API calls as that would’ve been a quick fix for us. As far as the @company.com remark… I just meant that because our google service account doesn’t have an @company.com account it gets an error when trying to authenticate with its access token.

I probably should know more about X.509 but that’s not something I’m familiar with… any sort of guide that walks through

  1. connecting spin to an external server (seems to default to localhost)
  2. authenticating with Google Oauth provider
  3. a basic pipeline setup using spin
  4. managing multiple pipelines
  5. is this the same concept as the “Pipeline Templates” i’ve read about? If so, even that topic was lacking a lot of examples and documentation last I checked…

… I will try to dig through these things on my own time but definitely don’t have time in my day to day to go beyond reading a tutorial or example that shows us how to manage this stuff.

As it stands already we are targeting the latest in master due to other issues with auth that have since been fixed so we do have the latest (I saw OAuth support was coming in 1.9 … according to the docs)

If you have any other guidance it’s greatly appreciated. I’ll try to post back if I am able to figure some of this stuff out.


#4

Your issue stems from mixing and matching GCP service accounts with Spinnaker service accounts. Your best bet for this may be to create a dummy/robot user using the Google Admin console and using that account in your automation.


#5

@ttomsu
I don’t believe I’m mixing them. To my knowledge they are completely unrelated. I had considered creating a real user but feel like that would cause a bigger hassle trying to manage oauth sessions for a real user in jenkins and to my knowledge that is the exact reason that GCP service accounts exist. I do use a spinnaker service account to run my pipelines as the accounts (k8s contexts) are locked down to google groups. My question from before was why I couldn’t use my spinnaker service account to “login” to spinnaker and make pipeline updates on its behalf… but it sounds like the Spinnaker team decided not to go in that direction.

Sorry if I wasn’t clear :slight_smile: I think my understanding is correct


#6

@briananderson1222 Are your pipelines gated by domain or by group membership? It is possible to put a Google service account into a Google Group, which I presume would work for Spinnaker.


#7

@morgantep Domain is a restriction of Authentication, Google Groups is for Spinnaker Account (k8s cluster) Authorization. It appears that the APIs first check the former (which even our Jenkins Service Account in GCP is not @company.com). And even if it were in a @company.com google group, would that get it past the domain authentication requirement? I guess I’m not 100% on how that actually works…


#8

We tried adding our Jenkins Service Account to a google group per your suggestion @morgantep. Unfortunately, this was not enough to get past the authentication flow in Spinnaker :confused:

Definitely open to other suggestions here.