I have successfully setup Kayenta and tested Canary Analysis feature, it’s a pretty awesome and fascinating product. I just got 1 question here:
How can I make more sophisticated comparison between Baseline and Canary deployment? I can only see “increase” “decrease” “either” in my metric, this is quite basic. How can I use it to validate a scenario like “If Canary CPU usage is 20% higher than Baseline CPU usage”?
I think it may be related to ACA Judge module? I’m using default NetflixACAJudge-v1.0, maybe some 3rd party Judge can provide this kind of advanced function? But I don’t know where to find them, is there a community for Judges?