Kubernetes canary deploy with spinnaker keyenta


#1

canary config with FILTER TEMPLATES

{
  "applications": [
    "jimi"
  ],
  "classifier": {
    "groupWeights": {
      "CPU": 100
    },
    "scoreThresholds": {
      "marginal": 50,
      "pass": 75
    }
  },
  "configVersion": "1",
  "createdTimestamp": 1528446568129,
  "createdTimestampIso": "2018-06-08T08:29:28.129Z",
  "description": "canary test",
  "judge": {
    "judgeConfigurations": {},
    "name": "NetflixACAJudge-v1.0"
  },
  "metrics": [
    {
      "analysisConfigurations": {
        "canary": {
          "direction": "decrease"
        }
      },
      "groups": [
        "CPU"
      ],
      "name": "norma-use",
      "query": {
        "customFilterTemplate": "tomcat-select",
        "groupByFields": [],
        "labelBindings": [],
        "metricName": "container_cpu_usage_seconds_total",
        "serviceType": "prometheus",
        "type": "prometheus"
      },
      "scopeName": "default"
    }
  ],
  "name": "my-first-canary",
  "templates": {
    "tomcat-select": "container_name=\"${container_name}\""
  },
  "updatedTimestamp": 1528879983613,
  "updatedTimestampIso": "2018-06-13T08:53:03.613Z"
}

and my canary pipeline

{
  "analysisType": "realTime",
  "canaryConfig": {
"beginCanaryAnalysisAfterMins": "0",
"canaryAnalysisIntervalMins": "1",
"canaryConfigId": "c8a3d9af-5391-4cbb-816f-34d338b7eae0",
"combinedCanaryResultStrategy": "LOWEST",
"lifetimeHours": "1",
"metricsAccountName": "ACCOUNT",
"scopes": [
  {
    "controlLocation": "default",
    "controlScope": "tomcat-baseline-deployment",
    "experimentLocation": "default",
    "experimentScope": "tomcat-canary-deployment",
    "extendedScopeParams": {
      "container_name": "tomcat-baseline-deployment"
    },
    "scopeName": "default",
    "step": 30
  }
],
"scoreThresholds": {
  "marginal": "50",
  "pass": "75"
},
"storageAccountName": "mionio-s3"
  },
  "completeOtherBranchesThenFail": false,
  "continuePipeline": true,
  "failPipeline": false,
  "name": "Canary Analysis",
  "type": "kayentaCanary"
}

but the kayenta log do not request the baseline and canary, that request the tomcat-baseline-deployment twice.

2018-06-13 16:54:08.784  INFO 26095 --- [0.1-8090-exec-3] com.amazonaws.latency                    : ServiceName=[Amazon S3], StatusCode=[200], ServiceEndpoint=[http://192.168.105.2:9001], RequestType=[ListObjectsRequest], AWSRequestID=[1537AC50343762D8], HttpClientPoolPendingCount=0, RetryCapacityConsumed=0, HttpClientPoolAvailableCount=1, RequestCount=1, HttpClientPoolLeasedCount=0, ResponseProcessingTime=[1.26], ClientExecuteTime=[5.402], HttpClientSendRequestTime=[0.117], HttpRequestTime=[3.226], RequestSigningTime=[0.347], CredentialsRequestTime=[0.007, 0.003], HttpClientReceiveResponseTime=[1.353], 
2018-06-13 16:54:08.787  INFO 26095 --- [0.1-8090-exec-3] com.amazonaws.latency                    : ServiceName=[Amazon S3], StatusCode=[200], ServiceEndpoint=[http://192.168.105.2:9001], RequestType=[GetObjectRequest], AWSRequestID=[1537AC50347411B4], HttpClientPoolPendingCount=0, RetryCapacityConsumed=0, HttpClientPoolAvailableCount=1, RequestCount=1, HttpClientPoolLeasedCount=0, ResponseProcessingTime=[0.192], ClientExecuteTime=[3.032], HttpClientSendRequestTime=[0.141], HttpRequestTime=[2.192], RequestSigningTime=[0.237], CredentialsRequestTime=[0.005, 0.003], HttpClientReceiveResponseTime=[1.709], 
2018-06-13 16:54:08.806  INFO 26095 --- [    scheduler-3] com.netflix.spinnaker.q.QueueProcessor   : Received message StartExecution(executionType=pipeline, executionId=0858b9b4-d5d6-48c5-b30d-1322a3f8f58c, application=jimi)
2018-06-13 16:54:08.822  INFO 26095 --- [    scheduler-3] com.netflix.spinnaker.q.QueueProcessor   : Received message StartStage(executionType=pipeline, executionId=0858b9b4-d5d6-48c5-b30d-1322a3f8f58c, application=jimi, stageId=12b06c8c-5d7c-47c2-80e2-2badd6145a05)
2018-06-13 16:54:08.851  INFO 26095 --- [    scheduler-9] com.netflix.spinnaker.q.QueueProcessor   : Received message StartTask(executionType=pipeline, executionId=0858b9b4-d5d6-48c5-b30d-1322a3f8f58c, application=jimi, stageId=12b06c8c-5d7c-47c2-80e2-2badd6145a05, taskId=1)
2018-06-13 16:54:08.881  INFO 26095 --- [    scheduler-9] com.netflix.spinnaker.q.QueueProcessor   : Received message RunTask(executionType=pipeline, executionId=0858b9b4-d5d6-48c5-b30d-1322a3f8f58c, application=jimi, stageId=12b06c8c-5d7c-47c2-80e2-2badd6145a05, taskId=1, taskType=class com.netflix.kayenta.canary.orca.SetupCanaryTask)
2018-06-13 16:54:08.906  INFO 26095 --- [    scheduler-9] com.netflix.spinnaker.q.QueueProcessor   : Received message CompleteTask(executionType=pipeline, executionId=0858b9b4-d5d6-48c5-b30d-1322a3f8f58c, application=jimi, stageId=12b06c8c-5d7c-47c2-80e2-2badd6145a05, taskId=1, status=SUCCEEDED)
2018-06-13 16:54:08.931  INFO 26095 --- [    scheduler-3] com.netflix.spinnaker.q.QueueProcessor   : Received message CompleteStage(executionType=pipeline, executionId=0858b9b4-d5d6-48c5-b30d-1322a3f8f58c, application=jimi, stageId=12b06c8c-5d7c-47c2-80e2-2badd6145a05)
2018-06-13 16:54:08.944  INFO 26095 --- [    scheduler-3] com.netflix.spinnaker.q.QueueProcessor   : Received message StartStage(executionType=pipeline, executionId=0858b9b4-d5d6-48c5-b30d-1322a3f8f58c, application=jimi, stageId=6c60d9d2-57b7-4372-8be2-92debbae9829)
2018-06-13 16:54:08.959  INFO 26095 --- [    scheduler-3] com.netflix.spinnaker.q.QueueProcessor   : Received message StartStage(executionType=pipeline, executionId=0858b9b4-d5d6-48c5-b30d-1322a3f8f58c, application=jimi, stageId=0528600a-2368-4c8a-80c2-732d5272956f)
2018-06-13 16:54:08.972  INFO 26095 --- [    scheduler-3] com.netflix.spinnaker.q.QueueProcessor   : Received message StartTask(executionType=pipeline, executionId=0858b9b4-d5d6-48c5-b30d-1322a3f8f58c, application=jimi, stageId=6c60d9d2-57b7-4372-8be2-92debbae9829, taskId=1)
2018-06-13 16:54:08.984  INFO 26095 --- [    scheduler-3] com.netflix.spinnaker.q.QueueProcessor   : Received message StartTask(executionType=pipeline, executionId=0858b9b4-d5d6-48c5-b30d-1322a3f8f58c, application=jimi, stageId=0528600a-2368-4c8a-80c2-732d5272956f, taskId=1)
2018-06-13 16:54:08.996  INFO 26095 --- [    scheduler-3] com.netflix.spinnaker.q.QueueProcessor   : Received message RunTask(executionType=pipeline, executionId=0858b9b4-d5d6-48c5-b30d-1322a3f8f58c, application=jimi, stageId=6c60d9d2-57b7-4372-8be2-92debbae9829, taskId=1, taskType=class com.netflix.kayenta.prometheus.orca.PrometheusFetchTask)
2018-06-13 16:54:09.002  INFO 26095 --- [    handlers-19] c.n.k.p.service.PrometheusRemoteService  : ---> HTTP GET http://prometheus.ingress.dev.yohocorp.com/api/v1/query_range?query=avg%28container_cpu_usage_seconds_total%7Bcontainer_name%3D%22tomcat-baseline-deployment%22%7D%29&start=2018-06-13T08%3A53%3A07.712Z&end=2018-06-13T08%3A54%3A07.712Z&step=30
2018-06-13 16:54:09.008  INFO 26095 --- [    handlers-19] c.n.k.p.service.PrometheusRemoteService  : <--- HTTP 200 http://prometheus.ingress.dev.yohocorp.com/api/v1/query_range?query=avg%28container_cpu_usage_seconds_total%7Bcontainer_name%3D%22tomcat-baseline-deployment%22%7D%29&start=2018-06-13T08%3A53%3A07.712Z&end=2018-06-13T08%3A54%3A07.712Z&step=30 (5ms)
2018-06-13 16:54:09.009  INFO 26095 --- [    scheduler-3] com.netflix.spinnaker.q.QueueProcessor   : Received message RunTask(executionType=pipeline, executionId=0858b9b4-d5d6-48c5-b30d-1322a3f8f58c, application=jimi, stageId=0528600a-2368-4c8a-80c2-732d5272956f, taskId=1, taskType=class com.netflix.kayenta.prometheus.orca.PrometheusFetchTask)
2018-06-13 16:54:09.013  INFO 26095 --- [    handlers-19] com.amazonaws.latency                    : ServiceName=[Amazon S3], StatusCode=[200], ServiceEndpoint=[http://192.168.105.2:9001], RequestType=[HeadBucketRequest], AWSRequestID=[null], HttpClientPoolPendingCount=0, RetryCapacityConsumed=0, HttpClientPoolAvailableCount=1, RequestCount=1, HttpClientPoolLeasedCount=0, ResponseProcessingTime=[0.029], ClientExecuteTime=[2.328], HttpClientSendRequestTime=[0.062], HttpRequestTime=[1.453], RequestSigningTime=[0.363], CredentialsRequestTime=[0.007, 0.004], HttpClientReceiveResponseTime=[0.84], 
2018-06-13 16:54:09.014  INFO 26095 --- [    handlers-20] c.n.k.p.service.PrometheusRemoteService  : ---> HTTP GET http://prometheus.ingress.dev.yohocorp.com/api/v1/query_range?query=avg%28container_cpu_usage_seconds_total%7Bcontainer_name%3D%22tomcat-baseline-deployment%22%7D%29&start=2018-06-13T08%3A53%3A07.712Z&end=2018-06-13T08%3A54%3A07.712Z&step=30
2018-06-13 16:54:09.043  INFO 26095 --- [    handlers-19] com.amazonaws.latency                    : ServiceName=[Amazon S3], StatusCode=[200], ServiceEndpoint=[http://192.168.105.2:9001], RequestType=[PutObjectRequest], AWSRequestID=[1537AC5043B69D0C], HttpClientPoolPendingCount=0, RetryCapacityConsumed=0, HttpClientPoolAvailableCount=1, RequestCount=1, HttpClientPoolLeasedCount=0, ResponseProcessingTime=[0.056], ClientExecuteTime=[26.064], HttpClientSendRequestTime=[1.783], HttpRequestTime=[25.27], RequestSigningTime=[0.326], CredentialsRequestTime=[0.005, 0.003], HttpClientReceiveResponseTime=[23.098], 
2018-06-13 16:54:09.053  INFO 26095 --- [    scheduler-3] com.netflix.spinnaker.q.QueueProcessor   : Received message CompleteTask(executionType=pipeline, executionId=0858b9b4-d5d6-48c5-b30d-1322a3f8f58c, application=jimi, stageId=6c60d9d2-57b7-4372-8be2-92debbae9829, taskId=1, status=SUCCEEDED)
2018-06-13 16:54:09.064  INFO 26095 --- [    scheduler-3] com.netflix.spinnaker.q.QueueProcessor   : Received message CompleteStage(executionType=pipeline, executionId=0858b9b4-d5d6-48c5-b30d-1322a3f8f58c, application=jimi, stageId=6c60d9d2-57b7-4372-8be2-92debbae9829)
2018-06-13 16:54:09.072  INFO 26095 --- [    handlers-20] c.n.k.p.service.PrometheusRemoteService  : <--- HTTP 200 http://prometheus.ingress.dev.yohocorp.com/api/v1/query_range?query=avg%28container_cpu_usage_seconds_total%7Bcontainer_name%3D%22tomcat-baseline-deployment%22%7D%29&start=2018-06-13T08%3A53%3A07.712Z&end=2018-06-13T08%3A54%3A07.712Z&step=30 (57ms)
2018-06-13 16:54:09.075  INFO 26095 --- [    handlers-20] com.amazonaws.latency                    : ServiceName=[Amazon S3], StatusCode=[200], ServiceEndpoint=[http://192.168.105.2:9001], RequestType=[HeadBucketRequest], AWSRequestID=[null], HttpClientPoolPendingCount=0, RetryCapacityConsumed=0, HttpClientPoolAvailableCount=1, RequestCount=1, HttpClientPoolLeasedCount=0, ResponseProcessingTime=[0.021], ClientExecuteTime=[2.311], HttpClientSendRequestTime=[0.083], HttpRequestTime=[1.299], RequestSigningTime=[0.494], CredentialsRequestTime=[0.009, 0.005], HttpClientReceiveResponseTime=[0.734], 
2018-06-13 16:54:09.076  INFO 26095 --- [   scheduler-10] com.netflix.spinnaker.q.QueueProcessor   : Received message StartStage(executionType=pipeline, executionId=0858b9b4-d5d6-48c5-b30d-1322a3f8f58c, application=jimi, stageId=0f8e252c-9d6c-4cc3-abfc-b60c6eb8258d)
2018-06-13 16:54:09.080  WARN 26095 --- [    handlers-19] c.n.s.orca.q.handler.StartStageHandler   : Re-queuing StartStage(executionType=pipeline, executionId=0858b9b4-d5d6-48c5-b30d-1322a3f8f58c, application=jimi, stageId=0f8e252c-9d6c-4cc3-abfc-b60c6eb8258d) as upstream stages are not yet complete
2018-06-13 16:54:09.089  INFO 26095 --- [    handlers-20] com.amazonaws.latency                    : ServiceName=[Amazon S3], StatusCode=[200], ServiceEndpoint=[http://192.168.105.2:9001], RequestType=[PutObjectRequest], AWSRequestID=[1537AC50467AC6F9], HttpClientPoolPendingCount=0, RetryCapacityConsumed=0, HttpClientPoolAvailableCount=1, RequestCount=1, HttpClientPoolLeasedCount=0, ResponseProcessingTime=[0.107], ClientExecuteTime=[12.8], HttpClientSendRequestTime=[1.621], HttpRequestTime=[11.453], RequestSigningTime=[0.823], CredentialsRequestTime=[0.005, 0.004], HttpClientReceiveResponseTime=[9.482], 
2018-06-13 16:54:09.099  INFO 26095 --- [   scheduler-10] com.netflix.spinnaker.q.QueueProcessor   : Received message CompleteTask(executionType=pipeline, executionId=0858b9b4-d5d6-48c5-b30d-1322a3f8f58c, application=jimi, stageId=0528600a-2368-4c8a-80c2-732d5272956f, taskId=1, status=SUCCEEDED)
2018-06-13 16:54:09.110  INFO 26095 --- [   scheduler-10] com.netflix.spinnaker.q.QueueProcessor   : Received message CompleteStage(executionType=pipeline, executionId=0858b9b4-d5d6-48c5-b30d-1322a3f8f58c, application=jimi, stageId=0528600a-2368-4c8a-80c2-732d5272956f)
2018-06-13 16:54:09.117  WARN 26095 --- [    handlers-20] c.netflix.spinnaker.q.redis.RedisQueue   : Re-prioritizing message as an identical one is already on the queue: cee7d51cc06daa183e5fd71e4f1d1739, message: StartStage(executionType=pipeline, executionId=0858b9b4-d5d6-48c5-b30d-1322a3f8f58c, application=jimi, stageId=0f8e252c-9d6c-4cc3-abfc-b60c6eb8258d)
2018-06-13 16:54:09.122  INFO 26095 --- [   scheduler-10] com.netflix.spinnaker.q.QueueProcessor   : Received message StartStage(executionType=pipeline, executionId=0858b9b4-d5d6-48c5-b30d-1322a3f8f58c, application=jimi, stageId=0f8e252c-9d6c-4cc3-abfc-b60c6eb8258d)
2018-06-13 16:54:09.134  INFO 26095 --- [   scheduler-10] com.netflix.spinnaker.q.QueueProcessor   : Received message StartTask(executionType=pipeline, executionId=0858b9b4-d5d6-48c5-b30d-1322a3f8f58c, application=jimi, stageId=0f8e252c-9d6c-4cc3-abfc-b60c6eb8258d, taskId=1)
2018-06-13 16:54:09.146  INFO 26095 --- [   scheduler-10] com.netflix.spinnaker.q.QueueProcessor   : Received message RunTask(executionType=pipeline, executionId=0858b9b4-d5d6-48c5-b30d-1322a3f8f58c, application=jimi, stageId=0f8e252c-9d6c-4cc3-abfc-b60c6eb8258d, taskId=1, taskType=class com.netflix.kayenta.metrics.orca.MetricSetMixerServiceTask)
2018-06-13 16:54:09.157  INFO 26095 --- [    handlers-20] com.amazonaws.latency                    : ServiceName=[Amazon S3], StatusCode=[200], ServiceEndpoint=[http://192.168.105.2:9001], RequestType=[ListObjectsRequest], AWSRequestID=[1537AC504A76908F], HttpClientPoolPendingCount=0, RetryCapacityConsumed=0, HttpClientPoolAvailableCount=1, RequestCount=1, HttpClientPoolLeasedCount=0, ResponseProcessingTime=[1.151], ClientExecuteTime=[4.707], HttpClientSendRequestTime=[0.07], HttpRequestTime=[2.619], RequestSigningTime=[0.298], CredentialsRequestTime=[0.007, 0.003], HttpClientReceiveResponseTime=[1.969], 
2018-06-13 16:54:09.159  INFO 26095 --- [    handlers-20] com.amazonaws.latency                    : ServiceName=[Amazon S3], StatusCode=[200], ServiceEndpoint=[http://192.168.105.2:9001], RequestType=[GetObjectRequest], AWSRequestID=[1537AC504AAB8C43], HttpClientPoolPendingCount=0, RetryCapacityConsumed=0, HttpClientPoolAvailableCount=1, RequestCount=1, HttpClientPoolLeasedCount=0, ResponseProcessingTime=[0.148], ClientExecuteTime=[2.25], HttpClientSendRequestTime=[0.054], HttpRequestTime=[1.544], RequestSigningTime=[0.238], CredentialsRequestTime=[0.004, 0.004], HttpClientReceiveResponseTime=[1.22], 
2018-06-13 16:54:09.168  INFO 26095 --- [    handlers-20] com.amazonaws.latency                    : ServiceName=[Amazon S3], StatusCode=[200], ServiceEndpoint=[http://192.168.105.2:9001], RequestType=[ListObjectsRequest], AWSRequestID=[1537AC504B18D87B], HttpClientPoolPendingCount=0, RetryCapacityConsumed=0, HttpClientPoolAvailableCount=1, RequestCount=1, HttpClientPoolLeasedCount=0, ResponseProcessingTime=[1.32], ClientExecuteTime=[4.449], HttpClientSendRequestTime=[0.059], HttpRequestTime=[2.46], RequestSigningTime=[0.28], CredentialsRequestTime=[0.005, 0.003], HttpClientReceiveResponseTime=[1.981], 
2018-06-13 16:54:09.171  INFO 26095 --- [    handlers-20] com.amazonaws.latency                    : ServiceName=[Amazon S3], StatusCode=[200], ServiceEndpoint=[http://192.168.105.2:9001], RequestType=[GetObjectRequest], AWSRequestID=[1537AC504B53F752], HttpClientPoolPendingCount=0, RetryCapacityConsumed=0, HttpClientPoolAvailableCount=1, RequestCount=1, HttpClientPoolLeasedCount=0, ResponseProcessingTime=[0.158], ClientExecuteTime=[2.474], HttpClientSendRequestTime=[0.062], HttpRequestTime=[1.605], RequestSigningTime=[0.274], CredentialsRequestTime=[0.004, 0.003], HttpClientReceiveResponseTime=[1.221], 
2018-06-13 16:54:09.180  INFO 26095 --- [    handlers-20] com.amazonaws.latency                    : ServiceName=[Amazon S3], StatusCode=[200], ServiceEndpoint=[http://192.168.105.2:9001], RequestType=[HeadBucketRequest], AWSRequestID=[null], HttpClientPoolPendingCount=0, RetryCapacityConsumed=0, HttpClientPoolAvailableCount=1, RequestCount=1, HttpClientPoolLeasedCount=0, ResponseProcessingTime=[0.021], ClientExecuteTime=[1.887], HttpClientSendRequestTime=[0.063], HttpRequestTime=[1.202], RequestSigningTime=[0.28], CredentialsRequestTime=[0.006, 0.003], HttpClientReceiveResponseTime=[0.747], 
2018-06-13 16:54:09.201  INFO 26095 --- [    handlers-20] com.amazonaws.latency                    : ServiceName=[Amazon S3], StatusCode=[200], ServiceEndpoint=[http://192.168.105.2:9001], RequestType=[PutObjectRequest], AWSRequestID=[1537AC504D2528BA], HttpClientPoolPendingCount=0, RetryCapacityConsumed=0, HttpClientPoolAvailableCount=1, RequestCount=1, HttpClientPoolLeasedCount=0, ResponseProcessingTime=[0.054], ClientExecuteTime=[14.563], HttpClientSendRequestTime=[1.699], HttpRequestTime=[13.712], RequestSigningTime=[0.339], CredentialsRequestTime=[0.006, 0.003], HttpClientReceiveResponseTime=[11.605], 
2018-06-13 16:54:09.210  INFO 26095 --- [   scheduler-10] com.netflix.spinnaker.q.QueueProcessor   : Received message CompleteTask(executionType=pipeline, executionId=0858b9b4-d5d6-48c5-b30d-1322a3f8f58c, application=jimi, stageId=0f8e252c-9d6c-4cc3-abfc-b60c6eb8258d, taskId=1, status=SUCCEEDED)
2018-06-13 16:54:09.222  INFO 26095 --- [   scheduler-10] com.netflix.spinnaker.q.QueueProcessor   : Received message CompleteStage(executionType=pipeline, executionId=0858b9b4-d5d6-48c5-b30d-1322a3f8f58c, application=jimi, stageId=0f8e252c-9d6c-4cc3-abfc-b60c6eb8258d)
2018-06-13 16:54:09.234  INFO 26095 --- [   scheduler-10] com.netflix.spinnaker.q.QueueProcessor   : Received message StartStage(executionType=pipeline, executionId=0858b9b4-d5d6-48c5-b30d-1322a3f8f58c, application=jimi, stageId=449b68f1-a47c-440c-9355-c017fb1241bc)
2018-06-13 16:54:09.256  INFO 26095 --- [    handlers-20] c.n.s.o.p.u.ContextParameterProcessor    : Evaluated 1 expression(s) - (#stage( #root.execution, 'Mix Control and Experiment Results')['context']['metricSetPairListId']), 0 failed - ()
2018-06-13 16:54:09.269  INFO 26095 --- [   scheduler-10] com.netflix.spinnaker.q.QueueProcessor   : Received message StartTask(executionType=pipeline, executionId=0858b9b4-d5d6-48c5-b30d-1322a3f8f58c, application=jimi, stageId=449b68f1-a47c-440c-9355-c017fb1241bc, taskId=1)
2018-06-13 16:54:09.276  INFO 26095 --- [    handlers-20] c.n.s.o.p.u.ContextParameterProcessor    : Evaluated 1 expression(s) - (#stage( #root.execution, 'Mix Control and Experiment Results')['context']['metricSetPairListId']), 0 failed - ()
2018-06-13 16:54:09.293  INFO 26095 --- [   scheduler-10] com.netflix.spinnaker.q.QueueProcessor   : Received message RunTask(executionType=pipeline, executionId=0858b9b4-d5d6-48c5-b30d-1322a3f8f58c, application=jimi, stageId=449b68f1-a47c-440c-9355-c017fb1241bc, taskId=1, taskType=class com.netflix.kayenta.canary.orca.CanaryJudgeTask)
2018-06-13 16:54:09.306  INFO 26095 --- [    handlers-20] com.amazonaws.latency                    : ServiceName=[Amazon S3], StatusCode=[200], ServiceEndpoint=[http://192.168.105.2:9001], RequestType=[ListObjectsRequest], AWSRequestID=[1537AC50534CB68F], HttpClientPoolPendingCount=0, RetryCapacityConsumed=0, HttpClientPoolAvailableCount=1, RequestCount=1, HttpClientPoolLeasedCount=0, ResponseProcessingTime=[1.899], ClientExecuteTime=[5.175], HttpClientSendRequestTime=[0.073], HttpRequestTime=[2.381], RequestSigningTime=[0.346], CredentialsRequestTime=[0.007, 0.004], HttpClientReceiveResponseTime=[1.746], 
2018-06-13 16:54:09.309  INFO 26095 --- [    handlers-20] com.amazonaws.latency                    : ServiceName=[Amazon S3], StatusCode=[200], ServiceEndpoint=[http://192.168.105.2:9001], RequestType=[GetObjectRequest], AWSRequestID=[1537AC5053909EFE], HttpClientPoolPendingCount=0, RetryCapacityConsumed=0, HttpClientPoolAvailableCount=1, RequestCount=1, HttpClientPoolLeasedCount=0, ResponseProcessingTime=[0.198], ClientExecuteTime=[2.814], HttpClientSendRequestTime=[0.066], HttpRequestTime=[2.033], RequestSigningTime=[0.22], CredentialsRequestTime=[0.005, 0.003], HttpClientReceiveResponseTime=[1.63],

#2

Hi,

The issue is that your template does not use the scopes you’ve provided.

There are several ways you can use the scopes when expanding the templates and building the queries, but in the example you’ve given here I think the shortest path is to define your template like this:

"templates": {
  "tomcat-select": "container_name=\"${scope}\""
}

This works because there are several variables implicitly available for use in your templates: https://github.com/spinnaker/kayenta/blob/master/kayenta-prometheus/src/main/java/com/netflix/kayenta/prometheus/metrics/PrometheusMetricsService.java#L219

You can also drop the extendedScopeParams sections from your stage config since you are relying solely on implicitly-available variable bindings.

Thanks,
-Matt


#3

thanks,duftler

I have the other question,the CANARY ANALYSIS all way give the score :100 and 0,never give the score between 1 to 99 ,how to config it.

according to baseline and canary result kayenta give me a zero score.

baseline result show
{
“status”:“success”,
“data”:{
“resultType”:“matrix”,
“result”:[
{
“metric”:{

                },
                "values":[
                    [
                        1529460174.77,
                        "856.3229260945"
                    ],
                    [
                        1529460204.77,
                        "856.346932045"
                    ],
                    [
                        1529460234.77,
                        "856.4164676830001"
                    ]
                ]
            }
        ]
    }
}

canary result show

{
    "status":"success",
    "data":{
        "resultType":"matrix",
        "result":[
            {
                "metric":{

                },
                "values":[
                    [
                        1529460174.77,
                        "886.8335765444999"
                    ],
                    [
                        1529460204.77,
                        "886.9093274930001"
                    ],
                    [
                        1529460234.77,
                        "886.9352387665"
                    ]
                ]
            }
        ]
    }
}