lundi 17 août 2020

Artillery.io always return failure: ensure.p95 < 200

I'm use artillery.io for load test our server. After run test always returned failure: ansure.p95 < 200

My scenario:

config:
  target: "https://testing.ros.chat:8080"
  ensure:
    p95: 200
    maxErrorRate: 1
  socketio:
    transports: ["websocket"]
  phases:
    - duration: 1  # Test for 60 seconds
      arrivalRate: 1 # Every second, add 10 users
      maxVusers: 1
      #rampTo: 10 # And ramp it up to 100 users in total over the 60s period
      name: "Ramping up the load"
    #- duration: 5 # Then resume the load test for 120s
    #  arrivalRate: 1 # With those 100 users we ramped up to in the first phase
    #  maxVusers: 1
    #  rampTo: 1 # And keep it steady at 100 users
    #  name: "Pushing a constant load"
  ws:
      # Ignore SSL certificate errors
      # - useful in *development* with self-signed certs
      rejectUnauthorized: false
  payload:
    path: "./users0.csv"
    fields:
      - "login"
      - "password"
      - "level"
    skipHeader: true
    order: sequence
scenarios:
  - name: "Scenario login user and get user params"
    engine: socketio
    flow:
      - log: "Login users  on server"
      - emit:
          channel: "login"
          data: {"login":"","password":"","APILevel":""}
          acknowledge:
            match:
              json: "$.0.status"
              value: "ok"
      - emit:
          channel: "get-user-params"
          data: null
          acknowledge:
            match:
              json: "$.0.httpServer"
              value: "testing.ros.chat:80"
      - think: 2
      - emit:
          channel: "logout"

Results:

Started phase 0 (Ramping up the load), duration: 1s @ 13:06:44(+0300) 2020-08-17
    Login users 9000 on server
Report @ 13:06:49(+0300) 2020-08-17
Elapsed time: 4 seconds
  Scenarios launched:  1
  Scenarios completed: 1
  Requests completed:  3
  Mean response/sec: 0.86
  Response time (msec):
    min: 0.8
    max: 1434.4
    median: 3.2
    p95: 1434.4
    p99: 1434.4
  Codes:
    0: 3

All virtual users finished
Summary report @ 13:06:49(+0300) 2020-08-17
  Scenarios launched:  1
  Scenarios completed: 1
  Requests completed:  3
  Mean response/sec: 0.86
  Response time (msec):
    min: 0.8
    max: 1434.4
    median: 3.2
    p95: 1434.4
    p99: 1434.4
  Scenario counts:
    Scenario login user and get user params: 1 (100%)
  Codes:
    0: 3

Log file: result.json
ensure condition failed: ensure.p95 < 200

I tried different values of p95 (0, 10, 100) but the result is always the same

Artillery version:

Artillery: 1.6.1 Artillery Pro: not installed (https://artillery.io/pro) Node.js: v10.19.0 OS: linux/x64

Aucun commentaire:

Enregistrer un commentaire