Skip to end of metadata
Go to start of metadata

Introduction 

This page contains the description of the new benchmark based on the initial benchmark 2016.

The benchmark has been updated May 2017 and later in February 2018 to work with the latest version of the swagger-ui.

Software used for testing:

Background on benchmarking

Architecture of monitoring network and app activity by the Yona app and database. By filtering the relevant activities on each component the amount of data captured, transferred to and stored in the Yona database is reduced as much as possible.

  1. Yona app: captures data of relevant apps and network activity, No filtering is done here to reduce volume of messages. See for Swagger UI specification http://build.dev.yona.nu/swagger-ui/index.html.
  2. Smoothwall: Check every HTTP request, labels it with a relevant category and writes it to the access.log file.
  3. Perl script: filters the access.log from Smoothwall, when the Smootwall category occurs in the set of relevant Smoothwall categories (as exposed by the Analysis engine service), then the Perl script passes the relevant data of the log record on to the Analysis engine service on the Yona server.
  4. Yona server: incoming transactions are filtered, transaction are only relevant when the user has a challenge for that category. Then data reduction is done by evaluating if the transaction is relevant for challenge, if the challenge has a timeframe the transaction will only be processed if relevant. Further  reduction is done when writing the transaction in the database, only one record per relevant timeslot (1 minute, 15 minutes or 1 day) is stored. Regularly data compression runs to reduce data of previous periods. This is not yet documented. Currently, a data aggregation job runs at 3:00 AM.

Proposed network activity

It is useless to generate more data than the Yona server will have to process coming from the Perl script. Normal internet browsing will generate 50-500 http requests per minute! So the benchmark only needs to generate relevant app and network activity data.

The objective here is to create data, so the user can scroll back through their activity overview. For that, it's best to ingest this as app activities. That allows you to create a set of activities server side, each with a time span. That will make the data ingestion simpler and faster.

Benchmark Script

Part 1 of the script: Benchmark Initiation

In the Initiation phase the user and buddy are created, linked to each other and their goals are set.

  • Set Random Start and End DateTime, within a range defined via arguments in this BeanShell PreProcessor
  • User: Range via Variable (BeanShell ) between 10.000.000 - 39.999.999 (a simple Random variable is not good enough, too many duplicates to identical seed value)
  • User: Device is IOS so only Network Activity is sent (Feb 2018).
  • Buddy: Range via Variable (BeanShell ) between 40.000.000 - 59.999.999
  • Buddy: Device is Android so only App Activity and No Go Network Activity is sent (Feb 2018).
  • Set ServerName and Ports
    • serverName
    • analysisServicePort
    • appServicePort
    • batchServicePort
  • HTTP Header Manager
  • HTTP Request Defaults
  • Uniform Random Timer, Random delay max 5 sec, Constant delay 0.5 sec
  • Add User
  • Stop the thread if the user create fails due to existing user or whatever reason
  • HTTP Header Manager Set Yona-Password for user, after this each http request for user has this HTTP Header Manager
  • User Confirms Mobile Number

New part with Goals (Adult and Gambling have default a Budget Goal of 0 minutes)

  • User Set Goal on Communication (Budget)
  • User Set Goal on Multimedia (Budget)
  • User Set Goal on News (Budget)
  • User Set Goal on Social (TimeZone)
  • Add Buddy
  • Stop the thread if the user create fails due to existing user or whatever reason
  • HTTP Header Manager Set Yona-Password for buddy, after this each http request for buddy has this HTTP Header Manager
  • Buddy confirms mobile number

Set Goals for the Buddy (identical to above part but using the UserID of the Buddy)

  • Buddy Set Goal on Communication (Budget)
  • Buddy Set Goal on Multimedia (Budget)
  • Buddy Set Goal on News (Budget)
  • Buddy Set Goal on Social (TimeZone)
  • Uniform Random Timer, Random delay max 0.5 sec, Constant delay 0.5 sec
  • User requests Buddy to become his buddy
  • Buddy checks his messages
  • Buddy accepts User's buddy request
  • User checks his messages
  • Buddy checks his messages
  • User checks his messages
  • Write all variables into table - UserAndBuddyDetails.csv

 

Part 2 of the script: Data ingestion

The benchmark script consists of below activity on a specific day between X and Y days in the past for Z minutes as defined as argument in the "Set Random Start and End DateTime" above.

Loop for 20 times both for user:

  • Uniform Random Timer, Random delay max 0.050 sec, Constant delay 0.010 sec
  • Network Activity Adult
  • Network Activity Communication
  • Network Activity Gambling
  • Network Activity Multimedia
  • Network Activity News
  • Network Activity Social

Loop for 20 times both for user and for its buddy:

  • Uniform Random Timer, Random delay max 0.050 sec, Constant delay 0.010 sec
  • App Activity Adult
  • App Activity Communication
  • App Activity Gambling
  • App Activity Multimedia
  • App Activity News
  • App Activity Social
  • Network Activity Adult
  • Network Activity Gambling

Example of App Activity: /users/{userID}/appActivity/

{
  "deviceDateTime": "${__time(yyyy-MM-dd'T'HH:mm:ss.SSSZ)}",
  "activities": [
    {
      "application": "Bad app",
      "startTime": "${randomDateTimeStart}",
      "endTime": "${randomDateTimeEnd}"
    }
  ]
}

Example of Network Activity: /userAnonymized/${vpnLoginUserID_1}/networkActivity/ to analysisServicePort

{
   "eventTime": "${randomDateTimeStart}",
   "categories":["Adult Sites"],
   "url":"http://www.poker.com"
}

Now there is enough activity for the specific Goal or Goals to start benchmarking. 

Special action only for Thread = 1, the other Threads wait a couple of minutes:

  • Batch Aggregate Activities (/batch/aggregateActivities/ on the BatchServicePort)

Part 3 of the script: Benchmark performance

Here the performance is measured of reporting the activity, this emulates the activities the user does on the Yona App.

Don't set the Page and Size parameter but use the default and extract the NEXT link from the response to read additional pages.

The next link on the first request has page 1 (as the first page itself has page 0) and size 3

    "first" : {       "href" : "http://185.3.209.132/users/6d63a926-f468-4822-a982-ff3c8d99e580/activity/days/?page=0&size=3"     },

    "next" : {       "href" : "http://185.3.209.132/users/6d63a926-f468-4822-a982-ff3c8d99e580/activity/days/?page=1&size=3"     },

     "last" : {       "href" : "http://185.3.209.132/users/6d63a926-f468-4822-a982-ff3c8d99e580/activity/days/?page=52&size=3"     },

The following part both for user and for buddy 

  • Uniform Random Timer, Random delay max 1 sec, Constant delay 0.025 sec
  • Check messages User
  • Check User Day Activity
    • Extract UserDayDetailsUrl to extract Next link
  • Check User Week Activity
    • Extract UserWeekDetailsUrl to extract Next link
  • Check User and Buddies Day Activity
    • Extract UserandBuddyDayDetailsUrl to extract Next link

Now the next links are filled which is needed in below loop. 

First a Loop Controller which runs 5 times:

  • Uniform Random Timer, Random delay max 0.5 sec, Constant delay 0.050 sec
  • Random Controller
    • Loop Controller which runs once
      • Random Controller
        • Check User Day Activity
        • Check User Week Activity
        • Check User and Buddies Day Activity
        • Check User Day Activity Next Page
        • Check User Week Activity Next Page
        • Check User and Buddies Day Next Activity
    • Loop Controller which runs once
      • Random Controller
        • Check Buddy Day Activity
        • Check Buddy Week Activity
        • Check Buddy and Buddies Day Activity
        • Check Buddy Day Activity Next Page
        • Check Buddy Week Activity Next Page
        • Check Buddy and Buddies Day Next Activity
    • Loop Controller which runs once
      • Random Controller
        • Check messages User ( 6 times)
    • Loop Controller which runs once
      • Random Controller
        • Check messages Buddy( 6 times)
    • For Buddy 4 times a Loop controller which runs 5 times ( to spread load more random) 
      • Random Controller
        • App Activity Adult
        • App Activity Communication
        • App Activity Gambling
        • App Activity Multimedia
        • App Activity News
        • App Activity Social
    • For User 16 times a Loop controller which runs 20 times 
      • Random Controller
        • Network Activity Adult
        • Network Activity Communication
        • Network Activity Gambling
        • Network Activity Multimedia
        • Network Activity News
        • Network Activity Social
    • For Buddy 16 times a Loop controller which runs 20 times  
      • Random Controller
        • Network Activity Adult
        • Network Activity Gambling
  • Debug PostProcessor
  • View Results Tree - TREE.CSV
  • View Results in Table - TABLE.CSV
  • jp@gc - Hits per Second - HITS_PER_SECOND
  • jp@gc - Response Latencies Over Time - RESPONSETIMES
  • jp@gc - Graphs Generator - TREE.CSV

 

 

 

 

 

 

 

 

 

 

  • No labels