Skip to end of metadata
Go to start of metadata

Benchmark results 2018-10-08

Benchmark run 100 threads on 2018-10-08

benchmark run is comparable with previous run.

Benchmark loadUsersApp Activities Avg per minuteNetwork Activities Avg per minCheck Activities Avg per minCheck   Message Avg per minApp Activities / hour / userNetwork Activities / hour / user

Check Activities / hour / user

Check Message / hour / userTotal  / hour / user

Original benchmark 2017-09-06 V0.26

100550550585.5330330353698
Goal of new benchmark100     20660 684 

2017-09-14 V0.27

10075027507575450165045452190
2017-09-15 V0.281001555050181893303011113145
2018-02-21 V0.36 100 1603500 2518 962100 15 11 2222 
2018-10-08 V0.3710020042002221120252013122665


Benchmark results 2018-02-21

Benchmark run 100 threads on 2018-02-21

Benchmark is now change, User has IOS device and only Network Activity, Buddy has Android and App Activity and No Go Network Activity.

Total activity during benchmark

 


Benchmark 2017-09

Benchmark run 100 threads on 2017-09-15

On a simple test server the following results were achieved.

The benchmark server has 4 CPU, 8 GB memory and 32 GB Disk.

Run with 100 threads:

First part of benchmark generates:

  • 35280 Network Activities
  • 35280 App Activities
  • 2 * 35280 activities / 100 users = 705 activities spread over 30 days results in 23.5 per day per user.

Second part of benchmark generates:

  • 376320 Network Activities in total, 5050 per minute
  • 11760 App Activities in total, 155 per minute
  • 1469 Check Message in total, 18 per minute
  • 1764 Check Activities in total, 18 per minute
  • 3123 Network and App Activities generated per benchmark user in one hour.
  • 22 Check activities per benchmark user in one hour.


Benchmark run 100 threads on 2017-09-06 

On a simple test server the following results were achieved.

The benchmark server has 4 CPU, 8 GB memory and 32 GB Disk.

Run with 100 threads:

First part of benchmark generates per second:

  • 10 Network Activities
  • 10 App Activities
  • 2 * 33480 activities / 100 users = 670 activities over 30 days results in 22 per day per user
  • Activities  generated  ( 10 + 10 ) * 60 * 60 / 100 = 720 activities per benchmarkuser in one hour.

Second part of benchmark generates per second:

  • 9.2 Network Activities
  • 9.2 App Activities
  • 0.09 Check Message
  • 1 Check Activity requests
  • 660 Network and App Activities generated per benchmarkuser in one hour.
  • 42 Check activities per benchmarkuser in one hour.

Total activity during benchmark                                                                    

Count of Status  
BenchmarkPhaseActivityTypeTotal
1Init1681
1 Total 1681
2AppActivity33480
 NetActivity33480
2 Total 66960
3AppActivity32843
 NetActivity32324
 CheckMessage461
 CheckActivity3853
3 Total 69481
Grand Total 138122

 

 

Benchmark results 2017-07

Benchmark run 300 threads on 2017-07-12 

On a simple test server the following results were achieved.

The benchmark server has 4 CPU cores, 8 GB memory and 32 GB Disk.

Run with 300 threads in parallel results in 2% loss of response due to bad performance.

First part of benchmark generates:

  • 31 Network Activities / second = 1850 / minute = 111.000 / hour
  • 31 App Activities / second = 1850 / minute = 111.000 / hour
  • 2 * 102240 activities / 300 users = 681 activities over 30 days results in 23 per day per user
  • Activities  generated  ( 31 + 31 ) * 60 * 60 / 300 = 744 activities per benchmarkuser in one hour.

Second part of benchmark generates:

  • 28 Check Activity requests / second = 100.800 / hour
  • 14 Network Activities / second = 50.400 / hour
  • 14 App Activities / second = 50.400 / hour
  • 2.4 Check Message / second = 8640 / hour
  • Activities  generated  ( 14 + 14 ) * 60 * 60 / 300 = 336 activities per benchmarkuser in one hour.
  • Check activities ( 28 + 2.4 ) * 60 * 60 / 300 = 365 checks per benchmarkuser in one hour.

Total activity during benchmark

                                                                          

Count of Status  
BenchmarkPhaseActivityTypeTotal
1Init5134
1 Total 5134
2AppActivity102240
 NetActivity102240
 Batch1
2 Total 204481
3AppActivity51049
 NetActivity51049
 CheckMessage9072
 CheckActivity103798
3 Total 214968
Grand Total 424583

The average latency seems to fluctuate enormously per minute, so the distribution of response times per activity is reported. This shows that nearly all response times are below 200 millisecond only some hit the 9 seconds, which means some timeout on the Yona server.

 

Benchmark run 1000 threads on 2017-07-14

On a simple test server the following results were achieved.

The benchmark server has 4 CPU, 8 GB memory and 32 GB Disk.

Run with 1000 threads in parallel results in 5% - 20% loss of response due to bad performance.

First part of benchmark generates per second:

  • 90 Network Activities
  • 90 App Activities

Second part of benchmark generates per second:

  • 85 Check Activity requests
  • 45 Network Activities
  • 45 App Activities
  • 7 Check Message
  • Activities  generated  ( 45 + 45 ) * 60 * 60 / 1000 = 324 activities per benchmarkuser in one hour.
  • Check activities ( 10 + 1.0 ) * 60 * 60 / 1000 = 306 Checks per benchmarkuser in one hour.

Total activity during benchmark

                                                                          

Count of Status  
BenchmarkPhaseActivityTypeTotal
1Init17770
1 Total 17770
2AppActivity354240
 NetActivity354240
 Batch1
2 Total 708481
3AppActivity75502
 NetActivity75494
 CheckMessage14536
 CheckActivity156792
3 Total 322324
Grand Total 1048575

 

The average latency seems to fluctuate enormously per minute, so the distribution of response times per activity is reported. Now clearly the server is overloaded.

Benchmark run 500 threads on 2017-07-12 

On a simple test server the following results were achieved.

The benchmark server has 4 CPU, 8 GB memory and 32 GB Disk.

Run with 100 threads in parallel results in 5% loss of response due to bad performance.

First part of benchmark generates per second:

  • 50 Network Activities
  • 50 App Activities
  • 2 * 179280 / 500 = 710 activities over 30 days results in 23 per day per user
  • Activities  generated  ( 50 + 50) * 60 * 60 / 500 = 720 activities per benchmarkuser in one hour.

Second part of benchmark generates per second:

  • 50 Check Activity requests
  • 23 Network Activities
  • 23 App Activities
  • 4 Check Message
  • Activities  generated  ( 23 + 23 ) * 60 * 60 / 500 = 330 activities per benchmarkuser in one hour.
  • Check activities ( 50 + 4 ) * 60 * 60 / 500 = 388 Checks per benchmarkuser in one hour.

Total activity during benchmark

                                                                          

Count of Status  
BenchmarkPhaseActivityTypeTotal
1Init8966
1 Total 8966
2AppActivity179280
 NetActivity179280
 Batch1
2 Total 358561
3AppActivity86363
 NetActivity86473
 CheckMessage15384
 CheckActivity175851
3 Total 364071
Grand Total 731598

average latency looks like:

Benchmark run 100 threads on 2017-07-10 

On a simple test server the following results were achieved.

The benchmark server has 4 CPU, 8 GB memory and 32 GB Disk.

Run with 100 threads in parallel results in <1% loss of response due to bad performance.

First part of benchmark generates per second:

  • 11 Network Activities
  • 11 App Activities
  • 2 * 34560 activities / 100 users = 691 activities over 30 days results in 23 per day per user
  • Activities  generated  ( 11 + 11 ) * 60 * 60 / 100 = 792 activities per benchmarkuser in one hour.

Second part of benchmark generates per second:

  • 10 Check Activity requests
  • 5 Network Activities
  • 5 App Activities
  • 1 Check Message
  • Activities  generated  ( 5+ 5 ) * 60 * 60 / 100 = 360 activities per benchmarkuser in one hour.
  • Check activities ( 10 + 1.0 ) * 60 * 60 / 100 = 396 Checks per benchmarkuser in one hour.

Total activity during benchmark

                                                                          

Count of Status  
BenchmarkPhaseActivityTypeTotal
1Init1732
1 Total 1732
2AppActivity34560
 NetActivity34560
 Batch1
2 Total 69121
3AppActivity17265
 NetActivity17266
 CheckMessage3070
 CheckActivity35110
3 Total 72711
Grand Total 143564

 

The average latency seems to fluctuate enormously per minute, so the distribution of response times per activity is reported. This shows that nearly all response times are below 200 millisecond only some hit the 9 seconds, which means some timeout on the Yona server.

 

 

Benchmark run 180 threads on 2017-07-10 

On a simple test server the following results were achieved.

The benchmark server has 4 CPU, 8 GB memory and 32 GB Disk.

Run with 180 threads:

First part of benchmark generates per second:

  • 19 Network Activities
  • 19 App Activities
  • 2 * 64080 activities / 180 users = 712 activities over 30 days results in 23.7 per day per user. 
  • Activities  generated  ( 19 + 19 ) * 60 * 60 / 180 = 760 activities per benchmarkuser in one hour.

Second part of benchmark generates per second:

  • 17 Check Activity requests
  • 9 Network Activities
  • 9 App Activities
  • 1.5 Check Message
  • Activities  generated  ( 9 + 9 ) * 60 * 60 / 180 = 360 activities per benchmarkuser in one hour.
  • Check activities ( 17 + 1.5 ) * 60 * 60 / 180 = 370 Checks per benchmarkuser in one hour.

Total activity during benchmark

                                                                          

Count of Status  
BenchmarkPhaseActivityTypeTotal
1Init3206
1 Total 3206
2AppActivity64080
 NetActivity64080
 Batch1
2 Total 128161
3AppActivity31935
 NetActivity31936
 CheckMessage5679
 CheckActivity64918
3 Total 134468
Grand Total 265835

 

The average latency seems to fluctuate enormously per minute, so the distribution of response times per activity is reported. This shows that nearly all response times are below 200 millisecond but quite a bit hit the 9 seconds, which means some timeout on the Yona server.


How to read and process benchmark data

JMeter Output directory

When the benchmark is finished at the end of the loop the directory output is filled (not when the benchmark is stopped halfway). This output can be viewed in a browser to do investigation.

How to process benchmark data into Excel Template

To analyze the benchmark results the content of the file "ResultsTable.CSV" can be past into the SourceData worksheet of attached Excel template.

After pasting data, run Text to columns and split on "," delimiter, then the last two fields are calculated and the entire sheet can be "Refresh All" Ctrl-Alt-F5 to fill the Pivot tables.

BenchmarkResults-template.xltx

 

 

 

 

  • No labels