USED FOR INITIAL ANALYSIS, NOT ACCURATE ANYMORE
- The cluster must support 10000 mobile users who are generating moderate internet traffic. Moderate is defined as 250 http requests/hour.
- The cluster must be available 24x7x365, with 99.99% availability corresponding to approx. 5 minutes downtime per month. The Yona services for request analysis and status queries must be available 99.999%, and the Yona services for account management must be available 99.9%. The VPN server should be available 99.999%, as blocking user access to internet is considered a no-go.
- The cluster supports the Yona microservices architecture, in which three services are recognized
- Admin service. Provides administrative capabilities like maintaining the set of goals
- Analysis service. Provides SmoothWall with the necessary information and analyzes suspicious HTTP requests and creates appropriate entries for a person and their buddies.
- App service. Provides the services that back the mobile app: user sign up, account maintenance, buddy relationship maintenance, message retrieval, etc.
- The cluster avoids single points of failure
- The SmoothWall request forwarding to the Yona analysis service can only use Perl scripts.
- The cluster supports Docker deployment of all Yona services and all 3rd-party components.
- The cluster may be deployed on virtual machines in different ways:
- for DEV and TEST a cluster deployment on a single VM
- for ACC and PROD a cluster deployment on multiple VMs.
The split in various microservices is motivated by earlier discussions, the first prototype of the Yona services and a number of requirements mentioned in various emails and discussions. The criteria for dividing functionality in microservices include separation of concerns, domain boundaries, differences in scalability and performance requirements and app-facing versus background batch functionality.
The following components are foreseen for the cluster architecture
- Load Balancer for the web traffic. Alternatives are among others nginx and HAProxy . Proposal is to use nginx
- Public REST web services (microservices), implemented in Java/SpringBoot. Alternatives are Apache Tomcat and Spring Cloud. Current prototype uses Tomcat.
- Private REST backend services (microservices), implemented in Java/SpringBoot. See #2 for alternatives
- Message streams between SmoothWall and the Analysis service. Alternatives are no messaging (using POSTs from Perl onto Analysis endpoint), RabbitMQ, Redis messaging and Spring AMQP. Proposal is to use direct POSTs in the first iteration. If throttling is required an AMQP messaging solution may need to be included.
- Relational database for storing Yona data. Alternatives are MySQL, JavaDB, HSQLDB, MariaDB. Current prototype uses HSQLDB. Proposal is to use MariaDB with Galera multimaster cluster.
- Cache for Yona data. Alternatives are Redis and HazelCast. Proposal is to use HazelCast.
- A bastion host for secure access to the cluster.
- Logging infrastructure. Alternatives are ELK (ElasticSearch, LogStash, Kibana) stack. Proposal is to use ELK
- Perl script for posting HTTP requests to analysis services. See the details page.
For the components a few alternatives are listed. There are typically many others...
We may want to consider some broader infrastructure platforms, which offer a lot more functionality, but increase the learning curve...
- Rancher/Rancher OS: a Docker-based software platform, OS, infrastructure, load balancer, etc.
- SpringCloud: A cloud platform
- OpenStack: A generic cloud platform definition and implementations
- AWS: A mature cloud platform
A first draft of the cluster is shown below. Message streams and caching are currently not included. It assumes multiple Linux (CoreOS is a well-defined container-supporting version) VMs, residing in different availability zones within CloudVPS. PowerPoint is attached (no Gliffy alas)
First a more functional setup
Then a more technical setup, with specific components filled in
Detailed calculations of load
In order to discuss the cluster architecture based on facts, an Excel sheet with some calculations has been prepared. The calculations are assuming the original Yona goal: assessing the violations of common and user-level categories (goals). A new requirement for assessing the total time spent on Internet will affect the calculations, and the architecture, in a profound manner. The calculations should be used as input for performance and scalability testing.
|% of common level violations||10%|
|% of user level violations||30%|
|#Yona status calls/user/hour||10|
|#Yona account retrieval calls/user/hour||0.05|
|#Yona account update calls/user/hour||0.025|
|#Yona account creation calls/hour||5|
|#Yona buddy creation calls/hour||10|
|Crypto read/write operations||2|
|% of push notifications per violation||10%|
|# VPN servers||10||10||10|
|# http requests||2500000||41667||694|
|Perl script comparisons||2500000||41667||694|
|Analysis service POSTS||250000||4167||69|
|Crypto operations for write||150000||2500||42|
|Database write operations||150000||2500||42|
|Status service GETS / hour||100000||1667||28|
|Crypto operations for read||200000||3333||56|
|Account service GETS / hour||500||8||0|
|Account service PUTS / hour||250||4||0|
|Account service POSTS/hour||15||0||0|
|VPN Profile creation/hour||15||0||0|