Step Guide to Performance Testing in Kubernetes
All working files and examples can be found in our githup repo on this url
1.1 What is JMeter?
The Apache JMeter™ application is open source software, written in Java with primary goal to test functional behavior and measure performance of a web app and more.
1.2 What can JMeter do?
Apache JMeter can simulate heavy load on server, or multiple servers with the goal to test strength and/or analyze performance.
Some of Apache JMeter features are:
-Ability to load and perform test on many different applications or protocol types:
• Web - HTTP, HTTPS (Java, NodeJS, PHP, ASP.NET, …)
• SOAP / REST Webservices
• FTP, TCP
• Database - JDBC
• Message-oriented middleware - JMS
• Mail - SMTP, POP3 and IMAP
• Native commands or shell scripts
• Java Objects
-A complete report
-Extract data from most popular response formats, HTML, JSON , XML or any textual format
-Caching and offline analysis/replaying of test results.
• Pluggable Samplers
• Scriptable Samplers (JSR223-compatible languages like Groovy and BeanShell)
• Functions can be used to provide dynamic input or data manipulation.
• Easy Continuous Integration through 3rd party libraries for Maven, Jenkins and more
1.3 Prometheus Listener for Jmeter
Jmeter supports a various range of plugins. Official or 3rd party. Prometheus listener is highly configurable and allows us to define our own metrics and expose them to Prometheus API to be scraped from server. The plugin can be found at this url.
1.3.1 Listener example
Here's a simple example to get us started.
Columns one by one.
• Name: metric name
• Help: help message
• Labels: A comma seperated list of labels
• label is a keyword. In JMeter it means the name of the sampler.
• code is a keyword. It's the response code of the result.
• JMeter variables can be used here.
• Type: metric type
• For more information, we can check Prometheus official docs here.
• Success Ratio is something specific to this plugin.
• Buckets of Quantiles:
• Buckets are comma separated list of numbers. Can be integers or decimals.
• Quantiles are comma, separated pair of decimals separated by a vertical bar |. The first decimal being the quantile and the second being the error rating.
• Listen To: Drop down to listen to samples or assertions. This only applies to Counters and Success Ratio type metrics.
• Measuring: Drop down menu of all the things we can measure
1.3.2 Success Ratio
Success ratio is a concept specific to this plugin library. Often we want measure success rates of samplers and it's difficult to do so when the failure for a given metric or label set has never occurred. It's difficult because it involves computations with NaNs.
1.3.2 Type and Measuring compatibility matrix
This is a matrix of what metric types can measure what metrics. If we configure, say a histogram to measure count total, the plugin will likely do nothing to update that metric.
Bold types can listen to samples or assertions (not both at the same time). Note that if we don't use label when listening to assertions we may get strange results. This is because one sample can generate many assertion results which are then counted. When there's no label to distinguish those counts, they'll be summed together which may or may not be expected.
2.1 What is Prometheus?
Prometheus is a free software application used for event monitoring and alerting. It records real-time metrics in a time series database built using a HTTP pull model, with flexible queries. Prometheus written in Go and licensed under the Apache 2 License, with source code available on GitHub, and is a graduated project of the Cloud Native Computing Foundation, along with Kubernetes and Envoy.
3.1 What is Docker?
Docker is a set of platform as a service (PaaS) products that uses OS-level virtualization to deliver software in packages called containers. Containers are isolated from one another and bundle their own software, libraries and configuration files. They can communicate with each other through well-defined channels. All containers are run by a single operating-system kernel and are thus more lightweight than virtual machines.
The service has both free and premium tiers. The software that hosts the containers is called Docker Engine. It was first started in 2013 and is developed by Docker, Inc.
4.1. What is Kubernetes?
Kubernetes is an open-source container-orchestration system for automating application deployment, scaling, and management. Originally designed by Google, and is now maintained by the Cloud Native Computing Foundation. It aims to provide a "platform for automating deployment, scaling, and operations of application containers across clusters of hosts". It works with a range of container tools, including Docker. Many cloud services offer a Kubernetes-based platform or infrastructure as a service (PaaS or IaaS) on which Kubernetes can be deployed as a platform-providing service.
5.1. What is Jenkins?
Jenkins is a free and open source automation server. Jenkins helps to automate the non-human part of the software development process, with continuous integration and facilitating technical aspects of continuous delivery. It is a server-based system that runs in Apache Tomcat servlet. It supports version control tools.
Builds can be triggered by various means
• commit in a version control system,
• cron-like mechanism
• requesting a specific build URL.
• after the other builds in the queue have completed.
Jenkins functionality can be extended with plugins.