What’s the Max Number of Users You Can Test on JMeter?

Actually doesn’t have an exact answer - what’s the maximum number of users you can test on.
Limits usually depend on many factors, and the same goes for JMeter limitations.
JMeter limits depend on the capabilities of
--your machine and network,
--the complexity of your performance scripts,
--the targeted number of simulated users and so forth.

We are going to use a pretty ordinary local machine:

CPU: 1.4 GHz Intel Core i5
RAM: 4GB 1600 MHz

Run Your Load Test for 500 Users
Now let’s try to run the same script with 500 users. Change the number of users in the Thread Groups to 500 threads, and ensure the script ran successfully.

Run Your Load Test for 1,000 Users
A bad thing happened this time. While adding the 743rd user, my JMeter UI application got stuck and stopped responding.

Tip #1: Use JMeter Listeners for Debugging Purposes Only

We clicked the button and…... the script got stuck again. This time I found the script failed while adding the 921st user:

Tip #2: Run JMeter Tests in Non-GUI Mode

To run the test in Non-GUI mode you need to save it as a JMX file (let’s call it ‘JMeterLimitationsTest.jmx’) and execute this specified command:

Let’s run our script and… we can see that test execution went fine without any errors:

Run Your Load Test for 2,000 Users

We are out of memory again. This time it happened after simulating 1,191 users. Can we move further? Of course, the answer is yes! We have lots of useful advice.

Tip #3: Increase JMeter Heap Space to Generate a Larger Load

The JMeter heap has default limits that can be easily increased without any harmful effect for test execution. Just open your JMeter runner script (located in the JMeter bin folder) and replace the value of max heap space (which goes after -Xmx flag) that has these default values:

HEAP="-Xms512m -Xmx512m"
Allocate 2 GB (2048 MB) RAM
HEAP="-Xms512m -Xmx2048m"

Now we can run the same test again with 2,000 users. We clicked the button again and…
We don’t have any out of memory issues anymore but you can see that after we ran almost 2,000 users, we started getting lots of failed requests for our test.

Tip #4: Use LAN instead of Wi-Fi Connections When Running Load Tests

The switch is done, now let’s try to execute the test again. This time the test passed without any issues. We can confirm that a regular laptop can simulate 2,000 users and we should now try to go for the next limitation.

Run Your Load Test for 10,000 Users

We simulated JMeter for more and more threads. As a result, we found that the max value of users we could simulate was around 8000-9000 users. If we added more, we got errors about available memory right after test execution  This time it is not about heap space, but rather based on the limits of my local machine’s RAM.
At this point there might also be many random errors, rendering test execution irrelevant.

n conclusion, we can say that you can simulate up to 10,000 users on a JMeter load test even on a regular laptop. This depends on the following 4 simple rules:

Use JMeter listeners for debugging purposes only
Run JMeter performance tests in non-GUI mode
Increase JMeter heap space to generate a larger load
Use LAN instead of a Wi-Fi connection when running a load test

As you can see, the answer to the question ‘What’s the maximum number of users you can test on JMeter?’ varies. It depends on many factors: test complexity, script running method, internet and machine capabilities and so forth.

I have found the user limit on my local laptop is 8000-9000 users. But what if you need more? There are options to do that:

Distributed mode: JMeter enables running scripts in distributed mode by using a few slave machines with your own main one, and then combining and outputting the results. With this method, you can scripts horizontally without any limitations except for the number of slave machines you allocate for tests.

Cloud solutions: You can also always use cloud solutions. BlazeMeter provides an easy-to-use cloud testing platform, enabling you to run as many users as you wish.

Blazemeter plugin

API Monitoring and Testing

Application Programming Interfaces (APIs) testing is different than other testing as GUI is not available and this won't concentrate on look and feel of the application. API testing involves testing APIs directly. It enables communication and data exchange between two separate software systems and as part of the end-to-end transactions exercised during integration testing.

API testing commonly includes testing REST APIs or SOAP web services with JSON or XML message payloads being sent over HTTP, HTTPS, JMS. Since APIs lack a GUI, API testing is performed at the message layer.

Summary - RESTful vs SOAP Web Service

SOAP:
--XML based protocol
--Uses WSDL for communication between consumer and provider
--Does not return human readable result
--Transfer over HTTP, SMTP, FTP, etc.
--Javascript can call SOAP but difficult to implement
--Performance is not great compared to REST
--SOAP web services can be tested through programs or software such as Soap UI.


REST:
--Architectural style protocol
--Uses XML or JSON to send and receive data
--Result is readable which is just plain XML or JSON
--Transfer is over HTTP only
--Easy to call from Javascript
--Performance is better than SOAP. Amazon and Google are moving their APIs from SOAP to REST.
--REST can be easily tested through CURL command, Browsers and extensions such as Chrome Postman.


There are multiple API testing tools available

• SoapUI
• Runscope
• Postman (Chrome Extension)
• Advanced REST Client (Chrome Extension)
• Insomnia REST Client (Chrome Extension)

Suppose, there is a checkbox in user registration form and the system using API function which receives input as date range and return a list of users that was registered with checkbox marked at that time interval.

Input: Date interval (e.g, fromdate=2016-12-01&todate=2016-12-15)
Output: list of users registered that time interval with checkbox marked

First of all, install any of the API testing tools and here i will be using SoapUI / Postman / Insomnia / Advanced REST Client.

Download SoapUI from here according to your machine. Other Insomnia / Advanced REST Client can installed from google chrome extensions.


1. Open SOAP UI application

2. Create REST project and put request URL

 
3. Authenticate using Username and Password

Authentication is stated as the act of confirming the identity of API consumer. After authenticated, they are usually authorized to get access to desired APIs

• Authentication is used to determine who the user of an API is.
• Authorization is used to determine what resources the identified user has access to. 

There are multiple standards and technologies available for authenticating users, for example;

• Form-based - Web/HTML based authentication that commonly uses HTTP cookies.
• Basic/Digest - Uses HTTP headers to identify users.
• OAuth 1.x/2

4. Submit the request

 See image from Insomnia

Now you can move to advanced things :)

What should consider before implement Test Automation

Suppose, you are working in a project as manual tester and you are enjoying the working. One day, your QA manager share client's thinking and they are expecting to integrate Automation Testing in the projects as a regression test. So, your manager requested you to plan for Automation Testing. Now it's your turn how you can manage and plan for automation efficiently.

Before implement test automation, you have to face some of the questions and have to consider some of the factors whether it is best fit for the current system or not.

• Are you thinking of 100% automation?
• Is your application is stable enough to automate further testing work?
• Automate testing procedure when you have lot of regression work
• Think about skilled resources
• Identify what test cases to automate
  • Repetitive tasks
  • Stable enough application
  • Complex calculation
  • Which require regular set up of environment
  • Things which are difficult for human
  • Task requiring multiple data sets
  • Apply common sense!

Test Automation Introduction

As a career of software tester, we all need to know the basic understanding of software test automation. There are good numbers of tools (Selenium, Telerik Test Studio, Robotium, QTP, TestComplete) to do the automation and there are multiple language (Java, C-Sharp, Python, Ruby) support.

Here, I will go further with Selenium and the language is Java. Selenium is a software testing framework for web applications. There are multiple components (Selenium IDE, Selenium RC, Selenium WebDriver, Selenium Grid) of Selenium. First one, Selenium IDE which is implemented as a Firefox Add-On, and it allows recording, editing, and debugging tests. The work around of Selenium IDE is not vast. It is less maintained and is only compatible with Selenium RC, which is deprecated product right now.

Selenium WebDriver (Selenium 2.0) implemented as a successor of Selenium RC. It has browser-specific browser driver, which launch the browser, sends commands, and retrieves results from browser application. Selenium 2.0 is supported in Python, Ruby, Java, and C#. So, we can write code according to our familiar languages.

Happy Coding! :)