JVM Performance Tuning
If you are having issues with slow performance of your Java application, and who doesn't, the first thing to do is take thread dump. The jstack tool is your friend.
jstack -l
The output of this command can be visualized in the Thread Dump Analyzer. You can download this tool from https://tda.dev.java.net/
To run the the tool, unzip the downloaded file, cd to that folder
java -jar tda.jar
In the GUI, File -> Open and choose your threaddump.log in the TDA and you should see a lot of info. In a well running system, you should see a bunch of threads sleeping. Some Threads locking monitors is okay. But if the majority of threads are in 'locking' status means they are waiting for some resource to free up. And that;s what you should focus your performance tuning effort on.
What's on your Heap?
Use a tool like jmap to get a dump of the heap: Beware, that this file can be huge, like in Gigabytes depending upon how much memory you have given to JVM and current heap size.
jmap -dump:live,format=b,file=heap.bin
Then use JVisualVM (part of the JDK 1.6 and later) or VisualVM (download from sun).
To load the heapdump do the following:
File -> Load -> File Format choose .hprof, *.* and Open,
In the Classes tab of the main panel you should see a the list of classes sorted by most instances first.
From localhost connecting to jConsole running on remote server. Ensure that the app is running with the JVM_OTPS = -Dcom.sun.management.jmxremote, otherwise jConsole can't connect to the app.
ssh -X user@my.server.com
on the server:
> jconsole
You should see a window popup on your lcoalhost
Using the right collection class
Know the difference between the ArrayList and LinkedList, both are lists but ArrayList is not good if you are going to be modifying the list (add/remove), but really great for access like get(i). LinkedList is just the opposite.
Tomcat thread pool size
Based on the above configuration, 0 threads will be created after the tomcat startup, depending on the number of concurrent requests number of threads will be created in the thread pool, for instance if there are 5 concurrent requests then 5 threads will be created in the pool and these threads are further reused by subsequent requests until more concurrent requests come where further threads will be created. Because of the minSpareThreads value of 10 all the threads created up to 10 will not be discarded irrespective of whether the threads are idle or not. Thread pools shrink based on the maxIdleTime which by default is 60000 ms/ 60 secs / 1 minute meaning if a thread is idle for 60 secs tomcat will discard it from the pool.Suppose there is a spike in the number of concurrent requests say 50 then about 50 threads will be created in the pool to accommodate the spike. if your average is 1 request/sec except during the spike you would expect the threadpool to shrink after the spike, but unfortunately this is not how tomcat does. Tomcat uses the logic in such a way that the threads are picked and used in a round-robin manner from the pool, since you are getting about a average of 1 requests/sec the thread pool won't shrink since all the 50 existing threads in the pool will be used at least once within a minute when there are 1 requests/sec coming. Ideally, I would expect tomcat to reuse the same free/idle thread (Most Recently Used thread) as much as possible instead of the round-robin policy to be able to discard the idle thread. Since it's not working as expected the only way to control is to reduce the maxIdle time with an optimal value, but reducing the maxIdle time also possess some risks. This also becomes an inconvenience if your monitoring system is alerting based on the number of thread pool count, in such cases using activeCount value of the thread pool is a wise thing to avoid false alerts.
JDBC drivers and connection pooling
jTds driver to connect to MS sqlsevrer: Features
Connection Pooling Yes
Connection Pool Implementation No (recommended pool implementations: DBCP and c3p0)
with was a "deadlock" where threads were waiting for connections to be made available to them. Indiana switched to c3p0 to fix this problem, but UMich just tuned off "evictions" and "didn't let the pool cycle", which seemed to achieve the same thing with dbcp. Apparently, c3p0 just has better default settings for this, so this may be why people saw performance gains using it.
c3p0 comes with Hibernate.
was when I found that the entire pool was locked the whole time a new connection attempt is made to the database. So, if something happens to your database that results in slow connections or timeouts, other threads are blocked when they try to return a connection to the pool—even though they are done using a database.
jMeter recording a script
1. Test Plan -> Add -> Thread Group
1.1 Name this thread group, No. of threads = 100, rampup period 300secs, Loop Count: 10 or Forever
2. Workbench->Add->Non config elem -> HTTP Proxy
2.1 Port 8090, Grouping : Put each group in a new controller
2.2 Click Start
3. Setup your borwser to use the proxy
Firefox: Preference -> Advanced -> Connection -> Settings -> Choose Manyal proxy config -> 127.0.0.1 and Port 8090
3.1 Don't forget to undo this setting when done recording the test.
4. Do the recording of the scenario, from login to work to logout
5. Add delays at certain pages to simulate manual ops
5.1 At certain ppoints in the scenario right click on an item -> Add -> Timer -> Guassian Random Timer: Constant delay of 1000ms dev 100ms
6. Add reporting so you can see the result
6.1 On the test plan -> Add -> Listener -> Aggregate report, you can add multiple report types.
7. Run the test TopMeny -> Run -> Start