Who is using all my CPU?
We have all been asked the question, who is using all the CPU resource?
You notice that the CPU is running at near 100% for some time and you want to find the processes that are consuming all this CPU. Here are some command line options to help track down these processes and/or threads.
Top 10 Processes Consuming CPU in the last few seconds.
This command sorts the ps output by the C column. Header line added for clarity.
The C column is a realtime indicator of CPU intensive processes and threads. It is incremented each time the system clock ticks and the process or thread is found to be running. The value is decayed by the scheduler by dividing it by 2 once per second. The maximum value that a single threaded process can reach is 120, which effectively means that processes thread is spinning on a vCPU. Values above 120 can been seen when there are multi-threaded processes, as each thread could be running on different vCPUs.
# ps -fe | sort -nk4 | tail -10
UID PID PPID C STIME TTY TIME CMD
user1 63832220 15598760 26 09:45:44 - 1:18 /opt/IBM/Information/
user1 9306640 38666360 34 09:57:24 - 0:05 /opt/IBM/Information/
root 8847386 1 34 14 Aug - 2982:35 /usr/lpp/perf/bin/
user1 60293948 35193856 47 09:39:13 - 3:22 /opt/IBM/Information/
user2 44434162 15598760 50 09:45:44 - 3:08 /opt/IBM/Information/
user2 16778328 25756546 80 09:39:17 - 11:35 /opt/IBM/Information/
user3 37683234 33816662 99 09:39:17 - 11:39 /opt/IBM/Information/
user4 31328872 2688112 105 09:39:18 - 11:37 /opt/IBM/Information/
user4 46007130 35193856 115 09:39:14 - 11:40 /opt/IBM/Information/
user4 57804050 15598760 120 09:45:44 - 6:35 /opt/IBM/Information/
Top 10 Long Term CPU Consumer Processes.
This command sorts the ps output by the CPU minutes consumed column (TIME) relative to the processes start time. Processes listed above the 'wait' processes are generally more active on the CPU than those listed below 'wait'.
# ps guw | head -12
USER PID %CPU %MEM SZ RSS TTY STAT STIME TIME COMMAND
user1 27526570 0.7 0.0 67876 65224 - A 10:13:34 2:42 /opt/IBM/Info
user1 28377742 0.7 0.0 67948 65232 - A 10:13:34 2:42 /opt/IBM/Info
user1 49809810 0.7 0.0 67868 65216 - A 10:13:32 2:43 /opt/IBM/Info
user1 24118390 0.7 0.0 67860 65208 - A 10:13:31 2:43 /opt/IBM/Info
user2 66915324 0.5 0.0 10032 9940 - A 10:10:14 3:19 /opt/IBM/Infor
root 1901046 0.4 0.0 32896 32896 - A 14 Aug 28059:44 j2pg
user3 41158094 0.2 0.0 849004 849040 - A 10:10:53 1:26 /opt/IBM/In
user3 22022640 0.1 0.0 22360 22268 - A 10:13:30 0:36 /opt/IBM/Info
root 786464 0.1 0.0 4352 4352 - A 14 Aug 9152:43 vtiol
user4 23399314 0.1 0.0 2112 2084 - A 10:17:22 0:05 /opt/IBM/Inform
user5 6225922 0.1 0.0 76820 77688 - A 14 Aug 6027:15 ./Agent.
Show User Processes and Threads.
# ps -elmo THREAD | more
The CP column on the process line (line with user name) is the sum of all the CP's for its threads, and then each thread is listed below. This can help to identify the number of threads per process that are actually active. The SC column is the Suspended Count for the process or threads and generally represends the number of n
USER PID PPID TID ST CP PRI SC WCHAN F TT BND COMMAND
root 1 0 - A 0 60 1 - 200003 - - /etc/init
- - - 65539 S 0 60 1 - 410400 - - -
root 5505222 2949412 - A 0 60 1 - 240001 - - /usr/sbin/hostmibd
- - - 26018017 S 0 60 1 - 418400 - - -
root 5963892 2949412 - A 0 60 1 - 240001 - - /usr/sbin/sshd
- - - 26738843 S 0 60 1 - 418400 - - -
root 6029562 2949412 - A 0 32 1 f1000a02ccc858b0 240103 - - /usr/sbin/aso
- - - 26607785 S 0 32 1 f1000a02ccc858b0 410400 - - -
root 6225922 1 - A 29 60 110 * 42001 - - ./cybAgent.bin -a
- - - 25100447 S 0 65 1 f1000f0a10017f40 8410400 - - -
- - - 25690283 S 0 65 1 f1000f0a10018840 8410400 - - -
- - - 12059355 S 28 65 1 f1000f0a1010b840 8410400 - - -
- - - 26279985 S 1 65 1 f1000f0a10019140 8410400 - - -
Show Kernel Processes and Threads.
# ps -klmo THREAD | more
USER PID PPID TID ST CP PRI SC WCHAN F TT BND COMMAND
root 0 0 - A 120 16 1 - 303 - - swapper
- - - 3 S 120 16 1 - 1000 - - -
root 131076 0 - A 0 255 1 - 303 - 0 wait
- - - 131077 R 0 255 1 - 3000 - 0 -
root 196614 0 - A 120 17 1 - 303 - - sched
- - - 196615 S 120 17 1 - 1000 - - -
root 262160 0 - A 0 16 13 f1000f00095cee80 303 - - lrud
- - - 262161 S 0 16 1 f1000f00095cee80 1004 - - -
- - - 983079 S 0 16 1 - 1004 - - -
- - - 1048617 S 0 16 1 - 1004 - - -
- - - 1114155 S 0 16 1 - 1004 - - -
- - - 1179693 S 0 16 1 - 1004 - - -
- - - 1245231 S 0 16 1 - 1004 - - -
- - - 1310769 S 0 16 1 - 1004 - - -
- - - 1376307 S 0 16 1 - 1004 - - -
- - - 1441845 S 0 16 1 - 1004 - - -
- - - 1507383 S 0 16 1 - 1004 - - -
- - - 1638459 S 0 16 1 - 1004 - - -
root 327698 0 - A 0 40 1 - 303 - - vmptacrt
- - - 327699 S 0 40 1 - 1004 - - -