Ibm Iv99343: Slow Performance Of Unix Package Data

I was asked to look at an implementation of IBM Information Server (MDM Server) that was experiencing a very slow processing rate for “evergreening” (the process of ingesting and cleansing data from legacy systems into the Master Data Management model format). Performance profiling on Linux. I would also be tempted to use IBM/Rationals Quantify profiler as that makes it easier to view the profiling data, drill down to the line level or look at it in a '10000ft' level. Why does changing 0.1f to 0 slow down performance by 10x? Qlik Sense is a business intelligence tool that allows data to be discovered and visualized. Qlik Sense supports Hadoop environments as a data source. Read this article for details about how Qlik Sense was tested to integrate with and visualize data in Hortonworks Data Platform (HDP) on IBM POWER8.

Most Linux tools have 'stat' as the last four letters of their name, 'stat' being short for 'status.' Some of the most commonly used performance reporting commands are: vmstat, iostat, netstat, nfsstat, pqstat, lpstat and mailstat. There is, however, a second set of stat commands: stat, fstat, statfs and fstatfs. These other tools analyze files and filesystems on the disk. They aren't directly associated with performance.

Of the directly relevant tools, vmstat is the first port of call. The figures it reports expand on figures that appear at the top of the 'top' command's output. The vmstat tool gives a server-wide view of performance. Supply it with two arguments, like this: vmstat 5 3 When run, vmstat spends five seconds accumulating data and reports a single line of performance statistics.

It does this three times. At least five seconds is good for accuracy, because it means the average reported is based on a reasonable amount of data from the kernel. You'll find that vmstat reports system-wide process use, swapping, memory use, disk I/O and CPU use.

Supported Input Formats • Scalable Vector Graphics (*.svg) • Graphics Interchange Format (*.gif) • JPEG Image (*.jpg; *.jpeg) • Windows Bitmap Image (*.bmp) • Microsoft Icon (*.ico) • PNG Image (*.png) • Tagged Image File Format (*.tif; *.tiff; ) • Most major graphic file formats Supported Output Formats • PNG Image (*.png) • JPEG Image (.jpg; *.jpeg) • Windows Bitmap Image (*.bmp) • Microsoft Icon (*.ico) • Scalable Vector Graphics (*.svg) • Portable Document Format (*.pdf) • Encapsulated Postscript (*.eps).

If the server is slow, but you can still login, then vmstat tells you immediately if the CPU or memory is the current bottleneck. If it's the CPU, then use 'top' to find the guilty process. Interpreting vmstat output requires some appreciation of Unix architecture. Linux processes can be blocked (waiting for input), running (one per CPU at most) or runnable. If there are lots of runnable processes, (the Procs r vmstat column), then that means there is more to do than the CPU can manage. If runnable processes hover much over 1 or 2, a faster computer will speed up the server significantly., on the other hand, should always be in short supply. If there is initially a lot of free memory, then the I/O buffer and cache will expand to use most of it.

One shouldn't panic, therefore, if the amount of free memory drops shortly after booting. The buffer and cache memory pools will hand back some memory if new processes demand their share. Only when all amounts are reported to be small is there a potential memory issue. Swap is also part of the virtual memory system. It is designed to allow more processes to run than can fit in conventional memory. The overflow is temporarily saved to disk ('swapped out') until needed. There are two kinds of disk activity, 'paging' and 'swapping.'

A small amount of paging is safe. Linux never swaps (unless you have a very unusual flavor of Linux), and in any case, if it did, performance would be disastrously slow. In a well-tuned server, where standard, stable processes are running, paging should be minimal as well. If vmstat reports sudden paging, then either someone's running a report or the commands you yourself type are the cause. If paging activity happens all the time, buy more RAM.

Just vmstat by itself presents plenty to explore. It's a first test that your server is performing well. Test the other 'stat' commands at your leisure - they're all harmless. Iostat and netstat are vital for analysis of per-disk and per-network performance, respectively. The remainder let you peek at the performance of application-specific features of Linux like printing and mail.