Provided To ICES Committee For Service Level Computing Platform Evaluation/Recommendation

By

Jim Giuliani

* * * * *

The Ohio State University

March 6, 1998

The current supported software list for the service level courses, and reported performance issues with these software packages, are listed below:

Performance Issues

Software Package Load Execution

Eudora no no

Microsoft Word no no

Microsoft Excel no no

Microsoft Access no no

Microsoft Power Point no no

Turbo Pascal no no

Netscape yes no

Execution performance issues with Power Point were traced to network bandwidth problems. The network in Baker 310 has been upgraded to new Cisco hardware and this issue was resolved. Performance tests of Power Point execution show that 24 users running slide show and cycling through slides as fast as possible give a worst case of 3 seconds per slide.

The Netscape load time issue concerns the one week where Netscape is covered in closed lab. Load times are presented in this report for 1 to 24 users starting Netscape at the same time.

The Netscape execution issue has been traced to network and memory limitations. The network in Baker 310 has been upgraded and server memory increased to address these problems. Delays due to slow response from external web servers cannot be addressed internally.

The tests conducted were designed to determine the worst case effects of host loading on the performance of applications running under WinCenter as users are added to the system. In order to provide accurate timing and logging of events, the Microsoft Test software package was used to generate the test scenarios. Microsoft Test allows the automation of user keystroke entries, window focusing and mouse movements. Logging capabilities allow the time required for specific events to be recorded.

Once a sequence of events is defined, they are programmed into MS Test using a custom programming language. These programs are compiled so that they can be executed at a later time by a smaller Microsoft Test utility program. For each user presented in the test data, this utility runs in the background and sends the keystrokes and mouse movements to the systems just as if a person were sitting at the workstation.

It is important to understand that the performance data presented includes the overhead of the MS Test utility program. The Microsoft Test utility that must be run for each user actually adds about 10% of overhead to the computer system, as far as memory and CPU utilization are concerned, in the 24 user configuration.

It is also important to understand that Power Point slide show and Netscape page loading tests are worst case. They cycle through displaying the slides and web pages as fast as the system can respond. Normal users would not behave in this manner. Instead, they would load a page or slide and take several seconds to read the information. This delay would allow the system to service other users during the pause, resulting in improved overall system response. Actual response seen by users sitting in the lab will always be faster than the data reported because they cannot cycle through the information as fast as the automated MS Test software.

Two types of results are presented: actual lab usage statistics and MS Test data, which gives worst case response time for 1 to 24 users. Performance is measured by %CPU, an indicator of how busy the processor is, and application response time.

Performance issues center around two applications: Netscape and Microsoft Power Point. Turbo Pascal has raised some theoretical performance issues, which are discussed, but no reports of these issues occurring in the labs has been reported over the last two quarters.

Power Point issues relate to the presentation display option, which is a very graphic intensive utility and consumes a large amount of CPU time to process the animated charts. Execution performance issues with Power Point were traced to network bandwidth problems. The network in Baker 310 has been upgraded to new Cisco hardware and this issue was resolved.

Figure 1 shows the response time for the controlled test, where accounts running Power Point slide show cycle through slides as fast as possible. This represents a worst case scenario. With 24 students logged on, this chart shows the time it would take for X users to move into presentation mode and cycle through slides, not taking any time to actually read the information. A detailed description of the test procedure is given in the figure. A worst case of 3 seconds per slide was seen at 24 users.

Figure 2 shows the % CPU utilization measured during the controlled test.

To estimate what the actual response time seen during an average lab would be, we need to look at Fig. 3. Figure 3 shows the CPU and memory utilization from the autumn quarter Power Point lab. The time window shown is approximately 70 minutes with samples taken every 60 seconds. The average CPU utilization for this server is 27% with peaks around 70%. A 70% CPU utilization load corresponds to approximately 5 users from Fig. 2. This corresponds to a system response time of approximately 0.8 seconds/slide, obtained from Fig. 1.

Figure 4 shows the startup time for Power Point. Since the load times were minimal, test runs at 8, 12, 16 and 20 users were skipped. An interesting observation on the controlled test is that it was very difficult to have all sessions of Power Point start simultaneously. Since the load times were very quick, any delay in starting the applications would allow the first ones started to finish. If this happened, load times would fall in the 4 second range for all 24 accounts.

Netscape performance issues of load time and execution time have been traced to excessive memory swapping and CPU load. Additional reported problems with Netscape, such as slow or locked up browsers, are due to slow response from the remote host and/or slow network response outside of CIS. These problems can not be solved by the selection of either X-Terminals or PC workstations.

To examine the benefits of changes made since Autumn quarter, Fig. 5 shows the CPU utilization from a controlled test of 24 students running Netscape simultaneously. Here the white line represents memory swapping and the darker line represents % CPU utilization. The performance problem seen during this test was due to excessive memory swapping to disk. Memory is being swapped out an average of 1000 pages per second, with peaks over 3000 pages per second. Adding 128 meg of ram to the server cut the average number of page faults in half and reduced the maximum to under 1000.

Figure 6 shows what the expected response time users will see based on the upgraded server configuration. With 24 students logged on, this chart shows the average time it would take for X users to load a web page. These numbers were obtained with test accounts continually loading a mix of text and graphic web pages. This represents a worst case scenario since the time users would normally pause to read the information is not modeled.

Figure 7 shows the worse case load times for Netscape. With the previous server configuration, unofficial load times for loading 24 simultaneous Netscape sessions was in

the 10 minute range. An observation to note is that if Netscape sessions are started in intervals of 3 seconds, the average load time for 24 session’s drops down to 62 seconds.

It is also beneficial to look at what the current class statistics are. Memory was added to one of the Hagerty servers for winter quarter. Performance data from the 9:30am 2/14/98 and 11:30am 2/14/98 Netscape labs is presented in Fig. 8. During the Netscape portions of the lab, CPU utilization is between 50% and 75%. The white line is pages per second and is running around 50 to 100 pages per second, a much lighter load than 500 pages per second seen in the controlled tests.

Turbo Pascal on the NT system has been used for CIS 201 and 214 since September of 1997. To date, no performance complaints have been received about systems in Baker 310. A theoretical problem is what happens when a students program gets stuck in an infinite loop. To see how the system responds, 23 accounts were configured to run a 6 Power Point slide presentation. The time required to complete the 6 page presentation for the 23 accounts is graphed in Fig. 9. Note that in this test, the presentations do not cycle, the time recorded for each account to complete is simply recorded. One and then two infinite loops are started in Turbo Pascal and the tests rerun.

Because the servers are dual processor servers, one infinite loop does not significantly impact other users as the offending process only ties up one of the two processors. The impact is more sever when two infinite loops are started.

A possible reason that no reports of this problem have been received is that it is very easy for the user to kill the infinite loop. If a user would happen to get trapped in an infinite loop, the other system processor can carry the load until the user has time to kill the process.