Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to upgrade your browser.

Log In
Sign Up

Figure 37 – uploaded by JAYANT SHEKHAR

See full PDF downloadDownload figure

Figure 5-11: z/VM paging activity within same time interval as Figure 5-10. Data de- rived from CP (hypervisor) level using Performance Toolkit for VM. — Figure 5 -11: z/VM paging activity within same time interval as Figure 5-10. Data de- rived from CP (hypervisor) level using Performance Toolkit for VM.

Related Figures (64)

A complete mainframe box (as depictured in Figure: 2-2) is by many sim- ply called a “system”, but also very confusingly referred to as “processor”, a “CPU”, or a “CEC” (Central Electronic Complex). Figure: 2-2 Inside IBM z9-109 (S38 or S54) / System z9 EC. [20] The box measures 1,94 x 1,57 x 1,57m (H x W x D) including covers, and thereby occupy 2,49m?.

tion of the I/O ports within the box. CHPIDs are then mapped to PCHIDs completely transparent to running programs (see “Channel I/O SubSystem” in Figure 2-3). [36] Channels (and the connected control units and devices) can be shared between logical partitions / LPARs, which are self-contained and completely separated subsets of the machine running independent operating system (see section 2.2.7 below). Sharing of channels within an LCSS is enabled by the Multiple Image Facility (MIF). Channels can also be spanned across multiple LCSSs and thereby transparently be shared among the logical parti- tions within these LCSSs. This will however decrease the total number of channels. [36]

As mentioned above, channels (and connected devices) can be shared between LPARs or they can be dedicated to LPARs. Some restrictions exist depending of the channel and device type. Up to 15 LPARs can be associated with each Logical Channel SubSystem (LCSS). There are in other words 256 channels (or CHPIDs) to be shared between or individually assigned to the LPARs in each LCSS group. depending of the channel and device type. Up to 15 LPARs can be associated

Today most resources can be (re)allocated/(re)configured dynamically without power-on-reset (POR) or booting / initial-program-load’ing (IPL) the operating systems. The HCD (Hardware Configuration Defi- nition) program greatly simplifies this process. With HCD it is possible to have a single IODF (1/O definition file), which contain all relevant I/O configuration including data about the Channel SubSystem, logical partitions, processors, and even external switches, control units and devices. Actually the same IODF can contain information about several mainframe boxes. Figure 2-5: Hardware configuration is loaded into the “Hardware System Area” from the Support Element on Power-On. HCD (possibly the HCM GUI) can be used to make changes dynamically.

Figure 3-1: The evolution of VM (graphic from [16] modified according to [46}).

Type 1 hypervisors run directly on system hardware or in system hard- ware as firmware / millicode / microcode. When running on hardware they act as a low level operating system creating “virtual machines” for other oper- ating systems. The principle is illustrated in Figure 3-2, page 22. Type 1 ex- amples include WMware ESX Server, Xen, z/VM, and the earlier mentioned “PR/SM” (hardware implemented). [21]

CP provides a 3270 console interface to manage virtual machines and re- sources. This interface can for example be used to dynamically attach or de- tach resources to running virtual machines. It can also be used to enquire CP for information about the system and virtual machines. Queries are issued within virtual machines. CP commands can be entered directly, when the vir- tual machine runs in CP mode - without an operating system (Figure 3-3). CP commands can also be issued directly from CMS (the single user operating system within z/VM) and actually from within any guest operating system by prefixing the command with #CP.

Actually z/VM is the only z/Architecture operating system, which uses ex- panded storage. This is feasible since it can host guests running in 31-bit ad- dressing mode, which actually use it. Furthermore the VM paging sub system has over time been optimized to expanded storage and it therefore generally performs better when available. storage was dedicated hardware different from main storage. Today it merely a piece of ordinary central storage assigned to the logical partition as ex- panded storage.

Figure 3-7: Services like a basic TCP/IP stack, FTP and error recording (EREP) are provided as Service Virtual Machines, which in fact are no different from other VMs. As indicated earlier, CP is “only” a hypervisor and not a fully fledged oper- ating system. CP has no understanding of programs, processes or threads — basically all CP knows is how to run virtual machines. The consequence is that any extra functionality has to be implemented as virtual machines. 3.4.6 Service Virtual Machines

Figure 3-8: A section of the USER DIRECtory file. The user (virtual machine), LINX01, is an example of a Linux guest. The user directory is a flat “text file’ owned by the MAINT user (vdev 2cC). It contains all user (virtual machine) definitions: Their names, pass- word, accessible devices, number of virtual CPUs, CPU share, the privilege class, and many other options. An example of how a user is defined is given in Figure 3-8. The following section (3.6) touches upon several of USER DI- RECT statements, which are use to allocate virtual devices.

Figure 3-9: Syntax diagram for USER DIRECTory “SHARE” statement, which is used to control the percentage of processor recourses a user (VM) receives. [28] The share values can be specified either as absolute or relative. An abso- lute value denotes an absolute percentage (0.1% to 100%) of the available processor resources (dedicated processors excluded). If the sum of absolute shares exceeds 99%, CP normalizes the values internally: 99% of the proces- sor resources are proportionally distributed to the guests according to their absolute share value.

The concept of processing power is a fictive concept, which express the number of machine cycles a real processor is capable of performing in a unit time normalized too 100. This is in accordance with general mainframe per- ception, where a real processor is assigned the processing power of 100 (often 100 percent). The four sharable processors in the example yield a total of Pict=400. The processing power assigned to individual virtual processors can thereby been seen as the percentage of what a real processor could do. The processing power of a virtual processor newer exceeds 100, since a virtual processor never operate faster than its real counterparts.

Number of virtual processors in guest i. Any virtual processor has a static share or “weight” according to the ac- tual user configuration and the defined virtual CPUs. This weight can be cal- culated by dividing the share of the individual virtual machine evenly be- tween the virtual processors inside it. Absolute and relative weights, W* and W', are kept separately; for virtual processor j in virtual machine i: W", are kept separately; for virtual processor j in virtual machine i:

If the normalized weights alone were used to proportionally distribute the total amount of processing power, Pio:, between the virtual processors, some processor might receive more power than a real processor actually has. This is not acceptable. Another issue arises since an “active” virtual processor, might not be able to utilize a real processor for a complete unit of time. But since it still is in the dispatch list, possibly with a very high priority, it can still influ- ence the priority of the other virtual processors considerable. Availability and performance aspects for mainframe consolidated servers By Klaus Johansen, at IMM, DTU, for KMD A/S A= the set of active virtual processors VP(i, j)

z/VM provides native support for SCSI disks attached via FCP (Fibre Channel Protocol), as briefly mentioned in the mainframe presentation i1 chapter 2. As a result both z/VM itself and guest operating systems can be in stalled on and operate on SAN disks (Storage Area Network).

From operating system perspective both interface types appear as a whole range of I/O devices: a range real devices in z/VM that is. It requires a total of three consecutive devices per connection: The first device (an even num- bered) is used for read control, the next for write control, and the last for data (see Figure 3-13). Figure 3-13: QDIO based networking: OSA (and HyperSockets) interfaces appear as three I/O devices (z/VM RDEVs) per connection.

A virtual switch (VSWITCH) is virtual device provided by z/VM. VSWITCH’es can operate either at Layer 3 (network layer in OSI model) us- ing IP addressing; or at Layer 2 (data link layer in the OSI model) working at MAC address level. The technology also supports IEEE 802.1Q VLANs. It can be used to connect several virtual machines to an external LAN provided via an OSA interface. A switch can actually also run in disconnected mode (with- out an associated OSA port) in order to interconnect virtual machines though a virtual switch almost equivalent to a physical one (Layer 2). function as “controller” and to enable the connection to the OSA interface

Figure 3-15: The machine language format of the DIAGNOSE instruction. The first part, “83”, is the machine language instruction code (an assembler mnemonic does not exist). RxRy gives the general registers, which hold operand values or storage addresses to such. B is a displacement base register for DDD, which is the actual DI- AGNOSE code value specifying the service to be performed. (Graphic from [29, p. 3]) IBM have implemented a varietv of DIAGNOSE code into z/VM. Onlv a

tual machines move between these lists according to their current work en- gagement; see Figure 3-16. Figure 3-16: The z/VM scheduler/dispatching lists; and flow of virtual machines be- tween them. E1: Virtual machines expected to have short transactions (in z/VM a

Figure 3-17: z/VM is capable of running inside a z/VM virtual machines. Since the virtual machines created by z/VM control program (CP) fully re- sembles real machines, they also are capable of running the z/VM operating system itself. The first z/VM running directly on hardware (often in LPAR) is called as a “first level” system and it runs first level guests. A z/VM running in one of these guests is a “second level” system running second level guests (see Figure 3-17). 3.7.3 Running z/VM in z/VM

In Linux for zSeries directory structure provides information on the device type, device node name, and subchannel numbers. Hardware devices are first and foremost listed in /sys/devices/ccso/ (see Figure 4-1). In this directory all available channels (chpo.XX) and subchannel (0.0.xxxx) are given as sub directories. In the “subchannel subdirectories” yet another directory exists representing the associated device (0.0.xxxx). The other “views” of the hard- ware, e.g. sorted according to bus relationship or drivers, “reuses” this sub- channel and device representation using symbolic links. Availability and performance aspects for mainframe consolidated servers By Klaus Johansen, at IMM, DTU, for KMD A/S

to function (see section 3.6.4 on OSA devices in z/VM). The three files cdevo- cdev2 are symbolic links to devices 0408-040A in “/sys/devices/csso/”.

When running Linux under z/VM, it can be useful to have a good under- standing and an overall picture of the memory hierarchy for the combined environments. This can be useful in order to interpret performance meas- urements or peculiar system behaviour. Linux itself uses swap disks to “enlarge memory”. In many cases Linux prefers to move rarely used pages (e.g. portions of loaded application pro- grams or data) to the swap disks to facilitate space for file buffers and caches. That is why a smaller VDISK (virtual disk in storage) is often used as fast, high priority, swap devices under z/VM. (Actually to many VDISK might in- troduce a priority inversion problem, because CP prefers to steal pages from running guests rather than VDISKs, which are regarded as very important data areas for e.g. lock files. Special DCSS, Discontinuous Storage Seg- ments, can be used as alternative swap device to avoid this problem. [59])

The problem becomes more complicated, when this type of time meas- urement is used in a virtual environment, where the hypervisor dispatches the processor to other virtual machines. Consider the situation depictured in Figure 4-5. The actual time spent in each context is now considerable less, since real processor is occupied elsewhere several times (white areas).

Again consider the effect of tick based time measurements: The time ticks (interrupts), which occur while the real processor is dispatched elsewhere, are delayed and counted, when virtual processor regains control. The time “taken away” from the virtual processor is thereby not taken into account. Again de- pending on the exact timing conditions, the time measurement might be far from the truth: See Figure 4-6.

In never Linux kernels (e.g. the one in SLES10) the problem has been re- solved by introducing the concept of steal time and by eliminating time tick based accounting. The mainframe architecture offers a 64 bit timer register per virtual processor. It basically uses the same format as the “TOD (Time of Day) clock”: the mainframe’s Real Time Clock. But this virtual timer is only running as long as the virtual processor is running. The kernel can uses this register to keep exact track of the time, including the time spent in kernel space, in user space. But even more import it can keep track of the time, where the real processor has been dispatched elsewhere: Figure 4-7.

Figure 5-1: The monitoring setup constructed to deliver one of multiple sets of cus- tomized measurements to an arbitrary workstation via a TCP/IP socket. The actual monitoring process is instantiated and controlled by the client. It connects to the configured TCP port and xinetd invokes the mon.sh script, if the connection is successfully open. Now the client sends a character (or code) representing the wanted monitoring data, for instance a ‘t’ repre- senting information from top. The mon. sh script parses input and invokes a corresponding command. This command performs the necessary measure- ments and reformats the output into a comma separated string (here using an awk script), which is written to stdout and thereby returned to the client. Each time the client sends a new code, a corresponding data string is gen- erated and returned. The client can easily parse the comma separated string and potentially graph the values in near real-time. Writing the unaltered data directly to a text file automatically generates a “CSV” file, which is easily read- able for most spreadsheet programs for further analysis and later graphing. In this case, a simple client with graphing and logging capabilities have been developed in Java, but any type of program or script could be used. The GUI of the Java client program is presented in Figure 5-2 . (The source is not pro- vided, but closely resembles the program for test 2 given in Appendix B.3)

Figure 5-2: The GUI of the developed Java client program, which is used together with the qkumon monitor. (Graphs are based on a class from the JChart2D library from http://jchart2d.sourceforge.net.)

IBM Performance Toolkit is a licensed product, which has to be purchased separately, although shipped as a component of z/VM. The Toolkit provides a variety of system information, performance measurements, and logging op- ‘ions (see Figure 5-3). Like all other things it runs in a virtual machine, but it Jepends on special data provided by CP, which enables it to present a picture of the complete virtual environment from “hypervisor view” (see Figure 5-4).

Figure 5-4:The “IBM Performance Toolkit for VM” runs in its own virtual machine (typi- cally “perfsvm”) but the data is provided from CP. The toolkit CPU readings are used to confirm, that the workloads actually are single threaded and that the virtual machines generally only occupy a sin- gle processor regardless of the number of virtual processors. The toolkit is also used here to generally verify that the tests are completed in a “stable” en- vironment without big fluctuations caused by other guests.

Table 5-1: Test 1; combinations of CPU and “relative share” settings together witl workload name, the number of repetitions and monitor settings.

Table 5-2: Execution time savings in percent gained by using 1-CPU guests (in the test setups with two concurrently running guests). Generally no effect on workload 2.

Figure 5-8: “Munin-graph” showing a typical memory distribution for a (inactive) Linux guest. Linux swaps although most memory is used for file cache and buffers. In a virtual environment, where memory usually is overcommitted, there is not always a gain having large file system buffers in memory since the buff- ers risk being paged back to disk. Actually the effect might be a performance penalty since multiple guests suddenly are heavily dependent on the paging sub system and the same physical disks. 9.2 Test 2: Memory usage 5.2.1 Motivation 5.2 Test 2: Memory usage

When inputting “t” and “g” the program’s main-function respectively in- creases and decreases the number of working threads. These working threads continuously access the allocated memory. The purpose with the working threads is as mentioned to keep the memory active. Each work thread simply loop though the allocated area one word at the time, reading data, performing a simple arithmetic operation (addition) and writing back the result. The worker threads continuously increase their own individual counter variable, for every 128kb of memory they process. A monitor thread wakes up every second and calculates the progress of all the worker threads using these counter variables. The monitor thread calculates the throughput and sends it to stdout. It is presented as a comma separated string also including current time of day (hh:mm:ss), the program’s elapsed time (millisecond accuracy), the current number of worker threads, and the current amount of allocated memory. All other messages are directed to stderr, which makes is easy to gather data for analysis and plotting.

Figure 5-10: Memory allocation during the first 4 steps of the test procedure. Data derived from /proc/meminfo.

Figure 5-12: Linux swap activity within same time interval as Figure 5-10. Data de- rived from /proc/vmstat Throughput

Figure 5-14: Memory allocation during step 6 to 9 of the test procedure. Data derived from /proc/meminfo.

Figure 5-15: Linux swap activity within same time interval as Figure 5-14. Data de- rived from /proc/vmstat.

Figure 5-17: Plot of the transfer rates (read/write values) given in table above.

The mainframe hardware supports two levels of virtualization (SIE) with- out notable performance degradation (as mentioned in 3.7.4 on page 51). Since the logical partitioning provided by PR/SM make up the first level, a z/VM system like KMD’s production environment “KMDZVM”, automatically uses the 2"4 SIE level (refer to Figure 5-18). Any 2"4 level z/VM system within the production system (like the VMUDEMO system provided for the thesis pro- ject) will require a 3'4 level of SIE, which has to be emulated. VM performance expert Bill Bitner states in [50] that running three levels of SIE is “fairly expensive”. On the other hand he claims to have seen as much as 9 levels of virtualization, which imply that adding another layer shouldn’t be fatal. 5.4 Test 4: Z/VM in z/VM penalty 5.4.1 Motivation

Figure 5-22: Comparison of runtimes/the duration of the individual test steps in the two z/VM systems. Figure 5-21: Test T4.1 to T4.4 run in second level z/VM system (VMDEMO); meas- urements from the CP level (collected by Performance Monitor) in VUDEMO.

Figure 6-1: Centralised versus distributed monitor approach. Monitoring systems fall into two groups depending on method used to gather data. The first group relies on a centralised approach. Here a single program or a few processes in a single location collects data. The central monitoring system actively tries to determine the status of the monitored en- tities. This is accomplished simply by establishing some kind of connection to them or bv verforming operations involving them (Figure 6-1). The second group is based on the distributed approach. Here special monitoring programs are placed on or close to the monitored entities. These programs are often called “agents” and they perform the necessary tests and report the outcome to the relevant entities. These agents can range from sim- ple programs to complex intelligent autonomous mobile software elements.” 168]

Figure 6-2: The “alert chain” at KMD. Monitoring messages (alerts and warnings) are transformed to a uniform format and gathered in HP OvenView/Unix. In order to provide the central monitoring centre with single view of all platforms, KMD has chosen to gather all alerts in HP OpenView. The individ- ual monitoring solutions have a central server gathering data for their specific environment (see Figure 6-2). These central servers transform alerts into a uniform message format developed by KMD. HP OvenView agents on the

1.4.3 Workload characteristics

#!/usr/bin/awk -f

const unsigned int kb128iMax = 128*1024/sizeof (unsigned long int) ;

srand(5); // Seeding random number generator

Figure 3-1: The GUI of the MemMeter java client for the gkumon/mem.awk monitor.

Related topics:

Computer Science

Connect with 287M+ leading minds in your field

Discover breakthrough research and expand your academic network

Explore
Papers
Topics

Features
Mentions
Analytics
PDF Packages
Advanced Search
Search Alerts

Journals
Academia.edu Journals
My submissions
Reviewer Hub
Why publish with us
Testimonials

Company
About
Careers
Press
Help Center
Terms
Privacy
Copyright
Content Policy

580 California St., Suite 400

San Francisco, CA, 94104

© 2025 Academia. All rights reserved