Today, most servers are highly underutilized. While average utilization varies by customer and operating system, in the HP-UX environment it is often around 30%. There are myriad reasons for this, but one of the primary reasons is that customers often have one application per server, and they size that server for a peak load of typically three to five times the average utilization.

Resource optimization is one of the goals of HP's adaptive enterprise strategy-a strategy for helping customers to synchronize business and IT to capitalize on change. Virtualization is a cornerstone of HP's approach to helping customers realize the promise of becoming an adaptive enterprise. It is an approach to IT that pools and shares resources so utilization is optimized and supply automatically meets demand.

Part of the virtualization model is the Virtual Server Environment. It offers the broadest virtualization capabilities in the industry today and the only goal-based policy engine, HP-UX Workload Manager (WLM). WLM ties together virtualization techniques including partitioning, resource management, on-demand resources, and clustering, linking them back to business goals and service-level objectives (SLOs). It allows companies to consolidate multiple applications onto a single server, which decreases server administration expenses and makes more efficient use of available resources.

Key uses of HP-UX WLM include these:

• Using excess server capacity by consolidating multiple applications on fewer servers making sure that mission-critical applications still get the resources they need in times of peak demand
• Automatically re-allocating system resources in response to changing priorities, conditions that change over time (night/day, month-end processing, etc.), package movement in a cluster, resource demand, and application performance

HP-UX WLM provides this functionality by automating many features of HP's Process Resource Manager (PRM), HP-UX Processor Sets (psets), HP-UX virtual partitions (vPars), and hard partitions (nPars) that use Instant Capacity on Demand (iCOD).

When using WLM, the system administrator:

1. Defines workload groups for each workload
2. Places the applications or users that define the workloads into their own workload groups (the workload's processes share the resources WLM makes available to the workload group)
3. Creates one or more SLOs for each workload group

In defining an SLO, the system administrator specifies a priority. The administrator can also specify a goal to attain some specific measure (metric) or a targeted resource usage. Such goals can be thought of as metric goals or usage goals. As the applications run, WLM compares the application metrics or usage against the goals. WLM then automatically adjusts processor allocations for the workload groups to achieve the goals.

This document explains the capabilities and benefits of HP UX WLM version A.03.02, which is available for HP 9000 servers running HP UX 11.0 or HP UX 11i v1.0 (B.11.11) and for HP Integrity servers running HP UX 11i v2.0 (B.11.23).

HP-UX Workload Manager is a software product that provides automatic resource allocation and application performance management through the use of prioritized SLOs.

WLMmanages workload groups as defined in its configuration file. The WLMadministrator assigns applications and users to workload groups and sets up one or more SLOs for each group. WLMthen manages each workload group's processor resources according to the SLOs. WLMcan also manage real memory, although not in response to SLO performance. Disk bandwidth can be statically allocated in the configuration file. If multiple users or applications within a workload group are competing for resources, standard HP-UX resource management determines the resource allocation.

WLM is integrated with a variety of products to provide better interoperability. These products include Oracle, Apache, HP UX virtual partitions, HP UX processor sets, GlancePlus, iCAP, Pay per Use, and Serviceguard.

The traditional open systems usage model has been one application running per server. This has led to a proliferation of servers-too many to manage effectively. Generally, each server is sized to provide "headroom" for peak capacity and future growth. With each new server, the surplus capacity grows-and no ability exists to share this excess capacity among the applications. Therefore, many companies want to reduce administration expenses and use computing resources more efficiently by consolidating their data centers onto fewer systems and consolidating multiple applications onto a single server.

With the tools in the HP Partitioning Continuum, there is less need to provide the same degree of headroom, allowing the excess capacity to be shared among the different applications. As one of the partitioning tools, WLM provides the ability to:

• Prioritize workloads on a single system, adjusting the associated workload groups' processor resources based on their goals
• Manage by service-level objectives within and across virtual partitions (vPars) or node partitions (nPars)
• Adjust resource allocations by automatically enabling or disabling SLOs based on time of day, system events, or application metrics
• Enable SLOs associated with a Serviceguard package failover
• Make sure critical workload groups have sufficient resources to perform at desired levels
• Adjust the number of processors in a processor set or virtual partition to meet SLOs
  
• Grant a workload group processor resources in direct proportion to a metric, such as number of processes in the workload group
  
• Grant a workload group dedicated processor and memory resources in the form of a processor set
  
• Set and manage user expectations for performance
  
• Run multiple workloads on a single system and maintain performance of each workload
  
• Monitor resource consumption by applications or users through HP GlancePlus, WLM tools, or PRM tools
  
• Set minimum and maximum amounts of processor(s) and memory available to a workload group

A key reason for using WLM is its ability to manage service level objectives. After defining a workload group, the system administrator specifies one or more SLOs for each workload group. WLM allocates processor resources to workload groups based on whether the group's application is under performing, meeting, or over performing its SLOs.

SLOs can be shares based or goal based. With a shares based SLO, WLM simply tries to grant the associated workload group a certain amount of the processor by allocating processor shares for it. (A processor share is 1/100 of a single processor or 1/100 of each processor on a system, depending on WLM's mode of operation.) With goal based SLOs, WLM actively changes the associated workload group's processor allocation to best meet the SLO. These SLOs are based on one of two goal types:

• Metric goals-goals based on a metric, such as processing at least x transactions per minute or having a response time under y seconds
  
• Usage goals-goals based on a workload group's utilization of its allocated processor: If the processes in a workload group are not using a certain amount of the group's allocation, the allocation is decreased; similarly, if the processes are using a high percentage of the group's allocation, the allocation is increased.
  

A goal-based SLO consists of:

• A workload group
• A goal
• A priority
• Constraints (minimum/maximum processor)
• Conditions (time of day, an event, etc.), if desired

A shares-based SLO contains most of the above elements, but does not include a goal.

Another great reason for using WLM is that it allows the system administrator to prioritize the SLOs. Valid priorities start at 1, with 1 being the highest priority.

SLO priorities do not have to be uniquely assigned-multiple SLOs can be granted the same priority, allowing more than one workload's objective to be top priority. This can be beneficial when multiple workloads are equally important. Typically, all the SLOs in a given configuration should not be assigned the same priority; otherwise, under heavy system load, WLM may not be able to allocate processor resources effectively when there are not enough to meet all SLOs.

A single workload can have multiple SLOs, each with a different priority. One SLO would be the high-priority, "must meet" goal and the remaining SLOs would be lower-priority, "meet if possible" goals (stretch goals). For example, a priority 1 goal might be for the workload to keep response time under three seconds. The priority 2 goal might then be to keep response time under one second. This lower-priority goal might be met, but only after the priority 1 SLOs of its associated workload group and all other workload groups are met.

You will benefit most from WLM if your environment meets one or more of the following conditions:

• You run more than one workload concurrently on a server. The workloads may all run under one instance of HP-UX or in separate partitions, each with its own instance of HP UX. These workloads could be multiple database servers, a database server and an applications server, or any other combination of workloads, provided that they are on PA-RISC servers running HP-UX 11.0 or later or on Itanium-based servers running HP-UX 11i v2.0 (B.11.23).
• You have workloads that can be prioritized.
• You have an important workload with end-user performance requirements.
• You desire consistent performance from applications under varying application and system loads.
• You run Serviceguard and need proper prioritization of workloads after a failover.
• You want more control over resource allocation than PRM provides.

WLM automatically allocates system resources to maintain application performance, even during changing system conditions and fluctuations in workload demand. For WLM to determine the appropriate resource allocation, the system administrator places applications in workload groups and then creates one or more SLOs for each workload group. Among other items, an SLO includes a priority and an optional metric or usage goal.

Along with each metric goal, the WLM system administrator selects metrics to measure the extent to which the goal is being met, exceeded, or underachieved. In addition, the system administrator chooses how to send the data to WLM. Utilities that send data to WLM are called data collectors.

As the applications run, WLM compares the metrics and the goals for each application to determine the appropriate processor allocations.

In addition to metric goals, the system administrator can specify usage goals. With usage goals, WLM gives a workload group more processor resources when it is busy and takes them away when it is idle. WLM does this by tracking an internally collected processor utilization metric.

If you are using WLM on a system without partitions, focus on nPar 0 to understand how WLM controls resources on your system to make sure SLOs are met.

You can use WLM within and across:

• vPars
• nPars that use iCOD software

The functional process flow for the WLM design is:

• The WLM configuration file specifies the goal-based or shares-based SLOs for each workload group. This file also provides the pathnames for the data collectors. WLM reads the configuration file and starts the data collectors.
• With the WLM configuration file activated using the -t option to wlmd, WLM produces audit data in /var/opt/wlm/audit/.
• For each application with a metric goal, as the application runs, a data collector for that application reports the application's metrics. The measurement, for example, might be transaction response times for an online transaction processing (OLTP) application.
• For each metric goal, WLM creates a controller. The controllers are an internal component of WLM. Each controller receives the metric from the respective data collector. The metric is compared to the metric's goal to determine if the application is over performing or under performing. The controllers then request increases to the processor allocations of workload groups with under performing applications, or they request decreases to the processor allocations of workload groups with over performing applications.
  
• For each application with a usage goal, WLM creates a controller. Each controller tracks its application's actual processor usage (utilization of allocated processor resources); no user supplied metrics are required. The controller requests an increase or decrease to the workload group's number of allocated processor resources to bring the utilization percentage within a configurable range.
  
• For applications without goals, WLM requests processor based on the processor shares requested in the SLO definitions. These requests could be for fixed allocations or for shares per metric allocations, with the metric coming from a data collector.
  
• The arbiter, an internal module of WLM, collects all the requests for processor shares. These requests come from controllers or from the SLO definitions. The arbiter satisfies the requests based on priority. If there are not enough resources for every application to meet its goals, the arbiter satisfies the highest priority requests first. If multiple SLOs at the same priority cannot be satisfied, WLM raises the processor allocation for each SLO's associated workload group to the same level, or to the SLO's processor request-whichever is smaller.
  
• Optionally, WLM determines how much memory to distribute to meet the minimum memory requests and then, if any memory remains, divides it among the groups with active SLOs.
  
• WLM then creates a new PRM configuration that applies the new processor and (optional) memory shares. Also, WLM adds data to the statistics log /var/opt/wlm/wlmdstats if enabled through the wlmd l option. The data collectors continue to feed application metrics to WLM, which at intervals calculates new resource allocations and performs any needed PRM reconfiguration.
  
• If configured for partition management, the WLM instance in each partition regularly requests from the WLM global arbiter a certain number of processors for its partition.
  
• The global arbiter adds data to the statistics log /var/opt/wlm/wlmpardstats if enabled through the wlmpard -l option.
  With the WLM global arbiter configuration file activated using the -t option to wlmpard, WLM produces audit data in /var/opt/wlm/audit/.
• The WLM monitoring command-line utility wlminfo or the graphical user interface allow you to get a variety of types of WLM information.
• The status of the SLOs and information about the performance of WLM are sent to the Event Monitoring Service (EMS). Using an EMS client such as System Administration Manager (SAM), a system administrator can choose from a number of notification methods (such as email, SNMP traps, TCP, UDP, and OPC Messaging) for receiving events of specific interest.
• WLM keeps you up to date on the operations of its daemon by updating the message log /var/opt/wlm/msglog.

WLM provides automatic PRM, pset, vPar, and nPar settings based on the defined SLOs. The supported types of SLOs are:

• Shares based SLO-This SLO type allows an administrator to specify a fixed number of shares or a shares per metric allocation for a workload group without specifying a goal. The actual amount of processor resources granted to the workload group is subject to the availability of processor resources after the needs of higher priority SLOs have been met.
  
• Goal based SLO-Goal based SLOs cause WLM to grant more processor resources or take away processor resources based on metrics or usage. These SLOs have either metric goals or usage goals. Metric goal based SLOs are suitable for applications that can generate metrics. For example, OLTP applications are good candidates for metric goal based SLOs.
  
• Usage goal based SLOs attempt to keep a workload group's utilization of its processor allocation within a given percentage range. With a usage goal, a workload group's processor allocation is reduced if its processes are not consuming enough of the group's currently allotted processor, allowing other workload groups to consume more processor if needed. Similarly, if the workload group's processes are using a high percentage of the group's allocation, more processor is requested for it.
  
• WLM automatically changes processor allocations for goal based SLOs to better achieve their stated goals. The actual processor allocation granted is based on the amount of processor needed to meet the goal as determined by the controller, the limits placed on the SLO, and the availability of processor resources after the needs of all higher priority SLOs have been met.
  

When a workload group has to accommodate a "must meet" goal and optional, lower-priority stretch goals, you can assign multiple SLOs, as long each SLO is a fixed-allocation or goal-based SLO.

When consolidating servers, one of the challenges is resource allocation. How do you make sure that one application will not steal needed resources from another?

WLM allows you to specify fixed allocations to make sure each application gets a defined resource amount. Additionally, the system administrator can define and prioritize one or more SLOs for each application's associated workload group. The workload groups are then given the resources they need to achieve the SLOs. This is significant for several reasons:

• Higher system utilization—By giving each application's workload only what it needs, when it needs it, the excess capacity is shared more efficiently.
• Service-level objectives are met even during peak demand—Resources are applied when needed so that even during times of peak demand, SLOs are met.
• Prioritization of SLOs—A workload with a higher-priority SLO is given what it needs to achieve its goal first, then lower-priority SLOs are met. Each workload has at least one SLO so it can be prioritized against other workloads. Furthermore, a workload can have multiple SLOs of various priorities so that it meets a minimal goal when resources are tight, but can achieve greater goals when resources are available.

Using the defined SLOs, WLM determines the amount of processor needed to achieve each SLO. It then allocates processor to each SLO's workload group based on the SLO's priority. This is important for:

• Server consolidation
• Higher resource utilization
• Ease of server management
• Sharing of excess capacity
  
• Application/Workload prioritization

Using WLM with Serviceguard, you can:

• Decrease the performance impact of a failover caused by software being unavailable in active active configurations
  
• Increase utilization of a system that receives failover packages
  
• Simplify the configuration of failover scripts
  
• Simplify routine/scheduled maintenance
  

WLM provides a number of features that support integration with Serviceguard. For more information, as well as step-by-step procedures for integrating the products, see the white paper "More efficient high availability through manageability (integrating Serviceguard and HP-UX Workload Manager)," available from the Information Library, which can be reached via http://www.hp.com/go/wlm.

WLM produces audit information when you activate a configuration using the -t option with either the WLM daemon wlmd or the WLM global arbiter daemon wlmpard.

Once you've activated a configuration using -t, use the wlmaudit command to display the audit data. The wlmaudit command allows you to specify a date range for the data to display. By default, the output is plain text; however, you can display output in formatted HTML as well.

HP-UX WLM Oracle Database Toolkit (ODBTK) simplifies getting metrics on Oracle database instances into WLM. This allows you to better manage Oracle instances. ODBTK is part of the WLM Toolkits product, which is included with WLM. Benefits of ODBTK include the ability to:

• Keep response times for your transactions below a given level by setting response-time SLOs
• Increase an instance's available processor when a particular user connects to the instance
  
• Increase an instance's available processor when more than n users are connected
  
• Increase an instance's available processor when a particular job is active
  
• Give an instance n processor shares for each process in the instance
  
• Give an instance n processor shares for each user connection to the instance

HP-UX 11i v1.0 (B.11.11) and later versions offer a feature known as processor sets, or psets. A processor set represents one or more processors grouped together for exclusive access by applications assigned to that processor set.

WLM allows you to define your workload groups based on processor sets. These groups have dedicated processors. Furthermore, WLM allows you to dedicate physical memory to a workload group that is based on a processor set. This isolates the workload's memory from other workloads running on the system. Rather than simply isolating the group, though, you can configure WLM to adjust the number of processors in your pset based workload groups, giving and taking away processors as indicated by your SLOs.

iCAP, or Instant Capacity, offers the ability to activate reserve processor capacity that is already on your system. You do not pay for the iCAP processors until they are activated. You would activate the iCAP processors either to increase the active processor capacity on the system or to replace the processor capacity of a failing processor on that system. In either case, action can be taken-with no waiting for new processors to arrive-because the iCAP processors are already on the system.

If you have HP UX WLM on a system with the iCAP software, you can configure HP UX WLM to let you know that SLOs are failing and the reserves are needed. By itself, WLM only notifies you of the situation; you must manually activate the reserves. However, if you use WLM with the Pay per use Toolkit (PPUTK, part of the WLM Toolkits product) on an iCAP system, you can have processors automatically activated when SLOs are failing.

HP offers a Pay per Use (PPU) feature that provides the capacity to support peak anticipated demand, but with payment for the HP server based on actual metered or monitored usage of that capacity. This capacity can be increased or decreased, as needed, on a per processor basis.

Using WLM on a system with the PPU software (up to and including version B.04.01), this capacity is increased or decreased automatically. After you set your SLOs in the WLM configuration file, WLM and its Pay per use Toolkit (PPUTK) automatically adjust the number of active processors to the smallest number of processors needed to satisfy the SLOs. With PPU, you pay only for the amount of processor you use. Therefore, decreasing the number of active processors reduces your costs.

• HP Systems Insight Manager provides a single point of administration for multiple HP UX systems. The WLM user interface can be launched from HP Systems Insight Manager, allowing easy management of WLM on multiple systems.

WLM can help you manage and prioritize Apache-based workloads through the use of the WLM Apache Toolkit (ApacheTK). ApacheTK, a component of the WLM Toolkits product that comes with WLM, shows you how to:

• Separate Apache from Oracle database instances
• Separate Apache from batch work
• Isolate a resource-intensive CGI workload
• Separate all Apache Tomcat workloads from other Apache workloads
• Separate two departments' applications using two Apache instances
• Separate module-based workloads with two Apache instances
• Manage Apache processor allocations by performance goal

For more information, see the white paper "Using HP-UX Workload Manager with Apache-based Applications," which offers various use cases. This paper is installed on systems with WLM under /opt/wlm/toolkits It is also available online at http://www.hp.com/go/wlm.

WLM and its Duration Management Toolkit (DMTK) allow you to manage the duration of your applications. Because many SAS jobs run from a few minutes to a few hours, DMTK can help manage your SAS environments, decreasing the impact of these jobs and eliminating the need to purchase more hardware. Given the usefulness of DMTK in these environments, HP also produced the WLM Toolkit for SAS Software (SASTK). This toolkit provides a SAS macro that you place in your SAS jobs to fine-tune duration management. (Both these toolkits come with the WLM Toolkits product, which is included with WLM.)

WLM BEA WebLogic Server Toolkit (WebLogicTK) provides a data collector called wlmwlsdc that tracks metrics indicating how busy WebLogic Server instances are. The toolkit is a component of the WLM Toolkits product, which is included with WLM.
WebLogicTK offers the ability to:

• Manually provide a single instance with an increasing amount of processor resources in the form of a dynamic pset (processor set) for benchmarking
• Separate an instance from other workloads, as well as from other instances, while automatically maintaining performance using a dynamic pset based on:
  • Group processor usage
  • Server instance queue metrics

For more information, see the white paper "Using HP-UX Workload Manager with BEA WebLogic Server," which offers various use cases. This paper is installed on systems with WLM under /opt/wlm/toolkits It is also available online at http://www.hp.com/go/wlm.

WLM manages workloads on individual servers. To manage workloads on multiple servers, install and configure WLM on each of the servers.

WLM can be integrated with HP Serviceguard by storing the WLM configuration file in a file system shared by all nodes in the cluster, then activating the configuration on each node independently.

The HP Partitioning Continuum offers three layers of partitioning. These layers are:

• Hard partitions—These partitions are implemented through hardware. The first form of hard partitions is a complete server, which can be clustered in a HP Serviceguard configuration. Another form of hard partitions is known as nPartitions, which are portions of a single server. Each hard partition runs its own instance of HP UX.
• Virtual partitions—These partitions are created with software, with each virtual partition running its own instance of the HP-UX operating system. Virtual partitions can be used within hard partitions.
  
• Resource partitions—PRM combined with processor sets provides resource partitions. These run within, but not across, hard partitions and virtual partitions. These partitions run within a single instance of HP-UX.

WLM can be used within both hard partitions and virtual partitions. Furthermore, you can use WLM across partitions, automatically moving processors between partitions based on SLOs in the partitions. (Given the physical nature of hard partitions, the "movement" of processors between partitions is achieved using iCAP software: a processor is deactivated on one nPar, and then a processor on another nPar is activated.)

For more information go to http://www.hp.com/go/wlm or see the HP-UX Workload Manager User's Guide.