|
|
Overview
| HP Integrity Superdome Servers: 16- processor, 32- processor, and 64- processor Systems |
![]() |
![]() |
| At A Glance | |||
| The latest release of
Superdome, HP Integrity Superdome supports the new and improved sx2000 chip set.
The Integrity Superdome with the sx2000 chipset supported the Mad9M Itanium processor at initial release. HP Integrity Superdome currently supports the following processors: sx2000
sx1000 (no longer offered
as new systems)
The November 2007 release of Montvale was accompanied by the release of a new I/O chassis for Superdome. The mixed interface I/O chassis supports six slots of PCI-Express (PCIe) and six slots of PCI-X. The abbreviation PCIeX is used below to discuss this product. HP
sx1000 Integrity Superdome supports mixing the Intel Itanium 1.6-GHz/9M processor
and the HP mx2 processor module in the same system, but in different partitions,
as long as they have the same chipset. All cell boards to be mixed in a system
must contain the same chipset, sx1000 only. HP Integrity sx1000 Superdome supports
mixing the Itanium 1.6-GHz/9M processor, PA 8800 and PA 8900 processors in the
same system, but in different partitions, again only with the same chipset, sx1000.
HP Integrity sx2000 Superdome supports mixing the Intel Itanium series 9000 and
series 9100 , Itanium 1.6 GHz processor (Mad9M) and PA 8900 processors in the
same system, but in different partitions, again only with the same chipset, sx2000. Throughout the rest of this document, the term HP sx1000 Integrity Superdome with Intel Itanium 1.6 GHz processors or mx2 processor modules will be referred to as simply "Superdome sx1000." The HP sx2000 Integrity Superdome with Intel Itanium series 9000 or series 9100 processors or single core Intel Itanium processors is refered to as "Superdome sx2000." Integrity Superdome
(with Itanium processors) showcases HP's commitment to delivering a 64 processor
Intel Itanium server and superior investment protection. It is the dawn of a new
era in high end computing with the emergence of commodity based hardware. | |||
| | |||
| HP-UX 11i version 2 (sx1000 & sx2000 systems) |
|
||
|
|
|||
| Microsoft Windows Server 2008 for Itanium- based Systems (sx2000 Superdome servers) |
NOTE: Customers can purchase Windows Server 2008 for Itanium based systems license and later downgrade to Windows Server 2003 which is supported on the Superdome. HP no longer sells Windows Server 2003 under the OEM agreement with Microsoft. | ||
| | |||
| Red Hat Enterprise Linux
4 and 5, SUSE Linux Enterprise Server 11 and 10 (sx2000 systems) Red Hat Enterprise Linux 3 and 4, SUSE Linux Enterprise Server 9 and 10 (sx1000 Itanium systems) |
|
||
|
|
|||
| OpenVMS |
|
||
|
|
|||
| Superdome Service Solutions |
Superdome continues to provide the same positive Total Customer Experience via industry-leading HP Services, as with existing Superdome servers. The HP Services component of Superdome is as follows:
| ||
|
|
Standard Features
| NOTE:
HP Integrity Superdome (sx2000) supports all 4 OEs with either Montvale or
Montecito processors NOTE: HP Superdome (sx2000) systems (Itanium 2 (1.6 GHz) '"Mad9M") support only HP-UX and Windows. |
| Minimum/Maximum Configurations for Superdome (sx2000) with Intel Itanium Processors (both Montvale and Montecito) |
|
Minimum Configuration |
Maximum Configuration (One Partition) | |||||||||
|
HP UX 11i v2/v3 |
Microsoft Windows Server 2008 for Itanium-based Systems | SLES10 |
Red Hat R4 |
OpenVMS V8.3 (or higher)3 |
HP UX 11i v2/v3 |
Microsoft Windows Server 2008 for Itanium-based Systems |
SLES 10 |
Red Hat RHEL 4 |
OpenVMS version 8.3 (or higher)3 | |
| 16
Processors / 32 Cores | ||||||||||
| Processors/ cores | 1/1 |
1/1 |
1/1 |
1/1 |
1/1 |
16/32 |
Datacenter 16/32; Enterprise 8/16 |
16/32 |
16/32 |
16/32 |
| Memory |
16 GB |
16 GB |
16 GB |
16 GB |
16 GB |
512 GB |
Datacenter 512 GB; Enterprise 256 GB |
512 GB |
512 GB |
512 GB |
| Cell Boards | 1 |
1 |
1 |
1 |
1 |
4 |
Datacenter 4; Enterprise 2 |
4 |
4 |
4 |
| PCIeX Chassis | 1 |
1 |
1 |
1 |
1 |
4 |
Datacenter 4; Enterprise 2 |
4 |
4 |
4 |
| PCI X Chassis | 1 |
1 |
1 |
1 |
1 |
4 |
Datacenter 4; Enterprise 2 |
4 |
4 |
4 |
| Maximum nPars | 1 |
1 |
1 |
1 |
1 |
4 |
Datacenter 4; Enterprise 2 |
4 |
4 |
4 |
|
32 Processors / 64 Cores | ||||||||||
| Processors / Cores | 1/1 |
1/1 |
1/1 |
1/1 |
1/1 |
32/64 |
Datacenter 32/64; Enterprise 8/16 |
32/64 |
32/64 |
16/32 |
| Memory |
16 GB |
16 GB |
16 GB |
16 GB |
16 GB |
1024 GB |
Datacenter 1024 GB; Enterprise 256 GB |
1024 GB |
1024 GB |
512 GB |
| Cell Boards | 1 |
1 |
1 |
1 |
1 |
8 |
Datacenter 8; Enterprise 2 |
8 |
8 |
4 |
| PCIeX Chassis | 1 |
1 |
1 |
1 |
1 |
8 |
Datacenter 8; Enterprise 2 |
8 |
4 |
4 |
| PCI X Chassis | 1 |
1 |
1 |
1 |
1 |
8 |
Datacenter 8; Enterprise 2 |
8 |
4 |
4 |
| Maximum nPars | N/A |
N/A |
1 |
N/A |
N/A |
81 |
Datacenter 81; Enterprise 2 |
81 |
81 |
81 |
| 64 Processors / 128 Cores | ||||||||||
| Processors / Cores | 2/2 |
2/2 |
2/2 |
2/2 |
1/1 |
64/128 |
Datacenter 32/64; Enterprise 8/16 |
64/128 |
64/128 |
16/32 |
| Memory |
16 GB |
16 GB |
16 GB |
16 GB |
16 GB |
2048 GB |
Datacenter 1024 GB; Enterprise 256 GB |
2048 GB |
2048 GB |
512 GB |
| Cell Boards | 2 |
2 |
2 |
2 |
1 |
16 |
Datacenter 16; Enterprise 2 |
16 |
16 |
4 |
| PCIeX Chassis | 1 |
1 |
1 |
1 |
1 |
16 |
Datacenter 16; Enterprise 2 |
8 |
6 |
4 |
| PCI X Chassis | 1 |
1 |
1 |
1 |
1 |
16 |
Datacenter 16; Enterprise 2 |
8 |
6 |
4 |
| Maximum nPars | 1 |
1 |
1 |
1 |
1 |
162 |
Datacenter 162; Enterprise 2 |
16 2 |
16 2 |
162 |
| 1Requires
I/O expansion cabinet if more than 4 nPars 2 Requires I/O expansion cabinet if more than 8 nPars 3 Dual core Montecito processors only on sx2000 configurations for OpenVMS and Linux | ||||||||||
| Standard Hardware Features |
Superdome with Intel Itanium Processors Standard Hardware Features
| ||||||||
|
|
Configuration
| There are three basic building blocks in the Superdome system architecture: the cell, the crossbar backplane, and the PCI X (or PCIeX) based I/O subsystem. Below is a high level illustration of this architecture. The figures below depict the system architecture: |
|
|
| Cabinets (sx2000) |
Starting with the sx1000 chip set, Superdome servers have been released with the graphite color. A Superdome system will consist of up to four different types of cabinet assemblies:
Superdome cabinets will be serviced from the front and rear of the cabinet only. This will enable customers to arrange the cabinets of their Superdome system in the traditional row fashion found in most computer rooms. The width of the cabinet will accommodate moving it through common doorways in the U.S. and Europe. The intake air to the main (cell) card cage will be filtered. This filter is removable for cleaning/replacement while the system is fully operational. A status display is located on the outside of the front and rear doors of each cabinet. The customer and field engineers can therefore determine basic status of each cabinet without opening any cabinet doors. Superdome 16 processor and Superdome 32 processor systems are available in single cabinets. Superdome 64 processor systems are available in dual cabinets. Each cabinet may contain a specific number of cell boards (consisting of processors and memory) and I/O. See the following sections for configuration rules pertaining to each cabinet. |
||
|
|
|||
| Cells (Processors and Memory) | A cell, or cell
board, is the basic building block of a Superdome system. It is a symmetric
multi processor (SMP), containing up to four processor modules and up to
16 GB of main memory using 512-MB DIMMs (sx1000 only) in 32 memory slots,
up to 32 GB of main memory using 1 GB DIMMs, up to 64 GB of main memory
using 2 GB DIMMs, and up to 128 GB of main memory using 4-GB DIMMS (sx2000
only). It is also possible to mix DIMM sizes on the same cell board. A connection
to a 12 slot PCI X card cage is optional for each cell. Each hard partition
must be connected to at least one PCI-X I/O card cage.
The Superdome cell boards shipped from the factory are offered with 1, 2, 3, or 4 processors. The Superdome cell board contains:
|
||
|
|
|||
| Crossbar Backplane | sx2000: Each backplane contains
three independent crossbar fabrics resulting in a robust highly available
system. A High speed serial (HSS) link technology is employed for crossbars
interconnect. Each backplane cabinet can support up to eight cells. Eight
cells support a 32 processor module Superdome. Two backplanes (two cabinets)
can be linked together with flex cables to produce a complex that can support
up to 16 cells resulting in a 64 processor module Superdome (128 core dual
core Itanium or 64 core Itanium (Mad9M)).
sx1000: Each crossbar backplane contains two sets of two crossbar chips that provide a non blocking connection between eight cells and the other backplane. Each backplane cabinet can support up to eight cells. A backplane supporting four cells results in a 16 processor Superdome (16 core Itanium Madison or 32 core Itanium mx2). Similarly, a backplane supporting eight cells would result in a 32 processor Superdome (32 core Itanium Madison or 64 core Itanium mx2). Two backplanes can be linked together with flex cables to produce a cabinet that can support up to 16 cells resulting in a 64 processor Superdome (64 core Itanium Madison or 128 core Itanium mx2). |
|
|
|
| I/O Subsystem |
Each I/O chassis provides twelve I/O slots.
|
|
|
|
| Core I/O (required by sx1000 systems only, sx2000 systems do not have a core I/O) | The core I/O in Superdome provides
the base set of I/O functions required by every Superdome partition. Each
partition must have at least one core I/O card in order to boot. sx2000
systems must have I/O cards with at least LAN connectivity as its "core"
I/O capability in each partition. Multiple core I/O cards may be present
within a partition (one core I/O card is supported per I/O backplane); however,
only one may be active at a time. Core I/O will utilize the standard long
card PCI X form factor but will add a second card cage connection to the
I/O backplane for additional non PCI X signals (USB and utilities). This
secondary connector will not impede the ability to support standard PCI
X cards in the core slot when a core I/O card is not installed.
Any I/O chassis can support a Core I/O card that is required for each independent partition. A system configured with 16 cells, each with its own I/O chassis and core I/O card could support up to 16 independent partitions. Note that cells can be configured without I/O chassis attached, but I/O chassis cannot be configured in the system unless attached to a cell. |
|
|
|
| HP-UX Core I/O (A6865A)(sx1000 systems only, sx2000 systems do not have a core I/O) |
The core I/O card's primary functions are:
|
|||||
| Other common functions, such as Ultra/Ultra2 SCSI, Fibre Channel, and Gigabit Ethernet, are not included on the core I/O card. These functions are, of course, supported as normal PCI-X add-in cards. | ||||||
| The unified 100Base-T Core LAN driver code searches to verify whether there is a cable connection on an RJ-45 port or on an AUI port. If no cable connection is found on the RJ-45 port, there is a busy wait pause of 150 ms when checking for an AUI connection. By installing the loopback connector (description below) in the RJ-45 port, the driver would think an RJ-45 cable was connected and would not continue to search for an AUI connection, hence eliminate the 150 ms busy wait state | ||||||
|
||||||
|
|
||||||
| Core I/O (A6865A and optional VGA/USB A6869A/A6869B)- (sx1000 systems only, sx2000 systems do not have core I/O) | For Windows Server 2003 and Windows Server 2008, Windows does not support the 10/100 LAN on the A6865A core I/O card, a separate Gigabit Ethernet card such as the A7061A, A7073A, A9899A, or A9900A is required. The use of Graphics/USB card (A6869A/A6869B) is optional and not required. NOTE: A6869A is not supported on systems with the sx2000 chipset. |
|
|
|
| Linux Core I/O (A6865A)(sx1000 systems only, sx2000 systems do not have core I/O) |
The core I/O card's primary functions are:
Other common functions, such as Ultra/Ultra2 SCSI, Fibre Channel, and Gigabit Ethernet, are not included on the core I/O card. These functions are supported as normal PCI-X add-in cards. |
|
|
|
| OpenVMS Core I/O (A6865A)(sx1000 systems only, sx2000 systems do not have core I/O) |
The core I/O card's primary functions are:
Other common functions, such as Ultra320 SCSI, Fibre Channel, and Gigabit Ethernet, are not included on the core I/O card. These functions are supported as normal PCI X add in cards. |
|
|
|
| I/O Expansion Cabinet | The I/O expansion functionality is physically partitioned into four rack-mounted chassisthe I/O expansion utilities chassis (XUC), the I/O expansion rear display module (RDM), the I/O expansion power chassis (XPC) and the I/O chassis enclosure (ICE). Each ICE supports up to two 12-slot PCI-X chassis. |
|
|
|
| Field Racking |
The only field rackable I/O expansion components are the ICE and the
12 slot I/O chassis. Either component would be field installed when the
customer has ordered additional I/O capability for a previously installed
I/O expansion cabinet. |
|
|
|
| I/O Chassis Enclosure (ICE) | The I/O chassis enclosure (ICE)
provides expanded I/O capability for Superdome. Each ICE supports up to
24 I/O slots by using two 12 slot Superdome I/O chassis. The I/O chassis
installation in the ICE puts the I/O cards in a horizontal position. An
ICE supports one or two 12 slot I/O chassis. The I/O chassis enclosure (ICE)
is designed to mount in an HP Universal 10K G2 rack and consumes 9U of vertical
rack space. To provide online addition/replacement/deletion access to I/O cards and hot swap access for I/O fans, all I/O chassis are mounted on a sliding shelf inside the ICE. Four (N+1) I/O fans mounted in the rear of the ICE provide cooling for the chassis. Air is pulled through the front as well as the I/O chassis lid (on the side of the ICE) and exhausted out the rear. The I/O fan assembly is hot swappable. An LED on each I/O fan assembly indicates that the fan is operating. |
|
|
|
| Cabinet Height and Configuration Limitations |
Although the individual I/O expansion cabinet components are designed for installation in an HP Universal 10K G2 rack, rack size limitations have been agreed upon. In order to allay service access concerns, the factory will not install IOX components higher than 1.6 meters from the floor. Open space in an IOX cabinet will be available for peripheral installation. Refer to the 10K G2 Series Rack Best Practices Guide for information on rack deployment, stabilization, and transportation. Go to: http://www.hp.com/go/rackandpower for more information. |
| | |
| Peripheral Support | All peripherals qualified for use with Superdome and/or for use in a Universal 10K G2 are supported in the I/O expansion cabinet as long as there is available space. Peripherals not connected to or associated with the Superdome system to which the I/O expansion cabinet is attached may be installed in the I/O expansion cabinet. |
|
|
|
| Server Support |
No servers except those required for Superdome system management such as Superdome Support Management Station or ISEE may be installed in an I/O expansion. Peripherals installed in the I/O expansion cabinet cannot be powered by the XPC. Provisions for peripheral AC power must be provided by a PDU or other means. |
|
|
|
| Standalone I/O Expansion Cabinet | If an I/O expansion cabinet is ordered alone, its field installation can be ordered via option 750 in the ordering guide. |
|
|
|
| DVD Solution | The DVD solution for Superdome requires the following components. These components are required per partition. An external rack AF004A or AF014A must also be ordered with the DVD solution. | |||||||||||||||||||||||
| NOTE: One DVD is required and one DAT is recommended per nPartition. | ||||||||||||||||||||||||
| NOTE: For OpenVMS, only the one DVD is required per nPartition. | ||||||||||||||||||||||||
| Superdome DVD Solutions | ||||||||||||||||||||||||
|
||||||||||||||||||||||||
|
10.5-meter HD HDTS68 is required if DDS-4 or DDS-5 is used. |
||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| Partitions |
Superdome can be configured with hardware partitions, (nPars). Superdome systems running Microsoft Windows Server 2008 for Itanium-based Systems, SUSE Linux Enterprise Server, Red Hat Enterprise Linux, or OpenVMS, do not support virtual partitions (vPars). A hardware partition (nPar) consists of one or more cells that communicate coherently over a high bandwidth, low latency crossbar fabric. Individual processors on a single cell board cannot be separately partitioned. Hardware partitions are logically and electrically isolated from each other such that transactions in one partition are not visible to the other hardware partitions within the same complex. Each nPar runs its own independent operating system. Different
nPars may be executing the same or different revisions of an operating system,
or they may be executing different operating systems altogether. Superdome supports
HP UX 11i version 2 and 3, Microsoft Windows Server 2008 for Itanium based Systems,
SUSE SLES 9 ,10 amd 11, Red Hat RHEL AS 3,4 and 5, and OpenVMS operating systems.
The diagram below shows a multi OS environment within Superdome. NOTE: The sx2000 Superdome
with Mad9M processors only supports HP UX 11i version2 and 3 , and Microsoft Windows
Server 2008 for Itanium-based Systems, (does not support any version of Linux
or OpenVMS) Each nPar has its own independent processors, memory and I/O resources consisting of the resources of the cells that make up the partition. Resources (cell boards and/or I/O chassis) may be removed from one nPar and added to another without having to physically manipulate the hardware, but rather by using commands that are part of the System Management interface. The table below shows the maximum size of nPars per operating system: | ||||||||||||||||||||
| |||||||||||||||||||||
| For information on type of I/O cards for networking and mass storage for each operating environment, please refer to the Technical Specifications section of this document. For licensing information for each operating system, please refer to the Ordering Guide. Superdome,supports static hard partitions (nPars) on
all operating environments. Static partitions imply that any nPar configuration
change requires a reboot of the nPar. | |||||||||||||||||||||
|
| |||||||||||||||||||||
| Microsoft Windows Server 2008, Datacenter edition for Itanium-based systems - HP Product Structure | Product Number
T8704A Options:
|
| | |
| Mixing of PA-RISC and Itanium Cells in Superdome |
1) Which processors can be mixed in a Superdome? | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 2) In order to run an Itanium-based partition in an Integrity server, what changes are required? In order to add a new partition with Itanium 9M processors on an HP 9000 Superdome, the following steps are required: Step #1: Upgrade firmware on PA-RISC based partitions Step #2: Create a new hard partition in the Superdome for Itanium based cell (s) Step #3: Plug in cell boards for Itanium based cells Step #4: Some I/O cards may need to be added for that specific hard partition (Windows does not support the identical set of I/O cards that HP-UX 11i supports) Step #5: Load operating system for Itanium based partition Upgrading a PA-RISC partition to support Itanium processors would require similar steps: Step #1: Upgrade firmware on PA-RISC based partitions Step #2: Pull out existing PA-RISC cell boards Step #3: Swap existing memory into cell boards for Intel Itanium processor (protects investment in current memory) Step #4: Plug in cell boards for Itanium based cell boards Step #5: Some I/O cards may need to be added for that specific hard partition (Windows does not support the identical set of I/O cards that HP-UX 11i supports) Step #6: Load operating system for Itanium based partition
3) Is mixing of PA-RISC and Itanium processors factory configurable? No. HP supports PA RISC systems that shipped from the factory adding Itanium partitions later on in the field. In addition, HP offers the ability to enable Itanium based systems that shipped from the factory to add PA RISC partitions later on in the field (this will happen less frequently) HP does not allow PA RISC partitions to be added to Intel Itanium-based
Integrity systems that shipped from the factory. Mixing of sx1000 and sx2000 chipsets
in the same system is NOT supported. 4) How long does it take to add an Itanium-based partition in an HP 9000 customers? The effort required to add an Itanium-based partition to an HP 9000 Superdome is slightly more than what is required to add a new PA-RISC-based partition. In both cases you have to create a new hard partition in the Superdome, plug in cell boards for the new processors and load the operating system. There are no hardware changes required to support mixing of processor types. The only known difference at this time is that a customer would need to upgrade the firmware to support the Itanium architecture. 5) Will mixing impact the overall performance of a Superdome (i.e., would an Itanium based partition have lower performance in a mixed Superdome as in a Superdome only populated with Itanium processors)? No. For example, suppose you have a hard partition with 16 Intel Itanium 9M processors. This partition would have the same performance in a system with a mix of PA RISC and Itanium based hard partitions as a server where all of the other partitions are Itanium based. This is assuming that besides the type of processors, all of the other configuration options are the same (memory, cell board locations, number of partitions, etc). 6) Can I add PA RISC processors to an Itanium based Integrity Superdome? Yes. HP offers the ability to enable Itanium-based systems that shipped from the factory to add PA-RISC partitions later on in the field. This serves as a safety net/insurance policy. If the customer determines after the fact that a specific application is not ready for the Itanium technology, they can simply add a PA-RISC hard partition to that Integrity Superdome and continue the execution of the application while it is certified. 7) Which versions of the HP UX 11i operating system are supported? HP supports HP UX 11.11 (HP UX 11i v1) for the PA RISC architecture as well as 11.23 (HP UX 11i v2) and 11.31 (HP-UX 11i v3) for Itanium processors and PA RISC processors in a mixed environment. NOTE: HP UX 11i v2
and v3 are the only HP UX versions supported on sx2000 based systems. 8) Does a customer have to power down when they add an Itanium-based partition to a PA-RISC-based Superdome? It depends. Please refer to
the Superdome mixed configuration documents for these details. The documents for
sx1000 can be found at: http://docs.hp.com/en/6906/SuperdomeMixingRev1.3.pdf
The sx2000 information is included in the service guide for sx2000 Superdome here:
9) What are the minimum firmware requirements for mixing various operating environments? The matrix below describes the minimum firmware requirements for sx1000 systems: | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Single System Reliability/Availability Features | Superdome high availability offering is as follows: |
| NOTE: Online addition/replacement for cell boards is not currently supported and will be available in a future HP UX release. Online addition/replacement of individual processors and memory DIMMs will never be supported.) | |
| |
| | |
| Multi-system High Availability | HP-UX 11i v2 |
|
Any Superdome partition that is protected by Serviceguard or Serviceguard Extension for RAC can be configured in a cluster with:
|
|
| Separate partitions within the same Superdome system can be configured as part of different Serviceguard clusters. | |
|
|
|
| Geographically Dispersed Cluster Configurations |
The following Geographically Dispersed Cluster solutions fully support cluster configurations using Superdome systems. The existing configuration requirements for non-Superdome systems also apply to configurations that include Superdome systems. An additional recommendation, when possible, is to configure the nodes of cluster in each datacenter within multiple cabinets to allow for local failover in the case of a single cabinet failure. Local failover is always preferred over a remote failover to the other datacenter. The importance of this recommendation increases as the geographic distance between datacenters increases.
From an HA perspective, it is always better to have the nodes of an HA cluster spread across as many system cabinets (Superdome and non-Superdome systems) as possible. This approach maximizes redundancy to further reduce the chance of a failure causing down time. |
|
|
|
| Windows Server Clustering | |
| Microsoft
Windows Server 2008 for Itanium-based Systems Clustering |
HP has qualified and supports the following capabilities with Integrity Superdome and MSCS:
For high availability purposes with MSCS, it is recommended (but not required) that customers also use HP SecurePath software (v4.0c SP1) with HP storage for multi pathing and load balancing capabilities in conjunction with the fiber channel HBA, AB466A or AB467A. Additionally, the NCU (NIC Configuration Utility), which is provided from HP on the SmartSetup CD that ships with Windows partitions, can also be used in conjunction with MSCS clusters with the HP supported Windows NIC cards. Additionally, customers can see the completion of our certification
for the Microsoft Windows catalog at the following URL: http://www.windowsservercatalog.com/results.aspx?&chtext=&cstext=&csttext=&chbtext= Microsoft requires hardware vendors to complete this certification-also called "Windows logo ing." |
| OpenVMS Multi-System High Availability |
An HP OpenVMS Cluster is a highly integrated organization of AlphaServer and HP Integrity server systems, applications, operating systems, and storage devices. These systems can be connected to each other and storage components in a variety of ways. HP OpenVMS Cluster software is an integral part of the OpenVMS operating system, providing the basis for many of the key capabilities utilized by OpenVMS enterprise solutions. A full "shared everything" cluster design that has been in existence for more than 20 years, OpenVMS Cluster software allows for the maximum in expandability, scalability, and availability for mission-critical applications. Qualified for up to 96 computer nodes and more than 3,000 processors, OpenVMS clusters afford virtually 100 percent uptime and expand the multiprocessing capabilities of the computing environment. For information on OpenVMS Cluster software, refer to the OpenVMS Cluster Software page http://h71000.www7.hp.com/openvms/products/clusters/index.html and the OpenVMS Cluster Software product description at http://h18000.www1.hp.com/info/SP2978/SP2978PF.PDF. For specific information on supported cluster and storage options for OpenVMS, refer to the OpenVMS Operating System product description at http://h18000.www1.hp.com/info/XAV12X/XAV12XPF.PDF. For information on specific Cluster Configuration guidelines, refer to the OpenVMS documentation at http://h71000.www7.hp.com/doc/, specifically the latest version of the Guidelines to OpenVMS Cluster Configuration manual. |
|
|
|
| Management Features |
| Integrity Superdome now supports the Console and Support Management Station in one device. |
|
|
| Service Processor (MP) | The service processor
(MP) utility hardware is an independent support system for nPartition servers.
It provides a way for you to connect to a server complex and perform administration
or monitoring tasks for the server hardware and its nPartitions. The main features of the service processor include the Command menu, nPartition consoles, console logs, chassis code viewers, and partition Virtual Front Panels (live displays of nPartition and cell states). Access to the MP is restricted by user accounts. Each user account is
password protected and provides a specific level of access to the Superdome
complex and service processor commands. Multiple users can independently
interact with the service processor because each service processor login
session is private. In general, the service processor (MP) on Superdome servers is similar to the service processor on other HP servers, while providing enhanced features necessary for managing a multiple-nPartition server. For example, the service processor manages the complex profile, which defines nPartition configurations as well as complex-wide settings for the server. The service processor also controls power, reset, and TOC capabilities, displays and records system events (chassis codes), and can display detailed information about the various internal subsystems. |
|
| Functional capabilities |
The primary features available through the service processor are:
|
|
|
|
||
| Support Management Station (SMS) | The Support Management
Station (SMS) runs the Superdome scan tools that enhance the diagnosis and
testability of Superdome. The SMS and associated tools also provide for
faster and easier upgrades and hardware replacement.
The purpose of the SMS is to provide Customer Engineers with an industry leading set of support tools, and thereby enable faster troubleshooting and more precise problem root cause analysis. It also enables remote support by factory experts who consult with and back up the HP Customer Engineer. The SMS complements the proactive role of HP's Instant Support Enterprise Edition (ISEE) that is offered to Mission Critical customers by focusing on reactive diagnosis for both mission critical and non mission critical Superdome customers. The user of the SMS is the HP Customer Engineer and HP Factory Support Engineer. The Superdome customer benefits from their use of the SMS by receiving faster return to normal operation of their Superdome server and improved accuracy of fault diagnosis, resulting in fewer callbacks. HP can offer better service through reduced installation time. |
|
| Functional Capabilities: |
The SMS basic functional capabilities are:
|
|
|
|
||
| Console Access |
The optimal configuration of console device(s) depends on a number of
factors, including the customer's data center layout, console security
needs, customer engineer access needs, and the degree with which an operator
must interact with server or peripheral hardware and a partition (i.e.
changing disks, tapes). This section provides a few guidelines. However
the configuration that makes best sense should be designed as part of
site preparation, after consulting with the customer's system administration
staff and the field engineering staff. Three common data center styles are:
These can each drive different solutions to the console access requirement. The considerations listed below apply to the design of provision of console access to the server. These must be considered during site preparation.
|
|
|
Functional capabilities |
|
|
|
|
||
| Windows Server 2008 |
For Windows Server 2008 customers desiring uninterrupted visibility to the Superdome Windows partition, it is recommended that customers purchase an IP console solution separately to view the partition while the OS is rebooting (in addition to normal Windows desktop if desired). Windows Terminal Services, which is a standard feature in Windows Server 2008, should be the recommended method to provide remote access, but is lacking in displaying VGA output during reboot. For customers who mandate VGA access during reboot, the IP console switch (AF6002), used in conjunction with a VGA/USB card in the partition (A6869B) is the solution. These IP console solutions are available "off the shelf" with resellers or the ProLiant supply chain. The features of this switch are as follows:
If the full graphical console access is needed, the following must be ordered with the Integrity Superdome purchase (it will not be integrated in the factory, but will ship with the system): |
|||||||||
|
||||||||||
|
For additional information, please visit: |
||||||||||
|
|
||||||||||
| Support |
SMS and Console Support |
|
|
|
| Hardware Requirements |
There is only one choice of SMS for Integrity Superdome: HP Superdome SMS PC solution (A9802D) NOTE: As of 10/1/2007, the rx2620 based
SMS is no longer offered. The Windows PC based
SMS:
The Windows PC based SMS/Console is comprised of a HP rp5700 desktop
PC and a TFT7600 retractable display monitor/keyboard/mouse to enable
console access. This solution also requires (and includes) a switch. This
is because scan diagnostics will not work properly if more than one IP
address exists on the PC SMS/Console. The PC based SMS, by default, provides
console functionality. A customer may not substitute any PC running Windows Server Vista for the PC SMS/Console due to the specialized software applications that have been qualified on the hardware and OS. Utilizing any other device as the SMS will void the warranty on the Superdome system and degrade the ability to service the system. The Windows PC SMS bundle is comprised of:
NOTE: If full graphical access to the SMS is needed, the PS/2 Interface Adapter (262588 B21) will allow the SMS to share the IP Console Switch with other OS instances. |
|
|
|
| Software Requirements |
All SMS software is preloaded in the factory and delivered to the customer as a complete solution. Current versions of the SMS software have not been qualified for 64 bit Windows. The Windows PC SMS/Console runs Windows Vista as the default operating system. The PC SMS/Console will follow the Windows OS roadmap and support later versions of this operating system as needed. The version of the scan tools used on the sx1000 Superdomes also does not require scan traffic to be isolated from console traffic. |
|
|
|
| SMS/Console Connectivity | Console capabilities are integrated
with the Windows PC SMS solution. The PC SMS includes the necessary display,
keyboard, and mouse, and only one LAN port on the MP is required to be connected.
The Windows PC SMS can support and connect to either of the MP LAN interfaces
(known as the Private and Customer LAN ports). Both LAN ports on the MP
have identical functionality so there is no preference in using one over
the other. Only one IP port on the Superdome MP is required to be connected
to the PC SMS. Since scan and console traffic can co exist on the same network,
only one IP address exists (and is supported) on the PC SMS.
The Integrity Superdome scan tools use TCP/IP, not UDP, for scan diagnostics. Therefore it is not necessary to isolate scan and console traffic on Integrity Superdomes. The Ethernet I/O cards from each nPar can optionally be connected to the Ethernet switch in order to facilitate graphical console functionality (i.e., parmgr). However, security concerns may dictate that a partition NIC not be connected to the management LAN. Alternatives are to access from a management station to a partition LAN through a secure router, or to use text mode access to commands via the console. Additional Integrity Superdomes can be added
to this configuration. Each new Integrity Superdome will require only one CAT5e
LAN cable for connection of the Customer/Private LAN port on the Superdome MP
to the existing switch. Additional Integrity Superdomes require only a CAT5e LAN cable for connection of the Private/Customer LAN port to the existing switch. PA-8700 Superdomes can also be added to this configuration, but require separate scan & console networks. Details on mixed Superdome environments follow later in this section. Windows Terminal Services (standard in Windows Server 2008) should be the recommended method to provide remote access for Windows partitions, but is lacking in displaying VGA output during reboot. For customers who mandate VGA access during reboot, configure the IP console switch (AF6002) used in conjunction with a VGA/USB card in the partition (/A6869B). |
| | |
| Mixed Superdome Environments |
| Some important rules regarding SMS/Console support in mixed Superdome environments: |
|
|
|
| Upgrading a PA-8700 Superdome to Integrity |
Upgrading a PA-8700 Superdome to Integrity After performing the software upgrade, scan and console traffic from the SMS to the newly upgraded Superdome can co-exist on the same network. Note that if there are other PA-8700 Superdomes still connected to the SMS, those Superdomes will still require the two separate networks (see Adding a PA-8800/PA-8900 Superdome to a PA-8700 Environment, below). |
|
|
|
| Adding an Integrity Superdome to a PA-8700 Environment |
When adding an Integrity Superdome to a PA-8700 environment, the Customer can choose to purchase a new SMS/Console (either the Windows or UNIX version) to manage the new Integrity Superdome, and continue to use their existing legacy SMS to manage the PA-8700 Superdomes. A simpler solution is to upgrade the software on the legacy UNIX SMS so that it may manage the new Integrity Superdome. The upgraded UNIX SMS is able to determine which Superdomes are PA-8700 and which are Integrity, and will separate scan traffic from console traffic on the PA-8700 Superdomes. To support this, the separate scan and console networks should be maintained for all PA-8700 Superdomes. The new Integrity Superdome requires only one connection to the SMS from its Private/Customer LAN port on the MP. |
|
|
|
| System Management Features | HP-UX (sx1000 & sx2000)
In addition, the Network Node Manager (NNM) management station will run on HP-UX Itanium 2 based servers. NNM automatically discovers, draws (maps), and monitors networks and the systems connected to them. All other OpenView management tools, such as OpenView Operations, Service Desk, and Service Reporter, will be able to collect and process information from the agents running on Itanium 2-based servers running HP-UX. Microsoft Windows Server 2008 for Itanium-based Systems (sx1000 and sx2000)
Red Hat Enterprise Linux and SUSE Linux Enterprise Server (sx1000 and sx2000)
|
| OpenVMS
Configuration
Workload Management
System Management for OpenVMS
OpenView for OpenVMS
| |
|
| |
| General Site Preparation Rules | AC Power Requirements The modular, N+1 power shelf assembly is called the Front End Power Subsystem (FEPS). The redundancy of the FEPS is achieved with 6 internal Bulk Power Supplies (BPS), any five of which can support the load and performance requirements. Input Options |
|||||||||||||||||
| Input Power Options | ||||||||||||||||||
|
||||||||||||||||||
| a. A dedicated branch is required
for each PDCA installed. b. In the U.S.A, site power is 60 Amps; in Europe site power is 63 Amps. c. Refer to the Option 006 and 007 Specifics Table for detailed specifics related to this option. d. In the U.S.A. site power is 30 Amps; in Europe site power is 32 Amps. |
||||||||||||||||||
| Option 006 and 007 Specifics* | ||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||
| a. In-line connector is available
from HP by purchasing A6440A, Option 401. b. Panel mount receptacles must be purchased by the customer from a local Mennekes supplier. c. In-line connector is available from HP by purchasing A6440A, Option 501. |
||||||||||||||||||||||||||||||||||||
| NOTE: A qualified electrician must wire the PDCA in line connector to site power using copper wire and in compliance with all local codes. | ||||||||||||||||||||||||||||||||||||
| Input Requirements Reference the Site Preparation Guide for detailed power configuration requirements. |
||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||
| Cooling Requirements | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
| DataCool is a custom-engineered overhead solution for both new data center construction and for data room upgrades for high heat loads. It is based on Liebert's TeleCool product, which has been installed in 600 telecommunications equipment rooms throughout the world. The system utilizes heat exchanger pump units to distribute fluid in a closed system through patented cooling coils throughout the data center. The overhead cooling coils are highly efficient heat exchangers with blowers that direct the cooling where it is needed. The blowers are adjustable to allow flexibility for changing equipment placement or room configurations. Equipment is protected from possible leaks in the cooling coils by the patented monitoring system and purge function that detects any leak and safely purges all fluid from the affected coils. DataCool has interleaved cooling coils to enable the system to withstand a single point of failure and maintain cooling capability. | ||||||||||||||||||||||||||||||||||||||||
| Features and Benefits | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
| For More Information | ||||||||||||||||||||||||||||||||||||||||
| http://www.liebert.com/assets/products/english/products/env/datacool/ 60hz/bro_8pg/acrobat/sl_16700.pdf |
||||||||||||||||||||||||||||||||||||||||
|
HP has entered into an agreement with Liebert to reference sell the DataCool solution
|
||||||||||||||||||||||||||||||||||||||||
| Environmental | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
| Uninterruptible Power Supplies (UPS) | ||||||||||||||||||||||||||||||||||||||||
HP will be reselling-high end
(10 kW and above) three-phase UPS systems from our partners. We will test
and qualify a three-phase UPS for Superdome.
|
||||||||||||||||||||||||||||||||||||||||
| APC Uninterruptible Power Supplies for Superdome | ||||||||||||||||||||||||||||||||||||||||
| The Superdome team has qualified
the APC Silcon 3-phase 20 kW UPS for Superdome.
There are several configurations that can be utilized depending on the Superdome configuration your customer is deploying. They range from a 64 processor Superdome with dual cord and dual UPS with main tie main to a 32 processor Superdome with single cord and single UPS. In all configurations the APC Silcon SL20KFB2 has been tested and qualified by the Superdome engineers to ensure interoperability. |
||||||||||||||||||||||||||||||||||||||||
| HP UPS Solutions | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
| NOTE: The APC Silcon 3-phase UPS solutions for Superdome must be ordered directly from APC. Please contact Ron Seredian at rseredia@apcc.com. | ||||||||||||||||||||||||||||||||||||||||
| Superdome Server Watt Ratings for UPS loading | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
| Power Protection |
Runtimes Power Conditioning Continuous Power during Short Interruptions of Input Power UPS Configuration Guidelines Superdome Integration/Installation |
|||||||||
| Power Connections with the APC SL20KFB2 | ||||||||||
|
||||||||||
| Communications Connections A DB-25 RS-232 Contact Closure connection is standard on all APC SL20KFB2 UPS. A WEB/SNMP card is also included. |
||||||||||
| Power Management | ||||||||||
|
||||||||||
| Type of UPSs | |
| Some customers may experience chronic "brown-out" situations or have power sources that are consistently at the lower spectrum of the standard voltage range. For example, the AC power may come in consistently at 92 VAC in a 110 VAC area. Heavy-load electrical equipment or power rationing are some of the reasons these situations arise. The APC SL20KFB2 units are designed to kick in before the AC power drops below the operating range of the HP Superdome Enterprise Server. Therefore, these UPS units may run on battery frequently if the AC power source consistently dips below the threshold voltage. This may result in frequent system shutdowns and will eventually wear out the battery. Although the on-line units can compensate for the AC power shortfall, the battery life may be shortened. The best solution is to use a good quality boost transformer to "correct" the power source before it enters the UPS unit. | |
| Ordering Guidelines | |
|
|
| Power Redundancy | |
| Superdome servers, by default, provide an additional power supply for N+1 protection. As a result, Superdome servers will continue to operate in the event of a single power supply failure. The failed power supply can be replaced without taking the system down. | |
|
|
|
| Multi-cabinet Configurations |
When configuring Superdome systems that consist of more then one cabinet and include I/O expansion cabinets, certain guidelines must be followed, specifically the I/O interface cabling between the Superdome cabinet and the I/O expansion cabinet can only cross one additional cabinet due to cable length restrictions. |
|
|
|
| Configuration Guidelines/Rules | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Superdome Configuration Guidelines/Rules | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| *
Superdome 32 processor system requires an I/O expansion cabinet for greater
than 4 nodes. Superdome 64 processor system requires an I/O expansion cabinet
for greater than 8 nodes.
NOTE: |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Instant Capacity (iCAP)-formerly
known as Instant Capacity on Demand (iCOD) and Pay Per Use Programs. For
a complete description of how to configure Instant Capacity, please refer
to the following URL: http://h18000.www1.hp.com/products/quickspecs/11723_div/11723_div.html |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
OpenVMS |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
Upgrades
| For information on Superdome System Upgrades, please refer to the Superdome Server Upgrades QuickSpec. |
|
|
Memory
| sx2000 Superdome Memory Configurations | |||||||||||||||||||
| Memory per Cell (GBytes) |
Number of DIMMs
|
Echelon
|
|||||||||||||||||
|
1 GB
|
2 GB
|
4 GB
|
0
|
1
|
2
|
3
|
4
|
5
|
6
|
7
|
8
|
9
|
A
|
B
|
C
|
D
|
E
|
F
|
|
|
8 |
8
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
|
|
|
|
|
|
|
|
|
|
|
|||
|
16
|
8
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
|
|
|
|
|
|
|
|
|
|
|
|||
|
16
|
16 |
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
|
|
|
|
|
|
|
|||
|
24
|
8 |
8 |
2 GB
|
2 GB
|
2 GB
|
2 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
|
|
|
|
|
|
|
||
|
24
|
24 |
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
||||||
|
32
|
|
8
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
|||||||||||||
|
32
|
16 |
8 |
2 GB
|
2 GB
|
2 GB
|
2 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
|
|
|
||
|
32
|
|
16
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
|||||||||
|
32
|
32
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
||
|
40
|
8 |
8
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
|||||||||
|
40
|
8 |
16
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
|||||
|
40
|
24
|
8
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
|
|
48
|
8
|
8
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
|
|
|
||
|
48
|
16
|
8
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
|||||
|
48
|
24
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
||||||
|
48
|
16
|
16
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
|
|
56
|
8
|
8
|
8
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
||||
|
56
|
8
|
24
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
|
|
56
|
24
|
8
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
|
|
64
|
16
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
||||||||||
|
64
|
16
|
8
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
|||||
|
64
|
16
|
8
|
8
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
|
64
|
32
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
||
|
72
|
8
|
16
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
|||||
|
72
|
8
|
16
|
8
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
|
80
|
8
|
16
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
|||||
|
80
|
24
|
8
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
|
|
80
|
16
|
16
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
|
|
88
|
8
|
8
|
16
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
|
96
|
24
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
||||||
|
96
|
16
|
16
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
|
|
104
|
8
|
24
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
1 GB
|
1 GB
|
1 GB
|
1 GB
|
|
|
112
|
8
|
24
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
2 GB
|
2 GB
|
2 GB
|
2 GB
|
|
|
128
|
32
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
4 GB
|
||
| *NOTE: These configurations will result in the best performance | |||||||||||||||||||
|
|
Technical Specifications
|
Superdome sx2000 Specifications |
SPU Description |
Superdome 16 processor* |
Superdome 32 processor* |
Superdome 64 processor* |
| SPU Product Number |
A9833A |
A9834A |
A9834A+A9835A | |
|
Number of Dual core Intel Itanium processors/cores NOTE: Single core active requires purchase of iCAP processor |
1/1-16/32 |
1/1-32/64 |
2/2-64/128 Microsoft Windows Server 2008 for Itanium-based Systems can support a maximum of 32 dual core Itanium processors with 64 cores per partition if hyper threading is turned off; or a maximum of 16 dual core Itanium processors with 32 cores and 64 threads per partition if hyper threading is turned on. RHEL 5, RHEL 4, and SLES 10 can support a maximum of 64 dual core Itanium processors with 128 cores per partition if hyper threading is turned off or a maximum of 64 dual core Itanium processors with 128 cores and 256 threads per partition if hyper threading is turned on. | |
| Number
of Itanium 2 1.6 GHz processors | 2-16 | 2-32 | 4-64 | |
| Supported Processors | ||||
| Dual core Intel Itanium processor series 9000 and 9100 |
1.6 GHz, 24 MB & 18MB cache |
1.6 GHz, 24 MB & 18MB cache |
1.6 GHz, 24 MB & 18MB cache | |
| Intel Itanium processor | 1.6
GHz, 9 MB cache | 1.6
GHz, 9 MB cache | 1.6
GHz, 9 MB cache | |
| Memory | ||||
| 2-GB DIMMs (DDRII) |
16-256 GB |
16-512 GB |
32-1024 GB | |
| 4-GB DIMMs (DDRII) | 32-512
GB | 32-1024
GB | 64
2048 GB (Windows Server 2008 support up to 2048 GB; 2048 GB for Linux) | |
| Cell Boards | ||||
| 1-processor, 2 processor, 3-processor, 4 processor cells |
1-4 |
1-8 |
2-16 | |
| Software Support | ||||
| HP-UX revision |
HP-UX 11i version 2 and 3 | |||
| Windows revision | Microsoft
Windows Server 2008 for Itanium based Systems. Hyper threading supported with
dual core Itanium processors (default from factory hyper threading off). | |||
| Linux
| RHEL 4, RHEL 5 and SLES 10 support for dual core (Montecito and Montvale) processors only on sx2000 configurations; hyper threading supported on sx2000 (default from factory is hyper threading off). Mad9M is not supported by Linux in the sx2000 platform. For minimum release information see the Linux Integrity Certification and Support: Matrix at www.hp.com/go/lxintegritycert | |||
| OpenVMS | OpenVMS V8.3 minimum; dual core (Montecito) processors only on sx2000 configurations; hyperthreading support on sx2000 | |||
| Expandability / Connectivity | ||||
| 12-slot
PCI-X I/O chassis SPU cabinet must be filled first before placing I/O chassis in I/O expansion cabinet | 1-4 No I/O expansion cabinet allowed |
1-8 I/O expansion cabinet required if number of I/O chassis is more greater than 4. |
1-16 I/O expansion cabinet required if number of I/O chassis is greater than 8. A second I/O expansion cabinet is required if the number of I/O chassis is greater than 14. NOTE: Both Linux releases support up to 8 I/O chassis. | |
| 12-slot
PCI X I/O chassis SPU cabinet must be filled first before placing I/O chassis in I/O expansion cabinet SKUs: A9836A and A9836AZ |
1-4 No I/O expansion cabinet allowed |
1-8 I/O expansion cabinet required if number of I/O chassis is more greater than 4. |
1-16 I/O expansion cabinet required if number of I/O chassis is greater than 8. A second I/O expansion cabinet is required if the number of I/O chassis is greater than 14. Note: Both Linux releases support up to 8 I/O chassis. | |
| Maximum
number of partitions without I/O expansion cabinet |
1 - 4 |
1 - 4 |
1 - 8 | |
| Maximum
number of partitions with I/O expansion cabinet |
Not Applicable |
1 - 8 |
1- 16 | |
| RS-232C
serial ports | Yes |
Yes |
Yes | |
| 10/100Base-T Ethernet | Yes Windows does not support 10/100Base-T Ethernet. Windows requires Gigabit Ethernet card | Yes Windows does not support 10/100Base-T Ethernet. Windows requires Gigabit Ethernet card | Yes Windows does not support 10/100Base-T Ethernet. Windows requires Gigabit Ethernet card | |
| | ||||
|
sx2000 Superdome Specifications |
SPU Description |
Superdome 16 processor*
|
Superdome 32 processor*
|
Superdome 64 processor*
|
| SPU Product Number |
A9833A (sx2000)
|
A9834A (sx2000)
|
A9834A+A9835A (sx2000)
|
|
| Maximum I/O Cards |
||||
| Mass storage |
16-48
|
16-96
|
16-192
|
|
| LAN |
2-32
|
2-48
|
2-96
|
|
| WAN |
8
|
16
|
32
|
|
| Multi function (Mass Storage/LAN) |
8-48
|
16-96
|
32-192
|
|
| Additional interface cards |
8
|
8-14
|
8-14
|
|
| Typical Electrical Characteristics by Configuration | ||||
| AC input power |
200-240 VAC, 50/60 Hz
|
200-240 VAC, 50/60 Hz
|
200-240 VAC, 50/60 Hz
|
|
| Typical input power dissipation Itanium 2 (watts) |
5,900
4 cells, 32 GB, 4 I/O chassis with 6 PCI each |
9,800
8 cells, 32 GB, 4 I/O chassis with 6 PCI each |
19,600
16 cells, 32 GB, 4 I/O chassis with 6 PCI each |
|
| Typical power dissipation mx2
(watts) |
5,730
|
9,490
|
18,980
|
|
| Typical 3 phase 4 wire current
requirements at 200V 240V |
17A/Phase
|
29A/Phase
|
Two cabinets 29A/Phase
|
|
| Typical 3 phase 5 wire current requirements at 200V 240V |
10A/Phase
|
17A/Phase
|
Two cabinets 17A/Phase
|
|
| Maximum Electrical Characteristics | ||||
| Maximum input power (VA) cabinet marked electrical |
8,200 | 12,196 | 24,392 | |
| AC input power-Option 7: 3 phase
5 wire input |
200 240 VAC phase to neutral, 5 wire, 50/60 Hz | |||
| AC input power-Option 6: 3 phase 4 wire input |
200 240 VAC phase to phase, 4 wire, 50/60 Hz | |||
| Maximum current requirements at
220V 240V: |
||||
|
24 A/Phase | 24 A/Phase | 24 A/Phase | |
|
44 A/Phase | 44 A/Phase | 44 A/Phase | |
| Required power receptacle | Option 6: IEC Compliant
4 wire 60A 250V Ph Ph Option 5: IEC Compliant 5 wire 32A 480V Ph Ph |
|||
| Heat Characteristics | ||||
| Typical heat dissipation (BTUs/hour) | 20,131 | 33,439 | 66,877 | |
| Maximum heat dissipation (BTUs/hour) | 28,969 |
41,614 |
83,288 | |
| Site Preparation | ||||
| Site planning and installation included | Yes | Yes | Yes | |
| Depth (mm/inches) | 1,220/48.1 |
1,220/48.1 |
1,220/48.1 | |
| Width (mm/inches) | 762/30 |
762/30 |
1,524/60 | |
| Height (mm/inches) | 1,960/77.2 |
1,960/77.2 |
1,960/77.2 | |
| Weight (Kg/lbs) | 500/1,123 | 598/1,343 | 1,196/2,685 | |
| Environmental Characteristics | ||||
| Acoustics | 65 dB | |||
| Operating temperature |
+20°C to +30°C | |||
| Non operating temperature |
-40°C to +70°C | |||
| Maximum rate of temperature change |
20°C/hour | |||
| Operating relative humidity |
15% to 80% @ 30°C | |||
| Operating altitude | 0 3.1 km (0 10,000 ft) | |||
| Non operating altitude | 0 4.6 km (0 15,000 ft) | |||
| Regulatory Compliance | ||||
| Safety | IEC 950:1991+A1, +A2, +A3, +A4; EN60950:1992+A1, +A2, +A3, +A4, +A11; UL 1950, 3rd edition; cUL CSA C22.2 No. 950 95 | |||
| More detailed regulatory documents and certifications | http://docs.hp.com/en/hw.html | |||
| Key Dates | ||||
| First CPL date |
6/03 (sx1000); 3/06 (sx2000) | |||
| First ship date | 3Q03 (sx1000); 2Q06 (sx2000) | |||
| * Given that Itanium 2 1.5 GHz is a single core processor and mx2 is a dual core processor, the columns listed in this table refer to 16 processor, 32 processor and 64 processor. This terminology refers to 16 processor cores, 32 processor cores and 64 processor cores for Superdome Itanium 2 1.5 GHz systems and 32 processor cores, 64 processor cores and 128 processor cores for Superdome mx2 systems. | ||||
|
|
||||
|
Superdome I/O Expansion (IOX) Cabinet Specifications |
SPU Description |
Superdome 16 processor |
Superdome 32 processor
|
Superdome 64 processor
|
| SPU Product Number |
A9833A (sx2000)
|
A9834A (sx2000)
|
A9834A+A9835A (sx2000)
|
|
| Maximum Number of I/O Chassis Enclosures (ICEs)* |
Not Applicable
|
2
|
4
|
|
| Peripherals Supported |
Not Applicable
|
All peripherals qualified for use with Superdome and/or for use in a 10K G2 are supported in the I/O expansion cabinet as long as there is available space. Peripherals not connected to or associated with the Superdome system to which the I/O expansion cabinet is attached may be installed in the I/O expansion cabinet. | ||
| Servers Supported |
No servers except those required for Superdome system
or High Availability Observatory or ISEE may be installed in an I/O expansion
cabinet.
|
|||
| Dimensions | ||||
| Height (mm / inches) |
Not Applicable
|
1.6 meters or 2.0 meters | ||
| Depth (mm / inches) | 45.5 in (115.57 cm) (same depth as 32W) | |||
| Width (mm / inches) | 24.0 in (61 cm) | |||
| Environmental | ||||
|
Not Applicable
|
Same as Superdome
|
|||
| Relevant Product Numbers | ||||
| 12 slot PCI X chassis for HP rack
expansion cabinet |
Not Applicable
|
A9836AZ (sx2000)
|
A9836AZ (sx2000)
|
|
| I/O Expansion Cabinet Power and
Utilities Subsystem |
A5861A
|
A5861A
|
||
| I/O Expansion Power and Utilities
Subsystem Graphite color |
A5861D
|
A5861D
|
||
| I/O chassis enclosure for 12 slot PCI X chassis |
A9852A (sx2000)
|
A9852A (sx2000)
|
||
| NOTE: Each ICE holds two I/O card cages or 24 PCI X I/O slots. | ||||
|
|
||||
|
APC SL20KFB2 Specifications |
Description | APC Silcon 20000VA/20000W Input: 115/200 3PH, 120/208 3PH, 127/220 3PHV; Output: 115/200 3PH, 120/208 3PH, 127/220 3PHV; Interface port: DB 25 RS 232; Contact closure |
|
|
General features |
0 95% non condensing, 200% overload capability, audible alarms, built in static bypass switch, Delta Conversion online technology, environmental protection, event logging, extendable run time, full rated output available in kW, input power factor correction, intelligent battery management, LCD alphanumeric display, overload indicator, paralleling capability, sine wave output, SmartSlot, software, web management | ||
|
Includes |
Parallel card, triple chassis for three SmartSlots, user manual, Web/SNMP Management Card | ||
|
Spare parts kits |
See APC website http://www.apcc.com | ||
|
Documentation |
User Manual and Installation Guide | ||
|
Input |
Nominal input voltage |
115/200 3PH, 120/208 3PH, 127/220 3PH V | |
|
|
Input frequency |
50 Hz programmable +/- 0.5, 1, 2, 4, 6, 8%; 60 Hz programmable +/- 0.5, 1, 2, 4, 6, 8% | |
|
|
Input connection type |
Hardwire 5 wire (3PH+N+G) | |
|
|
Input voltage range for main operations |
170-230 (200 V), 177-239 (208 V), 187-242 (220 V) V | |
|
Batteries |
Typical backup time at half load |
36.7 minutes | |
| Typical backup time at full load | 10.7 minutes | ||
| Battery type | Maintenance free sealed lead acid battery with suspended electrolyte: leak proof | ||
| Typical recharge time1 | 2 hours | ||
| Physical | Maximum height dimensions | 55.12 in (140.00 cm) | |
| Maximum width dimensions | 39.37 in (100.00 cm) | ||
| Maximum depth dimensions | 31.50 in (80.01 cm) | ||
| Net weight | 1,290.00 lbs (586.36 kg) | ||
| Shipping weight |
1,340.00 lbs (609.09 kg) | ||
| Shipping height |
66.93 in (170.00 cm) | ||
| Shipping width |
43.31 in (110.00 cm) | ||
| Shipping depth | 35.43 in(90.00 cm) | ||
| Color | Dark green (NCS 7020 B50G), Light gray (NCS 2703 G84Y) | ||
| Units per pallet |
1.0 | ||
| Communications and Management | |||
| Communications and Management | Interface port | DB-25 RS-232, Contact Closure | |
| SmartSlot interface quantity |
2 | ||
| Pre installed SmartSlot cards | AP9606 | ||
| Control panel | Multi-function LCD status and control console | ||
| Audible alarm | Beep for each 52 alarm conditions | ||
| Emergency power off (EPO) |
Yes | ||
| Optional management device | See APC website http://www.apcc.com | ||
| Environmental | Operating environment |
32° to 104°F (0° to 40 °C) | |
| Operating relative humidity |
0% to 95% | ||
| Operating elevation |
0 to 3333 ft (0 to 999.9 m) | ||
| Storage temperature |
-58° to 104°F (-50° to 40 °C) | ||
| Storage relative humidity |
0% to 95% | ||
| Storage elevation | 0 to 50,000 ft (0 to 15,000 m) | ||
| Audible noise at 1 meter from surface of unit | 55 dBA | ||
| Protection class |
4,094 BTU/hour NEMA 1, NEMA 12 optional |
||
| Online thermal dissipation | 4,094 BTU/hour | ||
| Protection Class | NEMA 1, NEMA 12 | ||
| Conformance | Approvals | EN 55022 Class A, ISO 9001, ISO 14001, UL 1778, UL Listed, cUL Listed | |
| Standard warranty | One-year repair or replace, optional on-site warranties available, optional extended warranties available | ||
| Optional new service | See APC website http://www.apcc.com | ||
| 1The time to recharge to 90% of full battery capacity following a discharge to shutdown using a load rated for 1/2 the full load rating of the UPS | |||
|
|
|||
| sx2000 Superdome supported I/O | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
NOTE:
While ordering Windows Server 2008, do not mix differing brands of HBAs (Emulex
and Qlogic) within the same server configuration in an MPIO environment.This is
to avoid issues which occur when different Fibre Channel HBAs are used within
the same server which support different I/O max transfer packet sizes in an MPIO
environment. For example, the Emulex Fibre Channel HBAs support a max I/O transfer
packet size of 1MB and the Qlogic Fibre Channel HBAs support a max I/O transfer
packet size of 2MB. Mixing these two cards in an MPIO configuration can cause
the system to hit a BSOD 0X000000D1 with a reference to elxstor.sys Qlogic: | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1
Maximum cards are two per I/O chassis. Therefore: 16 processor=8 maximum, 32 processor=16
maximum, and 64 processor=32 maximum. 2 Operating Systems Legend H=HP UX 11i V2, W= Microsoft Windows Server 2008 for Itanium-based Systems, L= Linux (Red Hat RHELAS 4 and SLES 10), O= OpenVMS | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
The boot configuration for Windows Server 2008 Superdome partitions can be:
Windows
Server 2008 operating system comes with a software mirroring solution. However,
the majority of Windows customers use hardware based RAID solutions (such as the
industry leading Smart Array disk array controllers from HP) instead and do not
use this mirroring tool. Also note that the Smart Array controllers do not support
failover capability (customers cannot have 2 Smart Array cards connected to the
same boot partition on a StorageWorks 4300/4400 enclosure). RAID levels 0, 1,
5, 1+0, and ADG are supported as well as disk sparing. Note that booting from external storage arrays is supported (HP XP and EVA storage). In these cases, it is recommended by HP that the FC HBAs are configured in a redundant pair using HP Secure Path software for high availability. |
| Please refer to the table below as guidance for configuring your Windows Server 2003 or Windows Server 2008 partition on Superdome (note that "Watson" rules are in place that reflect these recommendations). Please note that if the VGA/USB card (A6869A/A6869B) is used, it would only be needed once per instance to the Windows OS instance. |
| PCI-X Technical Slotting Information for Windows Server 2008 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| NOTE: FC HBA (AB466A or AB467A) are to consume 8X slots first and then populate 4X slots (recommended for performance optimization) | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
To ensure Windows Server 2008 high availability for storage connectivity, it
is recommended to use HP SecurePath (with HP storage) and EMC PowerPath (with
EMC storage) for load balancing/redundancy between fiber channel HBAs (AB466A
or AB467A). External Storage HP has the broadest, most robust server and storage line up in the industry, providing exactly the right fit for every need. Refer to the Storage-Server matrix found in the introductory section of Chapter 4 of this configuration guide to see a matrix that highlights which storage device, server and operating system is interoperable. For
HP Storage connectivity, the webpage http://spock.corp.hp.com/index.aspx
has all of the detail for HP hardware. Please consult this matrix for HP supported
on line and near line storage. For EMC connectivity with Windows Server 2008 on HP Integrity servers, the EMC support matrix has detailed information concerning supported HP hardware: http://www.emc.com/interoperability/index.jsp. Please consult this matrix to determine if your customer's desired configuration is supported by EMC. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Superdome Supported Online Storage | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 DA-11717 57 - Version 57 - October 23, 2009 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||