APC 15000 RAID manual

SGI ® InfiniteS torage 15000 RAID User ’ s Guide 007-5510- 002[...]

COPYRIGHT © 2008 SGI. All righ ts reserved; provided porti ons may be copyri ght in third parties, as i ndicated elsewhere herein . No permission is granted to copy , distribute, or creat e derivative works from the c ontents of this electronic documentation in any manner, in whole or in part, without the prior written permission of SGI. LIMITED R[...]

Contents 007-5510-002 i 1 Introduction .............. ................. ................. .................... ................ ................. ............. ............. ...... 1 1.1 Controller Features ............. ... ................. ... ... ................... ................. ... ... ................. .. ... .. .... ......... 1 1.2 The [...]

ii 007-5510-002 3.4 Security Administration ........................ ........ ........ ................. ................. .................... ........... .... 72 3.4.1 Monit oring User Logins ................. .................... .................... ................... .................... .... 73 3.4.2 Zoning (Anonymous Access) ........ ......[...]

Contents 007-5510-002 iii 5.6.3 AC INPUT PCM ..................... .. ... ................. ... ... ................ ... ... ................. ... ................. .. 106 5.6.4 DC INPUT PCM ...................... ... ................. ... ................. .. ... ................. ... ... ................. .. 1 06 5.6.5 DC OUTPUT PCM .............[...]

iv 007-5510-002 8.9 Replacing the DEM .................. ................. .................... ................... .................... ............. ...... 127 Appendix A. Control ler Technical Specifications . . . . . . . . . . . . . . . . . . . . . . . 129 Appendix B. Dri ve Addressing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .[...]

Preface 007-5510-002 v Preface What is in this guide This user guide gives you step-b y-step instructions on how to install, configure, and con nect the SGI InfiniteStorage 15 000 system to your host computer system, as well as to use and maint ain the system. Who should use t his guide This user guide assumes that you have a wo rking know ledge of[...]

Preface vi 007-5510-002 European Regulations This equipment complies with Europ ean Regulations EN 55022 Class A: Limits and Methods of Measurement of Radio Disturban ce Characteristics of Informat ion Technology Equi pment and EN50082-1: Generic Immunit y. Qualified Personnel Qualified personnel are defined as follow s: • Service Person: A perso[...]

Preface 007-5510-002 vii Chassis Warning Label: Weight Hazard • Do not lift the drive enclosure by the handles on the power coolin g module (PCM); they are not designed to support the weight of the popul ated enclo sure. Safety Important SGI InfiniteSto rage 15000 drive enclosures mu st b e always installed in SGI InfiniteStorage 15000 racks. SGI[...]

Preface viii 007-5510-002 Power Cooling Module (PCM) Caution Label: Do not operate with modules miss ing Warning To ensure you r syste m has warning of a power failure please d isconnect the power fr om th e power supply, by either the switch (where pres ent) or by physically r emoving the power source , prior to removing the PCM from the enclosure[...]

Preface 007-5510-002 ix Warning Oper ation of the Enclosure w ith ANY modules missing will disru pt the airflow and the drives will not receive sufficient cooling. It is ESSENTIAL that all apertures are filled before operating the unit. • Drive Carrier Module Caution Label: Drive spin down time 30 seconds Recycling of W aste Elec trical and Elect[...]

Preface x 007-5510-002 • The electrical distribution system must provide a reliable earth ground for each unit and the rack. • Each power supply in each unit has an earth leakage current of 1.5mA. The design of the electrical distribution system must take into considerati on the total earth leakage current from all the power supplies in all the[...]

Preface 007-5510-002 xi • If the subsystem is used with modules or bl anking plates missing for more th an a few minutes, the enclosure can overheat, causing power failure and da ta loss. Such use may also invalidate the warranty. • If you remove any drive module, you may lose data. – If you remove a drive module, re place it immediately. If [...]

[...]

Introduction 007-5510-002 1 Chapter 1 Introduction The SGI InfiniteStorage 15000 controller is an inte lligent storage infrastructure device desi gned and optimized for the hi gh band width and capacity requirem ents of IT departme nts, rich media, an d high performance workgroup app lications. The controller plugs seamlessly into existing SAN envi[...]

2 007-5510-002 • Comprehensive, Centralized Managem ent Capability The controller provides a wide range of management capabilities: Con figuration Management, Performance Management, Logical Unit Nu mber (LUN) Management, Security Administration, and Firmware Update Management. • Management Options via RS-232 and Ethernet (T elnet) A RS-232 por[...]

Introduction 007-5510-002 3 Figure 1–1 SGI InfiniteStorage 15000 IB - Front and Rear Views The controller is a high-performance controller designed to be rack -mounted in standard 19" racks. Each controller is 3.5" in height, requiring 2U of rack space. The syst em uses 10 independent SAS drive channels to manage data distribution and s[...]

4 007-5510-002 1.2.1 Power Supply and Fan Modules Each controller is equipped with two (2) pow er su pply modules and one (1) fan module. The PSU (power supply unit) voltage operati ng ran ges are nominally 1 10V to 230V AC, and are autoranging. The two Power Su pply modules provide red undant power . If one module fails, the other w ill maintain t[...]

Introduction 007-5510-002 5 1.2.2 I/O Connectors a nd St atus LED Indicators Figure 1–3 shows the p orts at the back of th e controller 4 Infiniband (IB) u nit. Figure 1–3 I/O Ports on the Rear Panel of the Controller P S PS G H GH A B AB E F EF C D CD DISK CHANNELS AC FAIL SYSTEM STA TUS CTRL STA TUS TEMP STA TUS DISK STA TUS DC STA TUS FAN ST[...]

6 007-5510-002 Figure 1–4 Host Port LEDs The four HOST ports are used for IB or FC-8 ho st connec tions. Y ou can connect your host servers IB HCA port(s) or FC HCA port(s)directly to t hese ports . Additionally , you can connect these ports t o your IB or FC switches and hubs. The IB LEDs (the Infiniband LEDs) located betw een the host ports, wh[...]

Introduction 007-5510-002 7 There are two AC Fail LEDs. Each LED is connected to its power supply independen t of the other supply . The LEDs are green to indicate that the AC input to the supply is present. The LEDs turn re d if the AC input to the suppl y is not present. If this occurs, check the LEDs on the front side of the unit. If you lose AC[...]

8 007-5510-002 Table 1–1 LED Indicators Status In dic at or Led Activity Explanation IB Solid Green (Infiniband) Phys ical Connectivity with host Solid Amber Subnet Manage r communicating with host DISK ports Flashing Green Activity . Ther e is an LED for each of the ten ports/cha nnels (ABC DEFGPS) Unlit T elnet ACT Flashing Green Activi t y Unl[...]

Introduction 007-5510-002 9 1.2.3 Uninterruptible Power Supply (UPS) Using an Uninterrupt ible Power Supply (UPS) with the controller is highly recommended. The UPS can guarantee power to the system in th e event of a power failure for a short time, wh ich will allow the system to power down properly . SGI offers two types of UPS: basic and redunda[...]

[...]

Controller Installation 007-5510-002 11 Chapter 2 Controller Inst allation These steps provide an overview of the controller installation process. The steps are explained in detail in the following sections of this chapter . 1. Unpack the controller system. 2. If it is necessary to install the controller in th e 19-inch cabinet(s), contact your ser[...]

Controller Installation 12 007-5510-002 2.2 Unp acking the System Before you unpack y our controller , inspect the shipping cont ainer(s) for damage. If you detect damage, report it to your carrier im mediately . Retain all boxes and packing materials in case you need to store or ship the system in the future. While removing the components from the[...]

Controller Installation 007-5510-002 13 2.2.3 Connecting the Controller T o set up the disk enclosures and connect them to the controller, do the following. 1. There are 10 disk channels on the controller . They correspond with di sk ports. The disk ports are labeled as follows (Figure 2–1 ) : DISK A = Channel A DISK B = Channel B DISK C = Channe[...]

Controller Installation 14 007-5510-002 Configuration of disks in the enclosures must be in sets of complete tiers (Channels A through P). Allocating one spare drive per ti er gives you the best data p rotection but this is no t required. The spare drives on the controller are global hot spares. 2.2.6 Connecting the RS-232 T erminal For first time [...]

Controller Installation 007-5510-002 15 2.2.7 Powering On the Controller NOTE : Systems that have dual controllers (couplets) should have the controllers powered on simultaneously ensure corr ect system co nfiguration. 1. V erify that the power sw itches on the two (2) power supply modu le at the back of each controller are off. 2. Connect the two [...]

Controller Installation 16 007-5510-002 2.3 Configuring the Controller This section provides information on con figuring your controller . NOTE : The configuration examples pr ovided here represent only a general guideline. These examples should not be used directly to configure your particular control ler . The CLI (command line interface) comma n[...]

Controller Installation 007-5510-002 17 The controller supports various disk drive enclosures that can be used to populate the 10 <ABCDEFGHPS> disk channels in both SAS 1x and SAS 2x modes. Each chassis ha s a lim it to the tiers that can be created and suppo rted. Refer to the specific disk en closure user guides for further inform ation. Y [...]

Controller Installation 18 007-5510-002 For example, to change the system date to March 1, 2009, enter: date 3 1 2009 <Enter> T o set the system time, at the promp t, type: time hh:mm:ss <Enter> where hh represents the two digit value for hour (0 0 to 24), mm is the two digit value for m inutes, and ss represents the two digit value for[...]

Controller Installation 007-5510-002 19 2.3.6 Checking Tier S t atus and Configuration Use the tier command to display your current tier status. Figure 2– 5 illustr ates the status of a system containing 80 drives on 8 tiers with both parity modes of 8+1 and 8+ 2 tiers. The plus sign (+) adjacent to the tier number indicates that the tier is in 8[...]

Controller Installation 20 007-5510-002 Figure 2–6 Current T ier Configuration 2.3.6.1 Heading Definitions • T otal LUNs. LUNs that currently reside on the tier . • Healthy Disk. The “health” of the spare disk currently being used (if any is being used) to replace a disk on the listed tier . The health indication for the s pare channel th[...]

Controller Installation 007-5510-002 21 Figure 2–7 Du al Con troller Configuration If you require multi- pathing to the LUNs, enable cache coherency . If you do not requi re multi-pathing, disable cache coherency . T o enable/disable the cache coherenc y function, enter the following ( ON enables, OFF disables): dual coherency=on|off <Enter>[...]

Controller Installation 22 007-5510-002 NOTE : In dual mode, LUNs are “owned” by the contro ller unit where they are created. Hosts only see the LUNs on the unit to which that they are connected, unless cache coherency is enabled. 1. T o display the current cache settings, type: cache <Enter> 2. Select a cache segment size for your array [...]

Controller Installation 007-5510-002 23 After you have initiated LUN format, th e message Starting Format of LUN is displayed. Y ou can monitor the format progr ess by entering the command LUN (see Figure 2–9 ) . Upon completi on, this message : Finished Format of LUN 0 displays. Figure 2–9 Lo gical Unit S tatus - Formatting 7. Enter the comman[...]

Controller Installation 24 007-5510-002 2.3.9 Setting Security Levels After you have formatted all the LU Ns, you can define users’ access right s. Configurations come in two types: • authorized user • host port zoning The Authorized User configuration is highly recommended for use in a SAN environment-- your data is completely secured and no[...]

Controller Installation 007-5510-002 25 2. Check to ensure that the LUN Zo ning chart is empty (Figure 2–11 ) . Figure 2–1 1 LUN Zoning Screen T o add a user : 1. Ty p e : user audit=on <Enter> The controller reports which users are connected. 2. Ty p e : user add <Enter> . 3. Specify a new Host User ’ s world wide name , ent e r [...]

Controller Installation 26 007-5510-002 10. Enter a new unique LUN mapping for this user . Opt ions are shown in T able 2–2 on page 26. 11 . Con nect user 2 and repeat steps 2--10 to specify the host port zoning and LUN mappings with the following changes: - For active host port (step 6), enter port 2 only. - For LUN ma pping: External LUN 0 is m[...]

Controller Installation 007-5510-002 27 2.3.9.2 Host Port Zon ing (Anonymous Access) Host Port Zoning (Anony mous Access) should only be used for non-SAN environment. Users are given “general admission” to the data. One zoning configura tion is supported for each of the host ports. Any unauthorized user accessing the storage is considered “an[...]

[...]

Controller Management 007-5510-002 29 Chapter 3 Controller Management 3.1 Managing the Controller The controller provides a set of t ools that enabl e administrators to cen trally manage the net work storage and resources that handle business- critical data. These include Conf iguration Management, Performance Management, Remote Login Management, S[...]

30 007-5510-002 3. W i th the controller ready , press <Enter> to get the controller prompt. NOTE : T o change the baud rate on controller , see section 3.8.3 "Changing Baud Rate for the CLI Interface" in this guide. Remotely - T elnet T o configure and monitor the controller remot ely , conn ect the controller to your Ethernet netw[...]

Controller Management 007-5510-002 31 NOTE : The default administrato r accoun t name is “adm in” and it s password is “password.” Similarly , the defau lt user account name is “user” and its password is “password.” Figure 3–2 Lo gin Screen 3.1.3.2 Logout T o lo gou t of the system, enter: logout <Enter> For a terminal sessi[...]

32 007-5510-002 Figure 3–4 WHOAMI Screen 3.2 Configuration Management The controller provides uniform configuration management across hete rogeneous SANs. Status of host ports and storage assets are continuously being monitored. Table 3–1 Controller Limits 3.2.1 Configure and Monitor St atus of Host Port s The status information of the host por[...]

Controller Management 007-5510-002 33 The PORT=X|ALL parameter specifies the specific host port(s) (1 to 4) to be affected when used in combination with any o f the other parameters: ID , TIMEOUT , SPEED (for FC only), or WWN . The default is to apply changes to ALL host ports. Figure 3–5 FC Host Ports Configuration Screen Figure 3–6 IB H ost P[...]

34 007-5510-002 HOST ST A TUSCLEAR resets the error counts. Figure 3–7 IB Host Ports S tatus Screen 3.2.1.3 Host IB Users HOST ibusers displays additional informati on on the Infiniband (IB) users logged into the controller . (Figure 3–8). Figure 3–8 Host IB Users Screen 3.2.1.4 Host Port Speed HOST SPEED lets you display and change the port [...]

Controller Management 007-5510-002 35 Figure 3–9 Di sk Channel Screen If the channel status is “acquiring loop synchronizat ion,” this may indicate a channe l problem. Refer to 4.2.2, "Recovering from Drive Fa ilures" for recovery information. Entering DISK INFO=<tier><channel> retrieves information about a specific disk[...]

36 007-5510-002 Figure 3–11 Disk Defect List Screen DISK F AIL=<tier><channel> instructs the system to fail the speci fied disk at the physical tier in the range of <1 ...125> and channel in the range of <ABCDEFGHPS>. When a non-SP ARE disk is specified and it is failing, the disk will not cause a m ulti-channe l failure. [...]

Controller Management 007-5510-002 37 Table 3–2 PHY Link Error Stat us Block Information NOTE : SA T A drives have an Active/Active MUX (A AMUX) installed. Erro r counts are read directly from the AAMUX. SA T A AAMUX PHY ERRORS H-RX The number of SA T A FIS CRC errors received on the host port of the AAMUX H-TX The number of SA T A R_ERR primitiv[...]

38 007-5510-002 Figure 3–12 Disk PLS T ier 1 S tatus Screen For other DISK param eters, see section 3.2.10 "Aut om atic Drive Rebuild" in this guide. 3.2.2.1 Tier View T iers (also known as RAID groups) are the basic building blocks of the control ler . In an 8+1 mode, a tier contains 10 drives: eig ht (8) data drives (Channels A th rou[...]

Controller Management 007-5510-002 39 3.2.2.2 Tier Configuration TIER CONFIG displays the detailed tier configuration in formation fo r all of the tiers (Figure 3–13). Figure 3–1 3 T ier Configura tion Screen The headings for the Tier Configuration screen indicate the following valu es or conditions for the tiers. T otal LUNs lists the number o[...]

40 007-5510-002 Figure 3–14 Tier Configuration ALL Screen 3.2.2.3 LUN View Entering the LUN command disp lays the curr ent status of the LUNs ( ) . “Ready” indicates that the LUN is in good condition. The percentage of com pletion is displayed if the LUN is being formatted or rebuilt. A status of “U navailable” may r esult from multip le [...]

Controller Management 007-5510-002 41 Figure 3–1 6 LUN List Screen 3.2.2.4 LUN Configuration LUN CONFIG displays the confi guration information for all the valid LUNs in the system ( Figure 3–17). Figure 3–1 7 LUN Configurati on Screen 3.2.2.5 LUN Reservations LUN RESERV A TIONS displays a list of all valid LUNs in the syst em and shows which[...]

42 007-5510-002 3.2.2.7 Status of Drive Enclosures The SES command display s the failure s reported by the enclosur e (Figure 3–19), through the SCSI Enclosur e Services (SES). It also prov ides a means to access SES speci fic functions such as disk, channel, and LUN. Drive failures are not disp l ayed usin g the SES command; you must use the TIE[...]

Controller Management 007-5510-002 43 TIER MAP displays the current mappi ng m ode fo r the disks in the array . TIER CHANGEMAP changes the current tier mapping for the di sks in the array . T o change the current tier mapping, do the followi ng: 1. Enter: tier change map <Enter> 2. Select the appropriate mapping mode fo r you r drive enclosu[...]

44 007-5510-002 NETWORK TELNETPO RT=<port number> changes the T elnet port number for th e curren t controller . The system must be restarted before the ch anges will take effect. V alid port numbers are 0 to 32768 ; however , the results may be unpred ictab le if the port number chosen is alread y in use (on this unit) by either the GUI or S[...]

Controller Management 007-5510-002 45 3.2.4.4 Displaying and Edit ing the Routing Table The ROUTE command displays the curren t routin g table of the system (Figure 3–20 ) and allows the administrator to change it. The routing table describes how the c ontroller can comm unicate with the hosts on other ne tworks. Figure 3–2 0 Routing T able ROU[...]

46 007-5510-002 REST ART DUAL rest arts bot h unit s. REST ART KILL stops a timed restar t that is in progres s. 3.2.5.2 System Shutdown SHUTDOWN shuts down the controller unit. If you need to power down the controller, use SHUTDOWN prior to shutting off power . This will cause the controller to flush it s cache, abort all format and rebuild operat[...]

Controller Management 007-5510-002 47 3.2.6.2 System Time TIME displays the current system time. Y ou can also change the system time. A t the prompt, type: time hh:mm:ss <Enter> where hh represents the two digit value for hour (00 to 24), mm represents th e two digit value for minutes, and ss represents the two dig it value for seconds. For [...]

48 007-5510-002 The system will halt all I/O requests, delete all th e LUNs and restore all the parameters back to their default values. This is a destructive operation which will delete all the data stored in the system. The system will ask if you want to erase all the co nfiguration information st ored on the disks. This will prevent the system f[...]

Controller Management 007-5510-002 49 T o add a 32-bit LUN that will not ex ceed 2 TB, type: lun add=X <Enter> T o add a 64-bit LUN, that exceeds 2 TB: lun add64=X <Enter> For both cases, “X” is the L ogical Unit with a range of < 0..1023 >. For either case, the system prompts you for all the necessary information to create th[...]

50 007-5510-002 Figure 3–24 Changing a LUN Label Screen 3.2.9.6 Moving a LUN (Dual Mode Only) T o change the ownership of a LUN from one cont roller to its par tner (when the uni ts are in dual mode), enter: lun move=x <Enter> where x is the Logical Unit number <0..1 023> (Figure 3–25) . If a LUN is on a tier that is shared by other[...]

Controller Management 007-5510-002 51 Use the TIER command to display th e current setting (Figure 3–26 ). The rebuild operation can take up to several hours to complete, depending on the size of the disk and rate of rebuild. Figure 3–2 6 Automatic Disk Rebuilding Parameter TIER AUTOREBUILD=ON|OFF enables/disables the automatic disk rebuild fun[...]

52 007-5510-002 SMART ENABLE enables SMAR T on all the disk drives in stalled in the system and updates the parameter blocks on the disk. This en ables the In formation Exception and th e T emperature warnings. However , the user can skip th e update part and enter SMAR T DISKUPDA TE later to write the parameter blocks to the disks. SMART DISABLE d[...]

Controller Management 007-5510-002 53 SMART TEST=ON enables the test bit in the Information exception m ode page for all the disks installed. Setting the test bit simulates a F ALSE SMAR T trip condition which raises a F ALSE check condition to the controller . Currently , this parameter is valid only with Fibre Channel disks. SMART TEST=OFF disabl[...]

54 007-5510-002 For example, a user is given access to internal LUNs 5, 6, and 7, wh ich are mapped to external LUNs 0, 1, and 2, respectively . controller 1 owns LUNs 0 and 1 while controller 2 owns LUN 2 . The user is physically connected to co ntroller 1, thus, they will only see LUNs 0 and 1. The user will not be able to access LUN 2. If the us[...]

Controller Management 007-5510-002 55 Figure 3–2 8 Labeling a Controller Uni t 3.2.12.4 Singlet The DUAL SINGLET command sets the system in the singlet mode. System recognizes only Unit 1. This command: • disables cache coherency • heals unit 1 if it is failed • fails unit 2 b efore attempting to remove it. T o set the system in si ngl et m[...]

56 007-5510-002 Figure 3–29 Cache Setting Screen Y ou can use the LUN=x option to specify which LUN to change. If no LUN is sp ecified, changes will be applied to all the LUNs. V alid LUN v alues are 0 to 1023. The defau lt value will apply changes to all LUNs. 3.3.1.1 Cache Segment Size A large cache segment size may give better performance for [...]

Controller Management 007-5510-002 57 If the MF (Multiplication Factor) parameter is OFF , the system w ill only prefetch the number of blo cks specified by PREFETCH after every read command. If the MF parameter is ON , then the system will multiply the transfer length of t he command by the prefetch value to determine how much data will be prefetc[...]

58 007-5510-002 3.3.2 Audio/Visual Settings of the System The audio and visua l ( AV ) settings of the system and the disk s can be tun ed to provide better performance and a lower latency . The writeback and prefet ch settings for each LUN are changed with the CACHE command. The AV command displays informat ion about the audio/visual settings of t[...]

Controller Management 007-5510-002 59 RC=ON|OFF en ables the Read Continuous (RC) option for Audio/V ideo streaming dat a; the sy stem starts the data transfer for read operations after RCTIMEOUT is r eached, even if the disks commands have not finished. Use this to reduce the latency for read operations in Audio/ V isual environments where latency[...]

60 007-5510-002 For example, LUNs 0 to 3 are locked in cache and all 50 % of the total cache has been filled by data from LUN 0, 1, and 2. When a host issues a read command for da ta from LUN 3, the following will occu r: • The controller unit determines which data to re move from the locked po rtion of cache, using the LRU algorithm. The LRU alg[...]

Controller Management 007-5510-002 61 Figure 3–3 3 System Performance S tatistics Screen Read Hits shows the percentage of Read I/O requ ests where the data was already in the cache. Prefetch Hits shows the percentage of Read I/O requests wh ere the data was already in the cache due to prefetching. Pr efetches shows the percentage of host Read I/[...]

62 007-5510-002 ST A TS DELA Y displays a histogram of the time it takes for the host and disk I/O requests to complete in 100 msec intervals (Figure 3–34). Figure 3–34 Command Delay S tatistics Screen Command Delay Statistics Time 15000 [1]: stats delay Host Host Disk Disk seconds Reads Writes Reads Writes 1690087 1446110 281633 253704 0.1 829[...]

Controller Management 007-5510-002 63 ST A TS HOSTDELA Y displays a histogram of th e time delay between when the last data transfer is set ready and the host command compl etes (Figure 3–35) . The host ready delay information is sho wn in 100 msec intervals. Figure 3–3 5 Host Delay S tatistics Screen ST A TS TIERDELA Y=<tier> displays a [...]

64 007-5510-002 ST A TS DISK displays a histogram of the disks in the system that have taken an unusually long time to complete an I/O req uest (Figure 3–37). The coun t is in cremented for a disk if that disk takes longer than the other disks to finish an I/O request. This command is used to determine i f a disk in the array is slowing down syst[...]

Controller Management 007-5510-002 65 ST A TS DUAL displays the statistics for the dual mode messages (Figure 3–38). Figure 3–3 8 Dual Message S tatistics Screen ST A TS LENG TH displays a histog ram of the length of the host I/O requests in 16 kb intervals ( Figure 3–39 ) . Figure 3–3 9 Command Length S tatistics Screen[...]

66 007-5510-002 ST A TS OFFSET displays a histogram of the offset of th e host I/O requests into the cache segments ( Figure 3–40 ) . Host I/O requ ests with offsets that are not in the 0x0 column may require blocking/ deblocking which can slow down the performan ce of the system. Figure 3–40 Host Command Offsets Screen ST A TS REPEA T=OFF|MBS|[...]

Controller Management 007-5510-002 67 TIER DELA Y=x is used to set the system rebuil d/form at delay . This value is in 100 millisecond increments. The range is 0 to 1000. The default setting is 30 mi lli seconds (3 seconds). The REBUILD EXTENT parameter determines how much dat a to rebuild or format at one time. A small EXTENT value will slow d ow[...]

68 007-5510-002 TIER VERIFY Displ ays a summary of verifications . T o enable Tier V erify (Figure 3–42): 1. At the prompt, type TIER VERIFY=ON <Enter> . 2. The system will ask which tier you wish to verify . Enter the tier number or type a for “All.” 3. The system will ask if you want ru n the Tier V erify operation cont inu ously or n[...]

Controller Management 007-5510-002 69 Figure 3–4 3 T ier V erify OFF Screen TIER VERIFY=X A specified tier will be verified if possible. These API and CLI commands will af f ect the TIER verification process: • The CLI commands TIER P AUSE , TIER RESUME, TIER STOP . • The API command TIER ST A TU SCHANGE . LUN operations (add, delete, m ove) [...]

70 007-5510-002 communication, such as a cable fail ure, can cause the controller to fa il numerous disk drives. O nce the loss of communication is resolved, the time to rebuild all the failed drives can take many weeks. The rebuild journals contain bitmaps that indicate which portions of the di sks in a tier have been updated with new data while a[...]

Controller Management 007-5510-002 71 T o display the information about the rebuild j ournal, use the TIER JOURNAL command (Figure 3–44). T o di splay th e information fo r a specified tier , use the TIER JOURNAL=t command, where t is the specified tier . This screen will give detailed inform ation abou t the status of the journal and a display o[...]

72 007-5510-002 NOTE : Improper use of the SES M_W AIT command can prevent the SE S monitors from detecting an enclosure fault before th e enclosure automatically shuts down. 3.3.4.5 Host Command Timeout The Host Command Ti meout parameter allows the system to free up resources and make them available to other users if the request from a particul a[...]

Controller Management 007-5510-002 73 3.4.1 Monitoring User Logins The AUDIT function continuously mon itors logins to the controller and provides al erts in the event of unauthorized logi n attem pts (Figure 3–47 ) . Figure 3–4 7 User Login Messages USER AUDIT=ON|OFF enables/disables the user auditing function. When enabled, the system will di[...]

74 007-5510-002 Figure 3–50 Edit Zoning Configurati on Scre en 3.4.3 User Authentication The controller creates correspond ence between users (W orld W ide Na me or GUIDs), storage LUNs, and permissions. The system can store co nfigurations for up to 512 users in total, and the settings app ly to all host ports. Each authorized user will only ha [...]

Controller Management 007-5510-002 75 3.5 Firmware Up date Management SGI periodically releases firmware updates to enhance features of th eir products. Contact SGI technical support to obtain the latest firmware files. 3.5.1 Displaying Current Firmware V ersion The VERSION command display s version information o f the controller ’ s hardware and[...]

76 007-5510-002 Figure 3–53 Downloading Controlle r Firmware 6. Enter the TFTP server ’ s IP address: TFTP <IP_addr ess> 7. Enter the firmware path and filename: TFTP <filename> 8. For the couplet controller configuration, connect and log into the other controller . Repeat Steps 4- 7 above to update the firmware. 9. Enter REST ART t[...]

Controller Management 007-5510-002 77 Figure 3–5 4 T elnet S tatistics The administrator is strongly advised to perform any commands af fecting th e system’ s configuration from the CLI UAR T only (and not from a T elnet session), and to only perform such command s after issuing the TELNET DISABLE command, so that remote users canno t log into [...]

78 007-5510-002 W ithin the CLI subshell, the TELNET command allo ws the administrator to view i nformation regarding the currently active T elnet session (Fig ure 3– 56). Figure 3–56 T elnet Session Info rmation TELNET KILL=m lets the system ad ministrator te rminate the remote T eln et session ( Figure 3–57 ) . The KILL parameter may also b[...]

Controller Management 007-5510-002 79 3.7 System Logs 3.7.1 Message Log All controller events are logged and saved in non-vol atile memory . The log will automati cally roll over when it is full. LOG displays the log of previous system messag es. LOG CLEAR clears the log of all previous messages. LOG CHECKCONDITION displays the Check Condition log.[...]

80 007-5510-002 T o disp lay the nu mber of LUN array pari ty errors detected by the system, us e the F AUL TS ARRA YP ARITY command. The system saves the counts for each tier of all the LUNs. T o clear the count of LUN array parity errors in the system, use the F AUL TS ARRA YP ARITYCLEAR command. F AUL TS BUSP ARITY displays the numb er of bu s p[...]

Controller Management 007-5510-002 81 3.8 Other Utilities 3.8.1 APC UPS SNMP T rap Monitor APC_UPS displays the status of the APC UPS SNMP trap monitor ( Figure 3–61 ) . Figure 3–6 1 APC UPS SNMP T rap Monitor St atus APC_UPS CLEAR_F AUL TS will delete all pending APC UPS faults from the fault li st. All APC UPS events that disabled writeb ack [...]

82 007-5510-002 Figure 3–64 Displaying th e Ser ial Console Sett in g CONSOLE BAUD changes the baud rate of the CONFIG port of the controller (Figure 3–65). Figure 3–65 Changing the Baud Rate 3.8.4 CLI/T elnet Session Control Settings Y ou may change the CLI’ s and T elnet’ s various session cont rol settings. The SETTINGS command display[...]

Controller Management 007-5510-002 83 3.8.6 Disk Reassignment and Miscellaneous Disk Commands The DISK REASSIGN=tc 0xh command all ows for the reassigning of defective logical blocks on a disk to an area of the disk reserved for this purpose. Th e disk is specified by its: tier (t) in the range <1..125>, and channel (c) in the range <ABCDE[...]

84 007-5510-002 test so it will not affect the performance of the system. Any changes applied to delay will affect tests in progress as well as future testing. This system spare diagnostic delay value is giv en in 100 millisecond increments. The valid range for 'x' is 0..100. The default is 0. The SP ARE P A TTERN=x comma nd sets the sy s[...]

Troubleshooting 007-5510-002 85 Chapter 4 Controller Remote Management and T roubleshooting 4.1 Remote Management of the Controller The controller can be managed locally through the RS-232 interface, or remotely via T elnet. The Administrative Utili ty is the same regardless of the ma nagement interface (RS-232 or T elnet). The controller supports [...]

86 007-5510-002 NOTE : T elnet connections are clear text. If T elnet connections are used, you may ex pose controller passwords to third parties. For higher security , we recommend that you disengage T elnet access if it is not required. 5. Decide whether the SNMP functionali ty shou ld be enabled. T o enable or disable SNMP , use the appropriate [...]

Troubleshooting 007-5510-002 87 - ROUTE GA TEW A Y=<aaa.bbb.ccc.ddd> sets the current gateway in th e network routing table to the specified Internet address. - ROUTE DEL=<aaa.bbb.ccc.ddd> GA T EW A Y=<aaa.bbb .ccc.ddd> deletes gateways from the routing table. - ROUTE ADD=<aaa.bbb.ccc.ddd> GA TEW A Y=<aaa.bbb.ccc.ddd> [...]

88 007-5510-002 4.1.4 SNMP Set Up on Host Computer The following procedure explains how to set up the host com puter to mo nito r the controller ’ s events (SNMP traps). NOTE : It is not possible to configure the controller th rough SNMP . Refer to Section 4.1.2 in this guide for in formation regarding network and SNMP config uration on the contr[...]

Troubleshooting 007-5510-002 89 4.1.4.2 Traps • Generic T raps - coldStart (occurs when the controller is restarted) • Enterprise Specific T raps - Any changes in controller MIB. If any variable in the controller MIB changes, a trap will be sent. Fo r exam ple, when a power supply fails or is replaced, a trap is sent. Trap includes ASCII string[...]

90 007-5510-002 4. Remove the module’ s thum bscrew s, then slide the module out of the bay . 5. Slide the module in to the bay making sure it is fully inserted. Install th e two thumbscrews to secure it. 6. T urn on the power switch. Check that the St atus LED is gree n , indicating t hat the module is operating normally . 7. Replace the cover p[...]

Troubleshooting 007-5510-002 91 NOTE : If more than one disk is failed in a single tier, on ly the first disk to fail will be replaced by the spare disk. Any disks which fail after the first in each tier will not be reconstructed using a hot spare. 4.2.2.1 Single Drive Failures A single drive failure in any tier does no t result in the data loss. T[...]

92 007-5510-002 DISK INFO will indicate what spare drive is re placing the failed drive (Figure 4–6). Figure 4–6 Obtaining Disk Information 4.2.2.2 Returning the System to a Fault-Tolerant State When you find a failed drive on a tier , replace it as soon as possible so that the tier can return to its optimal state. If you replace the failed dri[...]

Troubleshooting 007-5510-002 93 4.2.2.4 Changing the Rate of Rebuild The commands TIER DELA Y and TIER EXTENT control the percentage of processor time allocated to rebuild compared to I/ O handling. Rebuild is done in steps. DELA Y will control the amou nt of wait time before rebuilding the next chunk of data. EXTENT de term in es the size of data [...]

94 007-5510-002 4.2.3 Component Failure on Enclosures The controller implements the SCSI Enclosur e Services (SES) protocol to commun icate with its drive enclosures. If your enclosures provides SES comm unica tions, then enclosure stat us information of the , including power supply , fan, and presence of drive, will be obtained and evaluated. If a[...]

Drive Enclosure 007-5510-002 95 Chapter 5 Drive Enclosure System 5.1 The SGI InfiniteStor age 15000 Drive Enclosure The SGI InfiniteStorage 15000 drive enclosure storage system is an ultra dense 4U 60 disk drive enclosure. Figure 5–1 and Figure 5–2 show front and rear views of a drive enclosure, respectively. Module and major components and the[...]

96 007-5510-002 5.2 Enclosure Core Product The drive enclosure design co ncept is based on a subsystem togeth er with a set of plug-in modules and (as supplied) comprises: • Enclosure Chassis with integral Front Panel Drive Status Indicat or ( Figure 5–9 ). • Two (2) 1865W Power Cooling (PCM) plug-in m odules ( Figure 5–4 ). • Two (2) plu[...]

Drive Enclosure 007-5510-002 97 5.2.1 Enclosure Chassis The chassis assembly contains 60 drive bays at th e front, each of which accommodates a plug-in drive carrier module. The 60 drive bays ar e arranged in five rows of twelve drives (5x12). At the rear, the chassis assembly contains two (2) PCMs and two (2) I/O modules. The chassis is fitted wit[...]

98 007-5510-002 5.3.2 Input/Output (I/O) Module The drive enclosure storage subsyste m includes an enclosur e with rear facing bays whi ch house two (2) I/O modules ( Figure 5–5 ). Processors housed on the I/O modules provide enclosu re management and interface to devices on the Backplane, PCM, and Disp lay Panel in order to monitor internal func[...]

Drive Enclosure 007-5510-002 99 I Table 5–2 defines the LED indicators inco rporated on the I/O module. Figure 5–5 I/O Module Host Port 1 Expansion Port 1 Host Port 2 Expansion Port 2 Table 5–2 I/O Module LEDs LED Color STAT U S Definition Enclosure Audible Alarm Host Port 1 Expansion Port 1 Host Port 2 Expansion Port 2 Green ON OFF •Incomi[...]

100 007-5510-002 5.3.3 Drive Carrier Modul e and St atus Indicator The Drive Carrier Module comprises a hard disk mounted in a carrier ( Figure 5–6 ). Each drive bay can house a single Low Pr of ile 1.0 inch high, 3.5 in ch form fac tor disk drive in its carrier. The handle provides the following function s: • Camming of carrier into and out of[...]

Drive Enclosure 007-5510-002 101 Note The DEM is a serviceable PCB which may be replaced by trained pers onnel. Each DEM connects to a single port of the HDD based on the location within the enclosure within which it is inst alled ( Figure 5–8 ). Each provides power control signals to each driv e slot. It controls HDD identify/service LEDs and mo[...]

102 007-5510-002 cards are fitted to drives. Thes e LEDs indicate drive presence. Th e Front Disp lay Pan el LEDs are defined in Table 5–3 . Figure 5–9 Front Panel Drive Activity Indicators Panel Table 5–3 Display Panel LEDs ICON LABEL COLOR Definition Normal Behavior Enclosure Audible Alarm Shelf Identify BLUE enclos ure Identity ON -SES is [...]

Drive Enclosure 007-5510-002 103 . Drive Fault AMBER Drive fault ON -one or more HDDs are faulted. SES must determine exact HDD. OFF -no detectable drive faults Off Individually numbered HDDs GREEN HDD activity ON -indicates HDD activity OFF -no HDD activity Off Table 5–3 Display Panel LEDs ICON LABEL COLOR Definition Normal Behavio r Enclos ure [...]

104 007-5510-002 5.4.2 Internal Indicators Note In some components, the failure LED is internal to th e enclosure and visible only when the cover is open. The Internal LED indicators are explained in Table 5–4 . 5.4.3 Rear of Enclosur e Activity Indicators The PCMs and the I/O Modules are located on the rear of the enclosures. The LEDs on the rea[...]

Drive Enclosure 007-5510-002 105 5.5 V isible and Audible Alarms The functional modules have associated status LEDs. The Display Panel ( Table 5–3 ) shows a consolidated status for all modul es: • Constant GREEN or BLUE LEDs indicate good or positive indication . • Constant AMBER LEDs indicate that there is a fault present with in that module[...]

106 007-5510-002 5.6.2 W eight 5.6.3 AC INPUT PCM 5.6.4 DC INPUT PCM Item Quantity per System LBS (kg) Extended Weight Chassis Includes Base PCB & Midplane 18 3 8 3 PCM 2 12 24 I/O Module Drives 2 4 8 60 2 120 Miscellaneous 1 5 5 Total Weight 240 Input Parameter Value Input Voltage 190 - 264 VAC Input Frequency 47 - 63 Hz and 400 hz Maximum Inp[...]

Drive Enclosure 007-5510-002 107 5.6.5 DC OUTPUT PCM 5.6.6 PCM Safety and EMC Compliance 5.6.7 Power Cord (minimum requirements) Note Power cord is not included in th e standard drive enclosure package. 5.7 Environment The enclosure and all of its sub-co mp onents are compliant to the European Union RoHS (Restriction of Hazardous Substances) Direct[...]

[...]

Drive Enclosure Installation 007-5510-002 109 Chapter 6 Drive Enclosure Inst allation 6.1 Introduction Caution When connecting up the drive enclosure subsystem, use only t he cables supplied. 6.2 Planning Y our Inst allation Before you begin instal lation, you should beco me familiar with the config uration requirements of your drive enclosure syst[...]

110 007-5510-002 6.2.1 Enclosure Bay Numbering Convention Warning Operation of the Encl os ure with ANY of the plug-i n modu le s mis s ing from the rear of the enclosure will disrupt the airflow and the drives will not receive sufficient cooling. It is ESSENTIAL that all (rear) apertures are filled befor e operating the unit. The drive enclosure s[...]

Drive Enclosure Installation 007-5510-002 111 Figure 6–2 1X60 Drive Numb ering T able. Figure 6–3 2x30 Drive Numb ering T able[...]

112 007-5510-002 6.3 Enclosure Inst allation Procedures Important SGI InfiniteSto rage 15000 drive enclosur es sh ou ld only b e installed in SGI InfiniteSto rage 15000 racks. Mounting and insta lling these drive enclosures in any other rack is not authorized o r supported by SGI. Warning The drive enclosure with all its comp on ent pa rts installe[...]

Drive Enclosure Installation 007-5510-002 113 6.5 SAS DEM The drive enclosure contains eight (8) top-loadable DEMs. The DE Ms provide the SAS connectivity between the I/O module and the HDDs located within the enclosure. E ach DEM connects to a single port of the HDD based on the locat ion with in the enclosure in which it is installed. Each of the[...]

[...]

Drive Enclosure Operation 007-5510-002 115 Chapter 7 Drive Enclosure Operation 7.1 Before Y ou Begin Before powering up the enclosure, pl ease ensure that all the modules ar e firmly seated in their correct bays. 7.2 Power On / Power Down Caution Do not operate the subsystem until the ambient temperat ure is within the specifie d operating r ange. [...]

116 007-5510-002 7.2.2 I/O Panel LEDs The I/O Panel LEDs fault and status conditions are shown in Figure 7–1 . I Table 7–1 defines the LED indicator s in co rporated on the I/O module. Figure 7–1 I/O Panel LEDs Table 7–1 I/O Module LEDs LED Color STATU S Definition Host Port 1 Expansion Port 1 Host Port 2 Expansion Port 2 Green ON OFF •In[...]

Drive Enclosure Troubleshooting 007-5510-002 117 Chapter 8 Drive Enclosure T roubleshooting 8.1 Overview The SGI InfiniteStorage15000 Drive Enclosure includes an Enclosure Se rvices Processor and associated monitoring and control logic to enable it to diag nos e problem s within the encl osure’s power, cooling , and drive systems. The sensors for[...]

118 007-5510-002 3 Check that there is a valid SAS signa l present at the I/O connector. If there is no signal present, ensure the cable has been properly inserted. 4 Check the I/O module setup as follows: – Ensure the I/O module has been correctly installed and all external links and cables are securely fitted. – Ensure the maximum cable le ng[...]

Drive Enclosure Troubleshooting 007-5510-002 119 • Under Normal conditions, the LEDs should all be illum in a ted constan t GREEN • If a problem is detected, the color of the relevant LED will change to AMBER. 8.3.4 I/O Module The I/O fault LEDs are explained in Table 8–4 . • Under Normal conditions, the LEDs should all be illum in a ted co[...]

120 007-5510-002 8.3.5 Front Panel Drive Activity Indicators The Front Panel Drive Activity Indicators show the agg r egated status of all the modules. This panel and its associated LEDs are shown in Figure 8–1 . The Display Panel LEDs are defined in Table 8–5 . REAR Enclosure Green SAS Li nk Activity ON when a valid link is present on any of t[...]

Drive Enclosure Troubleshooting 007-5510-002 121 8.4 Audible Alarm When the Audible Alarm sounds, us e the appropriate SES CLI command. 8.4.1 T op Cover Open The Audible Alarm sounds anytime the top cover is open. The alarm mutes auto matically once the cover is closed. 8.4.2 SES Command The Audible Alarm can be set through SES communication. When [...]

122 007-5510-002 Symptom Cause Action If the ambient air is cool (below 25 °C) and t he fans are observed to increase in speed, then some restriction on airflow may be causing additional internal temperature rise. Note: This is not a fault condition. The first stage in the thermal control process is for the fans to automatically increase in speed [...]

Drive Enclosure Troubleshooting 007-5510-002 123 8.5.2 Thermal Alarm 8.5.3 Thermal Shut down Important An enclosur e will shut down when a critical temperature threshold is exceeded in order to prevent permanent dama ge to the di sk drives. 8.6 Dealing with Hardware Fault s Ensure that you have obtained a re placement module of the same type before[...]

124 007-5510-002 8.7 Continuous Operation During Replacement Depending on how the subsystem is set up, if a di sk unit fails, it can norm ally be replaced without interrupting the use of the system. 8.8 Replacing a Module Warning Whenever replacing a module NEVER leave an EMPTY bay in the rear of the enclosure, obtain a replacement before remo ving[...]

Drive Enclosure Troubleshooting 007-5510-002 125 To remove an AC PCM - Thumb Screw Model: Warning Do not remove the fa ulty PCM unless you have a rep la ceme nt unit of the correct type ready for insertion. If a power supply unit or its fan is faulty, you must re place the whole PCM. As there should always be two PCMs installed, you can continue wo[...]

126 007-5510-002 8.8.2 I/O Module Warning Do not remove this module unless a replacement ca n be immediately adde d. The system must no t be run withou t all units i n place. To remove the I/O Module: 1 Release the two latches on the bottom of the unit by simply pulling each latc h out and away from unit. 2 Pull the latches forward to cam the m odu[...]

Drive Enclosure Troubleshooting 007-5510-002 127 spindown the drives prior to remo val. If this is not possible, we recommend tha t you perform All steps of the following procedure to ensure that the drive has stopped prior to removal: To Insert the Dr ive Carrier Module 1 Release the carrier handle by sliding the latch back wards. 2 Insert the car[...]

[...]

007-5510-0 02 129 A Controller T echnical Specifications The following are the technical specifications of the SGI InfiniteS torage 15000 controller . NOTE : Specifications subject to change without notice. Configuration, Performance, & Capacity Host interface IB 4xDDR, FC-8 Drive interface SAS Management inte rface RS-232 and Ethe rnet (T elne[...]

130 007-5510-002 Physical, Power & Environmental Controller chassis (H × W × D, Weight) 3.5" × 19 .0 " × 31", 45 lbs Electrical/AC 1 00-120V / 200-240 V @ 47-63 Hz Power consumption (ma ximum current) 2.65A @ 208V AC Couplet: 8.0A @ 1 10V AC, 3.8A @ 230V AC Power consumption (a verage current) 3.0A @ 1 10V AC, 1.5A @ 230V AC [...]

007-5510-0 02 131 B Drive Addressing This appendix illustrates the drive addressing for the di sk enclosures. There are two drive numbering options: • 1 x 60 (one set of drives 1 throug h 60); (Figures B- 1and B-3). • 2 x 30 (two sets of drives 1 thro ugh 30); (Figures B-2 and B-4) Figure B–1 1x60 Drive Layout Configuration Figure B–1 2x30 [...]

132 007-5510-002 Figure B–3 Controller-Drive Enclosure Couplet (1x Configuration)[...]

007-5510-0 02 133 Figure B–4 Controller-Drive Enclosure Couplet (2x Configuration) P S P S A B C D E F G H P S E E F A B A C D G H G B D H F C A B C D E F G H P S Controller #1 Controller #2 Channel A/B Channel C/D Channel E/F Channel G/H Channel P/S[...]

[...]

007-5510-0 02 135 C Cabling Controllers and Drive Enclosures This appendix illustrates the proper cabling of the follow ing two configu rations: • 2 cont rol lers and 5 drive enclosures (Figures C–1 and C–2) • 2 cont rollers and 10 drive enclosures (Figures C–3 and C –4)[...]

136 007-5510-002 Figure C–1 Controll e r Cabli ng (2 C ontrollers and 5 Drive Enclosures) Figure C–2 Drive Enclosure Cabl ing (2 Controllers and 5 Drive Enclosures) A A Channel A/B C C Channel C/D E E Channel E/F G G Channel G/H P P Channel P/S A C E G P B D F H S Controller 2 A C E G P B D F H S Controller 1 BB DD FF HH SS Note: T o use these [...]

007-5510-0 02 137 Figure C–3 Controlle r Cabli ng (2 Controllers and 10 Drive Enclosures) A A Channel A B B Channel B C C Channel C D D Channel D E E Channel E F F Channel F G G Channel G H H Channel H P P Channel P S S Channel S A C E G P B D F H S Controller 2 A C E G P B D F H S Controller 1 Note: T o use these cable lengths, controllers and j[...]

138 007-5510-002 Figure C–4 D rive Enclosure Cabling (2 Controllers and 10 D rive Enclosures) A A B B C C D D E E F F G G H H P P S S A A B B C C D D E E F F G G H H P P S S[...]