Disk Redundancy in Exadata

Disk Redundancy in Exadata

Redundancy is fail-safe measure where data is duplicated more than one location.
Benefit of redundancy is, it protects against drive or storage server failure. But the downside of redundancy is it reduces actual storage available by half or more.

Lets look at how exadata storage layout.

Above illustration shows three cell servers with High redundancy (3-way mirror) of data. In case of physical disk failure or a storage failure occurs, data is spread in such a way that end users may not notice of failure due to this isolation of ASM failure groups.

Benefits of  High Redundancy is that it protects if 2 entire storage server fails or 2 disk failures and both disks from different storage servers. Rolling upgrade of Exadata storage server are possible with high redundancy. While normal Redundancy protects against 1 disk failure or an entire storage server fails.

ASM diskgroups are spread on grid disks. Grid disk are made up of cell disks.
Cell disk in turn comprised of physical disks and LUN. Based on exadata configuration,
it can have high performance disks (15k RPM) or high capacity disk 
Replacing failed disk becomes transparent to end users due to this ASM design
and it make easy to service exadata.



To check details about storage, you can use cellcli interface to query disk related info:

dcli -g cell_group -l root 'cellcli -e list physicaldisk'
dcli -g cell_group -l root 'cellcli -e list lun'
dcli -g cell_group -l root 'cellcli -e list celldisk'
dcli -g cell_group -l root 'cellcli -e list griddisk'

For example, +DATA diskgroup can be created by following command:

CREATE DISKGROUP data HIGH REDUNDANCY 
DISK 
'o/192.168.10.1/DATA*',
’o/192.168.10.2/DATA*’,
’o/192.168.10.3/DATA*’ 
ATTRIBUTE 
'content.type' = 'DATA', 
'AU_SIZE' = '4M', 
'cell.smart_scan_capable'='TRUE', 
'compatible.rdbms'='11.2.0.4', 
'compatible.asm'='11.2.0.4'; 

Exploring ASMLIB

What is Oracle ASM?

Oracle Automatic Storage Management (ASM) is a storage management solution. It consists of logically group disks, known as volumes or diskgroups, it holds database files.

What are benefits of ASM?

• It lets you add/remove disk without downtime by automatically redistributing disk contents.
• ASM provides uniform performance by spreading file content evenly on disks.
• Fault tolerance can be achived by ASM 2-way or 3-way mirroring
• Flexibilty to use storage management options provided by storage vendors.
• Easier storage management for DBAs 

What is ASMLIB 

ASMLIB is an Oracle kernel driver or library to enable ASM I/O in Linux for various disk type (IDE, SCSI, RAID, Multipath) providing better I/O performance by eliminating UNIX I/O API. ASMLIB provides interface between Linux operating system and database ASM devices. In database 12c, ASM Filter Driver is recommended in place of ASMLIB 

How to setup ASMLIB support? 

On Linux servers:

1. Download ASMLIB from OTN
2. Install set of RPMS (oracleasm) on Linux
3. Create mount point /dev/oracleasm with

/etc/init.d/oracleasm configure 

4. Setup proper owner(UID), group(GID) and permission on devices
5. Make disk available by running oracleasm createdisk 
6. Verify installation and make sure disks are visible on all RAC nodes 

/etc/init.d/oracleasm status  

Checking if ASM is loaded:  [  OK  ]
Checking if /dev/oracleasm is mounted: [  OK  ] 

List usable marked disks

oracleasm listdisks 

Query ASM library in use

SQL> select library, path from v$asm_disk; 

7. Set ASM init.ora parameter for asm_diskstring 
8. Startup ASM instance(s)