ASM Metadata Internals

A collection of facts and diagnostic queries on Oracle ASM metadata collected to complement existing documentation and for learning purposes. More on RAC and ASM configuration and performance at CERN Phsycs Db in HAandPerf.

ASM metadata, V$ and X$:

View Name	X$ Table name	Description
V$ASM_DISKGROUP	X$KFGRP	performs disk discovery and lists diskgroups
V$ASM_DISKGROUP_STAT	X$KFGRP_STAT	ists diskgroups
V$ASM_DISK	X$KFDSK, X$KFKID	performs disk discovery, lists disks and their usage metrics
V$ASM_DISK_STAT	X$KFDSK_STAT, X$KFKID	lists disks and their usage metrics
V$ASM_FILE	X$KFFIL	lists ASM files, including metadata/asmdisk files
V$ASM_ALIAS	X$KFALS	lists ASM aliases, files and directories
V$ASM_TEMPLATE	X$KFTMTA	lists the available templates and their properties
V$ASM_CLIENT	X$KFNCL	lists DB instances connected to ASM
V$ASM_OPERATION	X$KFGMG	lists rebalancing operations
N.A.	X$KFKLIB	available libraries, includes asmlib path
N.A.	X$KFDPARTNER	lists disk-to-partner relationships
N.A.	X$KFFXP	extent map table for all ASM files
N.A.	X$KFDAT	extent list for all ASM disks

This list is obtained querying v$fixed_view_definition where view_name like '%ASM%' which exposes all the v$ and gv$ views with their definition. Additionally querying v$fixed_table where name like 'X$KF%' and name like 'X$KF%' (ASM views start with the KF prefix).

Striping and Mirroring with ASM, extents and allocation units

An basic example, using ASM and normal redundancy: the available storage, say 64 HDs over FC SAN, are used to create the main DB diskgroup: DATADG. DATADG is logically divided into 2 evenly sized groups of disks: 32 disks in failgroup N.1 and 32 in failgroup N.2. Oracle datafiles created in DATADG are 'striped' into smaller pieces, extents of 1MB in size. Extents are allocated to the storage in 2 (mirrored) allocation units (AU): one AU in failgroup N.1 the other in failgroup N.2.

X$KFFXP

This X$ table contains the mapping between files and used allocation units. It allows to track the position of all the extents of a given file striped and mirrored across storage. Note: read operations are load-balanced over primary and mirror allocation units (for performance reasons). Write operations instead write both mirrored AU to disk.

X$KFFXP Column Name	Description
ADDR	x$ table address/identifier
INDX	row unique identifier
INST_ID	instance number (RAC)
NUMBER_KFFXP	ASM file number. Join with v$asm_file and v$asm_alias
COMPOUND_KFFXP	File identifier. Join with compound_index in v$asm_file
INCARN_KFFXP	File incarnation id. Join with incarnation in v$asm_file
PXN_KFFXP	file AU number
XNUM_KFFXP	ASM file extent number (mirrored AU have the same extent value)
GROUP_KFFXP	ASM disk group number. Join with v$asm_disk and v$asm_diskgroup
DISK_KFFXP	Disk number where the extent is allocated. Join with v$asm_disk
AU_KFFXP	Relative position of the allocation unit from the beginning of the disk. The allocation unit size (1 MB) in v$asm_diskgroup
LXN_KFFXP	0->primary extent, ->mirror extent, 2->2nd mirror copy (high redundancy and metadata)
FLAGS_KFFXP	N.K.
CHK_KFFXP	N.K.

Example1 - poking into ASM files with direct OS access

• Find the 2 mirrored extents of an ASM file (the spfile in this example)

sys@+ASM1> select GROUP_KFFXP,DISK_KFFXP,AU_KFFXP from x$kffxp where 
   number_kffxp=(select file_number from v$asm_alias where name='spfiletest1.ora');

GROUP_KFFXP DISK_KFFXP   AU_KFFXP
----------- ---------- ----------
          1         20        379
          1          3         101

• find the diskname

sys@+ASM1> select disk_number,path from v$asm_disk where 
    GROUP_NUMBER=1 and disk_number in  (3,20);

DISK_NUMBER PATH
----------- ----------------------------------------
          3    /dev/mpath/itstor417_2p1
         20   /dev/mpath/itstor419_2p1

• access the data directly from disk with dd

 dd if=/dev/mpath/itstor417_2p1 bs=1024k count=1 skip=101|strings|more

See also:

• https://twiki.cern.ch/twiki/pub/PSSGroup/HAandPerf/ASM_metadata_30012006.html
• http://www.freelists.org/archives/oracle-l/05-2006/msg00395.html]]

X$KFDAT

This X$ table contains details of all allocation units (free and used).

X$KFDAT Column Name	Description
ADDR	x$ table address/identifier
INDX	row unique identifier
INST_ID	instance number (RAC)
GROUP_KFDAT	diskgroup number, join with v$asm_diskgroup
NUMBER_KFDAT	disk number, join with v$asm_disk
COMPOUND_KFDAT	disk compund_index, join with v$asm_disk
AUNUM_KFDAT	Disk allocation unit (relative position from the beginning of the disk), join with x$kffxp.au_kffxp
V_KFDAT	V=this Allocation Unit is used; F=AU is free
FNUM_KFDAT	file number, join with v$asm_file
I_KFDAT	N.K.
XNUM_KFDAT	ASM file AU number join with x$kffxp.pxn_kffxp
RAW_KFDAT	raw format encoding of the disk,and file extent information

Example2 - list allocation units of a given file from x$kfdat

• same as example 1 above, another way to retrieve file allocation maps:

sys@+ASM1> select GROUP_KFDAT,NUMBER_KFDAT,AUNUM_KFDAT from x$kfdat where 
   fnum_kfdat=(select file_number from v$asm_alias where name='spfiletest1.ora');

GROUP_KFDAT NUMBER_KFDAT AUNUM_KFDAT
----------- ------------ -----------
          1            3         101
          1           20         379

Example3 - from strace data to file identification:

• The following line is take from strace of lmon:
  • pread64(22, "\25\302\0\0003\0\0\0\0\0\0\0\0\0\1\4P\323\0\0L\0\0\0\0"..., 16384, 248561664) = 16384
  • it is a read operation of 16KB at the offset 248561664 (=237 MB + 48KB) from file descriptor FD=22
• using /proc/PID/fd I find FD=22 is /dev/dm-19, that is /dev/mpath/itstor418_8p1
• I find the group and disk number of the file:

sys@+ASM1> select GROUP_NUMBER,DISK_NUMBER from v$asm_disk where path='/dev/mpath/itstor418_8p1';

GROUP_NUMBER DISK_NUMBER
------------ -----------
           1          14

• using the disk number, group number and offset (from strace above) I find the file number:

sys@+ASM1> select fnum_kfdat from x$kfdat where number_kfdat=14 and GROUP_KFDAT=1 and AUNUM_KFDAT=237;

FNUM_KFDAT
----------
       256

• from v$asm_file fnum=256 is the first copy of the controlfile (which matches with the fact that lmon read is 16KB, the controlfile block size)

X$KFDPARTNER

This X$ table contains the disk-to-partner (1-N) relationship. Two disks of a given ASM diskgroup are partners if they each contain a mirror copy of the same extent. Therefore partners must belong to different failgroups of the same diskgroup. From a few live examples I can see that typically disks have 10 partners each at diskgroup creation and fluctuate around 10 partners following ASM operations. This mechanism is in place to reduce the chance of losing both sides of the mirror in case of double disk failure.

X$KFDPARTNER Column Name	Description
ADDR	x$ table address/identifier
INDX	row unique identifier
INST_ID	instance number (RAC)
GRP	diskgroup number, join with v$asm_diskgroup
DISK	disk number, join with v$asm_disk
COMPOUND	disk identifier. Join with compound_index in v$asm_disk
NUMBER_KFDPARTNER	partner disk number, i.e. disk-to-partner (1-N) relationship
MIRROR_KFDPARNER	=1 in a healthy normal redundancy config
PARITY_KFDPARNER	=1 in a healthy normal redundancy config
ACTIVE_KFDPARNER	=1 in a healthy normal redundancy config

X$KFFIL and metadata files

Three types of metadata:

• diskgroup metadata: files with NUMBER_KFFIL <256 ASM metadata and ASMlog files. These files have high redundancy (3 copies) and block size =4KB.
  • ASM log files are used for ASM instance and crash recovery when a crash happens with metadata operations
• disk metadata: disk headers (typically the first 2 AU of each disk) are not listed in x$kffil (they appear as file number 0 in x$kfdat). Contain disk membership information. This part of the disk has to be 'zeroed out' before the disk can be added to ASM diskgroup as a new disk.
• file metadata: 3 mirrored extents with file metadata, visible from x$kffxp and x$kfdat

Tnsnames entries and ASM

TIP: An example of tnsnames entry to be used to connect to ASM instances via Oracle*NET (note the extra keyword (UR=A)). More generally UR=A allows to connect to 'blocked services'. Example connect sys/pass@ASM1 as sysdba (an asm password file is also needed on the server)

ASM1 =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = [hostname])(PORT = [portN]))
    (CONNECT_DATA =
      (SERVER = DEDICATED) (SERVICE_NAME = +ASM)  (INSTANCE_NAME = +ASM1)
      (UR=A)
    )  )

---

Revisions:
V1.0 Jan 2006, Luca.Canali@cernNOSPAMPLEASE.ch
Major additions, Jan 2007, L.C.
Added examples, Feb 2007, L.C.