2012 Failover Cluster Physical Disk (iSCSI) resources fail to come online on secondary node

hi,

here system background information:

vmware esx 5.5 hosts
shared iscsi luns on hp storevirtual p4000 san, presented esx hosts , guest vms raw device mappings
2 node sql 2014 failover cluster on windows server 2012 (non-r2)

began system build presenting 2 brand new luns 2012 guests, we'll call them vol_q (quorum/disk witness) , vol_s (for sql db files).  we installed sql on both nodes (i'll refer them node01 , node02 here) , tested failover of core resources new sql server resource, , testing proved successful in storage volumes , sql server failed on node01 node02 successfully.

our next step migrate old volumes our existing sql cluster new cluster, bring volumes in , add them new sql server resource, , test failover node01 node02.  this problem began; after adding first pre-existing volume (vol_p) new 2012 cluster successfully, failover attempted, , vol_q (the quorum/disk witness) volume come online successfully.  both of other volumes, vol_p, , volume came online successfully, vol_s, stayed in 'online pending' status. 

i did not have ample time allow resources go failed state before having bring system online again, after 2 minutes forced fail node02 node01, in case of resources came online expected.

here entries cluster log of node02:

00000f68.000015f8::2014/12/16-22:15:29.819 info  [res] physical disk <(s:) mssql>: resharddiskonlinev2: online request.
00000f68.000012d4::2014/12/16-22:15:30.048 info  [res] physical disk <(s:) mssql>: resharddiskarbitrateinternal request not space: uses fastpath
00000f68.000012d4::2014/12/16-22:15:30.048 info  [res] physical disk <(s:) mssql>: resharddiskarbitrateinternal: clusdisk driver handle or event handle null.
00000f68.000012d4::2014/12/16-22:15:30.059 info  [res] physical disk <(s:) mssql>: harddiskpquerydiskfromstm: clusterstmfinddisk returned device='\\?\scsi#disk&ven_lefthand&prod_iscsidisk#5&de2103a&0&000100#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}'
00000f68.000012d4::2014/12/16-22:15:30.091 info  [res] physical disk <(s:) mssql>: arbitrate - node using pr key 348c08690001734d
00000f68.000012d4::2014/12/16-22:15:30.521 info  [res] physical disk <(s:) mssql>: harddiskpprarbitrate: fast path arbitration...
00000f68.000012d4::2014/12/16-22:15:30.816 info  [res] physical disk <(s:) mssql>: successful reserve, key 348c08690001734d
00000f68.000012d4::2014/12/16-22:15:30.817 info  [res] physical disk <(s:) mssql>: disk offline
00000f68.000012d4::2014/12/16-22:15:30.818 info  [res] physical disk <(s:) mssql>: harddiskpsetunsetdiskflags(mask=0x00000007, setcluster=1, setcsv=0, setmaintenancemode=0, notify=1, update=1) device=2
00000f68.000012d4::2014/12/16-22:15:30.818 info  [res] physical disk <(s:) mssql>: harddiskpgetdiskhandle: exit, status 0
00000f68.000012d4::2014/12/16-22:15:30.818 warn  [res] physical disk <(s:) mssql>: harddiskpupdatevolumepropertiesindriver: disk offline.
00000f68.000012d4::2014/12/16-22:15:30.818 info  [res] physical disk <(s:) mssql>: onlinethread: cleared csv state partmgr.
00000f68.000012d4::2014/12/16-22:15:30.826 info  [res] physical disk <(s:) mssql>: harddiskpwaitforpartitionstoarrive: begin wait \\?\globalroot\device\harddisk2\partition0 partitions arrive
00000f68.000012d4::2014/12/16-22:15:30.827 info  [res] physical disk <(s:) mssql>: harddiskpwaitforpartitionstoarrive: status error_io_pending ioctl_disk_are_volumes_ready
00000f68.000012d4::2014/12/16-22:15:31.193 info  [res] physical disk <(s:) mssql>: harddiskpwaitforpartitionstoarrive: wait success , ioctl_disk_are_volumes_ready completed status=0
00000f68.000012d4::2014/12/16-22:15:31.193 info  [res] physical disk <(s:) mssql>: harddiskpwaitforpartitionstoarrive: wait volumes
00000f68.00001608::2014/12/16-22:18:33.880 info  [res] physical disk <(s:) mssql>: terminate request.

you notice last line forced failback node02 node01.  in contrast, here entry node01 fails back:

00001044.000017c8::2014/12/16-22:19:10.170 info  [res] physical disk <(s:) mssql>: resharddiskonlinev2: online request.
00001044.00000d8c::2014/12/16-22:19:10.186 info  [res] physical disk <(s:) mssql>: resharddiskarbitrateinternal request not space: uses fastpath
00001044.00000d8c::2014/12/16-22:19:10.186 info  [res] physical disk <(s:) mssql>: resharddiskarbitrateinternal: clusdisk driver handle or event handle null.
00001044.00000d8c::2014/12/16-22:19:10.186 info  [res] physical disk <(s:) mssql>: harddiskpquerydiskfromstm: clusterstmfinddisk returned device='\\?\scsi#disk&ven_lefthand&prod_iscsidisk#5&de2103a&0&000100#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}'
00001044.00000d8c::2014/12/16-22:19:10.186 info  [res] physical disk <(s:) mssql>: arbitrate - node using pr key 4f2847ef0002734d
00001044.00000d8c::2014/12/16-22:19:10.794 info  [res] physical disk <(s:) mssql>: successful reserve no need arbitrate, key 4f2847ef0002734d
00001044.00000d8c::2014/12/16-22:19:10.794 info  [res] physical disk <(s:) mssql>: disk offline
00001044.00000d8c::2014/12/16-22:19:10.794 info  [res] physical disk <(s:) mssql>: harddiskpsetunsetdiskflags(mask=0x00000007, setcluster=1, setcsv=0, setmaintenancemode=0, notify=1, update=1) device=2
00001044.00000d8c::2014/12/16-22:19:10.794 info  [res] physical disk <(s:) mssql>: harddiskpgetdiskhandle: exit, status 0
00001044.00000d8c::2014/12/16-22:19:10.794 warn  [res] physical disk <(s:) mssql>: harddiskpupdatevolumepropertiesindriver: disk offline.
00001044.00000d8c::2014/12/16-22:19:10.794 info  [res] physical disk <(s:) mssql>: onlinethread: cleared csv state partmgr.
00001044.00000d8c::2014/12/16-22:19:10.794 info  [res] physical disk <(s:) mssql>: harddiskpwaitforpartitionstoarrive: begin wait \\?\globalroot\device\harddisk2\partition0 partitions arrive
00001044.00000d8c::2014/12/16-22:19:10.794 info  [res] physical disk <(s:) mssql>: harddiskpwaitforpartitionstoarrive: status error_io_pending ioctl_disk_are_volumes_ready
00001044.00000d8c::2014/12/16-22:19:10.810 info  [res] physical disk <(s:) mssql>: harddiskpwaitforpartitionstoarrive: wait success , ioctl_disk_are_volumes_ready completed status=0
00001044.00000d8c::2014/12/16-22:19:10.810 info  [res] physical disk <(s:) mssql>: harddiskpwaitforpartitionstoarrive: wait volumes
00001044.00000d8c::2014/12/16-22:19:11.122 info  [res] physical disk <(s:) mssql>: harddiskpwaitforpartitionstoarrive: wait volumes completed
00001044.00000d8c::2014/12/16-22:19:11.122 info  [res] physical disk <(s:) mssql>: unlockvolumesifencryptionenabled
00001044.00000d8c::2014/12/16-22:19:11.122 warn  [res] physical disk <(s:) mssql>: fvelib load failed 7e fveloaded 0
00001044.00000d8c::2014/12/16-22:19:11.122 info  [res] physical disk <(s:) mssql>: resharddiskvolumeguidpathnameanddriveletterchecks. disk {2}.
00001044.00000d8c::2014/12/16-22:19:11.122 info  [res] physical disk <(s:) mssql>: resharddiskgetwin32pathnames: found 2 mount points disk {2}, partition {1}.
00001044.00000d8c::2014/12/16-22:19:11.122 info  [res] physical disk <(s:) mssql>: harddiskpdriveletterandvolumeguidreset: volume {\\?\volume{4a14bbe2-7995-11e4-9403-005056b846cb}\}, current drive letter {0x20}, cluster  drive letter {0x20}, point {000000530ed16f00}
00001044.00000d8c::2014/12/16-22:19:11.122 info  [res] physical disk <(s:) mssql>: volumeisntfs: volume \\?\globalroot\device\harddisk2\clusterpartition1\ has fs type ntfs
00001044.00000d8c::2014/12/16-22:19:11.122 info  [res] physical disk <(s:) mssql>: onlinethread: harddiskpverifyvolume part=1 returned 0, state=0x00000000
00001044.00000d8c::2014/12/16-22:19:11.122 info  [res] physical disk <(s:) mssql>: resharddiskverifymountfoldertargetvolumesareclustered: volume \\?\globalroot\device\harddisk2\clusterpartition1\ has fs type ntfs
00001044.00000d8c::2014/12/16-22:19:11.122 info  [res] physical disk <(s:) mssql>: harddiskpvalidatemountpoints - mount point \\?\volume{4a14bbe2-7995-11e4-9403-005056b846cb}\ returned invalid handle
00001044.00000d8c::2014/12/16-22:19:11.122 info  [res] physical disk <(s:) mssql>: harddiskpvalidatemountpoints - there no mounted folders.
00001044.00000d8c::2014/12/16-22:19:11.122 info  [res] physical disk <(s:) mssql>: mountpoint s:\ points volume \\?\volume{4a14bbe2-7995-11e4-9403-005056b846cb}\
00001044.000017f4::2014/12/16-22:20:11.122 info  [res] physical disk <(s:) mssql>: volumeisntfs: volume \\?\globalroot\device\harddisk2\clusterpartition1\ has fs type ntfs

one other thing note, on node02 during failover, after cluster service reserves disk, see entries in system event log, ntfs source:

"volume s: (\device\harddiskvolume2) healthy.  no action required."

so me looks os sees volume, yet not show information in windows explorer or otherwise, , if try such run diskpart or disk management view volumes, operations freeze up.

appreciated, i'm not sure @ point turn determining problem is; vmware, microsoft, hp, of above?  we have non sql cluster configured exact same way, using old volumes , all, performs way should, i'm confused going on specific situation i'm in now.   can provide logs of of involved systems.

thank time,
chad

hi chadren,

the vmware rdm storage solution not certified  windows server, therefore can more support vmware,  there have official support cluster storage can refer following kb,

failover clustering hardware requirements , storage options

http://technet.microsoft.com/en-us/library/jj612869.aspx

i found vmware kb raw device mapping may can please confirm won’t got incorrect disk format.

https://pubs.vmware.com/vsphere-4-esx-vcenter/index.jsp?topic=/com.vmware.vsphere.server_configclassic.doc_41/esx_server_config/raw_device_mapping/c_about_raw_device_mapping.html

more information:

the microsoft support policy windows server 2012 or windows server 2012 r2 failover clusters

http://support.microsoft.com/kb/2775067

i’m glad of you!

*** response contains reference third party world wide web site. microsoft providing information convenience you. microsoft not control these sites , has not tested software or information found on these sites; therefore, microsoft cannot make representations regarding quality, safety, or suitability of software or information found there. there inherent dangers in use of software found on internet, , microsoft cautions make sure understand risk before retrieving software internet. ***

---

please remember mark replies answers if , unmark them if provide no help. if have feedback technet support, contact tnmff@microsoft.com

Windows Server  >  High Availability (Clustering)