I was at a client site yesterday in the process of moving them off a NetApp Storage array and onto EMC Storage.  My colleague and I ran into a very interesting problem and have no idea why it was happening.  If anyone has any ideas please, do share.

The setup was two Windows 2008 Servers with Exchange 2007 CCR split between two datacenters.  The servers were using Snap Manager for Exchange and Snap Manager.   There were four LUNs from the NetApp presented, three for databasae and one for the logs.  The LUNs were attached via iSCSI. 

Since the Exchange CCR environment was running in the HQ Datacenter we started by getting the EMC agents installon the the Server in the DR Datacenter.  The iSCSI LUNs from the NetApp were presented via Drive letters, there were four, three for the database and one for the logs.  If you’re not familar with deploying EMC apps the first app to deploy is the EMC NaviHostAgent.  When I installed the agent the server requested for a reboot.  This is where things got really funky.  When the server came back from a reboot the iSCSI disk drives from the NetApp were gone!  No idea why.  We scratched our heads and couldn’t figure out what happened.  According to the NetApp management console the LUNs were presented.  When we tried to remap the LUNs via Snap Manager it would state that the volume was already mapped to a drive on the server, but the Server could not see them!  This was odd.  We backed out the EMC NaviHostAgent and the LUNs reappeared but the drives were off line.  I brought the drives back on line and everything worked again.

We then started by reinstalling the EMC NaviHostAgent, no reboot was required this time and installed the EMCPowerPath application.  When we came back from a reboot the drives from the NetApp were off line again, I brought them back on line and had both the EMC and the NetApp storage provisioned.  Great, this is nice.  We presented the EMC LUNs as mount points and started up replication again from the HQ Exchange Server.  Once replication was caught up moved the cluster to the DR server to begin the same process on the HQ server thinking we had a fluke.

Well, the exact same thing happened!  When we installed NaviHostAgent no reboot was required.  We then installed PowerPath, Had to reboot the server and the NetApp LUNs vanished!  WTF right?  Not again!  We did the same procedure I mentioned above, removed PowerPath, removed NaviHostAgent, rebooted the Server in HQ.  When it came back the NetApp LUNs were gone!  No drive letters, no detection.  Checked the iSCSI Initator configuration, and it was perfect, it was seeing the iSCSI targets for both the NetApp and the EMC.  From the NetApps perspective the LUNs were provisioned.  What in Windows 2008 was preventing the LUNs from being seen?  I checked the Network settings via IPCONFIG and there was a “Local Area Connection 9″ that was there along with a Prod network and a iSCSI network.  Looking at the network configuration only two NICs were active for the machine.  No idea where the third NIC was coming from and by this time I was too tired to try to figure it out.  Anyone out there have thoughts on that one? 

Anyway, after troubleshooting for about 5 hours between the two servers I proposed a Plan B.  Since we had the NetApp storage (Thank God) and the EMC storage on the DR server I could create new Storage Groups and Databases on the DR server to the new EMC LUNs.  Create new EMC LUNs on the HQ Exchange Server to reflect what was in DR and let CCR take care of the rest.  Once the Storage Group and Databases were created I enabled replication only for the EMC Storage Groups (since the NetApp was off line on the HQ Server I couldn’t re-enable CCR) and started moving mailboxes.  This worked fine, disaster averted. 

I was glad Plan B worked, but more complexing is why did the NetApp LUNs disappear?  Anyone have an idea what may have happened with it?  Again this was a Windows 2008 Server with Exchange 2007.  The iSCSI network was correct, the NetApp was presenting the LUNs, when going into snap manager the LUNs said they were mapped.  I tried to do a rescan of disks, nothing.  Tried to take away the LUN and remap the LUN via NetApp management console and nothing worked.  The Windows Firewall was turned off, disabled AV Software, nothing worked.  Nothing would bring those LUNs back to the Windows 2008 server.  Has anyone seen anything like this?  Anyone have an idea on what happened?  Please do share!

Thanks,