Hi

Since a few days i got the following error when i try to back up VMs with DPM 2010.

"Triggering synchronization on \Backup Using Child Partition Snapshot\servername failed:
Error 30148: DPM was unable to access the cluster resource group hostname.domain for \Backup Using Child Partition Snapshot\servername of protection group abcde
Recommended action: Verify that the cluster resource group hostname.domain is accessible by using the Cluster Administration console."

The Protection Group protects seven VMs. Four of them fail with the error mentioned above, the other three work fine. there are other protection groups which protect VMs without any problems. The failed VMs run on different CSVs and on different Hyper-V.

Some Information about our Infrastrucutre:

- two windows server 2008 r2 hyper-v, configured as a cluster (all updates installed)

- three CSVs

- one DPM 2010 running on windows server 2008 r2 (all updates installed)

can anyone help? i saw there is another topic for this error, but the answers there didn't help.

regards from switzerland

sandro

Hi,

Disk space is being eaten up quite quickly on my CSVs and I can't see where its going.  I think it might be VSS but how do I tell?

My environment is Server 2012 cluster with iSCSI storage on Dell EqualLogic holding the CSVs.  It is being backed up by DPM2012 SP1 RU1.

I am getting diskspace warnings from the SAN that don't match up with whats on the disk.  On a 3TB CSV, Explorer tells me there is 1.81 TB in use, Cluster manager tells me there is 777GB free (2.22 in use) but if I do properties on the volume it says 329GB free space which matches what the SAN is reporting.

How can I see what space is being used by VSS, and how can I recover some of this space, assuming it is DPM / VSS that is using it?

Cheers,

James

Does anyone know if DPM will be dropping the requirement to have an agent installed on the SMB server presenting storage shares to Hyper-V servers?

We want to back up our Hyper-V VMs and we are using a storage technology that presents SMB to the Hyper-V host but is not running windows. Since it is not running windows I can't install the DPM agent on it.

I have tried installing TP5 and testing but DPM seems to still want this agent installed on the SMB server... Any ideas?

A+, N+, MCP

Hi All,

First of all, I know it's a beta and these are the perils of being an early adopter, but I've got a serious problem.

I've upgraded our production Hyper-V cluster to Server 2012. The setup is a 4 node cluster running CSVs on an ISCSI SAN with MPIO via dual gigabit Ethernet networks. The SAN storage is provided byOpen-E DSS7 and replicated to another server in a different building.

Post the upgrade everything about the cluster seemed stable and to work as expected - live migrations etc all working. I then turned my attention to backups, and I discovered that Server 2012 wasn't supported by DPM. Fortunately there is a beta of DPM 2012 SP1 which adds support for Server 2012, unfortunately there is no upgrade path from the beta to RTM of SP1. Not wanting to upgrade our production DPM server to a beta, I installed a copy of DPM 2012 SP1 beta on a VM to provide a stopgap backup solution for VM level backups of certain machines that couldn't be backed up in other ways. I realise that running the backup server on the same cluster / SAN as the stuff that's being backed up is an odd thing to do, but this at least serves to provide snapshots, SAN replication provides resilience, and like I say, this is a stopgap.

Then I started noticing problems. First symptom was that on starting / rebooting VMs, sometimes other VMs would hang for perhaps 30s - 2m, people would start complaining that SharePoint had gone unresponsive etc. However, they would come back to life in a minute or two.On a couple of occasions we came in in the morning to find a number of VMs off or paused (backups ran overnight). Both of these problems occurred only when the DPM server was turned on. I thought the issue might be general load on the SAN, having both the backup server and the machines being backed up living on the same CSV / hardware. I moved the DPM server to a different ISCSI box and put on aggressive throttling (200Mbps) to try to reduce load, but the problem continues.

The event logs on the Hyper-V cluster suggest I/O timeouts to the SAN at the times of the backups. Lot's of event ID 1069, 1205, 1146, 1230,  (various cluster resources failed). The interesting one I think is 5120 Cluster Shared Volume 'Volume5' ('VOLUME NAME') is no longer available on this node because of 'STATUS_CLUSTER_CSV_AUTO_PAUSE_ERROR(c0130021)'. All I/O will temporarily be queued until a path to the volume is reestablished.

Is anyone else using SP1 beta to successfully backup a 2012 Hyper-V cluster? 

Is anyone seeing the same problem?

Is it likely that this is a problem with SP1 beta, will it be fixed at RTM?

Any suggestions for a stopgap solution?

I think I might try setting up a test physical DPM server to check the issue isn't in someway related to the fact that the DPM server sits on the same cluster it's backing up. I'm also happy to consider the problem could lie elsewhere i.e. with the SAN storage (this was upgraded from v6 to v7 at the same time as the 2012 upgrade, but as soon as I tell the vendor that the problem relates to running a beta of DPM they will be pointing fingers at that.

Thanks,

Tim

Does anyone have any tips for making my Azure Online Recovery Points take less time to complete?  I have some that take on average about 8 hours.  How can I make these shorter?  I imagine I need to create a better scheduling model for my protection groups, but I could use some advice.

Things to know about our env.:

• We have a fast site-to-site connection with Azure of 1 Gig.  Actual throughput is around 500 mbit up/down.
• We have our DPM servers virtualized in Hyper-V.  Each has a minimum of 8 GB of RAM, 4 CPUs, Win 2012 Std.
• We use Clustered Shared Volumes.
• Protection Groups like this one (shown below) creates an online recovery point every 24 hours. Some PGs are creating Hyper-V recovery points and others are creating SQL recovery points. I am not sure if that matters.  The below pic is filtered on one job that historically takes a long time.

Any advice is appreciated.

David Jenner IT Systems Engineer Colonial Williamsburg Foundation

Hi,

I've had issues with ILR on my DPM box for a while now. Its comes up with the eroro stating about dynamic volumes, no volumes, DPM role or auto mounting not being enabled. The ID is 958 and the error code is 0x80990F4E.

What I have found is that it wont do a ILR mount on a server with more than one disc. I have a couple of servers where I've stuck the data on a different VHDX and these aren't viewable in the recovery console. Single VHDX servers go in fine.

When I mount a single VHDX server I get in the event log

I don't get any log entries on servers with the issue.

Looking at the DPM  logs when it fails I see

15:28:46.345 22 nativevhdprovider.cpp(174) [0000000000A26A60] 99303538-624D-4F25-AB30-45F6D758E77D WARNING Failed: Hr: = [0x80070013] : c:\Program Files\Microsoft System Center 2012 R2\DPM\DPM\DpmVhds\bf56ca19-a372-4f2b-94fc-c0cfd3e861b5.diff.vhdx: lVal : pfnCreateVirtualDisk(&storageType, ssDiffVhdPath.PeekStr(), VIRTUAL_DISK_ACCESS_CREATE, NULL, CREATE_VIRTUAL_DISK_FLAG_NONE, 0, &createParam, NULL, &hDisk)
188C 2088 05/11 15:28:46.345 31 hypervvhdhelper.cpp(155) [00000000009C26A0] 99303538-624D-4F25-AB30-45F6D758E77D WARNING Failed: Hr: = [0x80070013] : m_pVHDProvider->CreateDifferencingVhd() path:[c:\Program Files\Microsoft System Center 2012 R2\DPM\DPM\DpmVhds\bf56ca19-a372-4f2b-94fc-c0cfd3e861b5.diff.vhdx], parent path:[c:\Program Files\Microsoft System Center 2012 R2\DPM\DPM\DpmVhds\bf56ca19-a372-4f2b-94fc-c0cfd3e861b5.vhdx]
188C 2088 05/11 15:28:46.345 31 hypervvhdmanager.cpp(340) [0000000000A32A90] 99303538-624D-4F25-AB30-45F6D758E77D WARNING Failed: Hr: = [0x80990e52] : CreateDifferencingVhd: ssParentPath [\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy389\76b21287-0f74-4580-bb7a-eb1f55c32a6c\Full\D-Vol\Email\Exchange2013\Skinner2_BEDFAB9E-DD04-417E-BE5C-8E042B21F78E.avhdx]
188C 2088 05/11 15:28:46.345 31 vhdmanager.cpp(270) [0000000000A32A90] 99303538-624D-4F25-AB30-45F6D758E77D WARNING Failed: Hr: = [0x80990e52] : Encountered Failure: : lVal : MountVhdInternal(ssVhdIdentifierInCache, ssParentPath, psErrorDetail)
188C 2088 05/11 15:28:46.345 61 vhdutils.cpp(78)  99303538-624D-4F25-AB30-45F6D758E77D NORMAL arrVhdChain[0] is [\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy389\76b21287-0f74-4580-bb7a-eb1f55c32a6c\Full\D-Vol\Email\Exchange2013\Skinner2_BEDFAB9E-DD04-417E-BE5C-8E042B21F78E.avhdx]
188C 2088 05/11 15:28:46.345 61 vhdutils.cpp(78)  99303538-624D-4F25-AB30-45F6D758E77D NORMAL arrVhdChain[1] is [\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy389\76b21287-0f74-4580-bb7a-eb1f55c32a6c\Full\D-Vol\Email\Exchange2013\Skinner2.vhdx]
188C 2088 05/11 15:28:46.345 61 inquirysubtask.cpp(1222) [0000000000A2C160] 99303538-624D-4F25-AB30-45F6D758E77D WARNING Failed: Hr: = [0x80990f4e] Ignoring processing error, processing continues for other nodes
188C 2088 05/11 15:28:46.345 61 inquirysubtask.cpp(1705) [0000000000A2C160] 99303538-624D-4F25-AB30-45F6D758E77D WARNING Failed: Hr: = [0x80990f4e] CInquirySubTask::CreateAndLogError::Adding inquiry error for component
188C 2088 05/11 15:28:46.345 61 inquirysubtask.cpp(1031) [0000000000A2C160] 99303538-624D-4F25-AB30-45F6D758E77D NORMAL <--CInquirySubTask::ExecuteInquiry
188C 13FC 05/11 15:28:46.345 61 inquirysubtask.cpp(458) [0000000000A2C160] 99303538-624D-4F25-AB30-45F6D758E77D NORMAL Sending final response with 1 records
188C 13FC 05/11 15:28:46.345 61 vhdutils.cpp(51)  99303538-624D-4F25-AB30-45F6D758E77D WARNING Failed: Hr: = [0x80990f4e] : \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy389\76b21287-0f74-4580-bb7a-eb1f55c32a6c\Full\D-Vol\Email\Exchange2013\Skinner2_BEDFAB9E-DD04-417E-BE5C-8E042B21F78E.avhdx: lVal : E_COORD_HYPERV_VHD_MOUNT_FAILED
188C 13FC 05/11 15:28:46.345 61 nevhd.cpp(124)  99303538-624D-4F25-AB30-45F6D758E77D WARNING Failed: Hr: = [0x80990f4e] : Encountered Failure: : lVal : VHDUtils::MountAndEnumerateVHD(ssId, arrVhdChain, arrVolLabel, arrVolGuid, arrVolId)
188C 13FC 05/11 15:28:46.345 61 nevhd.cpp(61) [0000000000A2BBA0] 99303538-624D-4F25-AB30-45F6D758E77D WARNING Failed: Hr: = [0x80990f4e] : Encountered Failure: : lVal : EnumerateVHDChildren(m_path, traversingReplica, callback)
188C 13FC 05/11 15:28:46.345 61 inquirysubtask.cpp(990) [0000000000A2C160] 99303538-624D-4F25-AB30-45F6D758E77D NORMAL CInquirySubTask::Inquiry finished with status [0000000000]
188C 13FC 05/11 15:28:46.345 03 workitem.cpp(296) [0000000000A2A880] 99303538-624D-4F25-AB30-45F6D758E77D ACTIVITY WorkItem stopping
188C 13FC 05/11 15:28:46.345 31 vsssnapshotrequestor.cpp(122) [0000000000A2C4A8] 99303538-624D-4F25-AB30-45F6D758E77D NORMAL CVssSnapshotRequestor::~CVssSnapshotRequestor [0000000000A2C4A8]
188C 13FC 05/11 15:28:46.345 31 vsssnapshotrequestor.cpp(2182) [0000000000A2C4A8] 99303538-624D-4F25-AB30-45F6D758E77D NORMAL CVssSnapshotRequestor::CleanUp [0000000000A2C4A8]
188C 13FC 05/11 15:28:46.345 31 vsssnapshotrequestor.cpp(1768) [0000000000A2C4A8] 99303538-624D-4F25-AB30-45F6D758E77D NORMAL CVssSnapshotRequestor::ReleaseVolumesForSnapshot [0000000000A2C4A8]
188C 13FC 05/11 15:28:46.345 31 vssbaserequestor.cpp(69) [0000000000A2C4A8] 99303538-624D-4F25-AB30-45F6D758E77D NORMAL CVssBaseRequestor: destructor [0000000000A2C4A8]
188C 110C 05/11 15:28:53.896 22 vdshelper.cpp(923) [0000000000A2D530]  WARNING Failed: Hr: = [0x80990fb0] Disk [\\?\PHYSICALDRIVE7] is not found as unknown
188C 110C 05/11 15:28:53.896 22 vdshelper.cpp(1892) [0000000000A2D530]  NORMAL Disk Name, fIsDiskFound = Port0Path0Target0Lun0, 0x0
188C 110C 05/11 15:28:53.896 22 vdshelper.cpp(1892) [0000000000A2D530]  NORMAL

It was working fine up until a couple of months ago.

Anyone got any ideas? I really hope it wasn't the new Anti-Virus I installed

Hi,

we are currently implementing DPM with the Hitachi VSS Hardware Provider. We have VSP Storage for our Hyper-V Cluster and AMS Storage for the Backup Data. The AMS Storage is presented as path-through disks to the DPM VM. When we run more than one backup in parallel we see the following error:

"DPM encountered a retryable VSS error. (ID 30112 Details: VssError:The writer's timeout expired between the Freeze and Thaw events. (0x800423F2))"

The backup will be started again and succeeds. Has anyone came accross this issue?

Regards Alex

Hi,

I have some VM's with dev databases, which not need to backup.

I run set-DPMGlobalProperty -DPMServerName dpm.mydomain.local -HyperVPagefileExclusions "Sql_server_E.vhdx"

if run: 

Get-DPMGlobalProperty –PropertyName HyperVPagefileExclusions

I get good results, but in DPM server recovery points I see this vhdx is backuped

Servers info:

DPM 2012 R2

Host - W2012R2

VM - W2012R2

I have DPM 2012 R2 running on server 2012 R2. Dpm backup vms through Hyper-V host. Some of the vms have this additional vhd drive with name "_DoNotBackup.vhd". Used this command Set-DPMGlobalProperty -DPMServerName dpmServer-HyperVPagefileExclusions"_DoNotBackup.vhd"  few days ago. Backup occurs very night. Today I restored one of the vm which has "_DoNotBackup.vhd" and found it has latest data which means HyperVPagefileExclusions not working.

Any clue?

We have Hyper-V 2012 R2 hosts and have been backing up Ubuntu VMs for a while.

We recently added a ubuntu 16.04 VM and the backups have failed since day 1.  Both full backups to tape and recovery point creation to disk are failing.

We upgraded another Ubuntu machine that was backing up fine and as soon as it was upgraded to 16.04, it started failing too.

Is this a known issue with this version of Ubuntu?

Thank you.

Hi,

I am currently configuring DPM 2012 R2 in a hosted environment. As part of this I am trying to test the recover of a Hyper-V VM to a different physical host from its original location using the "Recover as virtual machine to any host" option. However when using this option all it seems to do is move the virtual machine from 1 physical host to another and does not apply the recovery point.

Current environment configuration:

3 x Physical Server hosts (all Windows Server 2012 R2) configured as a failover cluster (inc storage)

Hyper-V 2012 R2 role installed on all 3 hosts

System Center VMM 2012 R2 used to manage Virtual Machines

Steps used in the process are as follows:

1. In the DPM console select the Recovery option

2. Browse to the VM you want to restore

3. Expand the VM node and select "All Protected Hyper-V Data"

4. Select Recoverable item in the Path list

5. Right-Click and select "Show all recovery points"

6. Select the recovery point to restore and click Recovery

7. On the Review Recovery Selection page click Next

8. Select the "Recovery as virtual machine to any host"

9. Click Next

10. Click Browse

11. Select the Hyper-V physical host and clustered storage location for where the VM is going to be recovered to.

12. Click Ok.

13. Click Next

14. Click Next again

15. On the Summary page click Recover

16. The recovery process with proceed and restore the virtual machine to a different physical host but will not recovery the virtual machine as per the recovery point.

Can anyone please confirm if I am using the correct process or whether I am doing something obviously wrong in this process?

I'm running a 3 node Hyper-V Cluster with approx. 25VMs and have an issue backing up 2 VMs. - All other VMs are backing up without issue.

The two VMs will show as replica inconsistent and will not perform a consistency check unless I manually create a snapshot/checkpoint within Hyper-V Manager.  Once created DPM will happily perform a consistency check and backup up the 2 VMs.

If I remove the snapshots/checkpoints - DPM will fail on the next scheduled backup and show the VMs as inconsistent again.

Anyone any idea?

Hi,

I have found that there is a script called AddNewClusteredVM.ps1, but this is for DPM2010. We have DPM2012, does the script work on 2012 also or is there another way to automatically add new VM to protectiongroups ??

Hi,

I am encountering an error when trying to backup a VM in SMB share. The error was (ID 2033 Details: A required privilege is not held by the client (0x80070522)). 

There is no VSS event logged on the host.

Does anyone face this error before? Appreciate if anyone could help to enlighten.

Thanks.

Hi guys,

I've been fighting for a long time (almost 6 years in multiple clients) with no luck to perform host level backups using SCDPM and a Hyper-V cluster without crashing everything. I've already opened a thread on this forum that can be reach athttps://goo.gl/qgz435

I've already tried multiple alternatives (hardware snapshots, softwares snapshots, serial backups, backup vms on the same CSV, and so on) ... In the end I always have a broken Hyper-V cluster no matter what I do ... some vm's don't start correctly, other vms loose configuration, others won't boot at all (no os found) ... a lot of time spent in support calls and last but not least a lot of client's criticisms about DPM (even the latest version 2k12R2 with UR 9) not being able to perform such a simple thing like a host level backup of a three win2k12 r2 cluster completely patched and using the best practices (management network, vm network and so son, iscs network, and so on)

I am really curious if someone is succeeding to perform this kind of thing without a crash and without a single event on the cluster manager, when I say crash I mean terrible things like vms that were booting not recognizing the c: drive anymore, vms that I need to delete snapshots because they can't start at all ... and so on.

My last attempt to perform a host level backup finished with:
1069 - error code 0x2 (system could not find the specified file)
5142 - The cluster shared volume could not be accessed by this node due to error 1460
1146 - the cluster RHS was finished and will be restarted
1230
5120
1254
and so on

Could someone share some experience?

Thank you very much!
--marcos

We recently purchased a Compellent SAN.   Are there any known conflicts or issues we should watch out for?

We are having strange issues when DPM backs up at night.  At times VMs seem to randomly lose connection.  Most nights at least 1 or 2 VMs get a disk error and have to be turned off the next morning and back on again.  Some nights this happens with many VMs.  One night we even had a VM get hung backing up and it froze the Hyper-V host it was on.

Any ideas what could be causing these issues?  Does DPM conflict with replays (compellent snapshots)?  Or is it something else?

One thing I did notice is that pretty much all the VMs are being backed up using Saved State rather than Online Backup.  I am not sure why this is.  They used to always use Online Backup.  Could that be causing issues?

Any thoughts or advice would be helpful.

Thank you.

Hello,

I'm having an issue when trying to backup to tape a Hyper-V guest VM residing on a Server 2012 R2 host by using DPM 2010 running on a Windows Server 2012( not R2) host. I am able to install the agent on the Hyper-V host server but when trying to add it to a protection group DPM displays "Offline\Hyper-V Name." When the backup runs it fails and gives no specifics. As far as I can tell, no event errors are created on the Hyper-V server or either of the host servers. The SCSI controller is set up in the guest VM and the Backup volume checkpoint is enabled.

If I install an agent on the guest VM itself instead of on the host server I can backup with no issues.

Does anyone have any idea why DPM reads the Hyper-V as "Offline\Hyper-V Name" or why I can not backup from the DPM agent installed on the host server?

We have both standard and enterprise DPM licenses available.

Any help is appreciated! 

Hi folks,

I've just setup a DPM 2012 R2 server on a Windows 2012 R2 server in order to replace our DPM2010 server!

Among the protected servers I need to backup, I've got a Hyper-v 2008 R2 failovercluster with 2 nodes (HYPER-A and HYPER-B servers)

I would like to protect each node with a Bare Metal Recovery (BMR) Backup.

In the protection group dedicated for BMR backup, I'm able to check the BMR option for the HYPER-B server Under system Protection but for the other server hyperv-A, I don't see any selection possible  for System Protection (neither for HyperV-> Initial Store)

AS you can see on the screenshot, BMR is not available for hyperv-A ?

How do you explain this?

Is there a particular action to do?

Thanks for the feedback

PS: HYPERV-A server is the current owner of the Cluster Shared Volume (CSV)

Hi

I have a DPM 2012 R2 server which is backing up a Virtual Machine. (Hyper-V Host based backup)

The Virtual machine is Windows 2012 R2 and has Exchange 2013 installed.

There is also an in guest Exchange databases backup performed by Evault. 

The issue I am facing is as part of the DPM VM backup it is truncating some of the exchange logs which causes the in guest  Evault exchange backup to fail as it doesn't have the full set of logs to work from.

I can see event logs within the VM being logged with log file being deleted as part of the dpm vm backup:

Event 224 ESE, Information Store - Exch2013Db01 (7204) Exch2013Db01: Deleting log files L:\ExchangeLogs\Exchdb01Logs\E00000E6E5C.log to L:\ExchangeLogs\Exchdb01Logs\E00000E6EB9.log

Question: How can I disable / stop a DPM VM based backup truncating exchange logs?

There are some referances on the internet to 2008 guests and creating and configuring registry settings which do not work for 2012 r2 VMs. https://social.technet.microsoft.com/Forums/windowsserver/en-US/8fa24259-e1bb-4cb0-b831-d42e1a720fdc/how-to-avoid-exchange-server-logs-truncation-when-hyperv-guest-os-gets-backed-up?forum=winserverhyperv

Thanks

John.

Hi,

I need to add the new VM added to my primary DPM server on my secondary DPM.

When I clic on the new VM already backed up on the primary DPM, I get the following error :

"DPM was unable to get the list of data sources on *****.Cluster.mydomain.local that are protected by the primary DPM Server.

Make sure that myprimarydpm.mydomain.local is protecting the the data sources for which you want to set this DPM server as the secondary server. If the problem persists, restart the DPMWriter service on myprimarydpm.mydomain.local.

ID: 33119"

Any idea ?

Bruno Capron - Administrateur systèmes & réseaux Microsoft MCSE/MCSA 2003 - MCITP-EA 2008