The error you are receiving is stating that you do not have enough free space allocated to store the backup. Unlike System State backups, BMR backups are streamed directly to the DPM server so there is no space requirement on the server being protected for a backup file. When configuring a BMR bakup, DPM does not calculate the size of the data source and just assumes 30GB will be enough. In your case it sounds like your BMT backup will be bigger than 30GB so you will need to increase the size of the replica volume.
You can increase the size of your replica volume by using the Disk Allocation page in the Modify Protection Group Wizard or the Get-DatasourceDiskAllocation and Set-DatasourceDiskAllocation cmdlets.
Hope that helps.