One of the new eagerly anticipated features of DPM 2010 is the ability to protect machines that are in a workgroup or in an untrusted domain. In the DPM 2010 Beta this feature was not available, however it is available in the DPM 2010 RC and it is very simple to configure.
First of all you need to install the DPM agent on the machine you want to protect. I got the agent installer from the Agents folder on the DPM 2010 install media and just ran the relevant architecture executable, in my case DPMAgentInstaller_x64.exe. You do not need to anything to install this agent, just double click the EXE and wait for the message to tell you that the agent was installed successfully. You then might need to reboot.
Once the agent is installed it needs to be configured. This is done using SetDPMServer.exe which is located here, <installation drive>\Program Files\Microsoft Data Protection Manager\DPM\bin. Run SetDPMServer.exe with following arguments, you need to specify the DPM server name and a user name for a new local account to be created.
SetDPMServer.exe -DPMServerName <DPMServerName> -IsNonDomainServer -UserName <NewUserName>
You will be prompted to give a password for the new user account and to confirm it, once done the agent is configured and the relevant changes have been made to the Windows firewall.

Now the agent needs to be attached to the DPM server. Attaching DPM agents to a DPM server is much simpler now as it can be done through the DPM GUI as I posted about previously (http://www.scdpmonline.org/dpm-2010-ndash-attach-manually-installed-agents.aspx). To attach a non-domain server agent simply open the DPM console and select the Management pane, click Install... from the Actions menu. When the Protection Agent Installation Wizard opens, select Attach agents and Computer in workgroup or untrusted domain.

Then enter the name of the untrusted computer that the DPM agent is installed on and the user name and password specified when running SetDPMServer.exe. Click "Add >" to add the computer to the selected computers list.

Then simply click Attach !

You should then receive a summary window advising that it attached successfully and that's it, done! With the agent attached you can create a protection group to protect the data on the machine in the usual manner.
Told you it was easy 