Using a private key for SSH connections is potentially risky if the client machine is not entirely secure. To limit damage to the repository host in case of compromise of your private key you can restrict that key's use to subversion. See the SSH notes' section on 'Restricting private-key-connections'.