If you are using are using the SSL VPN built into your IPBrick you may have some problems with slow downloads or in some cases no connectivity at all.
Virginmedia have some internal processing either at the premises router or within their backhaul network which will often manifest itself in slow download speeds or the failure to connect to the VPN either as a client of Server.
If possible I would recommend moving to an alternative provider.
Failing that the workaround which works at the time of this posting to make make sure that your SSL VPN is configured to use TCP instead of the default UDP.
Go into the menu of your IPBrick, configure
Security ->VPN->SSL, Select VPN SSL
Change the Protocol to TCP instead of UDP.
If you have already configured clients out on the internet, you will need to supply a new configuration file,