There are many cases when you get this type of error.
1. When you are access the SharePoint site using console application, from client machine. Means, for example, you have installed SharePoint on Server1, and you are developing your console application on your machine, let's say, devmachine1. This cause the error, because, using object model to access the site, your code needs to be on the same machine where sharepoint is installed.
2. You have not mentioned that which SharePoint version you are using. If you are using SharePoint 2007, then first point is relavent. If you are using SharePoint 2010, then you must have to use administrator privileges while running your console application. Otherwise SharePoint 2010, do not allow you to access the site and throws exception which you write. Also, consider point 1 in this case as well.
To, overcome this, for SharePoint 2007, you can use Web Services if you are developing application on other machine, and if you are using SharePoint 2010, use client object model.