If you aren’t handing out business cards and expecting them to type in the site name than it is not such a massive issue. Most of the traffic I get is either from searches or from online promoting and in all cases my website visitors just extensions hasn’t had a bad effect on my business.
Take the time to build your web presence, do not get bamboozled into assuming that going the ‘free’ route with a blog is the easiest way to go.