Chaley, have you downloaded the free getting started as a virtual assistant packet? Use the search feature in the main menu to track it down, that's a great place to start.
It sounds like you sort of got things started but gave up without any really consistent marketing efforts. Having a website and brochures is a really good start but you have to define your target market and then actively find and network with those people, get in touch with them, etc.
The bare minimum for marketing should be one hour a day - but really, if you anticipate 4 or 5 billable hours a day then you should be spending that much every weekday on developing your business, marketing your services, and getting in touch with your ideal clients in some way (at least until you have a full practice - but even then you should still be marketing every day at least an hour). It doesn't have to be the same marketing efforts every single day, in fact, you really should try a variety of methods for best results. Check out the
Promoting Your Virtual Assistance Business section of the forums - it's full of thousands of ways you can market your business as well as thoughts and ideas on networking locally and online.
If you really want it, you can absolutely make this happen - just don't give up and don't get mired down in things like a business plan. Your business plan doesn't have to be a lengthy, formal document (unless you're approaching a bank for financing) so let your plan be more organic for now, use a notebook to keep your ideas and policies organized.
There is no need to get it all perfect right out the door - some of this is bound to be a learning experience, so don't be too hard on yourself or expect too much right away.
Good luck!