My first thought in reading your post is..."Can you take them on as your first client?" Find a way that it would benefit them and sell your services to them...of course, I'm assuming that your current position entails something that you would perform in your VA practice.
On the other hand, an age old saying going something like this "Don't quit your current job, until you have another one to take it's place", if you can plan it out that way. I would start small if it's not an urgent reason why I'm looking to leave...if you can launch your practice and work for another 3-6 months (full-time) until it can
at least replace what you are bringing in now, then all the better. But you want to build your clientèle first.
That's just my 2 cents (and what I'm currently doing

).