It's unfortunate whenever this happens but especially depressing when you are just starting out in business.
I have, luckily, only had to deal with two clients like this over the last 4 years. The first, it took me 3 months to get paid but I had the advantage of knowing another VA who was having the same problem with this client, so we worked together to get us both paid.
The second, which was just 6 months ago, had a fairly small balance left to pay and was ignoring my emails & invoices. In my contract I state very clearly that they have 10 days to pay any invoice, after that time I charge a late fee DAILY. This client decided to send an email after he finally paid (45 days late but with the late charges) and ask if I thought it was a good business practice to charge such fees as it may turn off future clients. I had to laugh but responded nicely and thanked him for his concern. (FYI- The late fee policy has been included in every contract to every client since 2008)
You've gotten some great advice above. Look into every avenue to getting your money but, if it starts causing you undue stress it may be best to write it off as a learning experience and move on. Sometimes it's better to let things go rather than make yourself crazy.
Good Luck,

~C