I have no experience working for agencies, but the consensus seems to be that it's best to go it on your own if you're able to... 1) you'll probably get paid more, have more control over what you're earning and be able to keep those earnings for yourself (a lot of agencies don't pay very much) and 2) a lot of agencies treat you as an employee even though, technically, you are still responsible for all of your own expenses and taxes.
However, there is a lot more work involved in owning your own business because you'll be responsible for marketing yourself and attracting your own clients rather than being given assignments, so I guess it really depends on what you need and want to get out of a second job and how much of yourself you want to invest in it. One sort of "middle-of-the-road" way to go could be to become a sub-contractor to another VA. You'll still be a business owner in charge of yourself but your clients will be other VAs. You'd still have to get your name out there and build a reputation but it may be a bit easier than going completely off on your own.
As I said before, I've not dealt with anything like this and everything I've said is based on my impressions of what other people have discussed. I'm sure that other people will have totally different opinions.