Hey Collette,
As a VA that subs for others, I'll try to give you my perspective and what I do when a VA is having a hard time handing work off.
First off, one can usually tell by what the VA says as to how hard it is for them to outsource to a sub. Most are upfront about their fear or control issues, but you can also tell by their reluctance, too, or stating that "I just want you on board for future projects." Hear that quaver in their voice and they sound on the verge of tears? They're scared spitless.

(Just kidding with that last part - just a bit of my weird humor

)
The two reasons I have found (mentioned above) is that it is usually fear- or control-based issues. What I do is give them reassurances, and regular updates on the work that they DID hand off to me. By communicating and giving regular updates, I have found that to help with the issue of fear of the situation. I also offer to do a "test" project (paid, of course), for them to get a feel as to how I work and to show that I do top-notch, quality work. (I'm a perfectionist!) If it is a control issue, I just hang in there and be patient and let them know that when they are ready to perhaps just give me a small task to do at first, as in "baby steps, please or I'm going to panic!"
Also, when first contacted to sub, there is always the "get to know each other" phase, so developing a rapport beforehand can help. With the fearful or control oriented contractor, this seems to help with the not-sure-I-want-to-do-this contractor and we do have more than the "normal" amount of back and forth before work gets handed off. Just jumping in with a task without getting to know each other hasn't been my experience and it seems to calm the skittish, apprehensive VA contractor by interacting even more than usual. After all your interactions if you don't feel comfortable, for whatever reason, try another VA; you'll find your perfect "fit" that could make all the difference in the world. Who knows, your fears might just melt away, and you'll wonder why you were apprehensive to begin with.
Tell the VA what you need from them, let them know that you may need more interaction to feel comfortable, that you are apprehensive, are not sure what you can hand off, and any other things you are feeling - a good VA sub will help to allay your fears, and s/he will not make you feel foolish for feeling as you do.
It's getting late, so I'm not sure if this helped to have the "other side of the fence" perspective, or if I articulated what I wanted to get across in a cohesive manner, but hopefully this will help a bit.
P.S. To potential VAs that are thinking about subbing out: Don't be skeered, it will be alright!