It does take time! But as with anything the more you put into something the more you get out of it.
Link your Twitter to your profile here so people can easily find you on there ( I just read this post and went to your site to find your twitter username).
One thing that you can do to get the interaction going is to really read someone's time line. Pick something specific out of there and send them an @ about it. Read their bio's and ask them a question about where they live or something within their bio.
Check out some of the lists out there and begin to follow those people to build up your following. VAs are some of the most friendliest ones out there and find that most will return your follow no problem.
BTW I RT'd your Question of the Day! Good one!