A deposit and a signed contract are essential before you start work. Along with covering yourself to some degree, your insistence on this procedure shows the client that you're a professional business person.
As an example, I recently acquired a new retainer client, a small law firm some 500 miles away. After the initial contact they wanted to start sending me work right away ...

. As, for the number of lawyers in the firm, in my view they're overstaffed this was a bit of a red flag. I emailed them a client profile questionnaire and a contract for signature. Two days later I received the signed contract and "where/how would you like us to send you the first month's $".
Although a signed contract must be had, it's really not that much of a safeguard as, if the client turns out to be a deadbeat, trying to enforce it will cost you far more than you're owed. The $ up front is the essential.
Although I have a PayPal account, I detest it and only use it to make contributions to charity. I had some concerns with regard to security but, after speaking with my bank manager, I use direct deposit to my business account (no extra charge from my bank). If it continues to work well then I'm avoiding PayPal fees and the headaches of chargebacks and disputes. Over time I've also heard that PayPal can freeze your account on a whim and it's a nightmare trying to get it released.
Lynne