Can I continue to collect unemployment while setting up a company?
I see no reason that this should preclude you from "setting up a company", but, it all depends if you are engaged in a trade or business for a profit purpose! Clearly, if you were going to have some business transactions, my thinking would be that you would have to declare to your State Unemployment office that you are receiving some income. They would then have to determine whether your eligibility to continue receive unemployment benefit should continue or not!
It would be best to contact your State Unemployment Office to determine precisely whether or not you would qualify to receive these benefits based on your income earned.
And which would be best. S-Corp? C-Corp? LLC?
Based on the limited information you have supplied, one would think that you a LLC would be more appropriate to meet your needs as it would offer the same legal protection as an S or a C corporation, but in addition, in some States the S or C Corporation's minimum taxes are substantially higher than the LLC's minimum taxes.
Since, you would be sensitive towards spending monies whilst unemployed, I would steer you towards forming an LLC entity.