As a rule, startups operate with tight budgets, but this shouldn’t justify picking the cheapest option on the market. If you want your site to start making it big from the get-go, avoid shared packages and go off with a VPS plan instead. Shared plans suck big time in terms of speed, security and everything else, except for how cheap they are.
Important things to consider
First, you are probably going to tweak a lot around your site’s front end and back end since you are just starting it up. So, gauge the amiability and diligence the customer support team is ready to offer.
Second, you should determine the actual technologies you’re going to use for your startup, be it WordPress, PHP or Django. Make a list and ask the candidates whether they have a special plan that will suit you best. Picking a specialized plan will promise higher load speeds and easier development.
Third, look at the actual server machines being used. It’s these guys that make your site slower or faster, not marketese of your host’s site.