The first thing you need to do is have a good idea of what kind of business you want to start and where. Do you want to have a physical storefront? Do you want to be online? Do you want both? How big do you want this business to be? I know this seems like more questions than answers, but if you start with a good plan, you will be off to a good start.
Step2
Depending on the laws in your city and state, you will need to have a business license. These are not hard to obtain, and usually don't cost much, just go to your state website and figure out the steps. It usually requires a Business name, names of all owners, business type, estimated annual income, employees, and where you will be doing business.
Step3
Depending on what kind of business you plan to start, you will need supplies and equipment. But, the most important thing is customers. Start spreading the news of your business opening, preferably months before it opens. Start with friends and family, and expand to hanging flyers, sending out emails, maybe even calling those who might be interested in your services/goods.
Tips & Warnings
Get connected with other businesses in your area who might refer their customers to yours. For example, if you are a florist, talk to local wedding planners and stores and see if you can put some brochures/flyers in their place of business.
Talk to anyone you know how may have experience in your type of business.
Don't get discouraged if things don't take off right away - it takes a while for people to see that you're there.
Have some friends and family go through your business, brochures, look for your advertising, etc. to see if there is anything that can be improved. You might miss something that they can help you with.
Don't spend money on advertising unless it is a rock-solid business. There are many scams out there, especially for web-based businesses that claim to drive traffic to your site that are costly scams.