You really only need 3 things to get a website started:
A domain name:
You can purchase a domain name from multiple registrars. I used GoDaddy to purchase mine. It was $9.99 or something like that per year.
A web host:
You need a place to store your web content so people can visit your site. Unless you want to dedicate a computer at your home for your website, open the necessary ports on your router, and leave it on 24/7, you need a web host with a dedicated or shared server for your site content. I use GoDaddy for hosting for $4.99 per month (but there are several others that are just as good and even cheaper, I'm just too lazy to export all my content over).
An HTML or website editor:
Now the fun part... designing your website! There are probably hundreds of html editors out there ranging from literally only being able to edit html code to WYSIWYG (what you see is what you get) graphical interface editors. I started out using a tool called NVU. I don't think it is even in development anymore and it is very, very basic. Too basic by today's standards; but, I developed my entire website using just that tool. I did start out with a template that included a good CSS already and some stock images which made it easier to get everything set up initially.
Have fun! I find working on my website to be very rewarding, almost like painting a great picture or polishing out a swirled up car.
Regards,