I've not set up a studio proper, but I have done some recording, and have spent time haunting various forums covering recording and recording gear, watching and learning from the discussions there in order to develop a plan of what to implement when I payed off enough debt to have the time to build a proper studio. Do take what I say with a grain of salt, and spend your own time doing research; be it haunting various recording forums or reading various books on the subject (
Guerilla Home Recording: How to Get Great Sound from Any Studio (No Matter How Weird or Cheap Your Gear Is) is a recommended one). That said, here's my tips.
First off, you don't need to spend thousands of dollars in recording gear, be it a dedicated computer, digital/analog converters, microphones, preamps, signal processors, or the like. What do you need? A microphone, a preamp, a converter (to get the sound in and out of your computer, and monitors.
A simple USB/thunderbolt/firewire audio interface is plenty to get you started. These contain microphone preamps, line inputs & outputs, digital converters and a connection to get the sound in and out of your computer, all in one simple convenient package. Look for a USB audio interface from Mackie, Focusrite, Presonus, M-audio, or other brands in the same price range. These can also be found used on eBay, Craigslist, in the classified sections of recording forums, etc. (An aside, avoid Berhinger. They may be inexpensive, but their build quality suffers for it, and in light of that, a lot of their products actually aren't that great of a buy for the money). I am partial to Focusrite myself -- I've been using a Saffire Pro 24 for some years and have been pleased with it. Which one do you need? That mainly depends on how many things you plan to record at the same time. Even a drum kit can be recorded wtih few mics, (as little as 1!). If you know you aren't going to be recording more than a couple sources at once, go for one of the smaller models. You can always get another bigger model later.
The microphones that humbleMouse recommends are good, and inexpensive, and common enough that they can be had for cheap used.
Now to monitors. The purpose of monitors is to be a neutral, honest speaker that plays back what was actually recorded, whether it sounds good or not. By contrast, the speakers in many consumer setups are designed to sound nice, whether that's by hyping up the bass to make it sound thumpy and huge, sweetening the highs, pulling back the mid-range so it isn't as overbearing, or anything in between. If you record and mix your music on a set of consumer speakers, your recordings may well sound good on that set of speakers, but may not sound good on anything else, because the recording and mixing decisions you made on those speakers are only valid for the way that particular set of speakers colors the sound. Where if you made your recording and mixing decisions to sound good referencing a speaker that was sonically neutral, you can be reasonably confident your recording will sound good on other speakers as well.
You can get decent monitors in the $150-300 per speaker range new, cheaper used. I have a pair of JBL LSR308 that I like. They, and their smaller cousins, the LSR305 are frequently recommended on various recording forums as a great deal for the price. Other commonly recommend speakers in that price range are the Yamaha HS5 and the HS7. But please, do not take my word or anyone else's word for it. Take some recordings you like, and some you really don't like, and head down to Guitar Center or other place that carries pro-audio gear and audition them on various monitors. Listen for detail in the recordings, see if you can pick out what it is exactly about the recordings you like that makes you like or dislike them. Pick the one you think will help you make the best decisions on how to make things sound good because the *sound* sounds good, not the speaker. Another thing to keep in mind, as that many of the places that sell monitors have a 30 or so day return policy. So, you can take the time listening to various models, then return the ones that don't work for you. Obviously, this doesn't work so well with used stuff from eBay.
This leads me to another bit that doesn't get as much discussion as gear does: room treatment. The room that you're recording and mixing in can have quite an influence on the sound. Ever been in a small space, and found a note that resonated way louder than others when hummed or played over a speaker, or perhaps heard a short ringing echo to sounds when that room was mostly empty or had a lot of hard, flat surfaces in it? Same thing happens in the spare bedrooms and basements that people set up their personal project studios in, and even in the big professional studios. Sound, being vibrations in the air, bounces off of walls and hard surfaces like tabletops, and those reflected echos can then come back and interact with the original sound, cancelling some bits of it, and strengthening other bits of it, and then you're hearing the room as much as you are the original sound. Ever dropped a stone into a pool or pond and watched the waves ripple outward, then bounce off the edges to collide back with the original waves, creating new, more complex ones? Very much the same idea. When you're trying to record and mix, you're going to have an easier time of it if your basing your decisions on what the sounds themselves sound like, instead of what the sounds sound like, plus how the room is coloring them.
Good news, you don't have to spend thousands of dollars on room treatment buying fancy foam pads or absorbing panels -- you can DIY this stuff. One of the more popular methods is to buy semi-rigid fiberglass or rock wool insulation panels (available at your big-box home-improvement store), and stick them in the corners of your room. if you don't want exposed fiberglass just sitting around in your room, you can frame the panels in cheap 1x4 or 1x6, and wrap some cloth around them. Just that alone can help get stray lower mid-range and bass resonances in a room under control. There are sites Internet covering this, some with free plans even. Do some searches and read up.
Finally, we come to recording software. The most commonly used kind of program here is called a DAW - digital audio workstation. This takes the role that a big mixing console and multi-track tape recorder had in big professional studios -- you use it to record your raw sounds and mix and process them into complete song. Again, good news, you don't have to spend thousand of dollars on Pro Tools or fancy processing plug ins. Reaper has a free demo and offers a discount license for $50. Ardour is free and open source, and is now officially supported on Windows. And if you buy a new audio interface, it may well include a lite version of one of the other common DAWs.
I realize that was long-winded. tl;dr: What you need: Computer, audio interface, microphone, recording software. Optional but it really helps: sound treatment for the space(s) you're recording and mixing in. And if you look for deals, buy used, and DIY, you don't have to break the bank to do it.
