Welcome to cord-cutting!
Basically, there are three kinds of content you might like to watch on your television: current TV, new movies, and older TV and movies. There are two kinds of streaming devices--Apple, and non-Apple (Roku, the various Amazon widgets, other brands). Apple will work with Hulu and Netflix and sell/rent you content via the iTunes store. Non-Apple devices will work with Hulu, Netflix and Amazon Prime and sell/rent you content via Amazon or other providers. Apple TV will also let you play content from your iPhone or iPad on your TV. What you get depends on your needs.
Current TV has the most options but is the most complicated also. You can use a digital antenna (with a DVR like TiVo to record and watch later). A digital antenna is free but TiVo, for instance, has a monthly charge to keep track of the content. You could also use Hulu ($8/mo with ads, I think around $15 w. no ads) for most broadcast channels and occasional cable content. Shows are available the next day. Or you can purchase shows by the episode for $2-3 each from Apple or Amazon. This option can be great if there's just one or two shows (like Mad Men, for instance) that you especially want to keep up with from cable. CBS is an oddball; they keep their shows off Hulu and have a separate service with a monthly fee. Various cable channels offer apps that make their content available for people with a cable package; those are mostly meant for people who want to be able to watch on their phone or tablet.
If new movies are most important, you can rent those from Amazon or Apple; you could also use one of the Netflix DVD plans or Redbox kiosks if you have those nearby.
If you just want some kind of TV available to watch, but aren't too picky about it being up-to-date, you can get Netflix, or Amazon Prime, or to a lesser extent Hulu. They all have big libraries of older television and movies. Some content can be found on multiple services; some on just one.
Also, keep in mind that your cable company is legally required to offer you a package that is broadcast-equivalent...but they don't have to advertise or bundle it. So before you cut the cord, it might be worth pricing that, especially if you don't have good antenna reception and think you might want to watch live TV occasionally (during weather events, the Oscars, etc).