The shale gas boom in the US drove down prices, so OPEC actually *increased* production to drive prices low enough to knock the newcomers out of business. It's working to an extent--
But isn't going to work long term. Even if the current operators go bust we know the oil is there and have the technology so it's easy for another round of companies to turn the tap back on when prices increase.
Precisely, which is why OPEC is in such a pickle--if they slow their own production, prices will increase somewhat, but nowhere near as much as they might have in decades past.
Harvesting oil in any way other than pumping it from an underground reservoir is much more expensive. I'll use fake numbers to give an example.
Let's say the cost to pump one barrel of crude is $20 per barrel. Any time the market value of crude is greater than $20 per barrel, OPEC is making money.
At the same time, the cost to harvest oil from alternative methods may be in the realm of $70/barrel. When the price of oil spiked a few years ago, this made what would have otherwise been considered an expensive venture, profitable. As long as crude stayed above the (in this case, hypothetical) market value of $70/barrel, the alternative harvesting techniques were still profitable.
In the past, OPEC has come together and slowed production to maintain price stability and consistent profits for their member nations. In the late 80s, however, one or two country members of OPEC didn't follow along with the rest of the group and kept their production up while the rest of OPEC dropped their production. The result was that prices still went up, but the member nations that didn't do what they were supposed to made boatloads more than the ones that slowed production. This was viewed by the members of OPEC that followed the rules as essentially stealing from them (a big cause of some of the current tension in the middle east).
Back to the present, When north American companies started harvesting oil at $70/barrel, instead of slowing production to maintain the same market value, OPEC INCREASED production to force the prices down even lower with the idea that if the prices dropped below the operating costs, the new guys in the game would start losing money and be forced out of business.
It's started to work with one or two more expensive types of harvesting crude, but several of the "lower" cost alternative methods are still able to hang on for the time being. Keep in mind that OPEC is not hurting for money and they have plenty of crude already out of the ground sitting off of the market to help in their control of prices. They are taking a hit on profit margin now to win back the position of near exclusive petroleum supplier for the world in the future.