There is a train/greyhound that will get you to Yosemite Valley, but it's a pretty long ride, has some weird transfers, and, although I plan to take it someday, I can't say I'd recommend it. I don't know any way to Joshua Tree but by car.
In general, our area is famously underserved by any kind of logical public transit. I always take a train into LA proper, but I know my way around really well. Once out of the heart of the city, transit gets more and more sparse, unfortunately.