I recommend the Rick Steves guidebook and his walking tours (download through his app when you have wifi). The London Eye is good (but pricey) if the weather is nice. Another way to enjoy city views is to have lunch at the Ting Lounge (not restaurant, sit in the Lounge for the best views over the river) in the Shard. You will have to reserve ahead but for roughly the price of going to the top of the Shard or doing the London Eye, you will get great views and lunch!
As Telecaster said, the big museums are free! Only special exhibits require a ticket. The Natural History Museum can have quite a queue to enter, so if there's an exhibit you want to see there, it's worth paying for the ticket as you get to skip the queue and go straight to the main entrance (it doesn't say this on the ticket usually, but you can, just go straight up to the main door).
If you want to do a fancy afternoon tea but don't want to pay fancy afternoon tea prices, you can go to the 1st floor cafe + ice cream parlour at Fortnum & Mason, get their plate of two scones to share and 2 teas (same tea you get at their fancy afternoon tea!) for about £25 total.
For navigation in both cities, if you have international data on your phones, the app Citymapper is the best for getting around. Do you know how you're getting from London to Edinburgh? If you're taking the train, don't just look at standard fares--sometimes first class is nearly the same price and includes wifi, food, and drinks. If you're traveling in a pair or group by train you may want to look into a Two Together railcard--sometimes the discount on the fares to Scotland more than pay for the railcard.
In Edinburgh, the Castle is interesting and if the weather is decent, climbing Arthur's Seat gives you some great views. In Edinburgh and London, expect rain and wear layers. If you are short of space/weight, you can buy very small umbrellas on the cheap at grocery stores like Tesco and Sainsburys (my brollies were on sale for £6 from £8 at Sainsburys).
I used to live in Scotland and in London, so I feel like I could go on and on!