I actually worked at a Toyota dealer for a bit, and getting cars in that had never had oil changes was a surprisingly common thing. We got a Camry in one time, I think it was a 98 or 99 (this was in 2005/6) and it had never had an oil change. Engine was a 4 cylinder and was almost toast, but it did still run and drive. Unfortunately I don't recall how many miles were on it. Took the valve cover off and it literally had a SECOND valve cover inside made out of oil sludge that had gone rock-hard. Also had a Sienna come in that was around 3 or 4 years old at the time and had never had an oil change, had a 3L V6 that was completely shot. Needless to say I'm pretty religious about changing the oil in my cars.
I too worked at a Toyota dealership, and was a service manager for a national chain for pre-owned vehicles... Boy, what I have seen and repaired! Also what I have heard from customers, and seen like you have concerning oil changes, etc. I will say, I have seen 1987-1991 Toyota Camry engines nearly run dry of oil (3SF-FE), still run OK after nearly seizing... They do not IMO build them like this anymore.
Although I do believe Toyota did have some design flaws concerning piston design (skirt hole size, and ring design/material), and less than optimal PCV systems, piss poor maintenance really turbo-charged the situation. I'm lucky to have also been the training director for a chain of quick-lubes, and have an uncle who is a petroleum engineer... We used to talk a lot about the real nitty-gritty of the oil business, and without any doubt, true 100% synthetic oils with the proper additive package, using the proper viscosity will alleviate almost all of the problems.
I too have seen the v6 you mention (1MZ-FE) that I remember calling what you mention "valve cover brownies", because that is what it looked like with the valve cover pulled. Toyota did *try* to address the situation as you may recall, with there "abnormal gelling" campaign, but if you used good synthetic oils, changed at least on the normal service schedule, you would have been pretty much spared.
Also, the 2AZ-FE I mentioned, had some issues with pulled head bolts at the very back of the engine... You would see the foam insulator in this area soaked with coolant, and sure enough the middle three bolts would have pulled out of the block, with three stripped holes. I think after 2005/2006 a change was made to the block to have longer thread holes in the block. Toyota is a fine vehicle overall, but like them all, even the Japanese, they have plenty of problems such as the well known issues with Subaru and the RIDICULOUS problems Honda is having with their 1.5 turbo engines concerning oil dilution with fuel... Can you say wiped out bearings!
To be honest, I'm so effing sick of all of the BS newer vehicles deal with, and wish I still had my Japanese made 1994 Camry (owned 20 yrs when I donated it). To me it represented the nexus of true build quality, luxury level comfort, absolute reliability, and simple servicing w/o all the GD rolling i-phone crap everyone seems to want these days.