I'll be interested to see the responses, as I'm also in NZ and have struggled with the same question.
However, I can speak to one thing - private healthcare in NZ is *not* particularly useful for serious illnesses (such as cancer). If something is seriously wrong with you, you get moved up quickly, and the public system is better anyway. Where private is great is the non-urgent things, that's where you can end up languishing on waiting lists for months or years.
My uncle and father both had terminal cancer (one for 6ish years, one for 6ish months), both with private health insurance, and they both chose to use the public system for treatment. My father used private to access a CAT scan a week more quickly than through public (easier for us to schedule around Christmas holidays), but that didn't affect any treatment schedule. My uncle's recommendation was to use the public system, and I believe that's because they use the same doctors.