How is the rust for both of them (photo looks good, how about underneath)? How about what maintenance each has had? Although the 1ZZ-FE engine in the Vibe was supposed to be of a newer design that did away with oil consumption, I'd still ask the owner. This year also has throttle by wire, and to me a very weird (I have an 06 MT Matrix, as you know basically the same car) with hard to modulate shifting, and they also seem to experience low clutch life. No timing belt to worry about on this car either.
As for the Mazda I'd also look for a lot of rust in the hatch, and bad struts. The 2.3 engine is generally very reliable.
Now, on a what would I take... I'd probably go with the Mazda, especially if it has had good maintenance/nothing too wrong with it. The Vibe is about as numb and un-enjoyable a car that I have ever owned, despite insane reliability. The Vibe will also offer better useful capacity, and lower insurance cost too I'd bet. The Mazda will be a better driving experience IMO, more useful power, better transmission.
I bought a brand new Mazda 323, and loved the hell out of that car. No A/C, no power steering, roll-down windows, 5 sp manual, 1.6L fuel-injected engine, and damn was it a nice car to own, and peppy as hell. Could change the timing belt in about 25 minutes, and with way over 150k miles it still had its orig waterpump, starter, struts, wheel bearings, and steering rack. Exhaust issues were its vices, and I put a clutch in myself, after 130k miles (Surprised it lasted as long as it did, how I drove it). I'd love to have this car back,
Good luck, and hope you find the guidance you seek.