The Xterra would have been fine for us if we didn't haul around the gear we do. The added room of the 4Runner for that and the family, coupled with V8 power and very good fit and finish make me prefer the 4Runner.
If I only had one kid and didn't haul around 4 kayaks I probably would have kept the Xterra.
