Bottoming out of a fork CAN cause a failure of a fork. I'm speaking from my experience where I ruined 2 DH forks in a space of a week. Both by bottoming out on harsh landings. However, this might be a fault of the fork construction (both of them were the same brand/model).
While you are talking about XC and tame trail, I can't see the same things happening to you. I've bottomed out forks many times without failures. Bottoming out is not nice, but the forks survived in most cases (apart from the 2 above).
So go and play with the air pressure. You'll feel that air is lacking before you bottom out. Just pump some air in. Also, air forks are working as a progressive springs, so you'll need to try hard to bottom out 30% sag on 120mm travel.