No, there's no way to turn the "ringer" off on the phone. Generally when it rings your gang is inviting you to a "Survival" mission, and you should probably go. The survival missions spawn somewhat randomly, and they're required for the Saintsbook challenges (which is tied to an achievement). They'll only call every so often (not more frequently than 10-20 minutes or so, although that's still under investigation) and they won't call when you're on a mission or otherwise engaged in an activity.
If you're not interested in the activity or its associated challenge/achievement, you can always ignore the call, or go to the site and immediately leave the radius, which will cancel the survival mission.
As soon as you answer the phone, you can go back to whatever you were doing previously with that button - you don't have to wait for the call to end. Apparently the leader of the Saints sprung for a hands-free headset between Saints Row 2 and 3.