If you want to limit voice chat to party-only, or completely disable it, in XDefiant, then you can check out this guide at Seeking Tech to see how you can properly do so.
1. From the main menu of XDefiant, open the Settings menu. On a controller, you can do so by pressing down on the right analog stick.
2. You then need to head over to Audio & Voice Chat.
3. Scroll down to the Voice Chat Settings section.
4. Set Voice Chat to Off to disable it completely.
5. You can also set Voice Chat to On, and then set the Voice Chat Channel to Private to limit voice communications to your party only.
6. You can also use the Voice Chat Volume, Microphone Input Threshold and Microphone Volume Boost sliders to adjust your voice communication settings.