Wow, dropping BootStrap would be like starting over. But I applaud your efforts to make XC your own. You can unlink style sheets pretty easy, thanks to XC5's architecture.
Just look into:
public function getCSSFiles()
$list = parent::getCSSFiles();
$list = 'button/css/popup.css';
I'd loop through the $list, and see what I could remove. I do want to warn you though, Bootstrap is pretty foundational, removing it is going to mean a lot of work.