If determinism means the universe is pushing you around, and you also believe that you are made up of universe-stuff, that means it’s just the universe pushing the universe about. And being the universe you should be able to push right back.