I believe that sign-in-with-X implementations require actually authenticating with X in that context. So in order to sign in with Google to a website, I need to sign into Google itself in the same browser. So in that sense, you are enabling Google to track you in that browser - but no differently than if you just logged into gmail. You could sign back out of Google immediately afterwards.
But just the fact that an app uses Google authentication doesn't give Google any kind of privileged access to that app's data (beyond purely the data that they're logging in on this browser at this point in time) unless the app is pushing data actively back to Google (which they could do anyway for a gmail-managed email address, I guess) or you believe that Google will then subsequently forge authentication requests to that app and pull data out itself. Both of these things are hypothetical violations of your privacy enabled by using auth-with-Google, but for most use cases are rather unlikely, I would guess.
But just the fact that an app uses Google authentication doesn't give Google any kind of privileged access to that app's data (beyond purely the data that they're logging in on this browser at this point in time) unless the app is pushing data actively back to Google (which they could do anyway for a gmail-managed email address, I guess) or you believe that Google will then subsequently forge authentication requests to that app and pull data out itself. Both of these things are hypothetical violations of your privacy enabled by using auth-with-Google, but for most use cases are rather unlikely, I would guess.