Will OBS Server-Side Encryption Encrypt My Existing Objects That Are Unencrypted?

No. OBS encrypts only objects that are uploaded after the server-side encryption configuration is effective. If you want to encrypt existing objects, delete them and upload them again.