When I rebuilt my server I decided to run SSH on a non-standard port, just to deny casual automated probes. (Feel free to tell me it doesn't do any good...) But I also used the :ext: protocol of CVS to run it over SSH, and it didn't allow you to specify the port in the relevant file (CVS/Root).
The solution, as found on the info-cvs list: setup an alias for SSH. So in ~/.ssh/config I have something like:
Host cvs.cwinters.com HostKeyAlias cwinters.com Hostname cwinters.com Port 9765
The host cvs.cwinters.com doesn't actually exist to the world, it's just a pointer. Then I just ran my handy script to mass-update all CVS/Root files under the current directory with the updated host and everything just worked. Nice!
Note: apparently you need the nc binary to make this work properly.