Walter,
I used square section GRP battens for quite a long time, and over many ocean miles, the edges were chafing through the sail pockets.
Also, square tube is not as good as round tube in terms of strength per Kg.
My calculations give these results:
50 dia x 3 round tube: weight 1.2 kg/m section modulus Z = 4903
50 x 50 x 3 square tube: weight 1.52 kg/m section modulus Z = 5904 ( on the diagonal), 8340 (perpendicular to a face) - it's strange-but-true that square tube is weaker when loaded on its diagonal.
63.5 dia 3 round tube: weight 1.54 kg/m section modulus Z = 8222
A square batten tries to rotate to the orientation in which it can bend most easily, which is on the diagonal, so that presents an edge to the mast - not good.
So you'd get better value from a round tube. If your pockets will accommodate a 50 x 50 square tube, they will also accommodate a 63.5 round tube, and that's what I would go for. But Pacific Spray being so heavy and stiff, I'd make the upper battens from bigger tube still, if they'd fit in the pockets.