My experience is mostly with smaller boats on lakes and rivers - and here most wooden masts tend to be rounded indeed.
It seems that the easy swinging of the spars as described by Tony is the primary reason.

Most - but not all. For example, on a few occasions I had a chance to sail friends boat:

https://workingsail.com/messabout/boats/olive_oyl/index.html

which sails happily with junk rig on a square wooden mast with slightly rounded corners. It works just fine, I've barely noticed any difference.

One of reasons for a squarish mast may be simplicity of building square box-shaped hollow mast comparing to a birdsmouth.

I usually keep it simple though and  build my masts laminating a few  (usually 3 or 4) spruce or pine planks and make them somewhat round or elliptical in profile - for easy swinging of the spars and reducing wear and tear.

Epoxy is the golden standard for laminating masts of course, but I've seen plenty (and made a few) masts laminated with
waterproof PVA (like Titebond III) and never seen a failed glue line.

The masts I'm talking about are usually in 4-6 meter range though, not sure how all this would scale up to 8 meters...

In case you're interested in how I did it, I videoed the process here;

https://youtu.be/PqMa8ZMy3RE?si=D0Dg1FXmuceK0LNz

https://youtu.be/q1KYeeEiK_k?si=klWkHgCVWfcLlPTy

https://youtu.be/riRM6um01xQ?si=FfKoUMH0nb9zqMqj

https://youtu.be/K5X1hREf5wc?si=6Sxwph-nsGiE30v-

https://youtu.be/YBYxHMgxfPg?si=X8BMV-9-Qfvvl5g1

This may be a novice boat design question akin to asking a cyclist why they don't make square wheels, but what are the primary reasons for not just using a solid tapering square section mast? 

Random thoughts:

• Are masts round because trees are round, and we used to make masts from trees? Would a square and triangular mast be better, but we don't make them that way as they don't look tree-like?
• I suppose you might get more chafing with a square mast, but maybe not so much if the edges were rounded.
• Would it be harder for the sail to rotate around a square mast if the baton side of the mast was pressed onto the mast (creating a failure point?)

The reason for asking such a daft question is I'm discovering that although the junk rig appears to be the perfect rig for adventurous solo sailing (marginally less speed in return for robustness, think Formula Once race car vs John Deere tractor), the biggest hurdle is finding a mast to hang the rig from. This seems to be a universal issue, but when you live in Ireland, the options are very limited (I'm currently in Northern Spain, and I've found myself looking longingly at Spanish lampposts and utility lines (is this common?). Once you find someone who makes something that might work, the second they cotton on, you're not a certified municipal public illumination professional (I get the impression they're aware that people like us exist, and we're trouble), andthe communication stops. One thing that seems to expose our true nature is asking for wall thickness and type of aluminium used and telling them you don't want all the light/flag/net/radar/radio attachments on the mast... I mean pole. 

Finding the correct alloy tube to make a hybrid mast seems impossible outside of Europe or the UK (getting a big piece of pipe delivered is costly when you live on an island). 

Making a wooden mast is the only option, which is a cheap way to do it once you've purchased several thousand Euros of woodworking equipment and put yourself through night school to use it.

At the moment, I think I could manage to make a square section tapering mast (8 metres long, made from 4 lengths of squared Douglas fir) or just buy two scaffolding poles (48.3mm with 3,2mm thickness, zinc coated steel) and make some junk yawl affair (maybe sail it to the UK to find the pole of my dreams).

I don't need an answer to my question, as I am venting my frustration at a world that cares little for my junk rig dreaming.

Andy