This took a long time to figure out because
1) for testing purposes I set Icon before the window handle had been created
2) When the window handle was later created, mono would not actually apply the custom icon since the border style is FixedDialog (see CreateHandle in Form.cs)
3) I later set Icon again in multiple other places. However, this was essentially a noop because Mono checked if the new icon was the same as the same previously assigned icon and do nothing in that case.
However, the assumption in step 3) was incorrect because the previously assigned icon had not actually been applied to the window.
Unfortunately this meant the correct Icon never showed at all. So the correct solution to this is to only assign Icon once in the Load event of forms.
I spent way too much time on this