The tessellation tool is a vector-based tool that allows arbitrary forms to be created. Set a base height for the stroke and then draw a stroke. Set another height and draw another stroke and so on. After the basic strokes are in place, generating the surface will interpolate between the strokes.
Draw a desired coastline.
Draw from the outer to the center to make it easier to loft the ridges in the next stroke.
Select a stroke and then Tesselation>>Loft Stroke.
Click the Generate button. We’re still using the default map edges (+/- 1 in X and Y) so we’ll just use a small forced height to get a good level.
Use the gradient tool with radial gradient and a high value of 0.1 and operation of Add
.
Filter>>Mathematical>>Span with low = 1000 and high = 2000.
Filter>>Noise>>Percentage noise with 5% noise
Filter>>Fill>>Fill Basins with slope = -1
Filter>>Erosion>>Incise Flow with Blur = 1, Amount = 3, Flow Exponent = 0.25, Effect Blend = 0.2.
Filter>>Mathematical>>Span with low = -250 and high = 1000
Filter>>Erosion>>Precipiton-Based
Filter>>Mathematical>>Offset (Add) with a value of 30 to lift everything up a little.
Want more detail? Resample the surface. Surface>>Resample>>Simple
Filter>>Noise>>Absolute Magnitude Noise with magnitude = 25.
Filter>>Fill>>Fill Basins with slope = -1
Adds nice valleys
Filter>>Blur>>Gaussian Blur with sigma = 0.75