So, I worked around this with the following asciidoc config:
[xref2-inlinemacro]
<link linkend="{eval: "{1}".endswith('#') and "{1}".lstrip('/').rstrip('#').replace('/','-') + '-top' or '#' in "{1}" and "{1}".split('#')[1] or "{1}" }">{2}</link>
{2%}<xref linkend="{eval: "{1}".endswith('#') and "{1}".lstrip('/').rstrip('#').replace('/','-') + '-top' or '#' in "{1}" and "{1}".split('#')[1] or "{1}" }"/>
It assumes a [[foo-bar-baz-top]] id is inserted before each include::/foo/bar/baz.adoc[], and does this:
<</foo/bar/baz#,label>> -> linkend="foo-bar-baz-top"
<</foo/bar/baz#myanchor,label>> -> linkend="myanchor"
<<myanchor>> -> linkend="myanchor"
John