I found that if I destroy the resizable when the cursor hovers over the sortable handles then sortable suddenly works. Then I can recreate the resizable after sorting is stopped. The solution entails binding two event handlers to the moveButton class elements.
I imagine that this is a big performance overhead, but in the attached example with a relatively small DOM it is still very responsive. See attachment Sortables_NestedError_r4.html for workaround details.
It is a little flaky when you hover over a move button but do not begin to drag, because I have the autoHide option set on the resize handles. They disappear when I destroy the resizable, but do not reappear immediatly on mouseleave of the moveButton. If I move the mouse outside the resizable and back in then the resize handles reappear. This is more of an issue with the autoHide option. The jQuery bound mouseout event seems to be firing fine. If I do not use the autoHide option the handles all disappear and reappear as you would expect given this workaround. But without autoHide it is more obvious to the user that the resizable capability is being removed and added back.
It would be great if this compatibility issue between resizable and sortable could be worked out. It would then be a very robust and user friendly solution.