InP/GaSb core-shell nanowires: a novel hole-based platform with strong spin-orbit coupling for full-shell hybrid devices

I am very grateful to the authors for the extensive reply as well as the revised manuscript! These have helped to clear up some of my previous confusion - I was under the impression that the interaction described by alpha was within one subband, not between subbands. I understand now the authors' results, and I think they are interesting.

We sincerely appreciate the Referee's positive feedback and we are glad that our revised manuscript has helped clarify the nature of the interaction described by $\alpha$, thereby enhancing our manuscript readability. We are also pleased to hear that the Referee finds our results interesting. This aligns well with the feedback from the other Referees, who similarly highlighted the novelty and significance of our work.

My main question however is the following: The main result, a coupling between the two lowest $m_F=1/2$ subbands as in Eq. (1) has already been shown in Eq. (3) in Ref. [50] (Kloeffel et al., PRB 84, 195314 (2011)): the term with the C prefactor is identical to what the authors find.

This thus raises a major question: The paper positions itself mainly as showing strong spin-orbit in this proposed system. A very similar result (same order of magnitude) has previously been demonstrated for Ge, a platform that already exists. Is this sufficient to claim "Open a new pathway in an existing or a new research direction, with clear potential for multi-pronged follow-up work"?

Currently, Ref. [50] is somewhat dismissed in the beginning of the paper as "However, these favorable properties stem from the strain introduced at the Ge/Si heterostructure interface [50–52], which imposes specific design constraints." The term in Ref. [50] that corresponds to the author's results is without strain, however. Hence, I believe the relationship to this (and possibly other papers) is not properly discussed in this paper. This also hinders me in making an assessment where this paper should be published.

If the authors can make a compelling argument for why their proposal is fundamentally better than the previous literature, then I would suggest publication in SciPost Physics. If not, then I would propose to publish in SciPost Physics Core.

We thank the Referee for their insightful comments, which allow us to clarify the novelty of our work and its distinction from prior studies.

We agree with the Referee that the SOC in our proposed platform is similar to results previously reported in Ge-based heterostructures, including those in Ref. [50], and more notably Ref. [54], which employs a geometry more akin to ours. We have now explicitly acknowledged this in the main text, rather than only in the Appendix, and incorporated these references earlier in the manuscript.

However, we emphasize that the primary contribution of our work extends beyond simply demonstrating strong SOC. While SOC is an important ingredient, the novelty lies in the design of a realistic nanostructure that overcomes several critical challenges in the field and paves the way for experimentally feasible platforms for topological superconductivity. As we discuss in the Introduction, current full-shell hybrid NWs have significant limitations apart from small predicted SOCs, including the lack of control over the chemical potential and the proliferation of trivial subgap states (known as Caroli-de Gennes-Matricon analogs) with which potential Majorana zero modes have to coexist. Our proposed nanodevice addresses these challenges in several key ways. We propose to include an insulating InP core (absent in conventional full-shell NWs) that enables control over the Fermi level and confines the wavefunction to the outer GaSb shell, potentially reducing disorder. Moreover, in the presence of an outer SC shell, this confinement should enhance the induced superconductivity and, more importantly, should reduce the presence of trivial subgap states, as we discuss in the manuscript (see also the last answer below).

And we do so by leveraging the properties of the hole-bands of III-V SM compound heterostructures. While Ge-based (Group IV) heterostructures exhibit similar favorable SOC properties, they face several disadvantages compared to III-V compounds. Ge is more prone to dislocations and disorder due to its common growth methods and exhibits lower electron mobilities. Additionally, it is less versatile in tailoring band alignment, as III-V materials enable the use of ternary compounds to precisely engineer electronic properties. Growing a sharp, clean superconducting layer on Ge is also more challenging; its surface oxidizes quickly, requiring etching and preparation steps (leading to more disorder), and exhibits poorer chemical adhesion to SCs. Consequently, Ge-based SM-SC NW heterostructures are rarely studied, with most experiments focusing on planar nanostructures, which often suffer from irregular interfaces. This is why III-V SMs are widely studied and established in the field, particularly for partial-shell designs aimed at creating topological superconductivity. By identifying a III-V-based NW with strong hole-mediated SOC, our proposal provides an accessible and realistic alternative for many experimental groups that exclusively work in III-V platforms due to its advantages.

In summary, while the origin of the SOC that we report is not entirely new, the design and context of our nanodevice represent a significant step forward. Our proposal overcomes critical limitations of existing platforms, introduces a III-V-based alternative for realizing topological superconductivity, and opens up new opportunities for studying hole-mediated phenomena. As such, we believe our work fulfills SciPost Physics’ criterion of “opening a new pathway in an existing research direction, with clear potential for multi-pronged follow-up work.”

To address the Referee’s concern, we have revised the text to explicitly discuss the experimental challenges associated with existing platforms (with references), reinforcing the importance of finding an alternative platform outside Group IV materials. This clarifies what we initially meant by "specific design constraints," which, as we now realize, could be misinterpreted by a general reader. Furthermore, we have removed references to strain in Ge-based heterostructures, as we have verified that comparable SOC strengths can indeed be achieved in Ge without strain.

Other points with regards to this manuscript:

- I misunderstood the paper when I read it first. There is a reason for this: the authors use symbols as for the conduction band case ($\alpha$) and initially do not define what they mean with "spin-orbit" (this has only slightly improved in the revised version). In fact, Ref. [50] has the same term but does not call it spin-orbit interaction (Ref. [50] later derives a Rashba type SOI). I would suggest to the authors to (i) define what they call spin-orbit already in the beginning of Sec. 3.2, and not just in the appendix (and even with the appendix, I had to read all the previous papers to understand what was going on - it would be great to make this more self-contained), (ii) give an argument why they call this spin-orbit interaction.

We appreciate the Referee's thoughtful feedback and fully understand the confusion. In response to the Referee’s suggestion, we have now included an explicit definition of the SO interaction/coupling in the main text, specifically when we first introduce the parameter $\alpha$ in Section 3.2, in order to make the manuscript more self-contained and accessible to readers.

To completely clarify this point, we define the SO interaction, following the conventional definition, as an interaction that couples the momentum of a quasiparticle to its pseudo-spin. While the pseudo-spin in traditional cases typically refers to the electron's pure spin, in our work, it refers to a more complex band combination of spin-like degrees of freedom. Regardless of the precise nature of the pseudo-spin, the key point is that this interaction manifests in our case with a Rashba-type SO coupling, which is a scalar term that describes the strength of the coupling between the momentum and the pseudo-spin, i.e., the SO coupling $\alpha$. We hope this more precise definition resolves the confusion and makes the paper clearer for a broader audience.

- I find App. B a stronger argument for claiming that the effect is due to the Hamiltonian and not electric field than all the arguments in Sec. 3.2. Basically, Sec. 3.2 shows through numerics what does not affect the strength of the effect. App. B shows you get the effect from the Luttinger-Kohn Hamiltonian and confinement alone. To me, it would make more sense to state this fact first, and later show that the more detailed numerics only gives small corrections.

We appreciate the Referee’s valuable suggestion. We agree that the analytical derivation in Appendix B provides a stronger argument for attributing the effect to the bulk Hamiltonian structure together with the confinement, rather than to an electric field. However, this perception can depend on the reader's background field. We chose to present first the numerical results in Section 3.2, as they help to rule out other known sources of SO interactions before entering directly into a discussion of simplifications, approximations and notation. Nevertheless, we immediately after present the analytical calculations to justify our conclusions. Hence, since we believe that the Referee's proposed change does not significantly alter the overall understanding of our manuscript or the validity of our approach, we prefer to keep the current structure of our manuscript.

- Why do the authors consider an insulating core at all? It is clear from the results that the effect they want to show also will appear without an insulating core (as is the case in Ref. [50] for example).

It is important to understand that we are not just characterizing a hole-SM NW with strong SOC, we are proposing a specific SM nanostructure -the core-shell NW- with several crucial properties to improve the chances to find topological superconductivity in full-shell hybrid setups.

Although it is true that a strong SOC would be present without the insulating core (it actually slightly increases as the core diminishes for a fixed radius), the InP core serves additional purposes, as we discuss in Sections 2 and 4.

In particular, it helps to achieve the desired Fermi-level position. Notice that the Fermi level of GaSb does not naturally lie near the valence band. In contrast, the InP-GaSb heterostructure offers a type-II band alignment that places the Fermi level close to the GaSb valence band, as reported in Ref. [28]. Furthermore, the incorporation of dopants in the InP core can provide partial control over the Fermi-level position, without significantly affecting the electronic properties of the wire. This is important to achieve the proper electronic environment for topological phenomena.

In addition to this, the insulating core confines the wavefunction to the GaSb layer of thickness w, which then has a tubular shape. If the NW was then wrapped in a SC shell, the insulating core would help to enhance the proximity effect and, more importantly, in the presence of a threading flux, it should help to reduce the presence of trivial subgap states (known as Caroli-de Gennes-Matricon analogs) coexisting with potential Majorana zero modes and that are so detrimental for their topological protection, as discussed in Ref. 17.