Implementation is in pull request #291.

4\. Define how to indicate, that a Vectorcall-function supports the Stackless protocol.

Actually this turned out to be the hard part. First I thought, that I could simply use the tp_vectorcall_offset field in slp_methodflags, but first there is no such field and second the whole approach does not work. Unlike all other slots, the vectorcall function is a per instance property. Each instance of a type may use a different vectorcall function. When I discovered this difficulty I first feared this could be a show stopper for Stackless.

Then I found a way to modify the "Stackless-protocol" that is used to control, if C-function may return Py_UnwindToken. Traditionally we use the global variable int _PyStackless_TRY_STACKLESS and the local variable int stackless. Both were used as boolean variables with zero or non-zero values. The calling function needs to know whether the called function supports the Stackless protocol, because the called function must reset the global variable _PyStackless_TRY_STACKLESS.

For the vectorcall protocol the calling function does not know, if the called function supports the Stackless protocol. Therefore, for vectorcalls, the Stackless-protocol must work without the strict requirement for the called function to reset _PyStackless_TRY_STACKLESS. The solution is as follows:

1. The type of the global variable _PyStackless_TRY_STACKLESS is changed from int to intptr_t.

2. Possible values of _PyStackless_TRY_STACKLESS when calling a C-function.

   • 0: stackless calls are not possible, the called function MUST NOT return Py_UnwindToken.
   • 1: the called function MUST reset _PyStackless_TRY_STACKLESS and MAY return Py_UnwindToken.
   • Any other value: if the called function does not support the Stackless protocol, there are no requirements.
     Otherwise, the called function MUST reset _PyStackless_TRY_STACKLESS. If the value of
     _PyStackless_TRY_STACKLESS is the address of the called function, then the called function MAY return
     Py_UnwindToken.

3. To prevent leakage of a non zero value of _PyStackless_TRY_STACKLESS to other threads, a thread must reset
   _PyStackless_TRY_STACKLESS before it drops the GIL. This is done in the C-function drop_gil.

I implemented this solution in pull request #291. Seems to work well.

Upstream commit be718c3 causes assertion failures in prickelpit.c function static int init_type(PyTypeObject *t, int (*initchain)(PyObject *), PyObject * mod)

It is now required to copy a few fields from the base type: tp_descr_get, tp_vectorcall_offset and tp_call.

I merged the code changes (pull request #293). The documentation changes still need to be reviewed (pull request #291).

Interesting. The thoughts behind this certainly belong to the documentation. Is this perhaps a better protocol than the old one?
Some points

• "MAY return unwind token" should perhaps be "Is allowed to return ... (because the caller is prepared to handle it)"
• the issue witth the GIL (3 above), that's because the value is set without knowing if the called function supports it... would it be possible to use the thread state instead of a global value to avoid this?

Interesting. The thoughts behind this certainly belong to the documentation.

I added the information to the comment for the try_stackless field in pycore_slp_pystate.h. It is an implementation detail. This is why I didn't add it to the API documentation.

Is this perhaps a better protocol than the old one?

Not sure. It requires the address of a called function (woul prevent complete inlining functions) and requires two writes and one read for each function call. With the other protocol many simple wrapper functions do not touch try_stackless at all.

Some points

* "MAY return unwind token" should perhaps be "Is allowed to return ... (because the caller is prepared to handle it)"

I completely rewrote this part.

* the issue witth the GIL (3 above), that's because the value is set without knowing if the called function supports it... would it be possible to use the thread state instead of a global value to avoid this?

Sure, but a global variable has a known address and is always available. A unconditional write of 0 in drop_gil() is extremely fast.

And many thanks for your help. Good documentation is really hard.

On my way to the office this morning, I was thinking about this, and I realized that I never really understood the soft-switch mechanism in stackless.

If there isn't one, maybe a section on how it works would be helpful. Here is a rough outline from what I think happens, when A switches to B and then back to A: (without consulting code and my memory is rusty), e.g. when calling one of the switching functions, like tasklet_run

1. A sets ts.stackless.current to B
2. return the UnwindToken
3. Unwind token is passed on up to the top where the "Mario code" switches the current PyFrame state to the one of tasklet B
4. B runs its own frame execution function
5. B switches to A by same means (setting ts->stackless.next or current...), returns UnwindToken
6. code returns token back to the mario code...

This is where it gets fishy for me. How can we now, re-enter the C-stack of the function that return the original UnwindToken in 2. Maybe it was a stackless-aware function, and it needs to resume now. Is that even possible?

The documentation of vectorcall related changes (pull request #291) is merged. I opened issue #299 for the missing high level implementation documentation.