🔒 How do you deactivate GIL when using the python C lib?

[wise crater]

I need threading. Actual threading.

[drowsy jay]

@wise crater

Python help channel opened

Remember to:

• Ask your Python question, not if you can ask or if there's an expert who can help.
• Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
• Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.

[wise crater]

i already looked at this page https://docs.python.org/3/c-api/init.html#thread-state-and-the-global-interpreter-lock

but using the macros results in a crash

[sterile field]

Show your code?

And the crash you got

[civic kiln]

python 1.13t has an expiremetal features. one of which is no GIL.

Summary – Release Highlights
Python 3.13 is the latest stable release of the Python programming language ... The biggest changes include a new interactive interpreter, experimental support for running in a free-threaded mode (PEP 703), and a Just-In-Time compiler (PEP 744).

[wise crater]

seg fault

[sterile field]

You want to deactivate the GIL or release it?

[wise crater]

i mean sure but id like to not depend on a specific type of python instalation for my things to work

[wise crater]

threads. normal posix threads

as they work in any other language

[civic kiln]

https://docs.python.org/3/howto/free-threading-extensions.html#freethreading-extensions-howto

C API Extension Support for Free Threading
Starting with the 3.13 release, CPython has experimental support for running with the global interpreter lock (GIL) disabled in a configuration called free threading. This document describes how to adapt C API extensions to support free threading.

[sterile field]

Yeah python threading.Thread is a normal POSIX thread

[wise crater]

#include <string_view>
#define PY_SSIZE_T_CLEAN
#include <gtest/gtest.h>
#include <Python.h>

#include "LoggerManager.h"
#include "python_utils.hpp"
#include "TSocket.hpp"
int main(int argc, char** argv)
{
    iu::LoggerManager::LogToConsole();
    Py_Initialize();
    PyThreadState *_save;
    _save = PyEval_SaveThread();
    try
    {
        testing::InitGoogleTest(&argc, argv);
        return RUN_ALL_TESTS();
    }
    catch(const std::exception& ex)
    {
        Py_FinalizeEx();
        std::cerr << ex.what();
        return -1;
    }
    PyEval_RestoreThread(_save);
    Py_FinalizeEx();
    return 0;
}

[sterile field]

And the python code?

[wise crater]

import socket
import threading
import time
def connect_to_server(address, port):
    try:
        sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        sock.settimeout(10)
        sock.connect((address, port))
        return True
    except TimeoutError as e:
        print(str(e))
        return False
    except Exception as e:
        print(str(e))
        return False

class TSocket:
    def __init__(self):
        print("Initializing TSocket script");

    def SetUp(self):
        print("Setting up ...")

    def TearDown(self):
        print("Tearing down ...")

    def ListenAccept(self, address, port):
        print("ListenAccept -> IN")
        try:
            sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) 
            sock.connect((address,port))
        except Exception as e:
            print(str(e))
        print("ListenAccept -> OUT")

    def PollConnect(self, address, port):
        print("PollConnect -> IN")
        try:
            sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
            sock.connect((address, port))
        except Exception as e:
            print(str(e))
        print("PollConnect -> OUT")

    def AcceptQueue(self, address, port, maxConnections):
        print("AcceptQueue -> IN")
        connection_list = list();
        connection_count = 0;

        for i in range(0, maxConnections * 3):
            connection_list.append(threading.Thread(target=connect_to_server, args= [address, port]).start())
        for i in range(0, 10):
            if connect_to_server(address,port)== True:
                print("Connection succeeded when it shouldn't")
                return 0
            else:
                print("Connection failed. Incrementing count")
                connection_count +=1

        for t in connection_list:
            t.join(timeout=0)
        print("AcceptQueue -> OUT")
        return connection_count
``` imo this is kinda irrelevant since it works but threads are not threading

[desert pecan]

python threads are normal posix threads

[wise crater]

thanks i'll take a look at it

[desert pecan]

last time I tested, the free-threaded Python did not offer a performance advantage

[wise crater]

it's not about performance

[desert pecan]

what is it about then

[wise crater]

i need multiple connections going into this server to test some queue

[desert pecan]

...and?

[wise crater]

the way it works now it kinda squentially tries to connect the because of GIL or black magic

they neeed to happen at the exact same time

[desert pecan]

you're likely doing something wrong yourself

[wise crater]

i am sorry that this is how the thing works

[sterile field]

Why are you appending the result of .start()

[wise crater]

does start not return the object

ops

🙂

[desert pecan]

told u

[warm hawk]

How "exact"

[civic kiln]

this does sound like an xy problem. iirc, Python GIL has very little impact on running threads; and works almost identical to actual free threads. unless ofc, you're doing something that's very sensitive and/or resource hungry. which is not the case here.

!xy

[drowsy jay]

xy-problem

The XY problem can be summarised as asking about your attempted solution, rather than your actual problem.

Often programmers will get distracted with a potential solution they've come up with, and will try asking for help getting it to work. However, it's possible this solution either wouldn't work as they expect, or there's a much better solution instead.

For more information and examples, see http://xyproblem.info/.

[wise crater]

they just all need to attept to connect at about the same time and not timeout in the meantime

[warm hawk]

Like, you cannot even guarantee the processor using multiple core to process each thread exactly at the same time

[civic kiln]

I don't think "actual free threads" would bring any benefit here...

[sterile field]

Also looks like you're trying to use threads then you have a sequential for loop doing the same thing

Socket operations like connect release the gil

[warm hawk]

Doesn't seem to have any relevance to GIL
GIL only meant the processing of a bytecode take the lock, for a very short amount of time, to make sure 2 bytecode in 2 different location mess up the internal state (from my memory)

[warm hawk]

Also btw I sent you the code
I didn't noticed the message until 20mins later

[civic kiln]

where?

[warm hawk]

Old post

#1372646825466527765 message

[civic kiln]

^.^

didn't notice the ping >.<
Thank you ☕

[warm hawk]

Oh, it's because my default is not to ping lol

Image options is so if it's long you can generate an image that isn't text wrapping to make it easier to read

[drowsy jay]

Python help channel closed for inactivity

This help channel has been closed. Feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.