unit-testing

[limpid raft]

https://paste.debian.net/1377983/

not sure how to get a pycov html report

    while n < len(a) and n < len(b) and a[n] == b[n]:
        n += 1

does not have the problem.

fine by me

though it's not as nice as the for-zip

[river pilot]

--cov-report=html

You can have multiple --cov-report options

[river pilot]

i wouldn't change the code to avoid the warning, i'd add the comment pragma.

[limpid raft]

here is the hover:

i think i got it now

i think i should just take the a==b test handling out before the loop and let the loop complete at least once.

thank you @river pilot, I am always learning and there is so much thanks to you on this channel and elsewhere.

🧡

[ionic sedge]

Hi, this link in the pins does not work anymore

[proud nebula]

Looks like the entire server is down

[river pilot]

sadly, yes... they "upgraded" my VPS server, and I've been trying to get them to fix it.

[limpid raft]

With pytest, I can have generator fixtures such that I can yield an object, and control is returned to the fixture body to tear down when the object goes out of scope., Great!

However, when pytest encounters an error, it tears down the fixtures and then displays the error messages. If those include a call to e.g. (in my case) __repr__ of a lazily-loaded SQL-Alchemy object, I get a slew of errors about object-detached-from-session. I was wondering if there isn't a way to delay tear-down until after error messages have been generated?

[swift pewter]

If it's for a fixture you've written, you could catch the exception in the fixture and display it there

(Aware that isn't ideal, but the order in which things are called make the delay you imagine pretty much impossible, I think)

[limpid raft]

The exception is thrown in pytest when it calls __repr__ on an instance involved in an error, but the __repr__ method would actually result in a SELECT call to the database (it was lazily loaded only), but the DB Session has already been torn down. One answer is of course to eagerly load everything during testing.

[swift pewter]

Right, but you could at least catch the original error and print it where the session is still available. But yeah, that's not ideal and it will still have the same issue further out if you don't supress the error (which you don't want to do).

[marsh raft]

"eagerly load during testing" sounds scary -- it might be kind of complex, and now your tests aren't exercising your exact production code 😐

[limpid raft]

this is all integration testing anyway.

the unit tests have already all run

sorry if this is not the right forum 😉

[ember maple]

In general repr should never fail

Id recommend making a repr that supports partial state

[fiery arrow]

__repr__ executing a database query is definitely very spooky

is that something SQLAlchemy actually does by default?

[proud nebula]

I'm trying to fight against this in Django and getting crickets as reply :(

[silk tiger]

Can I (using pytest) have a fixture that has some internal state?

I have an object that is expensive to set up but cheap to copy

And I would like to have a fixture that creates the object once and when called after that just returns a deep copy of the object

[proud nebula]

You can just shove that state into a global variable.

[ember maple]

absolutely not - bad structure

@silk tiger have a fixture called mything_template thats high scoped, and then a fixture called mything that takes the template and returns the copy)

and if more/different needs arrive have something like a mything_manager and then different mything fixtures can ask the manager for a instance/copy

an example for that pattern is tmp_path <> tmp_path_factory

[proud nebula]

Meh. People are too squeamish about global variables in Python imo. I mean.. they're not even global like in C anyway. They're module level and it's fine.

But yea. That solution works and is pretty clean :)

[ember maple]

never start to introduce it in a codebase - the moment there is a example is the moment a fresh programmer will replicate the example and its gona bite 2-3 years down the line

[proud nebula]

If you fight it too hard, jr devs will follow that and get you foobarfactoryfactorybuilderdependencyinjector :) The dose makes the poison.

[ember maple]

Im typically handing those responsibly to a Java team for reuse before we hit that particular sweet spot

But letting people have their dirty global has repeatedly ruined a few months for me

So its turned into some sort of hill to stand on, not to die for tho

[proud nebula]

I haven't had that much bad luck. Or maybe my memory just sucks. 🤷‍♂️

[ember maple]

im dealing with >70 teams - so theres always someone new

[amber fulcrum]

I'm running pytest in an async package. I created a new test file with two simple async functions. When it runs that file it sais "async def functions are not natively supported and have been skipped". Yet it does work with all the other files. How can that happen?

[ember maple]

more context needed as we have to firgure why other async tets run while the new ones fail

[amber fulcrum]

Figured it out eventually. The other files had this little nifty line: pytestmark = pytest.mark.anyio. Such an easy thing to overlook.

[silk tiger]

Thank you I actually got there myself in the end

[supple mortar]

x problem: This isn't really a unit test (integration test?), but I don't see a more appropriate channel. I would like to test a module with pytest (not from the command line--the IDE says "run tests in tests" and I click that). The module requires a command-line argument (a path to a directory). A pytest fixture generates this argument by examining a directory full of test directories, yielding a list of paths to test. The test should run the module on each path in the list.

y attempted solution: use runpy.run_module(the_module)
and figure out how to set sys.argv[1]. Is that possible?

[ember maple]

more context needed - the usual way is to have a main function that takes args and invoke that - and if a framework is involved - use it (like click test runners)

[supple mortar]

In the module under development, I have:

if __name__ == "__main__":
    directory = sys.argv[1].strip()

I want to test it on several directories.

oops, premature send

[ember maple]

your pattern is broken then - you need a main function that defaults to sys.argv but can be passed somehing else

[river pilot]

you'll want to move logic like that into functions that can be tested

[ember maple]

monkeypatching sys.argv is a massive pain in the neck

[supple mortar]

I use the command line argument because it can be set in the IDE while developing...but I guess I don't need to be testing that.

[ember maple]

i storngly recommend a main function that takes a argparse namespace as input

[river pilot]

I'll go a step further than Ronny: a main function like this:

if __name__ == "__main__":
    sys.exit(main(sys.argv[1:]))

within that main(), you can do argparse, and call a deeper function that takes an argparse namespace.

[supple mortar]

This goes in the production code, not the test, correct?

I won't pretend I'll understand the suggestion for quite a while, but I can implement it.

[river pilot]

my two lines would be the production code, yes. Then you can write tests that call that main() function.

[supple mortar]

Okay, I think I see. Thank you both.

Is main a separate function, then, or does the ensuing code follow the sys.exit() line?

[ember maple]

basically ned is putting what i'd put into the if __main__ bit int othe main function
both vairants have some tradeoff

the important bit is that the actual application code should take higher level args and be tested with that level

[river pilot]

@supple mortar the big picture here is this: if you have some code that gets some data and does something with it, and it's hard to test because you are sure how to "get the data" during the test, then you can split the code into two parts: 1) get some data, then pass it to 2) do something with the data. Code part 2 can be a function that takes data in a convenient form and is easier to test.

(I'm not sure I explained that well)

[ember maple]

that was pretty spot on - splitting getting data and processing it in reasionable chunks safers a lot down the line

[supple mortar]

Well...this is the code that converts the data (mess) into a convenient form

So that I can write the code that is easier to test.

But the data comes in many variations.

[river pilot]

i saw the earlier description but didn't understand it, so I hope i haven't been giving misleading answers

[supple mortar]

I don't think so. I'm trying to write a converter that puts the data into a standardized format, and I'm trying to test the converter on different reference forms for the data. Making main a function that takes arguments makes sense, and if pytest can make use of that, it's what I need.

(also, I'm new at this)

The data typically arrive as a directory full of files, and I'm turning them into xArrays.

[amber fulcrum]

I found an interesting "feature" (bug?) during pytest. A mistake of one test run first affects all subsequent tests:

import pytest
pytestmark = pytest.mark.anyio

@pytest.fixture(scope='session')
async def common():
    print("\nSetup")
    yield "hello"
    print("\nTeardown")

# This mistake of not using async here cause common to be
# 'async_generator' object not the string "hello".
def test_bug1(common):
    print(f"\nBug1 {common}")
    assert common == "hello"

# But if the above test is runned first, it will cause
# "common" here to incorrectly be an 'async_generator' object too.
# Thus, it fails all tests that consume the "common" fixture.
async def test_bug2(common):
    print(f"\nBug2 {common}")
    assert common == "hello"

[cedar wharf]

hi
using pytest and pytest-mock
so i have a function called by asyncio.gather
but spy.await_count and spy.call_count are returning 0
while i'm sure the function is ran

is there a way i can test the call counts in this situation?

[tall brook]

I usually use plain monkeypatch and create mock classes adapted to specific needs

[cedar wharf]

i'm not sure how you mean
i want to make sure that this specific function is called, and not something else (as there is need to test this) ( by "called" i mean it gets passed to asyncio.gather so it runs it)
and i want it to be as close to reality as possible (hence i'm using a spy, not a mock)

[tall brook]

hm I see, sorry I'm not familiar implementing spy

[cedar wharf]

it's similar to mock
but it lets you run the actual code as well

[tall brook]

because all I understand is monkeypatch. What about creating a mock class that calls the actual call from within and also performs the counting? Probably is what pytest-mock spy does internally

[ember maple]

Sounds like a bug i the asyncio plugin

Pytest core doesn't support async fixtures

[toxic pivot]

could someone test a module? it was liked by 1/1 testers!

[limpid raft]

Do you know of a hook or the like in pytest that would execute after errors have been reported and everything is tearing down?

Fixture contexts tear down before errors are being reported.

[maiden pawn]

Fixture contexts tear down before errors are being reported.
try

@pytest.fixture
def your_fixture()
   # startup code
   yield smth
   # teardown code

[limpid raft]

If I do this for a fixture that is an SQLAlchemy ORM (integration tests), and an error occurs, and the error message includes a repr() call that causes another lazy load from the database, then I am told that the session has expired, i.e. it was closed before the error. If I use print() debugging, I can verify that (it seems at least as if) the fixture is torn down before the error message is evaluated.

the yield fixtures is what I mean with "fixture context"

[ember maple]

Again if you Sa repr breaks on certain object states its wrong

[limpid raft]

You are right. I am technically abusing repr. However, repr isn't strictly defined, so I am also not abusing it.

It is debatable that a Pytest error message is evaluated so lazily that every fixture has been torn down. Why isn't the error message evaluated in the context of the error, and later echoed?

[ember maple]

Among other tings it sometimes prevents oom kills of test processes not to do all reporting eagerly

[bitter wadi]

:incoming_envelope: :ok_hand: applied timeout to @cloud edge until <t:1749930984:f> (10 minutes) (reason: duplicates spam - sent 4 duplicate messages).

The <@&831776746206265384> have been alerted for review.

[silk tiger]

I have an issue with pytest fixtures that I don't really get
I have a library that requires logging to be initalized before functions of the library are called
I put the fixture in a file

from lib.utilities import setup_logging, LogLevel, set_level
import pytest


@pytest.fixture(scope="session", autouse=True)
def lib_logging_setup():
    setup_logging()
    set_level(LogLevel.off)

The file is next to all other test files but the fixture does not get called
How can I fix this?

It works when I call the file conftest.py but I don't get why test_logging.py does not get picked up

[proud nebula]

Why would it work when the filename isn't conftest.py?

Afaik the docs say that's the the name it will load.

[limpid raft]

How would you unit-test a coroutine that polls a file's mtime. It's an AsyncGenerator, i.e. it just yields None whenever the file's mtime increases, checking every couple of seconds (no asyncinotify on Windows). Do I mock pathlib.stat and have as side-effect e.g. [0,0,1,1,2,2] and verify that the function yields only on 0→1 and 1→2?

Similarly, how would you test the same function as used on a platform where asyncinotify works? Do I mock Inotify?

[proud nebula]

The first rule of mocking is to not do mocking. Maybe you can pass the function to use to poll for changes to the function under test? So by defualt stat, but in tests your own thing.

[limpid raft]

hm. not a bad idea, I had not considered DI at this level

[hexed cloak]

You could just use a tmp_path and change the mtime from another task

[limpid raft]

except that wouldn't be a unit test anymore… but yeah, that had occurred to me.

[hexed cloak]

Don't worry about 'unit' tests just write good small tests

[maiden pawn]

i will add extra comment to @hexed cloak

• just make that your test is preferably using only local controlled dependencies. (you can wipe it / reraise again)
• make sure the test is reproducable. If u run it again, it will give The Same result.
• when tests are run in parallel, it is still giving the same result, regardless of order in which test runs

Your Fileystem is under your control, so it is fine

as long as u follow such types of rules, that test is not very unit is not mattering

[limpid raft]

thank you!

[limpid raft]

yeah, you are right

I will go down the DI path I think though

one other question though: I've seen it in other tools: try-import some library, and fall back to bare bones handling on ImportError. How does one test this??

try:
    from asyncinotify import Inotify, Mask

except ImportError:
    _INOTIFY = False

else:
    _INOTIFY = True

[marsh raft]

just run it once and see if it works.

no need to automate that.

[limpid raft]

yeah but my 100% coverage 😉

[marsh raft]

😭

I think I'm around 70%-80%

life is short

[weary quarry]

You use a tool like tox or nox to run your tests in environments that don't have the imports. This also requires running coverage with partial tests and then compiling the final results once all variants are run.

Likewise, if you have cases that are python version specific, you can use docker or CI runners to run the tests for those cases.

[limpid raft]

omg that sounds like so many goats to shear and yaks to shave 😉

but yeah, you're right. that seems like the way

i fucking hate having to support windows

[weary quarry]

I like nox for that purpose and github actions for the CI.
https://github.com/Preocts/python-src-template/blob/58366b9a12ab102d315bcdd0d8b329aebad521af/noxfile.py#L86-L119

[limpid raft]

i don't use github but I get it

[weary quarry]

Locally I can just run nox -s test for my tests with coverage output. In the CI I can run nox -s test --partial-coverage, collect the coverage output files, and then nox -s coverage_combine once I have them all.

Actually setting up the implementation details of how to run with X dependencies versus Y dependencies becomes a small matter.

[limpid raft]

thanks for that input!

[proud nebula]

# pragma: no cover

[limpid raft]

so far, I have none of those. but yeah, I think I am possibly overdoing it a bit. Still learning though.

[drowsy hawk]

100% coverage is a nice merit badge but far from a make or break especially when cross platform

[twin shale]

100% line coverage is level 1, not worth much 👀

But it depends of course. How problematic would a bug in your product be?

[silk thorn]

Python noob here, what are some common frameworks used for unit testing python?

[swift pewter]

I'd recommend pytest, it's very commonly used.

[silk thorn]

Ah perfect, i'll have a peek thanks

[buoyant dune]

is it possible to create an automated test case "generator" by recording the testing process then converting it into codes of automated test cases?

like Selenium IDE, except the output will be a test case written as selenium python test case

or do I need AI for this?

[proud nebula]

I think you can. You will end up with a horribly slow test suite that is almost useless though.

[buoyant dune]

Ok so, I wanted to make "automated testing system" for my college project, but my professor said such automated test cases aren't enough to be considered a "system" or whatever it is, so maybe a test case generator might work but I'm not sure, any ideas?

[proud nebula]

How can you generate tests automatically? What would that mean logically? What would such a thing test?

[tall brook]

https://playwright.dev/docs/codegen

[proud nebula]

Before you use that tool, think critically about if you can answer the questions I posed.

[primal patrol]

how do you test your locks against race conditions in particular places in functions?

i'm about to mock a call to something in my function to side-call barrier.wait() (where barrier is my threading.Barrier(n))

an example, i guess...
https://paste.pythondiscord.com/P7TA

(no, i don't want to spawn a bunch of threads and hope a race occurs out of the blue)

[ember maple]

There's no sound testing for race conditions

They occur from misaligned design

The best course is to pick known algorithms/ implementations or to design based on well known state graphs

In particular awareness of thread behavior of builtin methods is key

For example im unaware if dict.setdefault maintains its invariants in free threading python

However one still need due diligence tests to show absence of basic rase conditions

[primal patrol]

actually, in this particular case, the test was quite easy, because we wanted to prevent a race condition on a passed-in object

i ended up implementing the test as follows:

https://paste.pythondiscord.com/ELBQ

the only scenario where this test breaks is when we get a false-positive from a timeout excess

and i couldn't use your advice in this particular case

[primal patrol]

For example im unaware if dict.setdefault maintains its invariants in free threading python
i believe it does, i checked that it opens a critical section before calling _impl

[primal patrol]

commented https://paste.pythondiscord.com/AHUQ

but 0.2 feels impossible

to be taken for just a little bunch of bytecode to execute

[primal patrol]

nevertheless, thanks a lot! (didn't want to be rude :-))

[ember maple]

@primal patrol my personal take on race conditions is that the best way to handle them is to remove the need for them by changing the design so races are no longer possible

this in particular make stuff like instance located caches a no go tho

[primal patrol]

what do you mean by a no go?

well, i just wanted to test a lock

[ember maple]

well -instance located caches are typically something where you need locks for thread safet

[primal patrol]

and i did it! if i remove the lock, the test fails

if i have the lock in place, it doesn't

[primal patrol]

yup

the nature of the functionality implies you write to instance-located caches

we considered some other approaches too, though

you either need __weakref__ in slots or __dict__

[primal patrol]

https://github.com/jaraco/jaraco.functools/issues/23

[primal patrol]

or, well, you can make a refleak by using the id() as the key and the instance somewhere around

or risk a weird error by not holding a ref at all

[ember maple]

one thing that helps a lot is using less subclassing and less complex object - going for composition more

[primal patrol]

yes, that's why I love rust

xd

but well, the issue was about subclasses so

i can't really help :-D

[ember maple]

well, its applicable in python as well - use less sub-classing is directly applicable, unfortunately legacy systesm may have a massive backlog

[primal patrol]

of course! I love using functional approach with namedtuples and some variants of DI / strategies if any classes are involved most often

which brings me back to https://github.com/bswck/make-typed-namedtuples-final

[ember maple]

i prever frozen dataclasses - not being a actual tuple is important for modeling

[gentle thicket]

driver.get("https://my.account.sony.com/sonyacct/signin/?#/signin/input/id")
driver.maximize_window()
driver.get_screenshot_as_file("screenshot.playstaion.test.png")

WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, 'Sign-In ID '))).send_keys(....)```


Need help not working Sgin-In ID

How i could know what it actually reading to where write the sign in automatically

F12 but then?

[twin shale]

When using pytest is there any reason to use hamcrest? Pytest usually seems to give equal or more readable output for plain asserts or pytest methods.

[thorn radish]

pytest : The term 'pytest' is not recognized as the name of a cmdlet, function, script file, or operable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1

• pytest test_calculator.py

•   + CategoryInfo          : ObjectNotFound: (pytest:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException
  
  

i did pip install pytest

anyone can help

[proud nebula]

try python -m pytest

[thorn radish]

ty

it works

[iron lark]

guys

I need a quick help please anyone help me with this

im applying for an internship

and in one of the fields they have asked "Did you find the apple pie"

What am i supposed to find/look at or know?

the page seems fine

looking at the code in inspect page, there is nth i did find this which was not visible in the ui

is this the "apple pie"?

Help me with thhis guys i really need this internship

https://hazesoft.co/career/qa-intern/

this is the website

[proud nebula]

Maybe they just want you to admit that you don't know. The worst employees are ones that can't admit not knowing or that they have failed.

School really sucks at teaching people this lesson, and it's as important as being able to read.

[iron lark]

I did check thoroughly and didn't find any clue. Im afraid there is smth and Im unable to find it. Thanks for your advice tho.

[blissful pivot]

checked the inspector and found a hidden image that isnt visible in the main UI that’s probably the apple pie

[blissful pivot]

no ther is some

[blissful pivot]

its probably a littel QA test

[bitter wadi]

Rules

10. Do not copy and paste answers from ChatGPT or similar AI tools.

[fallen tinsel]

Yes

[still orbit]

what exactly is unit testing?

[marsh raft]

I think of it as: tests which are meant to run very quickly, and not interact with the outside world, and which exercise reasonably small bits of code (a single function or method, say).

I don't think there's a really precise definition.

[proud nebula]

Sometimes people use that term when really they mean automated testing in general. 🤷‍♂️ That's why this channel has the name it has.

[ancient cove]

Hi Guys i just finished my frist full game on python i would appreciate u guys if u would test it for me and look in the code for bugs

[ancient cove]

Anyone?

[swift pewter]

What does this have to do with #unit-testing ?

[ancient cove]

idk someone told me paste here

@mighty willow

[swift pewter]

No, this is certainly the wrong channel for this. You can ask for a code review by opening a help post with the code review tag: #1035199133436354600

[ancient cove]

Okcsry

[mighty willow]

wha

[wind zenith]

Hello,
I'm still relatively new to testing. I'm adding tests to a personal project and am currently making tests for the service functions of my routes. Some of these functions call other functions like:

def write_ll_to_file(name:str):
    """Get the data of all nodes in the linked list as and write them to a JSON file with
    the given name
    
    Parameters:
        name: str - The root name of the file
    Returns:
        out - A confirmation message
    """
    root_filename = secure_filename(name) # convert the given name into a valid filename
    # logger.info(f"secured filename is {filename}")
    ll = get_ll(current_app)
    
    if is_unique_filename(root_filename): # if the filename is unique
        logger.info(f"filename validated")
        full_filename = get_full_filename(root_filename)
        curr_dir = os.path.dirname(__file__)
        path = f"Saves/{full_filename}"
        save_path = os.path.join(curr_dir, os.pardir, os.pardir, os.pardir, path)
        ll_jsonable:list[dict] = ll.getAll()

        with open(save_path, "w+", encoding="utf-8") as file:
            json.dump(ll_jsonable, file, ensure_ascii=False, indent=4)

        return f"Question group saved to {save_path}"
    else:
        raise ValueError("Name already exists")

Should I mock every call to another function and their return value? I know I should mock ll, ll_jsonable, and open at the least, but what about things like root_filenam, curr_dir, and path? I'm just a bit confused about to what degree I should mock things

[pulsar oracle]

Personally, I wouldn't mock anything to test something like this. This is python and it is very trivial to test code that uses the filesystem does what you want. If it is supposed to take a linked list and write it to a json file then the way to prove that it works, first it to make it easier to test in isolation by taking the linked list to write as an additional argument, and to test it by passing in a file path, and the linked list and seeing if it works correctly. Alternatively you could change your design to make a function to turn your linked list into a dictionary or list or whatever and just skip caring about json and files as much. But it's still important to check that a file is actually written as expected and I usually use the temp file module with TemporaryDirectory for that.

[wind zenith]

I see, I use get_ll() in several service functions, but it may make more sense to just turn it into a argument and have the endpoint call and pass it.
Thank you for your help!

[pulsar oracle]

No problem!

[marsh raft]

+1 to "try to avoid mocking". Mocking is occasionally useful, but is usually a pain in the ***.

[ember maple]

the major pita with mocking is that people think its okay to use it on complicated things they don't own - i have had to undo so many mock tests that instead of adding value where just locking obscure implementation details in place without truly testing functionality

[limpid raft]

I am wondering now if I am insane…

1. Argument of type "Entry" cannot be assigned to parameter "entry1" of type "BaseEntry" in function "set_entries"
     "Entry" is not assignable to "BaseEntry" [reportArgumentType]

The function set_entries takes BaseEntry instances. Class Entry derives from BaseEntry. Thus, an Entry "is-a" BaseEntry.

Where is my logic error?

It works, of course. Python is okay with it. Maybe Pyright and the folks at Microsoft have not heard of Polymorphism?

Pyright is really whacky at times

reveal_type(a<b) ● Pyright: Type of "a < b" is "Any"

[limpid raft]

sorry, this was meant for the typing channel.

[desert adder]

what is unit testing

[ember maple]

this commonly refers to the practice of testing part of a program in absolute isolation without external dependencies - over the years definitions have faced some weakening tho

[jolly vortex]

Hi, I have a codebase where in some places I use assertions to help type checkers understand the type of a variable; now the same codebase has some tests written in pytest too, is there a way to ignore all the assertion in the code that needs to be tested? (assertion are ignored at runtime because I run the project using the -O optimization mode)

[ember maple]

theres no builtin mechanism - but if your code fails those assertions to begin with - then stuff is likely broken or wrong - at least that was commonly the case whenever i dug into such a codebase - by now my default mode of operation is - always leave asserts on - if they fail something was broken and needs fixing - nothing is worse than silent errors accumulating broken data just because someone silentced the guardrails

[jolly vortex]

nevermind I fixed the issue

[jolly vortex]

the point is that that assertion wasn't any test or logic check at all, it wasn't there to provide security, it was there just to help typecheckers infer the correct type

[marsh raft]

I have tons of those, but they don't interfere with my unit tests.

[ember maple]

the point is - if they are nt impeeding anything in the test - just leave them in - pytest will not engage in removing assertions, as that tends to hide actual issues surprisingly often

[proud nebula]

Running without assertions is imo a very bad idea

[jolly vortex]

it's not like you can

I were just asking if there's a way in pytest to ignore them but I don't need it anymore

[jolly vortex]

like specifically ignore one assertion

[twin shale]

Know that you are ignoring asserts in not only your own code but also all code run from imports. -O is risky.

[pulsar oracle]

Maybe I am just inexperienced but I only use assertions as a stand in for exception handling, if I am not sure I want to handle something yet or not sure what I want to raise. If a bad value is meant to be prevented I am probably going to raise a value error, and have tests in place for the behaviors of that code. I have used them in places that the behavior might subtly change in the future breaking things and there aren't integration tests in place like for stuff that formats something and sends it over websockets. But the idea that you need assertions in your own code or need to run with them sounds wrong to me and like a result of not having enough tests to begin with. I can understand not wanting to mess with third party code that might go haywire due to preventing bad values with assertions rather than proper exception handling though I suppose. Actual example: I have a class which writes data to a websocket, it uses a method that writes base64 and xml, the design is to have that method add a null byte on the end, it might not in the future, if it doesn't this code will break and since there's no tests it'll become confusing, so to be absolutely sure I just threw an assertion statement in. But if I had integration tests this would be unnecessary and I would know it works as expected. The method in question is also fully tested, the design just might change.

[jolly vortex]

The assert keyword is not meant to be used as actual logic, it should be used only when testing / debugging https://docs.python.org/3/reference/simple_stmts.html#the-assert-statement

when you are in production you don't run code in debug mode

[twin shale]

Oh, my bad. I think I jumped to conclusion without thinking, that the -O flag would skip all raises. Which would very much not work. I've never run with this flag so I shouldn't have said anything before getting in touch with it.

[proud nebula]

That was the original assumption and maybe the docs say so. But not checking data in prod when it's ACTUALLY important is madness imo. If you can formally verify that it can't hit the assert sure, but that's about it imo.

You have to separate practice from theory.

[jolly vortex]

It's not important as I have said and if it were I wouldn't be using an assert because that's an abuse, it wasn't designed to be used like that.

[wanton axle]

Where should I start ? Official pytest docs? I get a lot of test code from GitHub copilot. It doesn't always make sense to me...

[river pilot]

do you have specific questions? Tests are definitely a different style of code, and pytest has a lot of power in unusual structures.

[wanton axle]

Sadly, but that's the current state, most of my testing code is genereated by github copilot. Sometimes i can make sense out of it and spot incorrect tests, but sometimes it's difficult because of pytest / unit test syntax, like mocks, magic mocks, fixtures, patches, mark.asyncio, etc. So i though it would be good for me to maybe start from scratch and follow some course, etc. So i can either write tests myself, or be able to judge GH copilot generated code better.

[pulsar oracle]

I'm not sure how to address this or that having AI generate your tests for you is a good idea. The value of a test is proven that your code isn't good enough, which for quality sake you should assume it isn't unless you've tested this, following this logic you can simplify your code and make decisions, even subtle ones like accepting lambdas in places to make it easier to test. If you want to get good at writing tests, write the signature of the function you want first, then write a test that will pass input to that function and check the output or check what it interacts with. If you aren't careful about this (typically code with a lot of mocking or patching isn't in my opinion) then you risk writing tests that test that the code you wrote is the code you wrote, not that it does what it is intended to. You could write the tests for what should go in and what should happen first, then have AI generate the implementation. If you're not sure how to write a test, using the unit test module it's simple. I personally use the unittest module but the same idea applies universally, setup/call your code, then assert that it does what you want, write clean tests that you can manage and understand over time. To give you an example: The other day I wanted to write something that would monitor how many requests per second a server was getting per second, so I wrote a class that can be notified of requests every second, then for the design of that class, I made where it gets timestamps from be passed in with a function. To test it I put in a function that would report a time I'd set and add time to so that I could test the notification/calculation logic, and even wrote a fixture (the TestCase in the unit test module is a fixture, you just put tests on it, there's only one, and it's setup for each test method on the class first). See the attached screenshot for an example of this. You should start by writing simple tests that prove your code works, you don't need much to do it at all, whatever you use, and you should practice this and prove that your code works, then get better at writing them.

[tall brook]

"obey the testing goat" book is a good starting point in my opinion.

[sleek valley]

can i use stress test ane benchmark together?

[modern hedge]

i am mocking db query / response in my tests w/ pytest. what is the best way to pass the query and response to the mocker fixture per test function?

[marsh raft]

dunno what you mean

[proud nebula]

Sounds like an xy problem. Mocking db interactions seems like a bad idea for example.

[tall dawn]

i made a text to ascii art thing
someone that has a cmd that supports ansi color codes can u test it

[bitter wadi]

Click here to see this code in our pastebin.

[tall dawn]

bruh

[marsh raft]

I wrote essentially that in BASIC in the 1970s

[lucid hull]

looks good

[forest jetty]

hypothesis is thread-safe now! https://github.com/HypothesisWorks/hypothesis/issues/4451

[river pilot]

someone in pygen was having trouble understanding parametrize, and looking around, i didn't see something that I thought explained it step by step. I wrote this: https://nedbatchelder.com/blog/202508/starting_with_pytests_parametrize.html

[swift pewter]

I always found it weird that parametrize is under pytest.mark. Like it seems weird that this one special mark also somehow generates test function variants.
I think it was in a workshop at some pycon where it finally clicked — pytest.mark.parametrize isn't some special function with two parameters. It's a mark like any other. But a built-in pytest plugin then finds test functions marked with it and then it generates the test cases.

[river pilot]

i'd never quite thought about it. interesting. fun fact: my first draft I just typed "@pytest.parametrize", and I had Claude review it, which pointed out I needed ".mark."

I feel like they should make an alias to @pytest.parametrize

[jade magnet]

@ember maple I recall you saying something to me about pytest adding native support for async testing in the future – any news on that? I am struggling with reconciling contextvar support and task groups in generator fixtures, so I'm hoping there could be some fresh ideas on your side.

[pearl cliff]

that would be nice. as l3v pointed out, it is indeed just a mark + a built-in plugin. but maybe that's also confusing to beginners, and comes across as a leaked implementation detail moreso than an elegant design

certainly that relationship is not emphasized in the docs

[river pilot]

and why do we need to know about that relationship? We'll never use it as a mark (run all the tests with parametrize? no.)

[pearl cliff]

It explains why the name has .mark in it

[river pilot]

no, i understand all that. from a user perspective, it isn't a mark. it's an implementation hack

[pearl cliff]

Yeah, agreed on that

I thinking is that, at least if they state "this is called mark because it's actually just a mark, click here if you want to know what that means in more detail", it makes the framework seem less mysterious and confusing

Which I think it is, to many beginners

It's one of those "work off of these examples and try not to think too hard about what it's doing" things for a lot of people who are starting out using it, especially people who are less advanced at programming in general

[ember maple]

i haven't gotten around finishing pluggy async support - so currently its on hiatus. maybe after im back from the summer vacation

[primal patrol]

have you ever found yourself having a nested doctest that didn't run?
https://github.com/python/cpython/issues/44538

[teal hinge]

Real basic setting up testing question.
I am in my env.
I have my base project folder, and in it:
tests -> test_core.py and
what_next-> __init__.py, core.py.

test_code.py has from what_next.core import ProjectClass.

On running pytest from the base project folder (one down from what_next the error message is saying:

============================= test session starts =============================
platform win32 -- Python 3.11.9, pytest-8.4.1, pluggy-1.6.0
rootdir: C:\Users\deleted\projects\what_next
collected 0 items / 1 error

=================================== ERRORS ====================================
______________________ ERROR collecting tests/test_ex.py ______________________
ImportError while importing test module 'C:\Users\deleted\projects\what_next\tests\test_ex.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
....\AppData\Local\Programs\Python\Python311\Lib\importlib_init_.py:126: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tests\test_ex.py:3: in <module>
from what_next.core import WhatNext
E ModuleNotFoundError: No module named 'what_next'
=========================== short test summary info ===========================
ERROR tests/test_ex.py
!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
============================== 1 error in 0.16s ===============================

Help me troubleshoot please!

Using pytest in the project level what_next folder, that contains the other two folders.

[teal hinge]

update: python -m pytest works, which I got from the reddit conversation https://www.reddit.com/r/learnpython/comments/11qtphw/my_pytest_code_doesnt_find_my_code/

[split field]

!unmute 323482775476895745

[bitter wadi]

:incoming_envelope: :ok_hand: pardoned infraction timeout for @fallow granite.

[split field]

!paste

[bitter wadi]

Pasting large amounts of code

If your code is too long to fit in a codeblock in Discord, you can paste your code here:
https://paste.pythondiscord.com/

After pasting your code, save it by clicking the Paste! button in the bottom left, or by pressing CTRL + S. After doing that, you will be navigated to the new paste's page. Copy the URL and post it here so others can see it.

[fallow granite]

thanks

[split field]

Use the paste bin for that much text

[fallow granite]

how do i recover all that text

[split field]

Otherwise it trips the bot's spam filters

[fallow granite]

alright, i wont

lesson learned

[split field]

Sec...

https://paste.pythondiscord.com/PP3Q

[fallow granite]

Thanks

What would be the good way to write the test for observer_type and a key_type?
Reading on internet i also came on that its bad to write test for private methods and instead in my case i should do add call for checking observer_type and key_type
I dont know much about writing test so help me out.
I heared that its not a bad approach to write a test only when it fails, is that true?
Main https://paste.pythondiscord.com/XIHQ
Test https://paste.pythondiscord.com/KLNA

[valid sun]

As I assume you've read, the reason it's bad to write tests for private functions is that it "locks in" the behavior of your implementation in your test. This means that if you want to change the implementation while keeping the behavior the same (literally the definition of refactoring), the test will have to be rewritten. What you want to do is to document the behavior of the program which you care about, as seen from the outside. The reason to write things using private methods or variables is that you'd be free to change it later without breaking the public interface. If you add tests which depend on the private stuff, you guarantee that at least one thing depends on it 🙃

Regarding writing tests which fail: You may be thinking of TDD (test driven development), which suggests that you define what you want your program to do by writing a test for it, then write the easiest possible code you can think of to solve that, then try improving the code without breaking the test. It can be a useful way to figure out what you'd want the code you're writing to look like in the first place, and a good way to ensure all your functionality has tests, but IMO having tests is better than not having them regardless of whether they were written first. I wouldn't assume that just because you didn't start with the tests, it means you shouldn't write tests 🙂

If you write a test which passes the first time, one tip I've heard is to break your implementation in such a way that the test should fail. This should help you gain confidence that your test actually tests what you think it does.

[fallow granite]

I was doing that 😄 didint know if it was good, i am doing that for mostly valid check. Not sure if should do for invalid

[valid sun]

You don't necessarily want to test every single behavior of your code. You want to test the behaviors that you actually care about. If some property of your implementation doesn't matter, leave it out of the tests.

[fallow granite]

oh ok

so i guess its not wrong that i read somewhere to test core functionality and write more when you get an error

[valid sun]

Sure, why not? You're talking about error handling, right? If the error handling behaves incorrectly, that should also cause your tests to fail, assuming you care about the error handling.

[fallow granite]

I mean like in general. I have heared about TDD write test first and make code later. Or writing test later when you have a code. But i also heared to write test when code only fails, i feel like this way wont allow to write too excessive test

[valid sun]

I think I see what you mean. Except that I'm not sure I know what you specifically mean with "when code fails". You might be thinking of the advice that you should write new tests when you encounter a bug, so that after fixing the bug, you now have a test for it that protects against regressions. It also helps to figure out what the "core functionality" of your code is. After all, if you had a bug which wasn't caught by a test, it means that you had not tested some of the behavior you cared about, since you labeled that incorrect behavior as a "bug".

[fallow granite]

"when code fails" = encounter a bug/not wanted feature 😛

[valid sun]

Tbh, test first or test after, whatever works for you. Both are better than no tests. But I would recommend trying out TDD if you haven't, and try to see if you can experience some of the benefits that people are talking about.

A good time to try TDD might be when you do encounter a bug. E.g. "the code I've written should do this when I call it in this way, but instead it actually does that". Write the test which documents the "this" behavior. Then implement it.

[fallow granite]

Im not sure how it would work, writing test first of wanted behaviour and than writing code, definetly worth trying cuz its new experience

Thank you for the talk. I learned some new aspects and will have to modify my test to them. Feel free to ping me if you missed anything usefull

[valid sun]

Anytime 🙂 I think my main recommendation for you would be, try different approaches to testing and see if you can spot benefits to those approaches. When I first started working as a software developer, tests were something that I just didn't see the utility of. It was something which I heard from all fronts were necessary, so I guess I just have to do them, but I didn't really buy it.

A few years later I'm actually on board with the idea, because I've experienced first-hand the benefit of having tests, and the pain of not having them. And I've also experienced some of the benefits of writing tests first. I'm by no means perfect at testing everything, but now when I test something it's because I think it will be better for me, not just something I do as a chore at the end. If I didn't really buy the benefits of testing, I probably wouldn't stick with it unless forced to from external pressure, and I'd probably write worse tests.

So try to experience what the tests actually give you 🙂

[fallow granite]

Makes sense, even if my test is not covering everything at least its covering something and it might benefit me, its a good thing that test dont fail and its there in case its gonna fail. At least i understand it like that

[fallow granite]

I want to ask about testing more. I learned that testing things marked as private is not good and i should test the function that use them. So i got a question if like 20 functions would use the same private thing, should i test that 20 times through that function? For example i got this

    def add(self, observer_id: Key, observer: Callable[[Event], None]) -> None:
        """Add new observer"""
        self._check_key_type(observer_id)
        self._check_observer_type(observer)
        self._observers[observer_id] = observer

    def send(self, observer_id: Key, event: Event) -> None:
        """Notify observer by ID"""
        self._check_key_type(observer_id)
        self._check_event_type(event)

If i want to check _check_key_type do i check it twice on add and send?
ig. test test_on_add_key_type, test_on_send_key_type

Also is there some sort of naming convention for testing? I only know that for pytest it required to start with test

[proud nebula]

The "don't test private" things is more of a guidline. You have to think for yourself. If that "private" code is cleanly understandable and reused, imo test it.

[fallow granite]

Ok. Thanks. By any chance maybe you know what is the common pattern of naming a test?

[proud nebula]

well.. in pytest you have to name them test_... so you are forced to do that :P But otherwise I like test_function_name_description.

[fallow granite]

i read that including function name is not advisable, because everytime you change the name you need to change in the test. But in my opinion i dont care much because i dont believe the name is change often

[pulsar oracle]

I use classes for this reason and sets up the methods to be named better. Like TestTheAdditionFunction then add methods like test_should_add_positive_and_positive_correctly and so on. The convention uses what it should do and should ideally get you to think about what it does rather than how it does it.

[fallow granite]

that should seems redundant but it does convey if test result is valid or not valid

[proud nebula]

Sure, but honestly, if you change the name of the function, maybe the behavior has changed enough that you need to rename a bunch of stuff anyway :P

[fallow granite]

That was my thought also

[proud nebula]

I think most people worry too much about those things, to the point of it affecting their productivity. That to me is a bigger risk than having a slightly incorrectly named thing down the line that you will immediately see is wrong and just rename.

Velocity > perfection.

[fallow granite]

Im from that lot 😄 I have speed but not velocity

Naming things is so hard and if structure is off im flipping tables untill its my way

[fallow granite]

How to check for not valid? Do i just do != and provide false value?
Like this?

def test_observer_not_added():
    def dummy():
        pass

    el = EventListener()
    el.add('key', dummy)
    assert lambda e: e != el.get('key')

seems like a pointless test to me but idk

[proud nebula]

What is the lambda doing there?

[atomic thistle]

I'm not sure what you're doing here? The lambda is never called, the code just asserts that lambda is truthy, which it always will be

[fallow granite]

I got rid of that test

dont know how to test not added properly but i know if its added i can call it

[proud nebula]

Eh. No. Think. Don't just give up. How did you get there?

[fallow granite]

Well i was writing a test for added for valid check

def test_observer_added():
    def dummy():
        pass

    el = EventListener()
    el.add('key', dummy)
    assert dummy == el.get('key')

[atomic thistle]

Maybe you want to write a test that asserts an exception is raised? https://docs.pytest.org/en/7.1.x/how-to/assert.html#assertions-about-expected-exceptions

[proud nebula]

That makes more sense. Why did you write a lambda?

[fallow granite]

I write to test if the returned observer is not this observer

    def add(self, observer_id:Key, observer: Callable) -> None:
        """Add new observer"""
        if not callable(observer):
            raise TypeError(f"{type(observer)} is not callable")

        self._observers[observer_id] = observer

   def send(self, observer_id:Key, event):
        return self._observers[observer_id](event)

Than i write those test and i thought they suffice for not added, if its not callable it means its not added

def test_observer_is_not_callable():
    el = EventListener()
    with pytest.raises(TypeError):
        el.add('key', 1)

def test_observer_is_callable():
    def dummy(event):
        return True

    el = EventListener()
    el.add('key', dummy)
    assert el.send('key', 'event') is True

[proud nebula]

You didn't answer the question

[fallow granite]

realy?

i thought im testing function != returned function

i dont know how else to explain

i just write lambda to avoid writing a second dummy function

[swift pewter]

Ohhh! You mean assert (lambda e: e) != el.get('key')

Not assert lambda e: (e != el.get('key'))

[fallow granite]

wait, it parse that as an action in lambda

now that i read it, makes so much more sense why its wrong

[swift pewter]

Don't you just want

assert el.get("key") is dummy

?

[fallow granite]

I want to check for not valid

[atomic thistle]

Can you expand? Maybe a code example

[fallow granite]

I got this test #unit-testing message for checking if observer is added, how to check if its not added?

[swift pewter]

assert dummy != el.get("key")

, then?

[atomic thistle]

Maybe you want to test the behaviour when the key doesn't exist?

[fallow granite]

its that simple?

Some test feels so simply that i doubt if im doing good

[atomic thistle]

assert el.get("not_valid") == ?

[fallow granite]

in this case i would get key error since get try to get value from dict

[atomic thistle]

Right, is that what you want to test? (I'd expect get to return None by convention)

[fallow granite]

I read that i want to test if add actualy adds to the dict internaly and if add actually works, i dont know what that means and it might confused me
I got all these 3 check for add

def test_observer_added_to_the_dict():
    el = EventListener()
    el.add('key', lambda e: e)
    assert 'key' in el.get_all()

def test_observer_not_added_to_the_dict():
    el = EventListener()
    el.add('key', lambda e: e)
    assert 'keys' not in el.get_all()

def test_observer_added():
    def dummy():
        pass

    el = EventListener()
    el.add('key', dummy)
    assert dummy == el.get('key')

2 is for if its in the dict and 1 for added

[atomic thistle]

The code before the assert in test_observer_added is the same as in test_observer_not_added. How should the not_added test be different?

[fallow granite]

I really dont know 😄

i feel like to check if its truly was added its needed to check if the key is in the dict and if the value in the dict is correct by that key

[atomic thistle]

I don't think you need to test this behaviour, you also don't need that second test. You're testing against the negative, a scenario that can't realistically happen. If you do that you'll be stuck writing tests forever.

It's like writing:

def test_dict_not_added():
    mydict = {'a': 1}
    assert 'b' not in mydict.keys()

[fallow granite]

So i dont need to test for not valid?

[atomic thistle]

Is there particular behaviour you expect in the case an invalid key is passed?

[fallow granite]

not really, its just if you try to call the non existant thing the error get raised

[swift pewter]

You could test that

with pytest.raises(KeyError):
    el.call_or_whatever()

[atomic thistle]

For example if I was writing tests for Python dictionaries I might write a test like this:

def test_dict_key_error():
    """The getter raises a KeyError if non existant key is passed."""
    mydict = {'a': 1}
    with pytest.raises(KeyError):
        mydict['b']

[fallow granite]

i do test that with test_observer_is_callable and test_ovserver_is_not_callable

[atomic thistle]

That's checking for a TypeError not a KeyError

[fallow granite]

hm, testing is hard 😄

writing pointless test is pain

i guess i take a piece of paper and try to think throught what i want to test

Is it ok if one test for one method covers other method?
So for example i got

    def get(self, observer_id:Key):
        """Return observer by id"""
        return self._observers[observer_id]

and if i want to check if its correct key i might try to catch key error, but do i have to do the same for like add, send or any method that requires the key?

!paste

Ok, i tried to write down what i think i should check and i skipped checking for not valid https://paste.pythondiscord.com/O2ZQ

[molten hollow]

Why would you write a test for something that's in stdlib?

[molten hollow]

Yes, that's okay - more over, it's unavoidable. Tests often reinforce oneanother.

For example, if you write a stack that has .pop() and .push() methods, then you need to use both in a test.

[proud nebula]

He was taking a well known data structure as an example to make the discussion clear. I don't think anyone was confused except you :P

[proud nebula]

Not if the test is for "push" of a single item onto an empty test :P

[molten hollow]

If you have only .push(), what are you going to assert? You need to call some other method to do assertion, check that is not empty, something like pop(), top(), size() or anything else.

Test that calls just .push() and nothing else doesn't assert anything.

[proud nebula]

Tests don't need to assert anything. Just not crashing is testing something.

Aaaanyway

[molten hollow]

You're right on this one, but these kinds of test aren't particularly useful. @fallow granite so in short, if your tested methods interweave each other, that's totally fine.

[fallow granite]

Okey, i got it. Gotta learn more about testing, because i can go and write many pointless test that actually in a sense test python data structure like dict. Like im starting to understand that my class might have add but if that method only do some_dict[key] = value i dont think i need to test that

[molten hollow]

Well, you're right, but I'd add to it.

In coding and tests, there's the WHAT and the HOW. The WHAt is want you want to do, your problem trying to solve, something the user needs.

The HOW is how you do it, the implementation, the exact function call, etc.

The tests should be more about that WHAT, than the HOW.

[fallow granite]

maybe having a small integration test would be better than unit test in such case

i hope thats how its called :d

[molten hollow]

It all comes down to what you're actually trying to code.

What are you trying to create? 😄

[fallow granite]

Its an event listener that hold mostly a single dictionary and i can add callables to it and than later call send method and provide a key to callable and an event

it usess observer pattern

[molten hollow]

Okay, so it sounds like you're trying to create an event listener, so that certain parts of your systems are decoupled from other parts of your system, so that one part can listen to something, and other can call it, right?

[fallow granite]

yea

[molten hollow]

Would you like some help? What tests do you already have? If you're interested, I could show you how I would test-drive that.

[fallow granite]

I had some test but i just deleted them all, most of them seemed pointless test the more i did research, like testing if the key or value was\wasnt in the dictionary, if the value was\wasnt callable. Its really simple class, but i only did that way that i might want to expand cuz else there is no point in using that over a dictionary

from collections.abc import Callable, Any

class EventListener[Key, Event]:
    """Listens events and notify the observers"""
    def __init__(self) -> None:
        self._observers:dict[Key, Callable[[Event], Any]] = {}

    def add(self, observer_id:Key, observer: Callable[[Event], None]) -> None:
        """Add new observer, if observer exist overwrites it"""
        if not callable(observer):
            raise TypeError(f"{type(observer)} is not callable")

        self._observers[observer_id] = observer

    def send(self, observer_id:Key, event):
        """Send event to the observer"""
        return self._observers[observer_id](event)

[molten hollow]

Well, I checked some of your earlier messages, and in certain areas, you're right. Testing private methods is not something you want to do.

[fallow granite]

I did lots of refactor and changes and some new methods was created just because testing was easier to it felt wrong, didint want in the end to make good api only for testing

[molten hollow]

If it's an event listener, than let's imagine the WHAT - what do you want it to do. The answer is clear, you want it to register a listener and call it.

So what are the behaviours this class should have:

• ability to register a new observer
• ability to call the listener, and when that happens, the observers should be called
• ability to pass some argument or an event from the caller to the observer

The dictionary is the internals of the class, so the test needn't know about it.

I can see you have additional behaviours:

• if trying to register an observer that's not callable, throw exception
• if trying to call a listener that's not registered, throw exception

[fallow granite]

i only added to throw not callable error because its better to throw error at the root cause rather than at the end

[molten hollow]

@grizzled nymph That's a very good design decision!

Worthy of a test.

[fallow granite]

What about testing good and bad cases? is it matter much

[molten hollow]

What about them? 😊

[fallow granite]

Should i write good cases and bad cases? Some just feels wrong like for example checking if key is in dict. Do i just test that with key in dict and key not in dict this test has funny vibes to me

[molten hollow]

You mean like error cases?

I would start with a todo list of what the behaviours are:

# Hello Kira, these are the cases we'll develop:
# 
# - ability to register a new observer
# - ability to call the listener, and when that happens, the observers should be called
# - ability to pass some argument or an event from the caller to the observer
# - if trying to register an observer that's not callable, throw exception
# - if trying to call a listener that's not registered, throw exception

def test_empty():
    assert True

[fallow granite]

that is great idea, making a test to do list

i also heared that return should be tested

[molten hollow]

I wrote a test like that:

from pytest import raises

# Hello Kira,
# these are the cases we'll develop:
#
# Todo:
# - ability to register a new observer
# - ability to call the listener, and when that happens, the observers should be called
# - ability to pass some argument or an event from the caller to the observer
# - if trying to register an observer that's not callable, throw exception
#
# Work in progress:
# - if trying to call a listener that's not registered, throw exception

def test_if_trying_to_call_observer_that_is_not_registered__throw_exception():
    listener = EventListener()
    with raises(NotRegistered):
        listener.send('not registered')

class EventListener:
    pass

By writing a test like that, I'm designing the interface of the class. With that test, I specified the expected behaviour - what should happen, when sending event to an unregistered listener - I want the exception NotRegistered to be thrown, that's the design decision I made, but it could be something else.

[fallow granite]

that is very long name 😄

[molten hollow]

It's just an item from the todo list 😉

# Todo:
# - if trying to call a listener that's not registered, throw exception

def test_if_trying_to_call_observer_that_is_not_registered__throw_exception():

You can chose a shorter one if you'd like, but make sure you're not removing too much information.

Just naming it test_send() would be very hard to understand.

When I run the test, it fails, because of a missing class:

I'm gonna add NotRegistered class

I added the class, now the test fails for different reason: I'm missing the send() method:

I'm gonna add it now.

I added the method, I run the tests again: It fails again, because the method doesn't throw the exception.

Now I threw the exception, and now it passes:

from pytest import raises

# Todo:
# - ability to register a new observer
# - ability to call the listener, and when that happens, the observers should be called
# - ability to pass some argument or an event from the caller to the observer
# - if trying to register an observer that's not callable, throw exception
#
# Done:
# - if trying to call a listener that's not registered, throw exception

def test_if_trying_to_call_observer_that_is_not_registered__throw_exception():
    listener = EventListener()
    with raises(NotRegistered):
        listener.send('not registered')

class EventListener[Key]:
    def send(self, key: Key):
        raise NotRegistered()

class NotRegistered(Exception):
    pass

[fallow granite]

The thing that you showed is called TDD(Test Driven Development)

[molten hollow]

Yup, exactly! Very nice.

Now, I would write another test from the todo list.

Maybe something like that

def test_throws_exception_for_register_not_callable():
    listener = EventListener()
    not_callable = 0  # anything that is not a callable
    with raises(ArgumentError):
        listener.add('key', not_callable)

[fallow granite]

so like in this case we using int to see if its not callable

[molten hollow]

Yes, exactly!

[fallow granite]

is it ok only to test with single type?

[molten hollow]

Great question! I had a question like that myself.

If somebody were to think of tests only as a regression mechanism, then even if you use 10 types, you can never be sure. Tests aren't a proof mechanism, they are a falsification mechanism. Tests can falsify a program (show that it lacks something), but they can never prove it.

The way I like to think about it is - how much doubt do you have in the code under test? Ask yourself that, check your emotions, how do you feel about the code? Are you worried about it? Add more checks. Do you have high confidence? You can go by with just one check.

I implemented the two tests like that:

def test_if_trying_to_call_observer_that_is_not_registered__throw_exception():
    listener = EventListener()
    with raises(NotRegistered):
        listener.send('not registered')

def test_if_trying_to_register_an_observer_that_is_not_callable__throw_exception():
    listener = EventListener()
    not_callable = 0  # anything that is not a callable
    with raises(NotCallable):
        listener.add('key', not_callable)

class EventListener[Key]:
    def send(self, key: Key):
        raise NotRegistered()

    def add(self, key: Key, observer: Callable):
        raise NotCallable()

class NotRegistered(Exception):
    pass

class NotCallable(Exception):
    pass

The send() and add() now only thwo exceptions, because that's the only thing we have specified the behaviour for with tests.

[fallow granite]

I also feel like that in such a small thing i could try to write the integrity test, dunno how its called but i could just write a working test with add and send. I never written before integrity test but i believe i just write a simple example and if something fails all test fail

[molten hollow]

Our todo list looks like that now:

# Todo:
# - ability to register a new observer
# - ability to call the listener, and when that happens, the observers should be called
# - ability to pass some argument or an event from the caller to the observer
#
# Done:
# - if trying to register an observer that's not callable, throw exception
# - if trying to call a listener that's not registered, throw exception

@fallow granite What could we do next? Maybe we could register and send a message to a listener.

I did it like that:

def test_register_a_new_observer():
    listener = EventListener()
    listener.add('registered', lambda: None)
    listener.send('registered')

class EventListener[Key]:
    def __init__(self):
        self.observers: list[Key] = []

    def add(self, key: Key, observer: Callable):
        if not callable(observer):
            raise NotCallable()
        self.observers.append(key)

    def send(self, key: Key):
        if key not in self.observers:
            raise NotRegistered()

class NotRegistered(Exception):
    pass

class NotCallable(Exception):
    pass

[fallow granite]

dont need to call assert in test_register_a_new_observer?

[molten hollow]

Okay, sorry. Maybe I was too fast.

First, I wrote a test, and without the implementation it looked like this:

def test_register_a_new_observer():
    listener = EventListener()
    listener.add('registered', lambda: None)
    listener.send('registered')

class EventListener[Key]:
    def __init__(self):
        self.observers: list[Key] = []

    def add(self, key: Key, observer: Callable):
        raise NotCallable()

    def send(self, key: Key):
        raise NotRegistered()

class NotRegistered(Exception):
    pass

class NotCallable(Exception):
    pass

I needed the test to fail, to demonstrate the lack of functionality.

And the tests did fail:

The tests did fail, because with other tests, I did drive them so they throw an exception.

[molten hollow]

So in short - you need to demonstrate the necessary behaviour you want from your code. Normally, you do that with assertions, but anything that will fail the tests will work.

Or in other words, I could add assert like that:

def test_register_a_new_observer():
    listener = EventListener()
    listener.add('registered', lambda: None)
    try:
        listener.send('registered')
        assert True
    except Exception:
        assert False

I could test for exception, but notice that pytest will fail the test if there's an exception anyway. So in that particular case, I didn't need it.

We have two items on todo list left

def test_call_observer_when_the_listener_is_sent_an_event():
    listener = EventListener()
    # we need to verify that when I send an event, the observer is called

[fallow granite]

I was doing something like that previously but i did it like that

def test_register_a_new_observer():
    listener = EventListener()
    listener.add('registered', lambda: None)
    try:
        listener.send('registered')
    except Exception:
        pytest.fail("Failed to register cuz...")

[molten hollow]

Yea, your gut feeling is correct. That's not right.

def test_call_observer_when_the_listener_is_sent_an_event():
    listener = EventListener()
    observer = lambda: None # TODO we need to somehow verify that this was called
    listener.add('registered', observer)
    listener.send('registered')

[fallow granite]

yea, that makes sense, if add raise an error of some sort there is no way we could call it and testing i only know few ways, try to call it or check with callable()

[molten hollow]

I thought maybe we can do something like that:

def test_call_observer_when_the_listener_is_sent_an_event():
    listener = EventListener()
    was_called = False
    observer = lambda: was_called = True  # TODO we need to somehow verify that this was called
    listener.add('registered', observer)
    listener.send('registered')

But that's a syntax error :/ You can't assign a variable in a lambda. So I guess we must use a regular function

def test_call_observer_when_the_listener_is_sent_an_event():
    was_called = False

    def observer():
        global was_called
        was_called = True

    listener = EventListener()
    listener.add('registered', observer)
    listener.send('registered')
    assert was_called

[fallow granite]

before i also tried to do like that

def test_call_observer_when_the_listener_is_sent_an_event():
    def observer():
        assert True

    listener = EventListener()
    listener.add('registered', observer)
    listener.send('registered')

[molten hollow]

That doesn't asser that the observer is called :/

It could work in other testing frameworks, like PhpUnit, which will verify if assert isn't called, but not in pytest.

[fallow granite]

well if it wasnt called then it shouldnt reach the assertion, that was my thought process

[molten hollow]

Yes, that's correct, but unfortunately in pytest, if the test doesn't reach assertion, it doesn't fail the test :/

[fallow granite]

tho i wasnt sure how to fail it

[molten hollow]

You need to somehow determine if the callback was called or not, and fail the test if it wasn't.

You need something like was_called or something similar.

[fallow granite]

well in my case send does return a value so, that might be valid

def test_call_observer_when_the_listener_is_sent_an_event():
    def observer():
        return True

    listener = EventListener()
    listener.add('registered', observer)
    result = listener.send('registered')
    assert result is True

[molten hollow]

I implemented it like that:

def test_call_observer_when_the_listener_is_sent_an_event():
    was_called = [False]

    def observer():
        was_called[0] = True

    listener = EventListener()
    listener.add('registered', observer)
    listener.send('registered')
    assert was_called[0]

class EventListener[Key]:
    def __init__(self):
        self.observers: dict[Key, Callable] = {}

    def add(self, key: Key, observer: Callable):
        if not callable(observer):
            raise NotCallable()
        self.observers[key] = observer

    def send(self, key: Key):
        if key not in self.observers:
            raise NotRegistered()
        self.observers[key]()

class NotRegistered(Exception):
    pass

class NotCallable(Exception):
    pass

functions can't set local variables, so I had to use a list for was_called.

[molten hollow]

That's half good, half bad. You do test that the return method is working properly; but you're still not testing that the observer is called.

[fallow granite]

if observer returned than it must be called right, unless there is a way to return without calling observer. Oh maybe it can be the side effect of send, like send return somethink else if the observer wasnt called so in that case it might not be valid way

[molten hollow]

if observer returned than it must be called right
That's what you're after, but if there's a bug, it doesn't necessarily happen.

@fallow granite that's the final tests and implementation that I would go with:

from typing import Callable
from pytest import raises

def test_if_trying_to_call_observer_that_is_not_registered__throw_exception():
    listener = EventListener()
    with raises(NotRegistered):
        listener.send('not registered', None)

def test_if_trying_to_register_an_observer_that_is_not_callable__throw_exception():
    listener = EventListener()
    not_callable = 0  # anything that is not a callable
    with raises(NotCallable):
        listener.add('key', not_callable)

def test_register_a_new_observer():
    listener = EventListener()
    listener.add('registered', lambda event: None)
    listener.send('registered', None)

def test_call_observer_when_the_listener_is_sent_an_event():
    was_called = [False]

    def observer(event):
        was_called[0] = True

    listener = EventListener()
    listener.add('registered', observer)
    listener.send('registered', None)
    assert was_called[0]

def test_ability_to_pass_some_argument_or_an_event_from_the_caller_to_the_observer():
    called_argument = [None]

    def observer(argument):
        called_argument[0] = argument

    listener = EventListener()
    listener.add('registered', observer)
    listener.send('registered', 'my argument')
    assert called_argument[0] == 'my argument'

class EventListener[Key]:
    def __init__(self):
        self.observers: dict[Key, Callable] = {}

    def add(self, key: Key, observer: Callable):
        if not callable(observer):
            raise NotCallable()
        self.observers[key] = observer

    def send(self, key: Key, argument):
        if key not in self.observers:
            raise NotRegistered()
        self.observers[key](argument)

class NotRegistered(Exception):
    pass

class NotCallable(Exception):
    pass

[fallow granite]

Thank you for your time, i will be checking this multiple times to get most out of this

[river pilot]

this looks like a really valuable learning session, I'm impressed! I'd make one change, which is to not use such long test names.

[molten hollow]

@river pilot I'm open to ideas! Feel free to suggest shorter names, I'd be happy to commit them 😄

Maybe instead of test_if_trying_to_call_observer_that_is_not_registered__throw_exception() I could use test_fail_when_called_not_registered_observer().

Maybe test_if_trying_to_register_an_observer_that_is_not_callable__throw_exception could be changed to test_fail_when_registering_non_callable_observer.

[fallow granite]

there is no way i have used such long names, i definetly can make it shorter

[molten hollow]

I pushed slightly shorter names: https://github.com/danon/kira-event-listener/commits/master/

[river pilot]

test_unregistered_observer and test_uncallable_observer. I don't think the names need the outcomes.

[molten hollow]

I understand your point of you, and I did write tests like that before; but I wouldn't go that road again; and I can share my perspective if you'd like.

[river pilot]

sure

[molten hollow]

When you want regression tests, you might want this 1:1 short naming. But when I write tests, I don't only want to test/check the behaviour, I want to actually specify what the behaviour should be.

[fallow granite]

i noticed that all this session we never tested obvious things, like if the key is correct and value is correct in the dict, i totally was testing that

[molten hollow]

In other words, I don't want to just exercise the behaviour, I want to codify what the words "correct behaviours" actually means.

e.g. what should the code do when the observer is unregistered? throw? ignore? provide a default? what is correct.

[river pilot]

can't you codify and specify with comments? Why does it have to be in the name?

the code itself shows what is correct.

[molten hollow]

So I can see it here 😄

Jokes aside, it doesn't necessarily have to be in the name, that's just much preference.

[fallow granite]

i still a believer that names shouldnt be overly specific because if i change a small thing that the name of a test implied, now i have to change it even if the test is correct

[molten hollow]

If I missed something in your example, I think I would write another test case that would demonstrate that.

[molten hollow]

Well, that's certainly true.

If you change what the test tests, then I guess we need to change the name too, yes.

Altought, IMHO that's equally true with shorter names

[fallow granite]

In the addwe add new key and value pair to the dict, what i would have totally done is tested if the key and value is correct when adding

[molten hollow]

Sure, we can add a test like that. What would the correct and incorrect keys be? 🤔

[fallow granite]

What i learned that this would be testing the built-in functionality, since we would be testing in this case if the key is the same in the dict as provided and if dict[key] == observer. Someone mentioned that this is over testing

[molten hollow]

Oh, so you're talking about registering an observer with the same key a second time?

def test_registering_same_key_second_time():
    listener = EventListener()
    listener.add('same key', lambda event: None)
    listener.add('same key', lambda event: None)

Something like that?

[molten hollow]

I agree with you, that the test should be readable enough that you can deduce what is going on. I agree 100%. That's just my preference to reinforce that behaviour in the name, but using comments is equally right as well!

[proper wind]

how to find cool libs in maven central?

[pulsar oracle]

This channel is about unit testing and this is the python discord server. Junit5 is probably pretty cool though and lets you write custom test runners or something along those lines easier than previous versions with good IDE integration, and junit6 is releasing soon. You should ask in a Java server. Maybe find a way to search the site by category and popularity and browse through? Or ask in a Java server for opinions peopleuu have of ones they think are cool.

https://blogs.oracle.com/javamagazine/post/junit-build-custom-test-engines-java

[jaunty ginkgo]

Is this the place for questions about pytest?

[tall brook]

yes

[jaunty ginkgo]

My question is about pytest exit codes. I volunteer on the cs50p server to assist students. CS50 has their own test software that calls pytest if/when needed. In the unit testing/pytest section of the class, students write tests blindly using only a specification and their own version of a program. Those tests then are run against staff versions...

When a student goes outside the specification and calls functions that are not specified and thus do not exist in the hidden staff version, pytest returns exit code 2.

I have directed them to the pytest documentation to show them why, and have them familiarize with the tool. But the documentation says:

Exit code 2:
Test execution was interrupted by the user

I know and have tested pytest by calling imports that do not exist to achieve exit code 2.

Is this exit code occurring this way documented somewhere and I am just not finding it?

(sorry for the long rambling post)

I see pytest actually does say interrupted, but is that what it should say?

[tall brook]

I was browsing through source code and it is not clear how the import process ends being marked as interrupted. Is it possible that you can run it again and increase the verbose?

[jaunty ginkgo]

Is running on my simple test file (above) okay?

[tall brook]

yeah. Just increase the verbosity. Maybe it helps us to localte where the exception is getting triggered

but it doesn't seem like there a too many error types

https://github.com/pytest-dev/pytest/blob/3d58e8d6d35017aa118952c5265860c5ed23aa4d/src/_pytest/config/__init__.py#L92

[bitter wadi]

src/_pytest/config/__init__.py line 92

class ExitCode(enum.IntEnum):```

[jaunty ginkgo]

I see SyntaxError at the bottom for some reason.

Oh my. Let me fix that and run again.

This is an old version of python but pytest is 8.3.2 because I'm on iSH on mobile at the moment. I wonder if the error messages change on current.

[jaunty ginkgo]

I jumped on my laptop. Python is 3.13.7, Pytest is 8.4.0. I made 3 tests.

1. A typo as : def_syntax_error(): -> exit code 1 exit code 2, SyntaxError in traceback
2. imports non-existent module -> exit code 2, ModuleNotFoundError in traceback.
3. imports empty module and calls non-existent function from module -> exit code 1, AttributeError in traceback.

[tall brook]

do you know how to debug? I would breakpoint every ExitCode instantiation, e.g. https://github.com/pytest-dev/pytest/blob/3d58e8d6d35017aa118952c5265860c5ed23aa4d/src/_pytest/config/__init__.py#L177

[bitter wadi]

src/_pytest/config/__init__.py line 177

return ExitCode(ret)```

[jaunty ginkgo]

How to debug pytest?

No, that is probably beyond my current level. I was just browsing the github repo and the number of lines that are def main(... are many.

[jaunty ginkgo]

This may be incorrect. My memory of the syntax error test file has it as exit code 2. I will retest when I can get back to my home.

[crisp hornet]

Test cases to catalog:

1. SyntaxError in global code of imported module
2. Import invalid module name
3. Import invalid member name of module
4. SyntaxError inside function code in module
5. Run time errors at global level in imported module, e.g. KeyError or Index error
6. Run time errors at global level in test file, same as above
7. SyntaxError in global level of test code (the invalid function header above counts)
8. SyntaxError inside test function code
9. Run time error inside test function probably not needed, handed by pytest.raises

[tall brook]

I think, place breakpoint() before the import, when the exception gets triggered, evaluate the traceback. Check this for details: https://docs.python.org/3/library/pdb.html

[jaunty ginkgo]

Thank you. I'll take a look.

I can now confirm that the test with only a typo returned exit code 2.

[crisp hornet]

Okay, so interruption is any error that occurs in the middle of collection phase

[jaunty ginkgo]

That's right. I thought I wrote that somewhere, I don't see it now though. That's the thread I want to pull on and see how many paths lead to it.

Oh I did write it, just not in this server. Serves me right for not sleeping and working on this.

[molten hollow]

i'm willing to share some knowledge about continuous delivery, testing and tdd, acceptance tests, continuous integration, etc.

[proud nebula]

So... like everyone else? :)

[pulsar oracle]

tell us about continuous delivery and acceptance tests

[molten hollow]

What would you like to know?

[pulsar oracle]

Tbh I've been studying this stuff very extensively (primarily acceptance testing) and have covered so much material and conferences and articles from Dave Farley, Martin Thompson and LMAX on it and just read the continuous delivery book yesterday. In my experience a lot of people tend to barely know what acceptance tests are, and confuse continuous delivery with automatically deploying into production after each commit, which the book makes extremely clear is not the case where it states testers should be able to pick a version and deploy it into their test environment, likewise for user acceptance testing. There's also the issue at face value of scalably writing acceptance tests, testing the application from the outside in and how to do that well, as the system under test changes, but that just involves using a layered architecture and a DSL as has stood the test of time at lmax. I guess I was just going to see if you knew about any of this as well. 😛

[proud nebula]

That all sounds more like tutorial hell than real experience.

[wind nova]

I want to figure out if I can/should abuse a fixture with pytest. I've got a dict where I want to iterate through it's keys using parametrize for a test. Is there a good way to do this that isn't horrible abuse?

[pulsar oracle]

It isn't really tutorial hell because I've been practicing and writing stuff actively as I would anything else, with stuff I've learned making it easier, practically. This isn't the kind of stuff you just follow a tutorial literally on, it's more about principles and ideas and then examples of what other people have done. I have literally applied this stuff on multiple occasions like I would anything else, granted like many projects, not finished and I didn't get spoonfed all of this at once. I literally learned hexagonal architecture and got the idea after having done the opposite of having a class where data access + decision logic were mixed together and very annoying to test. And I've written stuff time and time again, with and without acceptance tests in place, with them making development significantly easier. In a recent project of something I did seriously two years ago, I completed it in a week and it made life easier. None of this being in a state of tutorial hell. I can apply every bit of it pragmatically but seek out more to improve, and read up on a lot after I do stuff wrong.

[marsh raft]

So you've got an existing dict -- presumably defined after loadtime but before runtime -- and you want a test case for each key-value pair?

[wind nova]

you got it

I don't actually care about the values, i'm just leveraging the keys for something else

keys are used in my setup

[marsh raft]

#!/usr/bin/env -S uv run --with=pytest python3 -m pytest

import pytest

test_cases = {"this": "that", "cat": "tac"}

@pytest.mark.parametrize("key, value", test_cases.items())
def test_it(key: str, value: str) -> None:
    assert "".join(reversed(key)) == value

[wind nova]

wow.. I don't know why I didn't think about just expanding the dict in there

thanks! I think that'll do it

also love the uv use 😄

[marsh raft]

I try to make examples self-contained

[steel drift]

Who can help me clone my script and working again but change the GUI name

My mom also single

[molten hollow]

If I'm not mistaken it was actually Dave Farley and Jez Humble who conined the term "continuous delivery".

But everyone else mistook it for just creating jenkins pipelines.

[pulsar oracle]

Yea one of them definitely coined it and it was based on that one line from the agile manifesto.

Niceeeeee. The modern software engineering one doesn't interest me so much maybe because I've already binged the YouTube channel and various conference videos and get the ideas over and over again. This book is absolute gold though and so straight forward in so many aspects. For me personally I'm working on automating deployment and scripting stuff at the moment and the book addresses so much. Like I am actually shocked how much is in there and how hard hitting some of the lines are especially one about some people specifically in the agile community advocating for doing away with acceptance tests in favor of moore comprehensive component (basically integration tests). I definitely noticed Dave doesn't go into a lot of the technical details on his channel but in that book it explicitly calls out that being insufficient and says that it highlights issues quicker in event driven architectures and catches threading issues. (I used to be in that similar camp not understanding what acceptance tests were, with clean architecture since it eliminates logic errors at the unit level but doesn't get rid of plumbing and other ones that can still screw everything up)

[molten hollow]

Yup, exactly 😄 Even the term "deployment pipeline" is misunderstood very much. People hear it and think "ha! that's a github workflow".

[tight shale]

[pulsar oracle]

Exactly, that's just what shocked me so much reading it. My take away from the book was that deployment to any environment should be a single push button operation (I am not taking it literally, maybe a command line script and a lambda can do). I picked up the software developers guidebook and the CD pipelines book I think it's called and went over those two and it says for your acceptance stage you can do either, pick up the latest release candidate and release to production automatically, or make a web UI to select the version you want to deploy. In the book it says testers should be able to pick a version of the software and get it deployed into a test environment. I especially like the testing architecture behind this though and think that clean architecture practitioners and others ironically fail when it comes to that, you can test your system in a decoupled way basically for free, even your APIs, and leave the application flexible to rewrites, but instead people tend to just test from within the framework when they otherwise prescribe to architectures with similar principles.

Uncle Bobs acceptance tests are god awful and super ironic. 💀

[molten hollow]

Yea, I agree. I think one reason, is that when they use a framework, they tend to follow documentation (like docs for Django, Laravel, Spring), and these docs very likely will suggests tests that couple you to the framework, like laravel will not suggest using phpunit tests, but its own derived (which is an integration test). Another thing, is that people don't reflect on their ways of working. If some team in some company wasted 2 weeks of work because of some dumb mistake, they will not reflect on that to try and fix it; they would most likely accept the status quo and keep going. And additionally, when influencers like Dave, Jez, Nicole Forgsren, Martin Fowler, Michael Feathers, all of them, try to teach people something, I think what they try to convey most of the time is mindset and be outcome oriented, with some tools to help; but for people it's very hard and they tend to only pick the tools, and they apply them, but miss the mindset completely, and that leads to stupid processes.

Btw, what acceptance tests by Uncle Bob did you have in mind?

[pulsar oracle]

It's understandable for people to use their frameworks for testing honestly. If they don't vibe with these architectures and don't want to I'm not gonna argue. But I do find it ironic in clean architecture projects and from Robert Martin. If the use cases or service classes and everything are decoupled from the framework, then why shouldn't the tests be, I guess it's just that they don't know of a better way and it's well enough but I always did feel gaps in it. I struggled with the idea of acceptance tests until I stumbled upon a series of blog posts from lmax. Dave's introduction to bdd video piqued my interest though and eventually I saw a conference talk that went into the whole thing.

[pulsar oracle]

It's just sloppy, in an ironic way. It was part of one of those clean coders series of videos and he used rest assured or something and they were coupled to spring. Like I personally don't want to write code like that, I enjoy writing with the internal DSL/driver approach and the flexibility that offers. I like leaving the details separate. So when I see this guy advocate for some good stuff then violate it in terms of acceptance tests, I was just a bit disappointed. I have tests as part of a game server at the moment that register accounts sign in connect and spawn into a room with two used and verify they can see each other's messages. This is literally so simple but only with this internal DSL approach. I mixed Dave Farleys approach where he has a DSL object, and lmaxes where they just add an additional like endpoint for lack of better word, API, to the test case, dsl1, dsl2, covers a lot.

[molten hollow]

Yea, I know what you mean, I do that too.

[pulsar oracle]

I always wondered if any viewers of the channel paid attention. Because I've never seen anyone comment on it and couldn't find any examples elsewhere. Shocked again to see the driver part of it for sure in the continuous delivery book (I studied this very hard elsewhere and I just was too bored with the concept to read much of it again there). I like how flexible the book is too, and that's what I like about Dave, I used to be hung up on the idea like what if I don't want to release the software? What if I only want it for playing around (exploratory testing)? Only to not hear about this aspect of it and to see the book mention that just scripting deployments with scp and ssh as a valid option and recommending the fabric library. I love it. I guess the only thing I'm hung up on or question i have left (and this is less so because I believe I can tdd my way to a load generator/tester) is how they use the same drivers for the performance tests, at lmax they say they use them explicitly and the capacity tests section says the same. But mine use asyncio (locust is out of the question), and it's not like jmeter supports just writing code for tests I don't think so do you write your own then. (The book says use the acceptance tests for your capacity tests) And at lmax they use the driver layer.

[molten hollow]

You're not gonna find many CD geeks in python serwer 😄

[pulsar oracle]

That's true but I am not sure I'd find very serious ones in many servers (not that I go around too much), they're probably well employed, older and not on discord so much. I didn't expect to have a discussion like this here but here we are. Python is practically limitless in what you can do though, even if not everyone uses it to its full potential for a lot of stuff. You'll always find someone here who knows their stuff though, as evidenced by this conversation. But yeaaa, I feel pretty close to being able to do continuous delivery in a pretty true sense and that book and acceptance tests, and others, are awesome.

Also I gotta be honest some of this stuff feels like a real competitive advantage to have (it's like first using unit tests, the improvements are insane) so I wouldn't be surprised if other people are staying intentionally quiet about some of it.

[proud nebula]

And how much of this is at a real job?

[pulsar oracle]

So if it's not at a real job it's not valid? I've been spending my time working on a project in a grey area to build new servers for an old game that was shutdown and riddled with security vulnerabilities. I lived through it and took advantage (judge if you will different discussion) but I saw them refuse to fix anything and saw dysfunction on a place that I cared about. I did a lot on this game to impress others, get attention make bots, throw stuff together. Enjoying what I do like anyone else in this field, and now I am working to bring it back and not make a mess in the process. Do I need to work a whole job in order to feel the pain of my own decisions? How much experience in life is from one off things that are never finished? How many things must one learn. You can try to invalidate by saying I don't got one but for me I take this extremely seriously, obsessively and live for this. What I do I do for me and my pride and I do it how I enjoy it and working through simple yet complex enough problems that things like acceptance tests help with. I've spent so much of my life in different situations deploying stuff and it isn't fun. Is the pain I feel of deploying my own software not valid? The manual process being tedious and error prone. I am working though this and studying all of the time so that I can enjoy what I do and do it in a way that it is assured to be well and deliver something to my community in the process.

[proud nebula]

You don't have to write a novel to answer. Lol.

[pulsar oracle]

You're funny

[proud nebula]

I'm just pointing out that you two guys sound like I sounded 25 years ago before I had real professional experience. I've been there.

[pulsar oracle]

I'm not gonna play offense on this because I can't argue with something that probably comes down to if you had to get something done on a job some of these approaches you might throw out the window in a bind. But for me, some of this is about how I prefer starting out solving a problem to begin with, be it writing a function, writing tests, etc, as a practical feeling thing, and when something is complex enough writing these acceptance tests. When I put my code together and it doesn't work as expected or breaks, that's annoying. Now granted some of the errors experienced might be on account of not working within a rigid framework that prevents them but I'll take my chances and don't really expect to find employment for this anyway. I do what I like how I like, etc and by what I find feels well enough and acceptance tests and this approach speaks to me, as a philosophy at least that I'm not not going to write tests, I feel so much faster when I get that feedback.

[proud nebula]

Sure. It's a great tool to have. You just need to keep the goal in focus. Sometimes you don't really need tests, sometimes you need way more than unit+integration written by hand can give you. It's always a tradeoff, and it always depends.

Mutation testing, property based testing, and formal methods are the extreme. For when it really matters.

[pulsar oracle]

To me personally I'd always rather have tests, especially if I'm continuing to build something up. I'm flexible on where those tests are, like skipping unit tests and going for integration, testcontainers+database, or end to end tests with stubs. Like soon I am thinking about writing a lambda (serverless function wherever) to allow for deploying the software to a given environment and I don't imagine myself wanting to bring in ports and adapters for it, nor really testing much automatically at all while I'm playing around with the idea, likewise most of the code I've written to script stuff up isn't getting tests either, because the feedback from using it is rapid and errors are acceptable, but if I get something stable and get the idea out even though I can imagine myself using something like pydantic and some way to make it use a database (idk) I'd still want to make sure that when I go through the API digital ocean for example is interacted with as expected through stubs, "end to end". Like for me as a philosophy it's either rapid manual feedback and room for error, or prototyping, but automated tests somewhere as an automatic indicator that something is complete and isn't broken. And sometimes it's a mix of different tests for different places, skipping some and having stuff filled in with others.

[graceful cloak]

Hi, Can anyone recommend any books for learning writing tests or any other resources. My goal is to land a qa automation engineer role.

[tall brook]

I liked "Obey the Testing Goat"

[graceful cloak]

Thanks, will check it out

[shy wedge]

whats the point of unit tests if your just gonna mock and mocks are not real

[pulsar oracle]

It heavily depends. If you structure your code right then you test your own logic against something external without that thing where having the real thing doesn't matter to your logic and the implementation can vary. Other times there is no point in mocking and you shouldn't be keeping everything in memory, if your code is supposed to work with a database then testing your interactions with the database library doesn't prove that it does what you want, only spinning up a database and doing an integration test does, as with certain important code like that is the case. It depends on your level of abstraction.

[proud nebula]

I mean.. I have a lot of tests. I have zero mocks. So....

(also, "unit tests" as opposed to "automated tests" is ill defined and people use the terms very loosely)

[marsh raft]

ditto: #1208904428967432262 message

[shy wedge]

great point and just you mentioning that has me questioning how you have alot of test with zero mocks

[shy wedge]

any docs for proper definition of the 2

[marsh raft]

The way to not have mocks is to ... not create them 🙂

Seriously

Try to test just the data that comes back from some service, as opposed to interacting with that service

[shy wedge]

i see so asserting of the data coming back. ok thanks offby

[proud nebula]

Pure functions for example

[unkempt matrix]

hey guys. i need to be a test ninja in PyTest within a couple of weeks of pure torture. do you guys have any resources of best practices outside of the official documentation?

[atomic thistle]

Highly recommend this book: https://pragprog.com/titles/bopytest2/python-testing-with-pytest-second-edition/

[unkempt matrix]

thank you

[proud nebula]

Why would it be a problem? Ask about the specific problems you have, don't pile up problems and then hide them from the world.

[modern hedge]

i went through hell to figure out where exactly to patch my database client. i ended up patching the module in my code where it gets instantiated with patch.dict and that works but I am not sure its the correct way

one thing i get confused about the way i am architecturing my query / response mocker fixture is that i pass it data by indirect parametrization. i feel that its not the correct way and i should maybe expose it in such a way as a fixture that my test function mocks it

where should i keep classes and helper functions created for simplifying the query / response creation? its currently in conftest.py but that is already full of fixtures

should i prefer monkeypatching over unittest.mock.patch?

[marsh raft]

Have you read [the discussion just above](#unit-testing message)? I'd try to talk you out of using mocks at all

[atomic thistle]

You could separate your queries and your logic so that you can isolate the tests of your logic.
For example:

def combined_func(parameters):
    response = query().where(**parameters)
    # do some stuff with response
    result = ...
    return result

# becomes
def my_query(parameters):
    return query().where(**paramters)

def do_some_stuff_with_response(response):
    result = ...
    return result

[pearl cliff]

And sometimes that's not possible, or just not a good idea. In which case dependency injection is your best friend

Then you can run a database in a container or whatever alongside your test suite

[molten hollow]

It kinda depends if you're going with london school or chicago school.

• In london school, you want to use unit tests to drive the design of your system focusing on interactions - which class/function/module talks to which, and when. You use mocks not to stub 3rd party per se, but to specify what interactions need to happen. That way you can vary/refactor the data that flows through the system, but interactions tend to be more stable (cause changing them would require updating tests).
• In chicago school, you want to use the state of the class/module to drive the behaviour of the class/module, and hide the interactions from the tests, so they can vary without invalidating the tests. Here, you're right, mocks tend to be obstacles here, because they're another thing to update when the implementation changes, so we use state/returns/arguments to drive the desired behaviour.

Both try to achieve safety and flexibility, but london values safety a bit more, and chicago values flexibility a bit more. IMHO, I prefer chicago at lower level tests, and london at higher level tests.

Additionally, you can use mocks or fakes to stub out external systems, like stripe, oauth, network cache, etc.

[molten hollow]

If I were to suggest something, I'd go with not really focusing on tools as much (such as pytest), but on the practices more. Go for ideas like separation of concerns, tdd, outside-in/inside-out design, cohesion, good features of good tests (like tests desiderata), etc.

[molten hollow]

I think it depends whether your system is already well designed and testable, or is it a bug ball of mud. If it's a big ball of mud without tests there, where everything is coupled to everything, a database calls are mixed with logic everywhere, then I think no matter what you do, you won't be able to introduce valuable tests to that software, without refactoring it :/ It doesn't matter whether you monkeymatch it or not, it's unlikely the tests will give you any kind of reliance or safety net, because the code is probably too fragile, and any change in the code will invalidate the tests. That's why there's the tdd notion, to start with the tests, because that way, it's harder to create a system that's not testable.

If that's the case, then with any kind of good test you would write, the system would oppose testing so to speak, it would fight you. It's a big pain to try to retrofit good tests to an untestable system. It might make you feel like testing doesn't make sense at all. What is likely to happen, is writing tests that are so tight to the implementation, than they will actually make it harder to refactor the code in the future.

When it comes to monkeypatching itself, I tend to not use it, because it has all the same flaws as global variables - it's just overriding global state. I tend to use dependency injection to achieve separation from the database. Or better yet, design the application so that as little part of it as possible knows about persistence.

[molten hollow]

That's the thing about coupling to the testing framework, what pytest calls "fixture" is not something a testing community would call "fixture", so if you use that word, be prepared for some misunderstandings.

[river pilot]

what does fixture mean to the broader testing community?

[molten hollow]

They're sort of related, but not quite synonyms. For example, in pytest you can create a db fixture that gives you an opened connection/handle you can use to interact with a database, that the pytest will close at the conclusion of the test; but I don't think anyone in testing community would call a connection to the database a fixture.

[pulsar oracle]

I think in the case of pytest the fixture is the function and instead of setting up a class with the tests, it provides something and is passed through a dependency injection like mechanism rather than inheritance, it's not that the db connection is the fixture, it's that the fixture is used in a way where it provides something and is given/used by name.

[river pilot]

wikipedia has a pretty broad view of it: https://en.wikipedia.org/wiki/Test_fixture

[molten hollow]

Yea, you're right there. My point being, knowing a particular testing framework is a relatively small thing; and knowing testing in general is a relatively big thing; and we should focus more on the way we test something, rather than on specifics of any testing library.

[molten hollow]

So when I see messages like this one, when someone tries to tunnel vision for a particular test library; I think that's looking for your fruits in the wrong tree.

[pulsar oracle]

That's true, but it's easier to get into testing if you can get hands on, learn to do it, then learn to do it better. At the end of the day you just call code and make assertions, pick your poison for structuring it. I personally prefer a approach of the unittest module + pytest for even more elaborate test fixture classes, or even just the built in one sometimes.

[pulsar oracle]

Some people learn better with tunnel vision. I know I did, I did pick whichever one I thought was simpler though or liked more. Most people asking probably don't write many tests at all to begin with, and don't have enough experience to react to more elaborate advice.

[molten hollow]

At the end of the day you just call code and make assertions, pick your poison for structuring it
Yea, I believe that's the essence of it. However, I'm far from claiming that's an easy thing. Plus, when you do it for multiple applications, you tend to notice that some approaches work better than others - those are the ones that we tend to keep and reuse, and they very often tend to be irrelevant of testing libraries.
I personally prefer a approach of the unittest module + pytest for even more elaborate test fixture classes, or even just the built in one sometimes.
I'm happy to hear that this approach works for you, but because you told me only the tools and nothing more, it's very hard for me to actually learn your approach.

For example, if I told you that I use junit, assertj and run tests in intelliJ, that would tell you very little about my approach - because I'm focusing on the tools; and that's very opaque, and truth be told, not that relevant.

But if I told you I have about 80 acceptance tests that execute the business logic through the UI, the ui is abstracted so the tests speak the domain terms (that execute in about 10 minutes), and then I have 1000 unit tests which execute in about 2 seconds, and provide nice design on a finer level, and the database is abstracted; but is still being tested by the acceptance tests - you needn't know anything about the tools, but you already know much more about what to expect from those kinds of tests.

[pulsar oracle]

I wouldn't know how to achieve that without asking you more if I didn't have context. I at the very least had examples of a junit style testing library from showcasings before to know what that is expected to look like, and I wouldn't be able to achieve any of that very good because I lack the creativity or knowledge. And I think being told that this elaborate testing approach uses Junit would be helpful and was when I heard it before. Domain terms could be using gherkin and tying it to regular old code in a pytest like structure.

[pulsar oracle]

I could trivially explain my approach. By default I use the unittest module, the TestCase class doubles as a fixture + a test, multiple tests as methods are put onto it, and the fixture code is the teardown and setup method. I create a class for it and downgrade it per say into a fixture, then I share that between my test cases. It can have stuff on it accessible to all tests and methods to help with testing. It does use a TDD style approach in this case.

[molten hollow]

Okay, let's hit it from another perspective. If I learn that your tests are written in pytest, that could mean:

• either that your tests are very good. That they're small, execute fast, and have desired features, like: if you have a bug in a code, they will find it. If you refactor your code, they will still pass. They execute very fast, seconds. If they fail, you can very easily see what's wrong. They tend to be focused, if you break something, only a small portion of tests fail. The reveal intentions, if you read them, you can immediately know what they are testing for, and whether that behaviour is still necessary. They're easy to change and work with. They don't share much of a common state, so changing one test doesn't break the other. They are helpful, they are worth keeping and running.
• or it could mean that they are crap: they are big, long, you read them and have no idea what they're testing, if you have a bug, none of them fail - they still pass; if you refactor your code, suddently half of your tests is red. if they fail, you have no idea why, you see assertions like "expected true, but was false". They are destructive, not helpful, probably should be removed.

And I don't know which. And that's because tools are just that, tools. you can use them to create good tests or bad tests. The tools don't matter that much, but your approach to testing matters greatly.

[pulsar oracle]

That's true, but what If I can't write tests at all? And what if my code is good and written with TDD but my test cases suck because I never learned how to structure them and share stuff between them very good? I'd personally do better if I was shown examples of either and started with whatever works, even if starting out was bad, which mine originally were.

[molten hollow]

Yea, that looks like a good test. Me, myself wouldn't use inheritance to share behaviour, I would use composition, but there's nothing really wrong with that test. They even have great bdd comments. That's actually better than most tests I see. Good job @pulsar oracle !

And what if my code is good and written with TDD but my test cases suck because I never learned how to structure them and share stuff between them very good?
A skill to share stuff and reuse them between tests is exactly one of those "testing skills", that's not relevant to a particular testing library. If you learn how to share and reuse stuff properly in one library, you know how to do it in others. Of course, you need to think of things like: are these instances shared, is this recreated for every test, does this destroy connection on teardown, sure. But you don't have to worry about every framework implementation details.

I'm gonna say @pulsar oracle, I didn't expect such a well written test. Nice!

[pulsar oracle]

As a beginner (former beginner but remembering my perspective, and even for stuff new to me now) I'd prefer if possible that some resource could show me how to test stuff in general, with pytest or even unittest or anything else really to get the idea + actionable information. If a resource showed what could usually be an anti-pattern like monkey patching, and how it can be fixed with dependency injection or any form of inversion of control really. I find that it can be very hard to learn theory if you aren't ready to put the code into action. I'm as good as I am because as I was learning I was able to see examples of what other people have done, and I was shown dependency injection, code first. Dave Farley explaining the origin of BDD being a way to better teach TDD at the unit level is where I picked up this should convention and they also do it at lmax. I should note that the approach I have is something I've picked up over time, writing a lot of tests, and If I wasn't as obsessive as I am or study as hard there is no way I would be writing tests as clean as you saw.

[pulsar oracle]

Tyyyy

My approach is also inspired by internal dsl bsaed acceptance tests, and thorough reading of the documentation for both pytest and the unittest module. If test cases should be easy to write, then If possible I'd like to write a fixture that can be inherited to have everything on it to script stuff up and reuse it, like I do for acceptance.

[proud nebula]

@torn trout Don't PM users randomly. <@&831776746206265384> I believe this guy is spamming. Sorry about the ping in this channel, but I didn't find a specific channel that is only for non-channel moderation.

[fiery arrow]

If you want to report someone, message @void tulip

[trim jacinth]

Hey everyone! ,
I just published a new Medium article: “From Stress to Success: Load Testing Python Apps & Visualizing Performance”.
Would love your thoughts and feedback! You can check it out here: https://medium.com/@alleny244/from-stress-to-success-load-testing-python-apps-visualizing-performance-83ea4ff16c6c

[unkempt matrix]

i just want to specialize

[mellow geyser]

pytest is not a specialization

[molten hollow]

I read it, it's short; and to me it kinda feels like walking somebody on a leash. The article to my eyes looks a bit like: "do this because I say so". There is no real explanation what's happening, no reflection on practices and what we're doing; it's just a list of step to follow without real insight.

[molten hollow]

All right, that's very good you'd like to get better. If you'd like to increase your skills, why don't you start wit this thing: https://archive.org/details/est-driven-development-by-example/test-driven-development-by-example/page/n139/mode/2up

[mystic kraken]

I have a unix system information tool that runs a bunch of programs and examines a bunch of files. Learning from previous iterations, it has a "archive" feature to capture all this on a host and create a tarball so I can develop/debug on my workstation. I've got that working in pytest (could be better). But with the 3 hosts I have in the mocked data, I have to manually add a new host. Is there a good pattern to tell pytest (or my test code) - "hey, look in this dir for hosts to test against"? Is this ... tagging? (I'm still new to pytest)

[zenith peak]

Anyone familiar with testing using Responses? I'm having a hard time getting my head around how one does so without adding the testing code into their application as the examples seem to show. I understand I can record responses during regular usage and load those for testing but even that requires adding the testing code into my application. I feel like I'm missing something.

I'm trying to adhere to "don't mock what you don't own" as my application scrapes and returns HTML as well as downloads files from a third-party site and I would rather not hit it constantly in testing.

[pulsar oracle]

You want to confirm that when the remote server is actually requested and a response gotten it performs how you expect without just testing through the specific http libraries?

[zenith peak]

In my case the remote server is actually irrelevant, I have no control over and am not testing its functionality, I'm primarily checking for regressions in my own application in how it handles the data its pulling.

[pulsar oracle]

You can make the url to the server configurable then test it against a stubbed response of the actual server which you can setup with wiremock if you want to test the correctness of your code against one or more endpoints.

https://github.com/wiremock/python-wiremock

[zenith peak]

I'll have to look into this and see the level of complexity for getting this working with a github action to check PRs for regressions

[lilac ferry]

Hi everyone! I've been working on a tool that resizes models in Blender to be imported in Unreal. Yeah, it's a tool for game development and I'd like to use pyfakefs to fake an output folder where the resized model will be saved. Problem is that I need to use pytest to be able to use Blender Python API but when a run a test with it I get the following error:

ImportError while importing test module 'C:\Users\juank\dev\game_tools\grin_gate_studios\art\resizer\blender_scripts\tests\test_blender_resizer.py'.
Hint: make sure your test modules/packages have valid Python names.

I know that there are ways to set up pytest to let it import modules according to this website. I tried them all but still I get the same error I'm afraid.

Can anyone tell me what I might be missing here please?

Do I need a special set up for third party module for pytest?

[ember maple]

why is the test module a path name instead of a pyhton package name?

blender is a very different python environment - it may be necessary to have a pytest plugin to set things up correctly

[lilac ferry]

Hi! Thanks for taking time to read my question. I don't understand that either

[ember maple]

how do you invoke pytest inside blender

[lilac ferry]

Oh sorry, I should've explained that I'm writing this code outside blender, I mean with my code editor

I'm using pytest-blender to test blender with python

Weird thing is that pytest complains about pyfakefs import and not other modules such as bpy (for blender api autocompletion)

[ember maple]

ok, that loos a bit overwhelming at first - you need to install packages in the blender python as far as the docs of that go

i never used it (i wasnt even aware it exists until today)

[lilac ferry]

That's alright

[ember maple]

so if pyfakefs is not avaliable in the blender python (which is different from the normal python) things are going to fail

the first part of the readme tells how to install things - d

so pyfakefs needs to be installed in the blender python

[lilac ferry]

You are right!

Just read that

Oh I get it finally

pyfakefs must be installed inside blender python environment

[ember maple]

glad i could help, sometimes all thats needed is someone pointing at the top of the radme

[lilac ferry]

Yeah, thanks for that! I was stuck with this for a long time

[ember maple]

i recommend a beverage of your choise for processing missing the start of the readme ;P

[lilac ferry]

That's right, a bit of time away from the keyboard always helps

[ember maple]

also time for me to go off - my typo rate is trying for a new record

good night

[lilac ferry]

Good night 🙂

[molten hollow]

I think you're exactly right with don't mock what you don't own. I think it comes to good design and separation of concerns. Are you able to abstract the external system in your application in a clean way? Like maybe down to one interface/one abstract class/one function? If so, do that, and using polymorphism inject a test implementation into your tests, and in production use the real system.

I wouldn't go for mocking responsnse/http/urls, etc. all that, because it's too tightly coupled to the integration itself.

[limpid raft]

What am I doing wrong trying to mock httpx.URL like so?

from unittest.mock import MagicMock

uri = "http://example.org"

URL = MagicMock()
URL.is_absolute_url.return_value = True
URL.__str__.return_value = uri

url = URL(uri)
assert url.is_absolute_url()
assert str(url) == uri, f"str(url) == '{str(url)}'"

AssertionError: str(url) == '<MagicMock name='mock()' id='140242501020912'>'

shouldn't URL.__str__.return_value be returned by the str(url) call?

(https://docs.python.org/3/library/unittest.mock.html#mocking-magic-methods)

[proud nebula]

The first problem is that you're trying to use mocking ;)

It's generally a good idea to refactor so you get "functional core - imperative shell", then you don't need to mock.

[swift pewter]

The actual issue: url is not the same as URL. So while you intend for URL to behave like a class, it doesn't know that; it's __call__ just returns some other magic mock.

You can probably solve this by doing URL.return_value = MagicMock(); URL.return_value.__str__.return_value = uri

[limpid raft]

I have lots to learn. I do wonder if there are articles you could recommend to read?

In my case, I am trying to write and test a validation function for click. It gets strings and returns URLs, after checking that e.g. the strings are valid URLs. In order to do those checks, I want to use httpx.URL.

I already made it so the validation function takes an argument for the URL-like class to use, and it defaults to https.URL. So when testing, this is where I want to insert a mock instead, so that I am not testing https.URL, but just my use of it.

    URL = mocker.MagicMock()
    URL.is_absolute_url.return_value = is_absolute
    validated = validate_urls(mocked_click_context, "url", (inp,), url_cls=URL)

[proud nebula]

I don't know of any article per se. The idea is that you should prefer to write all functions as functional/pure functions that don't go out into the world and grab data themselves. You want to have some "shell" (aka the outermost code) that fetches the data and passes it into the functions.

If you showed the code that you are trying to mock I think it's easier to explain concretely how to restructure the code to make it testable.

[limpid raft]

The reason I am using mocked_click_context in the above is because the function signature requires it, and the validator is actually written for the click specs

def validate_urls(
    ctx: click.Context,
    param: str,
    value: tuple[str | None],
    *,
    url_cls: type[URL] = URL,
) -> list[URL]:
    _ = ctx, param
    ret = []
    for urlstr in value:
        if urlstr is None:
            continue

        try:
            url = url_cls(urlstr)

        except TypeError as err:
            raise click.BadParameter(
                f"URLs must be strings, not '{type(value).__name__}': {value}"
            ) from err

        except ValueError as err:
            raise click.BadParameter(f"Not a valid URL: {value}") from err

        else:
            if not url.is_absolute_url:
                raise click.BadParameter(f"URL is not absolute: {value}")
            ret.append(url)

    return ret

the signature is a given, i.e. click expects that. I have added url_cls with the purpose of dependency injection, am I doing this right?

[river pilot]

I'm wondering why you need to mock URL here at all.

[proud nebula]

Ah, I see the issue. You think MagicMock() returns some class-like thing. It does not. What you want is:

from unittest.mock import MagicMock

uri = "http://example.org"

url = MagicMock()
url.is_absolute_url.return_value = True
url.__str__.return_value = uri

assert url.is_absolute_url()
assert str(url) == uri, f"str(url) == '{str(url)}'"

Btw, those are not good variable names, I have a hard time seeing that url and uri are two different variables.

(That's what the example you cited from the documentation is doing)

[limpid raft]

oh doh!!

okay, that was really silly of me

[limpid raft]

yes bad variable names, this was just a quick case I hacked up to share here

do I just make a class FakeURL that returns static is_absolute_url and __str__ results?

and use that as theURL factory?

[river pilot]

why do you need to mock it?

[limpid raft]

because I don't want to unit-test functionality in httpx.URL, just my use of it?

[river pilot]

i think you should ask yourself: what would be bad about using the real URL class? Is it expensive? Is it unpredictable? The best thing is to not mock.

[limpid raft]

nothing bad about it. We're just leaving the domain of unit tests and entering the domain of integration testing, no?

I am still on the learning curve in case that wasn't obvious yet 😉

[river pilot]

those distinctions seem academic to me. The important thing is to test your code, and to not add complexity to your world.

[river pilot]

we are all on the learning curve.

[limpid raft]

true that of course

[limpid raft]

yeah that makes sense.

[proud nebula]

You can. But the magic mock you used now will also work fine no?

[molten hollow]

That's actually not a bad thing. You basically created a factory pattern, for the sake of dependency injection.

But my question is, why is the signature a given? Why can't you adapt it so it's more testable? 🤔

Because all your testability problems come from high coupling of the function to its dependencies, making it not very testable.

[proud nebula]

That part about the signature is a bit weird. He added url_cls to make it more testable so 🤷‍♂️

[proud nebula]

Well.. no actually not. He's made it very nicely testable. There was just a small confusion about how MagicMock works. You should read through before commenting 2 hours late 😉

[mossy path]

I'm using pytest and the parametrize marker for my unit tests as most of my tests consist of the same assertions on many different inputs. This works great except when a test fails, the output shows me the entire decorator containing every possible test input even though only one of them is relevant to the given case. That makes scrolling through the failing cases very tedious. Is there any way to programmatically avoid this? The ideal I suppose would be to show the parameters for just the failing case but it's not strictly necessary

[river pilot]

Can you make a pastebin with the output? i've only seen failures show the single input that failed. there's an id= thing you can use, but you shouldn't have to.

[mossy path]

I'd have to whip up an example. I'll try to do that but may forget 😅

[river pilot]

are there secrets in the test output?

[mossy path]

I would count company code as secret generally speaking even if there aren't Secrets. but also it's on a different computer without discord access

[mossy path]

I know you said pastebin but this seems small enough to just throw here with a slight change to keep the bars on one line :P
This is just running pytest without any additional arguments

_______ test_foo[ab-abab] _______

test_input = 'ab', expected = 'abab'

    @pytest.mark.parametrize(
        "test_input,expected",
        [
            ("x", "xxxxx"),
            (5, 25),
            ("ab", "abab"),
        ]
    )
    def test_foo(test_input, expected):
>       assert Foo(test_input).x == expected
E       AssertionError: assert 'ababababab' == 'abab'
E         
E         - abab
E         + ababababab

tests/test_foo.py:16: AssertionError
======= short test summary info =======
FAILED tests/test_foo.py::test_foo[ab-abab] - AssertionError: assert 'ababababab' == 'abab'
===== 1 failed, 2 passed in 0.05s =====

This is just one failing case but you can imagine that it gets really annoying when there's multiple failures, when your parameters contain classes with more to initialize, and when your list of parameters is more than 3 elements :P

[proud nebula]

That only contains relevant data that I can see?

[mossy path]

The only relevant data from pytest.mark.parametrize is ("ab", "abab"). 90% of the decorator is unnecessary information and takes up more space than the actually relevant information.

[river pilot]

i don't understand: it's not showing the passing inputs, which I thought you said it was showing?

or you mean the traceback shows the whole decorator? The FAILED line only shows the failed input.

(not traceback, the source listing)

[mossy path]

I'll call it the full output for the single test case because it also includes the error thrown and the specific test arguments.
The short test summary is frequently not useful to me because the repr of what's being asserted is long enough to get truncated in the short summary. So when I need more information, I need to scroll up. If my parameters list is 100 lines long because of formatting and I have five failing cases, that's 500+ lines to scroll up just to see the first failing test.

[mossy path]

But yes I was talking about the decorator

[river pilot]

I don't know if pytest has a way to fine-tune how much of the test function is shown. One way that I think will work is to make the list of inputs separately:

FOO_INPUTS = [
    ("x", "xxxxx"),
    (5, 25),
    ("ab", "abab"),
]

@pytest.mark.parametrize("test_input,expected", FOO_INPUTS)
def test_foo(test_input, expected):
    ...

[mossy path]

That may just have to be it.

[river pilot]

try adding --tb=short to the pytest command line

or even --tb=line

[mossy path]

short seems to be what I want

and I could probably create a custom thing here but it doesn't seem worth it when I can just add the option to pyproject :P

[cedar wraith]

I hate unit testing so much

It made me literally cry yesterday

[river pilot]

why in particular?

[cedar wraith]

You had to implement asserts to proof random numbers

[river pilot]

that sounds difficult

[cedar wraith]

And that task made me literally crashout

Like what am i supposed to do

Random seed function isnt even proof enough

So i had to go further beyond of that

[hollow basin]

What do you mean "proof a number" @cedar wraith ?

[cedar wraith]

Let’s say I had an assignment where I had to compare random value numbers from a JSON dictionary with others in order to check whether it is indeed the random number. However, I was required to write asserts that explicitly prove that this number is truly random. The random seed method essentially predetermines the sequence of numbers that will be generated. In other words, if you set the same seed value, the so-called ‘random’ numbers will always appear in the exact same order. This means that the randomness is not truly unpredictable, but rather deterministic and reproducible.

That‘s literally computer engineering level

And before implementing, my teacher especially said we had to do unit testing before like what‘s the point of that?

[fiery arrow]

Can you show the assignment text?

[proud nebula]

If you don't understand the assignment or the teacher is bad, don't randomly blame unit testing as a concept for that. That is clearly nonsense.

I am reminded of the song Blame it on the Boogie 🤣

[mellow geyser]

The point of writing tests first is to help you think about the goal before jumping straight to typing code as well as thinking through the interface you would expose to your users.

And the point of tests themselves is to ensure the actual behavior matches the expected behavior. It's helpful it not only documenting the behavior but also refactoring your code with confidence. And it's helpful to force you to think about what is even the expected behavior, and from whom (which user and context?)

So now, you have to think in terms of what are the expected behaviors you want to ensure?

• Is it that given the same seed, you do observe the same sequence?
• Is it that the sequence of numbers appear random enough?
  Or maybe is it both or more?

If you want to test the randomness itself, you may want to take a step back and think about the bigger picture:

• The main question is whether or not a sequence of numbers is random enough. You don't care and can't say if a number in isolation is random. So you want to rely on the properties of randomness, and the related statistics
  Based on that, you can search on google for randomness tests and notice there are statistical tests that are well documented and even class material about that: https://www.cs.fsu.edu/~mascagni/Testing.pdf. This will point you towards methods you can implement yourself or look for in python libraries

This can go pretty far and you may want to check with your teacher how far down the rabbit hole you want or need to go

[molten hollow]

No one would actually "test randomness". In most cases, we'd treat random numbers as an external dependency and either fake/mock it, or if you really must, use a seed so that the tests are deterministic. More over, you probably didn't write your own random number generator, but used one from standard library - testing standard library is also not something we'd normally do.

[molten hollow]

So that assignment doesn't make much sense to me. Maybe you can quote it in full here?

[molten hollow]

That actually makes a lot of sense, it's a good practice. However, because there's a lot of merit to it, he should've better described why that's useful.

[marsh raft]

I could imagine "testing" a RNG by running its output through some sort of statistical wozzit, and eyeballing it to check that it wasn't emitting all nines or something. But that'd be a manual test, not an automated test.

[mellow geyser]

check the link in my message above as it explains a few statistical tests one can apply. There is no need for manual eye balling

[marsh raft]

https://tenor.com/bsVTR.gif

[mellow geyser]

It's undergrad level math but you can intuit a lot of it.
For instance for the chi square test, you can see it runs a bunch of experiments and then computes the sum of the scaled differences between what it observes and what it expects. If you are testing the fairness of a die with 6 faces, you would assume the probability to land on a given face to 1/6. And if you throw a die a hundred times, you would expect to land on a given face around 100 * 1/6.
And so with that formula, if what you observe is too far away from what you expect, then that sum of differences will be larger and reach a threshold that is too big to be random

[molten hollow]

Every once in a while that test would produce a false-negative/false-positive, due to chance. There are easier ways to develop software that uses randomness.

[marsh raft]

Yes, hence "eyeballing it"

[molten hollow]

IMO it's simpler to just treat randomness like an external dependency, and fake it.

[proud nebula]

Well you missed the entire point then. He was implementing a randomness function.

[mellow geyser]

It doesn't work so much when you want to assess the qualities and performance of a random generator

[shut raft]

do people use AIs to write unit tests?
Any recommendations?

[proud nebula]

I mean.. if you want to use AI for coding I don't think anything beats Claude Code in reality. Some other systems have higher numbers on benchmarks but in practice fail on stuff. It's unclear how much for example Google Gemini has cheated on those benchmarks (I mean... has cheated more than Claude since AI is almost by definition to cheat and not actually think)

[weary quarry]

I've seen people try. My experience is that AI writes passing tests which is dangerous. If the test passes, you're tempting to smile, nod, and move on. But what is it testing?

If reviewing a code suggestion from an LLM is a time consuming process, then reviewing an LLM generated test is twice the work.

[shut raft]

Good point

[weary quarry]

-# Also just write your tests first and then write the code, but I'll see myself over this corner now.

[pulsar oracle]

I've used it when the tests I was writing was based on a set of data that was predictable. I'd already written 40+ of them and it was basically to make sure a dictionary was constructed with everything necessary for a specific protocol as necessary for a legacy system basically. But otherwise it's better to structure your code in a way that you can easily write a few asserts and have AI fill in the implementation.

[mellow geyser]

I do sometimes.
My main recommendations are:

• Keep it focused and narrow for your tasks. Be specific for what you want.
• Don't trust, and verify
• Keep your CLAUDE/AGENTS.md up to date with your wants and needs

Model it as a human amplifier, not a substitute. This means to use it in ways and when it makes you more efficient.
Beyond writing tests, it can be used to plan your tests (or testing strategy) and figuring out if you are missing cases, making documentation, refactoring or any other case that can go beyond just writing tests.

[molten hollow]

I tried that once, but I noticed that my prompts were very similar to actually writing the tests. I needed to put the same information into the prompt, that I did actually writing the code. Sometimes it was just simpler to write the code.

[proud nebula]

If someone blocks you, the only way to notice is that you can't react to their messages. I tried to agree with the above but... 🤣

[mellow geyser]

Indeed. It's a tool and sometimes it will be easier/faster to use AI and sometimes not.

[proud nebula]

Imo a lot of the value of AI is really not that AI writes code, but that APIs kinda suck so that the natural code isn't available and we have to write a lot of boilerplate to work with those crappy APIs.

Hmm... I wonder how APL programmers feel about AI.

[swift pewter]

The question is what one should do then: Have LLMs write boilerplate to deal with the bad APIs, or fix the bad APIs.

[proud nebula]

Yea. Making really good APIs is damn hard though. And I'm pretty sure AI won't help much ;)

[mellow geyser]

indeed, AI can be great at understanding existing API, refactoring them documenting them. It can also help critique new ideas, APIs and sometimes write some of it.
But as usual, only a sith deal in absolutes and still wonder why some folks still get hung up on it being either completely terrible all the time or being awesome all the time.

[surreal trellis]

Hey guys 👋🏾, do you know any complete tutorial for learning pytest that also covers fixtures 🔧 and explains the differences? Preferably a video course 🎥. I need a full guide because I want to write unit tests for my Multi-Purpose Calculator project 🧮.

Also, if the tutorial covers testing with dependency injection, that would be perfect. Does anyone know of such a resource? I really need it 🙏🏽.

By the way, I already searched on YouTube 🔍 and found a couple of relatively long videos ⏳, but they didn’t really cover the parts I was looking for and weren’t that useful for me.

[proud nebula]

I would highly recommend reading the official docs.

[surreal trellis]

Thanks! I’ll definitely go through the official docs 🙏🏽. Honestly, sometimes reading the docs is a bit hard for me, so I was just hoping for a structured video walkthrough too, since that way I can learn more easily.

[proud nebula]

I believe that feeling of learning from video is false. It feels easier because you are learning less. Real learning/thinking is hard.

[surreal trellis]

Got it, thanks for sharing your perspective.

[molten hollow]

By the way, I already searched on YouTube 🔍 and found a couple of relatively long videos ⏳, but they didn’t really cover the parts I was looking for and weren’t that useful for me.
Can you share which parts did you look at?

[proper marten]

hey guys good day. should i use pytest or unittest for django? thankyou, they say pytest is better since it got parameterize, and fixtures. but i wanna some of your ideas thankyou.

[molten hollow]

Both are fine, chose what you prefer.

[edgy nacelle]

Can someone help me? I am trying to use MySQL to make a small app but I get this dumb error that has no fix at all:

Process finished with exit code -1073741819 (0xC0000005)

Apparently there is no fix because google and chatgpt have provided no information on it and I am really annoyed.

I quit coding for today due to this stupid error

[proud nebula]

Ypu should first track down which line causes the error.

Oh. And this is absolutely the wrong channel.

[molten hollow]

Well, it could mean you don't have the necessary libraries installed. You can remove the unused imports. Then, the error could be due to the app not being able to connect to the mysql database. Try surrounding the call in a try-catch to catch the exception.

[clear vale]

Is AI good at writing unit tests?

[river pilot]

it's impressive what it can do, but it's flawed at that, as it is at many things.

[limpid raft]

I have a set of pytest fixtures, and I need to create another set of fixtures that are all identical, i.e. like this:

@pytest.fixture
def new(old):
    """make a new instance from an old instance"""
    return new.make_from_(old)

since I have well over 30 of these, I was wondering if there's not a way to loop and avoid all this redundancy? Can I programmatically/dynamically generate a fixture?

[river pilot]

can you say more about why you need to make new ones? The fixtures are re-done for every test, right?

[manic belfry]

Can I put a parametrizer "alias" in conftest.py?

For example:

# conftest.py
s_parametrizer = pytest.mark.parametrize("s", ["hello", "world", "pytest"])

# test_a.py
@s_parametrizer 
def test_a_sample(s: str) -> None:
    assert isinstance(s, str)

# test_b.py
@s_parametrizer 
def test_b_sample(s: str) -> None:
    assert isinstance(s, str)

[river pilot]

yes, did you try it?

[manic belfry]

❯ open tests\conftest.py
import pytest

s_parametrizer = pytest.mark.parametrize("s", ["hello", "world", "pytest"])

❯ open tests\test_a.py
@s_parametrizer
def test_example(s: str) -> None:
    assert isinstance(s, str)

https://privatebin.net/?aa9a6c327e78bd78#3PKPshzkfePQGqrF8nhtuBZyM5fwhK4Td44bf5FLmDX6

[river pilot]

you'll need to import s_parametrizer

[manic belfry]

Oh

I was expecting pytest will do so automagically (like fixtures)

[river pilot]

no, in this case it's just Python, not special pytest semantics

[manic belfry]

Understood, thank you!

[limpid raft]

I just have fixtures for all the SQLModel models I am dealing with, and now I need to add an 'export' layer. I decided not to export the SQLModel instances directly, nor to mess about too much with model_dump, but instead to create more or less 1:1 export models for the SQLModel models. And now I want fixtures for them. And since each export model instance is generated from an instance of SQLModel model, the code for each export model fixture is essentially the same, and only differs in the target class.

[tall brook]

have you checked this https://docs.pytest.org/en/stable/how-to/fixtures.html#parametrizing-fixtures

[limpid raft]

yes. But you cannot parametrise a fixture with another fixture

[river pilot]

you could definitely do this without writing out 30 fixtures. At the very least, you could write a loop to create a string with the Python you want to avoid writing, and then exec() it.

There are other ways to do it, but this low-tech approach might be easiest to write and debug.

[limpid raft]

Oh oh. In 20 years of Python I've so far avoided exec() like the pest 😉

[proud nebula]

(the "plague" in English)

You could also do some code gen or something.

[limpid raft]

what is "some code gen or something"?

I just want to write a factory generating pytest fixtures, or better even: use an existing factory 😉

[proud nebula]

You can generate code. So similar to what ned wrote above, but write that code to disk instead of generating it dynamically every time.

[limpid raft]

right.

[spare wedge]

I'm going to write some end to end tests with pytest and playwright. I'm testing a largish webapp with a login page and I want to test both, latency, functionality and freshness of the data diplayed. So somewhat of a largish scope. Does anyone have any resources for setting up such a test suite? I'm thinking code organization wise.

[marsh raft]

I tried using playwright with pytest-django's "live_server" fixture, and could never get it working. The live server refused to serve static files 😐.

shame, since I could really use some end-to-end tests, and selenium is a pain in the a**.

[spare wedge]

I used selenium last time I tried this (2 years ago maybe). After making the "log in" test work I just felt it would be entirely to much work to make the full thing work. Have high hopes for playwright though.

[marsh raft]

yep, that's roughly my experience with selenium too. See above under "pain in the a**" 🤣

if you ever get the live server to serve static files, tell me how you did it 😕

[molten hollow]

For things such as latency and speed of page, I would just profile it. I don't think you need acceptance tests for that

The reason is, such tests should only fail if there's something wrong with the solution. If your app has a hiccup, that shouldn't invalidate the tests.

Perhaps a better question is, what are you really trying to achieve now?

[marsh raft]

who, me? I want to catch crashes and stuff before users do.

pptt wants to test functionality as well as latency.

[proud nebula]

Testing latency is super hard. Adding an entire browser to the process is going to make it harded imo.

[marsh raft]

I just run the "lighthouse" thingy in google dev tools once in a while

that pointed out a few problems which were easy to fix, iirc

[ember maple]

at work we currnetly use a combination of sentaku, navmazing and widgetastic (which is in the process of migrating from selenium to playwright)

[odd walrus]

Did this make the rounds here? I don't think so given a quick search with has:link etc. Ignore inflammatory title, IMO this has a reasonable message.
https://nedbatchelder.com/blog/202509/testing_is_better_than_dsa.html

[river pilot]

"inflammatory" 🙂 I prefer "punchy"

"pithy"?

or should I use 18th-century conventions? "In Which I Argue that Beginning Learners Get More Benefit from Learning Testing than they will from Grinding DSA" ?

[odd walrus]

Hey man long time no see, we met at at least one RubyConf back in the day, unless I have you mixed up. Wilson.

Hot takes are always at least warm to the touch. 🙂

[river pilot]

thanks for the greeting, but I've never been to a RubyConf, maybe somewhere else?

[river pilot]

yes, and on the internet it helps to give the heat-seeking attention missiles a clear target.

[odd walrus]

Oh my bad, must be remembering a different “Ned B”

Unless Strange Loop

[river pilot]

if it wasn't PyCon or Boston Python, I wasn't there! 😄

[odd walrus]

Ohhhh

[odd walrus]

Hey are you friends maybe with Amy Woodward? Because I lived in PDX and hung out several times at the bars with PyCon people etc without having attended?

That's my last guess, after which I'll assume I'm just hallucinating a similar name 🙂

[marsh raft]

ooh ooh what 'hood in PDX? (I myself went to Beaverton High School)

[odd walrus]

Fremont mostly and then whatever we call 97217 now, maybe "Overlook"? Jarrett St, if you look that neighborhood up.

[marsh raft]

all I know about Fremont is Stanich's. Ooh, the bar that replaced them in the same spot also has a "The Special" burger.

[odd walrus]

Some cool stuff in that area, there was a really good local pub and a bunch of nice shops in the area at the time, I lived in PDX from I guess early 2011 until some time in 2017

[pulsar oracle]

I was literally able to put together code for the sliding window rate limiting algorithm without ever hearing the word and just knowing the behavior that I wanted out of it and writing tests to know when I got it.

[ember maple]

that one is a bit tricky - its easy to get side-tracked
DSA is important to a specific degree to even enable creating enough structure to have something to test and to refactor
however most people will not deal with most of the exotic datastructures to begin with

i'd argue for a circulum that gets one into a test driven approach, then adds performance tests and datastructure changes so see steady improvements that make sense

[river pilot]

i like that idea. I'm definitely not trying to do one to the exclusion of the other. The post includes a list of what I think is important to learn about DSA

[ember maple]

no more paper based battle of radix sort vs quicksort

[proud nebula]

Yea, like most things there's a 80/20 optimal thing and school often goes way past the optimal point.

[proper wind]

ok, so with mock_testing. patching, you mimic the api and does the library have to be there as well in the .py script with the other imports?

[river pilot]

ideally yes, because you should use autospec=True, which will look at the real objects and ensure the calls match the actual signatures.

[river pilot]

why wouldn't the real library be installed?

[proper wind]

I mean in the test suit, not installed, imported.

[river pilot]

you shouldn't need to import it

[proper wind]

do people ever unittest dataframes?

[odd walrus]

I test everything that isn't actually just impossible to test.

[river pilot]

you don't have to test all your code, only the parts that you want to work.

[proper wind]

ok, should a config test just be brief and if it is always used for all .py files, should it be a fixture?
also, should conftest.py contain a dummy_something of what one is going to use across all tests?

[marsh raft]

what is "a config test"?

I use conftest.py to hold fixtures that I need in more than one test

[proper wind]

if a class is only used in the .py part of the test suite you are testing and no other ones, then should that instance not be in the conftest.py?

[pearl cliff]

tests/
  groupA/
    pytest.ini
    conftest.py
    ...
  groupB/
    conftest.py
    ...

tests/groupA/pytest.ini contains testpaths = .. When I run pytest -c tests/groupA/pytest.ini I expect it to only find tests within tests/groupA/pytest.ini, but it also collects tests under test/groupB unless I also specify tests/groupA on the command line, or some other equivalent filter. Why does that happen, and can I configure Pytest to work the way I expected it to work? I can see that rootdir is being correctly set to tests/groupA

I even tried including --ignore=../groupB in addopts within tests/groupA/pytest.ini, but it semed to have no effect.

I see this old feature request but it was kind of an X-Y problem and it resulted in a solution that doesn't appear relevant for me https://github.com/pytest-dev/pytest/issues/12749

[proper wind]

ok. So it has to be included in more than one test

[proper wind]

would a config.py file be good for a conftest if it is used throughout the entire project and is used in all test_name.py files?

[pearl cliff]

I'm not sure what you're asking, sorry

what is config.py?

is this related to my question, or are you asking me a question?

[proper wind]

how do I mock that api and does the dummy config even need a patch?

[molten hollow]

@proper wind Did you try dependency injection?

[proud nebula]

And by that he means "pass in an object or function that is used to do the network call so it's easy to pass in something else in the test"

[limpid raft]

am I wrongly assuming that caplog should capture logs and inhibit their printing to stderr if such is configured?

caplog works, but the test output is also disturbed by all the log output

[lusty horizon]

Hi anybody online who has time to discuss how to design good tests?
I have some instrumentation, I would like to implement in code. Some parts are already implemented. But how do design good tests, testing for size, shape, materials? Part of this information is already implemented in the actual class of the instrument

[proud nebula]

Don't ask to ask, just ask.

Maybe you should paste a small bit of code and ask for some kind of code review?

[lusty horizon]

Should implement instrument properties in my actually code and then asser them with a test?

[proud nebula]

I didn't understand that. What is "instrument properties"?

[lusty horizon]

class _Lambda(Detector):

    MANUFACTURER = "X-Spectrum"
    # This detector does not exist but those are place-holder
    MODULE_SIZE = (256, 256)
    MODULE_GAP = (4, 4)
    DUMMY = 0
    force_pixel = True

For example that was implemented like this.
I would like now to add the class attribute SENSOR

And then there are more like

class Lambda60k(_Lambda):
    """
    LAMBDA 60k detector
    """
    MAX_SHAPE = (256, 256)
    aliases = ["Lambda 60k"]

@boxed: Before I was always in the situation, I have a function, I put in x, y comes out. I could calculate the solution, here I cannot.

[proud nebula]

Why do you say you want to add SENSOR?

[lusty horizon]

It will hold materials properties to improve the correct description of the instrument.

My question is how to write here code in a more TDD style. Is it the only way to implement all properties in the tests and then again in the actuall classes?

[proud nebula]

So purely informational, it has no behavioral impact on anything?

[lusty horizon]

Yes.

[proud nebula]

Yea, then there is nothing to test imo. TDD is a tool, you should only use the correct tool for a job, not some other tool that makes no sense.

[lusty horizon]

Can you explain this differently? TDD is test driven development. Why can I not test for proper implementation of constants?

[proud nebula]

Because it makes no sense. You can test that constants exists. For example if all subclasses of Detector must have a certain member, you can loop through them all and check that.

But testing for constants? No, it does not make sense. Like you said above: you will just write it in two places. If you can't write it correctly the first time, why would the second time be different? It makes no sense.

Some people treat TDD like a religion. Ignore those people.

[lusty horizon]

Ok, I see that point. So I could write a test, that asks if properties are in place and reasonable.

[proud nebula]

Yea. Checking ranges and types makes sense for example. Absolutely.

[lusty horizon]

Btw, not in science. People don't write tests and gone is the reproducibility (at least in my area)

Thank you. I will try to write a generic test that includes at least the properties

[proud nebula]

What? TDD isn't science. No one talked about science.

[lusty horizon]

I meant people in my field avoid writing tests . "Some people treat TDD like a religion. Ignore those people." is what you wrote

[proud nebula]

Aha, yea, I've done a little bit of work in science and the approach to code had me screaming internally. Horrible stuff. The lack of tests wasn't even the biggest problem 😱

[lusty horizon]

I do feel you. I try to better. But a prof told me he is not paid to write tests. His approach to reprodcuible code was horrible.

[proud nebula]

Yea, unfortunately a common attitude. A lot of scientists have the attitude that their job isn't even to find the truth but just to publish. That's why we've got this huge replication crisis....

[lusty horizon]

To be fair: I am never judged by good code. My profs ask only for results and with first author publication you get your next job

[pearl cliff]

Something to consider: you can write type hints on the base class but not provide values for them. You can rely on type checking as a separate layer of tests

the #1 reason I think type hints are useful in Python is that they significantly reduce the amount of "manual" testing you have to do

[brittle kiln]

.

[pearl cliff]

another option is to require, on the base class, that all subclasses implement certain attributes

for that you can just use the built-in abstract base class framework. or write your own easily with __init_subclass__ (but mypy won't understand it as well)

[molten hollow]

@lusty horizon Try to think of this in terms of behaviour. Like, you run your program and program does something. You can TDD that something. If you just declare constants, the behaviour of your system won't change, so it doesn't really make sense to test those.

Unless of course you're doing something like meta-programming, where declaring constants actually does change the output of your program somehow. It's weird, but it's possible. If that's the case, then you can TDD those outcomes, but not directly via constants, but via that behaviour that's now changed.

Some parts are already implemented. But how do design good tests, testing for size, shape, materials?
If you want to do TDD, you should really start without code. If the code already exists, it can be hard to introduce good tests to it.

My question is how to write here code in a more TDD style. Is it the only way to implement all properties in the tests and then again in the actuall classes?
It comse down to you asking yourself a question: "What do I want to happen?". Not in terms of implementation, but expected change in behaviour.

Thursday, 9 October 2025 14:06
Can you explain this differently? TDD is test driven development. Why can I not test for proper implementation of constants?
What does it mean "proper implementation of constants"? Who says that's the "proper" implementation?

So I could write a test, that asks if properties are in place and reasonable.
If they influence the behaviour of your system in some way, then yes. If they don't, they may as well be a comment.

I meant people in my field avoid writing tests . "Some people treat TDD like a religion. Ignore those people." is what you wrote
Well, there are these kinds of people, but that doesn't mean all of TDD is bad. Like any practice, TDD can be misused and misatributed. I'm a big proponent of TDD, but I've seen people pushing it in places where it doesn't make sense. That could be "religion".

I do feel you. I try to better. But a prof told me he is not paid to write tests. His approach to reprodcuible code was horrible.
And chefs aren't paid to sharpen their knifes, drivers aren't paid to wipe their windows, but they do it because it makes their job easier 😉 Same with tests. Every programmer, however against-tests he is, he will still test, but manually. He will run the program and click through it. I never saw a bad or a good programmer who wouldn't do at least that. So the question now becomes not whether to test, but whether to test manually or automatically? Which is cheaper, faster, better, more reliable? And automatic is cheaper, faster, better and more reliable in any program that takes more than 1h to write, in my experience.

[odd walrus]

IMO despite the hype, this is exactly why the right way to think of it is "Behavior-Driven Development", not "Test-Driven"; constants aren't good to test because they aren't "behavior"; look for something that USES that constant, the thing that made you add it, and ask/tell that how it works.

[proud nebula]

Just don't use BDD tooling, because it's mostly putting regex in between two layers of your test code. And no one wants that.

[pulsar oracle]

Yea even the creator of BDD (Dan North) just uses pytest and comments the given when and thens for stuff in the regular test code (he said so in a recent podcast).

[pulsar oracle]

I would argue that a constant on a class that is just supposed to provide information to the consumer is behavior though. If you were to TDD it, it might not be a function or method call result or side effect but you could say "There should be a lambda60k class and it should contain a constant with the max shape size" and if anyone ever changes that or it breaks some way somehow you have a test in place making sure it's still there for the user (another programmer) as it should be.

[pulsar oracle]

The real idea behind BDD if I've understood it correctly (outside of the business aspects) is that you don't think of the tests as tests, you think of them as specifications and you specify the externally verifiable behaviors you want which you can do at the unit level with normal code or whole system with acceptance tests. Generally specify what you want the code or system to do and not how it does it.

[odd walrus]

Yeah exactly, it's a tool to help you build as little as possible, and to have it be the right stuff, before it's anything about 'verification' etc.

[proud nebula]

Imo that was the idea of TDD too. Sometimes in the olden days "Top Down Development"!

[pulsar oracle]

I think it literally is/was as well. If I've understood the origin story of BDD correctly Dan North came up with this part of it to explain it better to some developers who didn't think it was their responsibility to test things, "as a way to get to the high value of TDD" and the other aspect which everyone obsesses over cucumber about is communication to get work done.

[odd walrus]

I do still highly recommend Kent Beck's original TDD book, despite also suggesting a look at a BDD tool like RSpec etc.

[molten hollow]

I would argue that a constant on a class that is just supposed to provide information to the consumer is behavior though.
There's a very simple check. If you add a constant and you run the program again, does it something differently? If yes, that's a behaviour. If not, then no.

You know, TDD and BDD aren't really in conflict with one another. And also, BDD isn't about the tools. As it was correctly pointed out before, you can use bdd in any testing framework, like pytest.

[odd walrus]

Yeah BDD is just a way of doing TDD

[pulsar oracle]

Maybe behavior is the wrong word. But a commit stage check in place that this class still contains what I want.

[odd walrus]

IMO

BDD is just telling you how to decide what the tests look like, in my view

I've seen other viable TDD approaches

I just like BDD a lot

but it maybe isn't the right way to test a boot loader for your OS

[pulsar oracle]

BDD doesn't say you have to use given when then. They can look like normal unit tests or use an internal DSL.

[molten hollow]

Just tell me what the constant actually influences, and I would write a test for that. I would allow myself to refactor the constant to something else, and the behaviour of the class should stay the same.

[odd walrus]

Yeah you can do BDD with a flint axe if you want, it's a mindset to me, not a prescription for tooling

[molten hollow]

If I remember correctly, Dan North and Chris Matts, who invented BDD, said that TDD was a great idea, but most people got it wrong. And they wanted to introduce something, that will better resonate with people. They didn't like that there are "tdd evangelist", so they wanted to get the words right.

test -> scenario
check -> specification
suite -> feature

etc.

[pulsar oracle]

It's also evolved into the other part where it's communication for work to get done for a feature. So usually bdd tests are end to end tests when people refer to them. It's muddy imo because it definitely started as just a better way to think of TDD

[molten hollow]

Yea, people misuse that all the time. They mix feature files, e2e tests and bdd, all the time, unfortunately.

[odd walrus]

This is my understanding also from having met a lot of the OGs at conferences; that it's not saying TDD is bad, it's saying "here's a good way to TDD"

None of those guys hate TDD etc

[molten hollow]

I don't think it's about practice, because when you look at someone from behind their shoulder, and you see them write a test, you can't really tell whether they're doing tdd or bdd. The practices themselves, are actually similar if not identical. It's about getting the words right, and communicate it to people.

When people hear "test", they think - I need to check/verify/assert. When they hear "specification", it's more probable that their brain will click into the right mindset.

[pulsar oracle]

They're practically identical if we're talking about unit tests and testing behaviors

[molten hollow]

Where aren't they identical? 😄

[pulsar oracle]

Given a user has registered an account and signed in
When they leave a comment on someones profile
Then they will be unable to login to their account when they try after

^^^ You'd never write a scenario for a feature like this normally. It's just synonmous with features, like whole features. This original one was focused on code or units of code that you wouldn't be able to even think about like that.

It's just a synonym for acceptance test driven development atp

[molten hollow]

I see a test written in a syntax of a feature file, but I don't think they're a synonym for BDD.

[pulsar oracle]

The idea is the same regardless though

[molten hollow]

You can use BDD with or without feature files; and you can write feature files with or without bdd.

They're orthogonal.

[pulsar oracle]

I agree but I figured it easier to convey the kind of functional test people do when they mean BDD rather than TDD, with the scope of everything, and a feature. Normally unit tests don't test or specify features and that's not really what I think anyone does with TDD even if it can get pretty close with stuff like DDD or clean architecture and the like.

It's morphed to take on additional meaning is what I'm trying to convey, and practically speaking it's turned into meaning a functional test, or an acceptance test.

[molten hollow]

You your point being that people think "bdd" means "high level acceptance tests", and "tdd" means "lower level unit test"?

[pulsar oracle]

Yea, anything for the code = TDD, practically speaking, anything for the system as a whole = BDD, even though BDD started initially as just a better way to communicate what the real value in TDD is.

[molten hollow]

Because, while I agree that most people who don't know what they're talking about might say...,

that's not really true. You can use BDD all the way down to the lowest level of your unit tests. And you can use TDD for higher level too.

You don't need feature files to use bdd, so you can take the smallest unit test you can think of , and bdd that thing 😄

[pulsar oracle]

The idea is indeed identical for both, which boils down to specify what not how

[molten hollow]

Yea.

[pulsar oracle]

You also don't need feature files to test a feature of a system, you can use normal unit test style tests

[molten hollow]

Yea. Still buffles me why people get this wrong.

When you play games, something you can buy a skin. Like in cs go , you can buy a skin for your AK17, or in league of legends you can get a skin for your character, but it doesn't change anything about it, other than its appearance. I think BDD is like a skin to TDD. You can continue to use it exactly as before, but with some better words around it.

[pulsar oracle]

Google is not helpful is all I can say really. I mean I got it fairly early on where the value in TDD was, and saw how to do it and started specifying what not how. But I didn't get what BDD was because it isn't, because though gherkin/the format you do them is, is irrelevant, it's not just about code anymore and thinking about it like this with how it started I don't think. A lot of it is identical in principle, even execution depending on how you do it for sure.

[molten hollow]

The problem is, when there is a new practice, that's 90% methodoloy and 10% tools; people will not get that, and pay attention to that 10% of the tools, and ignore the process completely.

Then you get buggers like "bdd is feature files" thing.

[pulsar oracle]

I ignored all the tools and I was still confused (even after hearing the origin explained by Dave Farley, I did get the point of TDD though, well some of it anyway!) 😂

[molten hollow]

Right now, when I try to simplify things:

• tdd: write test first, then code
• bdd: test for behaviour, not implementation

[odd walrus]

Basically I introduce BDD when someone asks me "why" do TDD. If they already get that and feel ready to try it, I just stay out of the way.

[molten hollow]

You could introduce them both at the same time I think. Most of what BDD suggests, Kent Beck already suggested in TDD.

Like both of TDD and BDD suggest that you should test the what, not the how.

[odd walrus]

Totally yes, and depending on how they ask about it I'll choose one or the other.

Like, I might talk about TDD but actually show them RSpec as the "test framework" and let them discover it themselves

[molten hollow]

I might chose to ignore the names, not to tilt someone 😄

[pulsar oracle]

I would royally confuse someone if I tried to explain this so I would probably just teach someone TDD and tell them we're testing behavior, what not how, and not mention BDD. and leave BDD for like acceptance tests or applying the same idea to features, or writing specifications and coordinating with other people for what we want the system to do before we implement it and have that same specification/test verify it.

[odd walrus]

my buddy Brian and I made RubySpec, the "test suite" for the Ruby language, and it's all in BDD style, and it worked really well there because we only care about the stuff the language "actually does" etc.

[molten hollow]

But this previous guy with the constants, I think he wanted to do one of two things:

• he either got some framework, that parses the constants and does something based on them, like meta programming. In which case you can totally TDD that, by specifying what you want to happen, and then implement that
• or he just wanted to add a constant first, before he uses it, and wanted to test that in small step. In which case I think he should just remove the constants, and first write a failing test

[odd walrus]

Now what is weird is how code-coverage tools often handle lines that define constants

and make them look uncovered

[molten hollow]

Coverage is a whole another topic, don't get me started on that 😄

[odd walrus]

You need a real dataflow code coverage tool to do a great job there probably etc

[molten hollow]

@odd walrus do you? If you TDD your application from the start, you very rarely get uncovered code.

[pulsar oracle]

Coverage isn't really something one should ever intentionally cover, but when I say this people go, "oh yea i don't test either it takes too much time"

[odd walrus]

What I'm saying is that even if it's covered, some tools make constant definitions look left out etc

[pulsar oracle]

I gotta be honest, If I'm developing a "web app" I move all the important logic to use case or service classes, ports and adapters style and test there, the driving adapter code can get screwed up for how I've been developing for awhile (only got into acceptance tests recently) but I've managed to cover a lot that way. I'm still going to skip those layers and save them for functional tests.

[pulsar oracle]

I think he probably wanted to know what the TDD approach would be to this problem as like a best practice.

[odd walrus]

I actually kinda like "presenter" models in web apps largely because of how testable it makes the front end code

View - Presenter - Controller - Model

The view ends up tiny

[proud nebula]

BDD is dead to me because it has become synonymous with Cucumber and similar bad ideas. That is just the semantic treadmill. Words drift and we need to move on.

[molten hollow]

Sure. Any kind of decoupling is a good idea there. If you really bite TDD, then that kind of code comes up always for free, because using TDD it's less probable that you create code that's hard to test.

Sure - if you get the code first, it's hard to make it testable; especially if it's coupled to the ui library.
but when you start with tests, its really hard to write untestable code 😄

[proud nebula]

Don't look at my framework iommi then. It goes hard in the other direction ;)

(Imo more testable and less need to test huge chunks too)

[pulsar oracle]

That's understandable. The word BDD is almost a red flag to me at this point. I personally like that Gherkin can describe something in natural language in a way that just writing code that tests something doesn't. I can't imagine using it for unit tests though (maybe some people find it helpful but I don't). Very very confusing to discuss too.

[odd walrus]

I'll check it out, maybe the other direction is even better etc.