unit-testing

[kind meadow]

INPUTS = [
    ("a", 1),
    ("b", 2),
]

@pytest.mark.parametrize(["inp", "expected"], INPUTS)
def test_do_something(inp, expected):
    actual = do_something(inp)

    assert actual == expected

And let's say I also wanted to apply a function before testing ```py
@pytest.mark.parametrize(["inp", "expected"], INPUTS)
def test_do_something_2(inp, expected):
inp = transform_inp(inp)
actual = do_something(inp)

assert actual == expected

But... without having two separate tests and without doing something like ```py
INPUTS += [(transform_inp(inp), expected) for inp, expected in INPUTS]

There probably isn't a simpler way

[potent quest]

hmmm, maybe something with fixtures

you can parametrize fixtures as well

[kind meadow]

Not sure. I'm just gonna stick with separate tests for now; might be nicer for making sense of the results anyway. But I'm still open to suggestions.

Maybe it's a bit of an x y problem. What I am actually doing is randomly inserting whitespace into the input. I want to test the input with no whitespace, and again with random whitespace. I just wrote my own simple function for inserting the whitespace but maybe there's a more generalised library for doing that sort of thing.

[fair elm]

can anyone help me with nosetests and sonarqube?

[fair elm]

https://stackoverflow.com/questions/71235710/getting-0-coverage-in-sonarqube-using-nosetests-for-generating-coverage-xml
Here is stackflow link , having my queries. Please do response, If you have some suggestions for me.

[scenic hawk]

how do folks handle prototype classes and test coverage?

[maiden pawn]

prototype classes? sounds like javascript.
Unless you speak about design pattern prototype

just to be sure you are in python

we have no prototypes

for inheritance python uses
Abstract classes
just Classes
We can inherit one class to another one, one class can even inherit from multiple parents.

class A:
  pass

class B:
  pass

class C(A, B):
  pass

and we can include one class to another one, of course

obviously we just need to test child classes in order to have them test covered

I use coverage library for pytest to check stuff

[meager fable]

I have a unit test written that is checking if requirements.txt is met before starting a flask website. Is this bad practice? I was going to try and execute the unit test at startup and if it fails don't start the website?

[misty peak]

Is there a way to unit test a web scraping script?

[kind meadow]

That test isn't really reassuring. You shouldn't rely on the assumption that it will work just cause dependencies are all present. You need more tests.

You should instead be testing that the features provided by those dependencies or that are reliant on them function correctly. But if you have those tests, then it transitively means dependencies have been satisfied.

Perhaps I misunderstood. If you're concerned about testing your deployment then that's outside the scope of unit testing.

[scarlet marlin]

Yes - you could download html from web pages where you know exactly what you want your scraper to output, then run your web scraper on those html files and assert the result

[scenic hawk]

Sorry, protocols. PEP 544 classes.

[misty peak]

im not sure how to unit test this script

[scarlet marlin]

Something like this. https://www.tutorialspoint.com/python_web_scraping/python_web_scraping_testing_with_scrapers.html

Except their example sucks because it’s using a real website connection instead of reading a static file

[misty peak]

oh

[scarlet marlin]

But otherwise it makes sense

Sorry if this comes off as rude, but do you understand my reasoning behind why I say their test example sucks @misty peak

[misty peak]

ya

nah it's fine im super new to python n im forced to do a project regarding web scraping and unit testing

import unittest
from bs4 import BeautifulSoup
from newscrapy import NewIan

class Test(unittest.TestCase):
    bs = None
    def setUpClass(self):
        url = '<a target="_blank" rel="nofollow" href="http://192.168.137.160/spicyx/ </a>'
        self.soup = BeautifulSoup(NewIan(url), 'html.parser')
    def test_titleText(self):
        pageTitle = Test.bs.find('h1').get_text()
        self.assertEqual('Python', pageTitle);
    def test_contentExists(self):
        content = Test.bs.find('div',{'id':'mw-content-text'})
        self.assertIsNotNone(content)
if name == 'main':
    unittest.main()
``` this is the new modified code

[warm plume]

how would you do test driven development for opencv drawing functions, so for example, a function whose purpose is to just literally draw a box on the screen. Right now I just do visual inspection to make sure the function draws properly but then what about automated testing in a CI pipeline?

[scarlet marlin]

If you want to be real intense you can take screen grabs and make sure they are pixel-for-pixel perfect. Probably easier would be to abstract “create an image of a box” away from the code for “draw an image on the screen”. Then your test isn’t “can I draw an image of a box on a screen” which would be pretty harder to write and very brittle, but “can I draw an image of a box”, which is easier and more reliable since you’re testing fewer things at once.

[warm plume]

can you explain this a bit better? Let's say I have a function that draws a bbox on an image:

def draw_track(track: Track, image) -> None:
    """
    draw Track object on image

    Args:
        track: Track object you wish to draw on each of its images
        image: image on which the track will be drawn
    Returns:
        None
    """
    logger.info(f'bbox coordinates: {*[track.x1, track.y1, track.x2, track.y2],}')
    cv2.rectangle(image, (track.x1, track.y1), (track.x2, track.y2), colors.green, 1)

the Track object is a dataclass object with several attributes including the top left and bottom right corner coordinates. The image is what I want to draw the box on.

Now how would you "create an image of a box" for this simple example

[scarlet marlin]

You appear to be already there

[warm plume]

hmm how do I test this function then?

right now I just look at the image with the box drawn on it

to verify it is ok

I have some very complicated drawing functions that actually project a 3d shape onto several images using a projection matrix

those are the types of functions I really wish to test

[scarlet marlin]

Take a very simple base image, like an all white background, then run draw_box, and inspect it yourself to see that it’s correct. Save that resulting image and the input image, and use them as input and expected result of ‘draw_box’ in a test

[warm plume]

I see, thank you, that is a very interesting idea

[scarlet marlin]

This is a pretty typical flow for how to write tests for complex stuff where you cannot just easily define what the output is

Without actually generating the output and looking at it with your own human eyeballs

Iterate your code until the output looks right, save the inputs and outputs, and encode them as a test

[severe raptor]

how does platforms like hacker rank or Gradescope test codes

I really want to understand the concept behind

[hexed cloak]

Usually a VM or container (or both)

Some sort of sandbox

[severe raptor]

thanks, I will do some research on them

[frigid basalt]

You can checkout and use Snekbox

https://git.pydis.com/snekbox

[severe raptor]

Thanks

[sick portal]

Thanks for your reply, Actually i solved the issue i was facing by delegating the generation of id to the repository that way i was easily able to mock it 😄

[frigid basalt]

I do some introspection in code of mine and I want to make sure it works with positional-only parameters - but I run my tests on 3.7. Is there anything I can do to skip these specific tests? Won't it fail with SyntaxError even if I skip it?

[scarlet marlin]

if you're using pytest you can use test marks: https://docs.pytest.org/en/6.2.x/mark.html

[frigid basalt]

Doesn't pytest need to load the file to read them though?

[scarlet marlin]

yeah. Tho if the test is encapsulated in a function where you aren't running the mark, I don't think it actually reads the code inside the function? I'm not sure if it would be able to ignore a syntax error tho

[hexed cloak]

You have to use eval - or make a new test file that's not collected on 3.7

[fallow turtle]

Can anyone give explain to me or give some kind of resource on how to write tests for requests? I haven't written a lot of tests in general and writting tests for aiohttp is a bit confusing

[pearl cliff]

are writing your own server application that will handle requests, or are you testing some code that makes requests to external systems?

[fallow turtle]

Im building an async apiwrapper as a learning project and I would like to test some of the methods there

[pearl cliff]

ah, and you are using requests to make calls to the external api? you have a few options

it also depends on how precisely you need/want to mimic the api

the easiest option imo is to use the "dependency injection" pattern, to avoid having to mock out the requests at all - you can just replace your requests Session object with one that returns pre-constructed response data

there are also a handful of libraries that specifically work with Requests that can help you make mock requests

you could even consider refactoring your application to use sans-io style, which imo is the best choice but has the least support from tooling and doesn't have enough information about it online

(i've done it, i love it, but it's hard and requires more boilerplate code up front)

[fallow turtle]

aiohttp but yea

[pearl cliff]

i think there is at least one mocking library that works with aiotthp

you'd have to search for it

[fallow turtle]

I dont even know what that is but will do reaserch on it

[pearl cliff]

but what technique you specifically use is a lot less important than thinking about what properties do i need to test? and how do i generate realistic fake data to test these properties?

sometimes if you really need very very realistic data, your only option is to actually use real requests instead of generating fake responses

for that you can use https://vcrpy.readthedocs.io/en/latest/

caveat: i havent personally used it, but it seems good if you really cannot avoid making real external calls. this way at least you just make the external requests once and cache them in your test suite, instead of upon every run

[fallow turtle]

My idea was to have some json examples of real requests but that might be a bit tedious to do/there might be a better way

My main testing concern is how those responses get turned into class objects and then work my way up

[fallow turtle]

I'll check it out

Thanks for the help

[royal orbit]

If you have an API schema, https://github.com/schemathesis/schemathesis can generate many tests for you 🙂

[sour creek]

hi, anyone know what it means when i run a program and it instantly closes but it shows for a flash? its downloaded from github and i have pythin 3.10 on windows 10

[pearl cliff]

this has been on my "to-look-at" list for a while 😆

well i have looked at it

but i haven't used it

[wild merlin]

Does anyone know a creative workaround to run some code after all tests / before rusttest is shutting down?

[hexed cloak]

rusttest?

https://pypi.org/project/rusttest/ ?

[wild merlin]

Oh damn. This went to the wrong discord

[hexed cloak]

Can't you just impl Drop?

[wild merlin]

Heh. Nope because i need to wait for all tests to end

[hexed cloak]

Just have a value all your tests borrow

Aparantly GitHub is down but https://github.com/la10736/rstest/commit/1fbf794dff88e2b8d64e6daf2fa6cd32215c16b9

Oh aparantly it's never dropped

[hexed cloak]

What teardown do you want to run?

[fading surge]

@untold ember please do not spam

[fair elm]

I have a .py file where database is linked and insertion, update is taking place from there into the database. I have test files written in Unittest for that particular .py file.

I wish to know , If I give command say coverage run test_doc.py and then coverage report and coverage.html , while getting coverage , will .py file which has database lined will also get invoked and insertion and update will take place?

[shrewd compass]

Hi I am new to testing in python. I have developed some script for mobile browser testing which I am trying to run on AWS-Device-farm, however, script abruptly stops after fetching the url page and do not do anything else, and outputs the test as passed. May I know what might be the issue here?

[empty dust]

could you paste your test code?

[shrewd compass]

Sure

[bitter wadi]

Hey @shrewd compass!

You either uploaded a .txt file or entered a message that was too long. Please use our paste bin instead.

[shrewd compass]

https://paste.pythondiscord.com/sevekajiwa

[empty dust]

The reason the test passes is that you don't technically test anything - you are running the various functions but you aren't comparing any observed behaviour to expected behaviour. Here's an example using pytest:

When you have

            '''Geting Title'''
            title = self.driver.title
            print("Title: ", title)

That print should be replaced with an assertion where you're checking the title against what you expect the title to be

[shrewd compass]

But when I run this script in my pc using Android Studio emulator and appium desktop server, it works fine. Also my major motive is to going to the picture uplaod line and then check whether pictures are getting uploaded or not? It's actually for face recognition.

And I am not even getting there with the script

[empty dust]

What does the exception say?
In general you should remove any try: excepts from a test because it will interfere with the testing framework (which already catches exceptions to assert the test has failed)

[shrewd compass]

There is no exception that is the problem. it says test passed.

[empty dust]

Remove the try: except and you might see the exception

[shrewd compass]

These are the aws logs:

[shrewd compass]

Yes let me try that.

[shrewd compass]

Hi yes thanks something worked, it started to atleast find elements. Will let you know how it goes.

[cobalt roost]

I have two (pytest) test cases that use async methods. Should i make my test cases also be async, or is it better to have them run a task with run_until_complete? I feel kind of iffy with making all the stuff that uses any kind of asyncio method async

[glass pivot]

you can do run_until_complete, but if you're going to have a lot of async tests eventually, I'd recommend looking into pytest-asyncio: https://github.com/pytest-dev/pytest-asyncio

with it, you can simply do: ```py
@pytest.mark.asyncio
async def test_some_asyncio_code():
res = await library.do_something()
assert b"expected result" == res

[cobalt roost]

so in general it'd be better to make the tests async (with the pytest-asyncio lib)

[proper wind]

What are the way I can unit test computer vision results? All I could think of doing is checking if the function return the correct stuff. However it's not as simple as that as I haven't yet programmed the text recognition algorithm.

Here is my program that I want to unit test

https://github.com/PissMan420/recipe-recognizer

I am thinking of splitting up the step of the reconnaitre_facture in different functions however I can't figure out how to unit test the splitted function. How can the computer know that something look ok if it doesn't have eyes?

[slender kernel]

I am using pytest to run my test. I have a file named script.py and a test called test_script.py. I'm importing script in the test and running the tests.

This works.

But when I move my test to a tests/ subdirectory, it shows an import error.

How do you guys deal with this?

[shrewd compass]

Hi need help with one issue while testing on the android browser I am using AWS and when I am trying to upload the picture, I am unable to go past the android permission pop-up. Any help? Pls, mention me. I have tried to use driver.switch_to.alert.accept(). Using selenium==3.141.0, Appium-Python-Client==1.3.0

[wanton axle]

a question on exceptions with pytest.
i want to make sure that say MyException is raised
i have functions like that:

def func0():
  raise MyException()

def func1():
  func0()

and pytest test looks like that

with pytest.raises(MyException):
  assert func1()

when i run tests i get test error, but when i change the test to:

with pytest.raises(Exception):
  assert func1()

then pyspak test passes
i guess there is something different when the exception is raised from func0 rather than func1?

[ember maple]

shouldnt make any difference, seems like something else is wrong

[wanton axle]

thanks! so most likely different exception is raised i guess. i should print that exception and see what is it. is it possible with pytest? something like that would work?

with pytest.raises(Exception) as e:
  assert func1()
  print(e)

[ember maple]

pytest,rasies typically tells it in the error, just make it specific again

[wanton axle]

oh, i see it now. thank you.

[tidal rock]

is there a way to make tox not stop at the first failure?

ah got it https://tox.wiki/en/latest/example/basic.html#ignoring-a-command-exit-code

[knotty pelican]

hi not sure if this is the place to ask this, anyone has reference on 'test automation plans' ?

[royal orbit]

No, that ignores the command failing, you want tox to run subsequent commands but still fail.
https://tox.wiki/en/latest/config.html#conf-ignore_errors

[tidal rock]

in my case ignoring the return code of some specific commands fits better. thanks anyway though, didnt know about that setting!

[south scaffold]

guys, i have a function in my main file def cuboid_volume(): return 1
and a

    assert (convert2yaml.cuboid_volume(), 1)```
in my test_convert2yaml.py file...

i keep getting this error: ERROR: not found: /home/krete77/PycharmProjects/Digital-Alchemy/tests/test_convert2yaml.py::test_cuboid_volume (no name '/home/krete77/PycharmProjects/Digital-Alchemy/tests/test_convert2yaml.py::test_cuboid_volume' in any of [<Module test_convert2yaml.py>])

i have my test.py file in my /tests dir,

this is just a simple test to get it to pass, i opened up another project with way more extensive testing, and runs fine,

im using pycharm

any idea?

i have pytest as my default test runner,

[ember maple]

those test ids look missmatched, im udner the impression something went very wrong, whats the output when you run pytest --collectonly -q on a terminal

[cursive shuttle]

7.(2 points)
def func7(m, n):
count = 1
for i in range(m):
count = count + 1
for j in range(n):
count = count + 1

whats runtime? and big O notation

[ember maple]

its not quite clear what you are asking for, at brief glance it looks like a linear time O(M+N) way to add up numbers, i would strongly recommend the usually (for non-bigint) O(1) + operator instead

[cursive shuttle]

awesome thx

[pearl cliff]

this sounds like a homework question. also this either belongs in a help channel (see #❓｜how-to-get-help ) or #algos-and-data-structs . please keep in mind our rules about homework questions:

!rules 8

[bitter wadi]

Rules

8. Do not help with ongoing exams. When helping with homework, help people learn how to do the assignment without doing it for them.

[ember maple]

🤦‍♂️ im a bit slow on the update on Friday noon

[pearl cliff]

i know, it's easy to get caught up in being helpful!

[lilac rover]

Good afternoon everyone. I am new to docker and kubernetes, Could anyone suggest me few resources for beginner level?

[deep dew]

Hello all! Can someone tell me how should I approach testing a private method which gets called in class __init__? I cannot test this method without creating the class containing it. This could result in errors happening in fixture returning this class object.

class TransactionRepo:
    """Class cointaining temporary transaction data and handling connection and queries to the DB during session"""

    def __init__(self):
        # POSTGRESCONFIG - contains DB connection keywords
        # USERMAP - map USER table columns to Transaction class
        # TRANSACTIONMAP - map TRANSACTION table columns to Transaction class
        self.POSTGRESCONFIG, self.USERMAP, self.TRANSACTIONMAP = self._readDatabaseConfig()

        # *snip*

        # Create a database connection with data from .ini file
        self.conn, self.cur = self._connectDB(self.POSTGRESCONFIG)
        self._decToFloat()
        self.bankMap: dict[str, int] = self._parseBankID()

    def _readDatabaseConfig(self) -> tuple[dict, dict, dict]:
        """Read .ini file and load DB config parameters and DB column name mappings

        Returns:
            tuple[dict, dict, dict]: key-value maps for DB config and DB column mappings
        """

        config = configparser.RawConfigParser()
        config.optionxform = lambda option: option # preserve case-sensitivity of keys/values
        
        configFilePath = pathlib.Path(__file__).parents[1].joinpath('config','db.in')
        config.read(configFilePath)

        postgresConfig: dict = {}
        transactionMap: dict = {}
        userMap: dict = {}
        # Read separate config blocks and extract key-value pairs
        try:
            for key in config['postgresql']:
                postgresConfig[key] = config['postgresql'][key]

            for key in config['transactions']:
                transactionMap[key] = config['transactions'][key]

            for key in config['users']:
                userMap[key] = config['users'][key]

            #for key in config['banks']:
            #    userMap[key] = config['banks'][key]
        except:
            print('Corrupted db.ini file')
            raise

        return postgresConfig, userMap, transactionMap

What I'd like to test is that:

1. the .ini file is read while retaining case-sensitivity
2. that dicts created from 'postgresql', 'transactions', 'users' sections contain some key-value pairs and are not empty

[pearl cliff]

if you cannot refactor this code to avoid putting i/o in the init method, then you need to patch this method to return a mock database object. but this is no longer really a "unit" test and more of an "end-to-end" or "integration" test... depending on your perspective, at least

it also looks like you need to patch out the config parser reading data from a config file

another alternative is to not try any mocking, and actually use a config file and database for the test suite

in a lot of my work nowadays i don't bother mocking, i just expect each developer to have a local database setup that they can use to run the tests

[deep dew]

Do some further explaining for a complete beginner. Why would I need a db mock for testing just _readDatabaseConfig method? I understand it would be necessary for further tests regarding e.g. _connectDB

[ember maple]

what i lke to do these days is to have named constuctors that pass the done objects to init

that way you dont have to mock around , you can jsut pass different objects in if you want / need

[deep dew]

I could refactor the code to run these methods outside of __init__, but considering the whole class is centered on DB connection, it felt like a good idea to establish db connection in __init__

[pearl cliff]

you don't need a mock then, but you still need to patch out the method, even if it just returns None, or if you just don't use the mock

[ember maple]

thats a common key mistake ^^

[pearl cliff]

why not just pass in the db connection as an instance attribute in __init__?

[ember maple]

creating resoruces in a init is almost always the wrong thing to do

[deep dew]

Please, explain the wrong logic here, I'm happy to do it the correct way

[ember maple]

a db conneection is a resource, so the way to make them and controll them is something like context managers which clean up after themselfes

[deep dew]

to my feeling, then i have to create a new function somewhere else, which would create my connection and cursor object

[pearl cliff]

sure, i think that's a good approach

then you can test each function separately

[deep dew]

Hm, I see, that's a fair point.

[ember maple]

if you make a contextmanager classmethod that makes those, then yields the object and finally closes the connection, you have a maned constuctor as context manager that handles your config and resources

[deep dew]

Could you please write me a short syntax structure of how it would look like? I'm getting slightly confused.

[deep dew]

My context manager handling DB connection would be located in my TransactionRepo ?

Second way how i interpret what you wrote is that i have a contextmanager yielding connection/cursor object to my TransactionRepo __init__

[ember maple]

# untested
class myRepo:
  @contextmanager
  @classmethod
  def with_config(cls, ...):
      config = mymagic()
      with db_connection(config.dburi) as conn:
         yield cls(conn)

i always forget which order the decorators have to have

[deep dew]

Ok, please correct me if i understand the mechanism wrong. My Repo class will run a classmethod with_config yielding a connection/cursors which can be used by all instances of Repo class. When all Repo objects get deleted, it will take care of closing the DB connection. It also allows me to move my config parsing outside of __init__, so i can unittest it

[ember maple]

no, the classmethod will return you a conextmanager for a single repo, which will clean up the connection after you are done with use

[deep dew]

why for only single repo when it's defined as a class method? Is it due to @contextmanager decorator? It's a completely new decorator for me, so i need to read about it first, but I'd like to get a general understanding.

Gosh, doing anything correctly is such a rabbit hole

[ember maple]

@deep dew the context managr is what youd use to make your repo objects and use themm the responsibility of the context manager is to prepar e things, and create/dispose resources

[deep dew]

Yes. I understand your whole point about using a class contextmanager for handling my resources (in this case DB connection objects). This makes a perfect sense. What I'm stuck on is technical implementation of it (syntax).

  def with_config(cls, ...):
      config = mymagic() <------------

mymagic() should be defined as another class method, correct?

[ember maple]

i putthat in as a placeholder, make whatever code you need for config loading there

[deep dew]

All right, I don't want to bother you anymore, thanks a lot for you help, this clarified a lot

[ember maple]

butt then in the end you use it like this

with myrepo.with_config() as repo:
    repo.somework()

[deep dew]

I see, thanks!

[verbal steeple]

Hullo, I'm writing a unittest with pytest and I wish to verify if a function (which has 3 params) was called with two parameters and the last one can be ignored.
Can anyone point me out the best way to do this ?? I tried Any but clearly it didn't workout

Expected: mail_commission('[e-mail]', 'Test Files', typing.Any)
E           Actual: mail_commission('[e-mail]', 'Test Files', <memory at 0x7f470c619940>)

[pearl cliff]

typing.Any is for type hints, not for unittest assertions. i assume you are using the assert_called_with method of unittest.mock; unfortunately there's no "wildcard" sentinel for that method (although i agree it would be useful). you will have to manually inspect the arguments used to make the call and check the first 2 args. although i will offer that perhaps this is not a good test; mocking like this is usually a last resort when all other testing techniques cannot work

https://docs.python.org/3/library/unittest.mock.html#unittest.mock.Mock.call_args

call_args = mock_mail_commission.call_args
self.assertEqual(len(call_args), 3)
self.assertEqual(call_args[0], '[e-mail]')
self.assertEqual(call_args[1], 'Test Files')

like that, i guess

the only time i'd say that this is a good idea, is when you need to test 3rd party code that performs i/o for which you cannot set up a proper fixture, or some expensive operation that you do not want to run in each test

[verbal steeple]

Kinda, the third field is tempfile that is generated dynamically, so I can't really predict the value there.

Thanks for the snippet*

[pearl cliff]

what does this function do, that you are mocking and checking the args of?

[verbal steeple]

Just, dynamically generating a PDF with reportlab to send it out as an email attachment

[molten plinth]

Any suggestions on how to make pytest ignore warnings that are in packages coming from venv?

[pearl cliff]

i see, fair enough. that's a reasonable thing to mock

[hexed cloak]

where else would packages come from?

@molten plinth It's best set filterwarnings = error. Initially ignore just the warnings you have then gradually whittle it down to none

eg:

filterwarnings = [
    'error',
    """ignore:Some regexy .* bad happened:DeprecationWarning""",
]

[ember maple]

i strongly recommend against warnings as errors in testrunning, unfortunately we don't have a error the testrun if there are warnings/specific warnings flag

[molten plinth]

==================================== warnings summary ====================================
venv/lib/python3.9/site-packages/django/apps/registry.py:91
  /Users/jvacek/Git/webdatamap/venv/lib/python3.9/site-packages/django/apps/registry.py:91: RemovedInDjango41Warning: 'versionfield' defines default_app_config = 'versionfield.apps.VersionFieldConfig'. Django now detects this configuration automatically. You can remove default_app_config.
    app_config = AppConfig.create(entry)

for example this is a warning coming from a dependency I have in the project

all my dependencies like this are in my venv, and they're basically outside of my control as I want to keep using the published ones and not my wackyhacky

[hexed cloak]

You will have to add that warning to the filterwarnings setting

[molten plinth]

Can I intelligently filter based on the filepath?

[hexed cloak]

I'd recommend contributing a fix to the dependency and then commenting that PR URL above the filterwarnings config line

No the warning is coming from django.apps.registry

[molten plinth]

That's a given!

just would like to focus on a different part of my codebase right now

[hexed cloak]

Right so ignore the warning with filterwarnings and comment it with a github ticket opened to report the problem

[molten plinth]

Can I ignore that specifically? that takes care of the dependencies in django too

this package's github issues are closed 😐

only way to communicate to them is via a PR, or probably sending the author an email

[hexed cloak]

URL?

[molten plinth]

https://pypi.org/project/django-versionfield/ // https://github.com/mcldev/django-versionfield

hasn't been touched since 2020

I'm planning to fork it and actually make some PRs against this one, as it's the one that's actually published. There are many other forks

but yeah...

[hexed cloak]

Ah the fork doesn't have issues enabled

2020 is quite recent imho

@molten plinth make a PR to avoid setting the attribute on new django versions

[molten plinth]

will do!

Gotta first figure out how to if around django version

[static grove]

using Pytest

@pytest.mark.parametrize(
    ('array', 'output'), (
        ([3, 4, 2], [[1, 2], [1, 3], [2, 2]])
    )
)
def test(array: list[int], output: list[list[int]]) -> None:
    assert get_above_avg_subarrays(array) == output

gives the following error:

it's probably the list of lists, but how to fix it? the function to test returns a list of lists

[pallid dirge]

lol

[empty dust]

I think it should look more like this

@pytest.mark.parametrize("array, output", [
                        ([3, 4, 2], [[1, 2], [1, 3], [2, 2]]),
                        ([7, 6, 5], [[5, 4], [3, 8], [1, 7]])
                        ])
def test(array, output):

[delicate basin]

I recently watched this recorded presentation on test driven development: https://www.youtube.com/watch?v=58jGpV2Cg50

I have started trying to apply this to my own Python projects, but there's one stumbling block that I'm running into at every turn: how do I decide which part of the application I should start with?

[hexed cloak]

Start with the main function

And see how much coverage you can gain

[delicate basin]

You should in theory have 100% coverage if you do it right.

[mellow geyser]

Reaching 100% coverage often implies doing a lot of tests for the sake of doing them. So in practice you wouldn't really aim for 100% as it would just not be worth it nor be meaningful

[hexed cloak]

I think you should get 100% coverage - and use # pragma: no cov on lines that don't get hit

[delicate basin]

If you're doing test driven development you won't write code that isn't tested, by definition.

class Result:

    def __init__(self, value, steps):
        self._value = value
        self._tested = steps

    def __repr__(self):
        return f"<Result object: {self._value}"

    @property
    def tested(self):
        return self._tested

    def __eq__(self, other):
        return self._value == other

def binary_search(elements, search_term):

    tested = []

    min_bound = 0
    max_bound = len(elements)

    last_mid = -1
    mid = (max_bound + 1) // 2 - 1
        
    while mid != last_mid:
        element = elements[mid]
        tested.append(element)

        last_mid = mid

        if element < search_term:
            min_bound = mid
            mid += (max_bound - mid) // 2
        elif element > search_term:
            max_bound = mid
            mid -= (mid - min_bound + 1) // 2
        else:
            return Result(mid, tested)

    return Result(-1, tested)

I came up with this, this morning. 🤔
This isn't exactly the way I'd normally write binary search lol.

But it was the way I came up with that I could use test driven development for.

I have 15 tests, and they primarily prove that it actually does a binary search and not a linear search for example. And I think I may have an off-by-one error still, but I haven't quite figured out how to prove it.

[pearl cliff]

this is probably a silly question, but does unittest support the -f option without using unittest discover? i swear it used to, but now i get the error Unrecognized command line option: 'f'

$ /opt/venv/bin/python3 -m unittest -f tests.integration.test_guides.TestGuideSubscriptions

Usage: python3 -m unittest [OPTIONS]

Options:

  --help                           show this help information

/opt/venv/lib/python3.8/site-packages/tornado/log.py options:

  --log-file-max-size              max size of log files before rollover
                                   (default 100000000)
  --log-file-num-backups           number of log files to keep (default 10)
  --log-file-prefix=PATH           Path prefix for log files. Note that if you
                                   are running multiple tornado processes,
                                   log_file_prefix must be different for each
                                   of them (e.g. include the port number)
  --log-rotate-interval            The interval value of timed rotating
                                   (default 1)
  --log-rotate-mode                The mode of rotating files(time or size)
                                   (default size)
  --log-rotate-when                specify the type of TimedRotatingFileHandler
                                   interval other options:('S', 'M', 'H', 'D',
                                   'W0'-'W6') (default midnight)
  --log-to-stderr                  Send log output to stderr (colorized if
                                   possible). By default use stderr if
                                   --log_file_prefix is not set and no other
                                   logging is configured.
  --logging=debug|info|warning|error|none
                                   Set the Python log level. If 'none', tornado
                                   won't touch the logging configuration.
                                   (default info)

🤔

does tornado somehow take over the unittest command in a way that disables the usual cli options?

[hexed cloak]

I hope not

[delicate basin]

How does test driven development handle algorithms where there are many possible solutions and the only difference is performance?

[pearl cliff]

implement it the naive way in the test suite, and assert that the result is identical to your complicated implementation

[delicate basin]

You misunderstood. One of the rules of TDD is that you don't write production code unless it's to make a failing test pass.

If I have am building a test suite for say, merge sort, that's a complicated algorithm. And if I obey said rule, it's impossible to implement, is it not?

[mellow geyser]

Sounds like you would care about the tests so that you can refactor with confidence and ensure you won't break anything as you optimize your code.

But the performance itself would most likely be handled by another suite of tests which highlight the specific characteristics you want to measure

[delicate basin]

Awesome, thanks.

So my unit tests would check that the behavior is correct, and another test would check that my performance requirements are within the requirements?

[mellow geyser]

That's the idea. But note that performance testing is a beast in itself. There are considerations to be aware of in terms of environment, technologies and ensuring you are measuring the right thing. In addition, some of the metrics may be very specific, especially when it comes to system level metrics

You also have to be mindful about what you are trying to test the performance of. That could go into micro-benchmark or a whole application performance testing

You also have to be mindful about the intent. Is it to ensure you meet a specific baseline in a specific environment and conditions or is it about helping you sizing your system and finding the next bottleneck.

[rotund palm]

Hey If I wanna learn testing how do I start? I know basics of python

[royal orbit]

https://realpython.com/python-testing/ is a fantastic introduction for beginning testers. (just swap out unittest for pytest, and travis for github actions, when you get to those sections)

[rotund palm]

Thanks

[versed aspen]

Huh
I never gave pytest the proper attention it requires, mostly testing things at the module level with the if __name__ == '__main__': idiom

[maiden pawn]

Huh... Using name for testing,? That is creative way to approach it

Btw people who dont do pytest, usually use unittest... Or in more fun cases doctest

[inner sedge]

isnt pytest built on top of unittest

[maiden pawn]

Unlikely
May be u confused pytest having compatibility with unittest. All unittest tests are valid for pytest.

[spark sable]

anyone can help me make loop?

@ me if you are free for 5min

[royal orbit]

Nope. Pytest can run unittest tests, but it's not implemented using unittest.
(IIRC nose was, years ago)

[steep yacht]

what's the benefit of doing fuzzing like https://github.com/Rog3rSm1th/Frelatage using a genetic algorithm approach over something like hypothesis?

if a mutation survives (thus pointing towards a bug), it would only generate a zoo of examples hitting the same spot, wouldn't it?

[pearl cliff]

at some point you have to decide if you need/want to test your tests. of course, if you do have known "exact" test cases, then you can use those too

[pearl cliff]

https://sqa.stackexchange.com/q/18751

you can maybe think of fuzzing as a specific case of property based testing: you are testing the property that the program does not crash or behave otherwise "badly" in a macro sense upon unexpected inputs

the two techniques have different goals, but in practice look very similar and have a lot of overlap

[steep yacht]

tbh i don't think i understand the value of fuzz testing.. it seems to me property based testing already covers the whole space

but if it's mostly about checking for crashes, maybe i'm looking at this the wrong way

[pearl cliff]

yeah, frankly i think it's a case of very similar techniques being invented independently by two different groups

property-based testing was popularized by the haskell/FP community, fuzzing comes from the security community

[steep yacht]

i see, that makes sense

[pearl cliff]

but one wouldn't employ fuzzing in a regular test suite run

property-based testing frameworks usually employ "shrinkage" which as far as i know fuzzers do not

fuzzing is an integration-level test of very broad "macro" properties of the application

i found a few interesting blog posts on this distinction too

https://blog.nelhage.com/post/property-testing-is-fuzzing/
https://ruanmartinelli.com/posts/notes-property-based-testing

[steep yacht]

i only recently learned about mutation testing, so i was surprised to learn the two are very different, even though their techniques are similar

[pearl cliff]

you could flip this around and argue that property-based testing is just fuzzing applied to smaller units than the entire application

[steep yacht]

https://talkpython.fm/episodes/show/63/validating-python-tests-with-mutation-testing <- that's how i learned about it

[pearl cliff]

there was a user in this server who wrote a mutation testing framework for python and was a big advocate of the technique

i don't remember their username

i remember not quite understanding how the technique could work well, and i shelved it in my brain

now that my job is slowing down and i have more time to think, i might take another look

[steep yacht]

it's basically a test for your tests, not for your code

[pearl cliff]

i have seen people advocating for using it on your code too

but that's a really interesting idea

the more i "get into" testing, the more i find myself writing lots of testing utilities, fixtures, and mini-frameworks

all of which in principle should be tested. although in practice usually a bug in one of those things means that the tests fail anyway... but not always

[steep yacht]

yeah, me too 😄

you really should check the podcast, it was quite fascinating to me

[pearl cliff]

sure, i will

[proven elm]

Meta testing lol

[steep yacht]

so to say, it's a pretty cool idea imo. checking if the test cases would hold if bugs were introduced into the code

[proven elm]

Then make test for the meta test lmao

[steep yacht]

basically fighting the idea that you need 100% coverage, leading to overly simple test cases only to cover all branches but without thinking about the logic

[proven elm]

Ok I see the role fuzzing and property based testing plays into this

[royal orbit]

Hypothesis author here! The boundaries between fuzzing and PBT are, heh, pretty fuzzy - it's usually more a matter of workflow than anything fundamental.

• PBT is designed for use by devs, looks like unit tests, happens in the dev cycle, includes shrinking, rarely uses invalid inputs, checks specific behaviours.
• Fuzzing is usually done after development, often by different people, generates anything representable, leaves you with many redundant crashes to understand. (though in recent years both have been converging)

But to answer your actual question, the advantage of a genetic algorithm (ie coverage guided fuzzing) is that it can quickly and reliably "evolve" an input to meet several independent conditions which trigger a bug, and are exponentially unlikely to happen by random chance. https://www.fuzzingbook.org/html/GreyboxFuzzer.html is a fantastic resource if you want to read more.

[steep yacht]

oh, hello 🙂 thanks for the answer!

[royal orbit]

But you don't actually have to choose, you can write Hypothesis tests and then drive them with the default random generation, or with a fuzzer! https://hypothesis.readthedocs.io/en/latest/details.html#use-with-external-fuzzers

(we actually tried doing greybox fuzzing by default, but it takes about a minute per test before that's better than random-with-good-heuristics, which is too slow for unit test like workloads. So separate interfaces it is!)

[steep yacht]

that sounds like you could use hypothesis to cover all bases and then use fuzzing for debugging, kind of?

[royal orbit]

I tend to think of it as "use Hypothesis to write tests, use fuzzers to search for security issues from misinterpreted/invalid inputs"

[steep yacht]

okay

[royal orbit]

With a side of "use coverage data to find extra bugs using your property-based tests", eg with https://hypofuzz.com/example-dashboard/

[steep yacht]

i was wondering, since i mentioned it earlier, do you have experience with mutation testing against hypothesis? do both play well together?

something like https://cosmic-ray.readthedocs.io/en/latest/ maybe

[royal orbit]

I've tried it a few times; the main issue is just performance - so turn down the max_examples setting to maybe 20.
Otherwise just expect immature tooling, eg weird perf issues, broken windows support, etc. IIRC most python mutation testing tools focus on rigour rather than speed, which is not what I'd want.

(eg Google's internal system checks only a few heuristically chosen mutants per line, and can run only on the diff of a pr)

[steep yacht]

then i might run it when i do mass testing anyway

wouldn't matter whether it takes 3 or 5 hours ^^

[fiery arrow]

In pytest, is there an equivalent of --durations but for fixtures?

I want to know which fixture is taking so much time

[jolly tangle]

I am using python flask.

What is the user object?

https://speakerdeck.com/patkennedy79/testing-flask-applications-with-pytest?slide=10

Is the user object equal to the code below in sqlalchemy?

def repr(self):
return '<User %r>' % self.username

Here is the video from the slides
https://www.youtube.com/watch?v=OcD52lXq0e8

[pearl cliff]

@jolly tangle flask itself doesn't have a "user" object, so this must be something defined elsewhere in the application

[jolly tangle]

What is a user object?

[full meadow]

using pytest if there’s a fail how can i have it do something

for example if it fails i want to write the number to a file

[jolly tangle]

I am sorry I am brand new to unit testing

so it is just a way to test if pytest fails

[jolly tangle]

is this just used in a database?

[jolly tangle]

Also what is the difference between slide 10 and 11

?

[maiden pawn]

use

try:
  # code
except TypeOfError as error:
  # writing to file
  raise Exception("Writing error to file ;)") from error

[real moat]

from idk import something // --return ""

def closer():
    return something()

How do I mock something? I want to check if something is called when I call closer

[empty dust]

https://docs.python.org/3/library/unittest.mock.html#unittest.mock.patch

[pearl cliff]

a.py:

def something(): ...

b.py:

from a import something
def closer():
    return something()

test_b.py:

from unittest import mock
from b import closer
class TestB:
    def test_b(self):
        with mock.patch('b.something') as mock_something:
            closer()
            mock_something.assert_called_once_with(())

something like that

required viewing/listening imo:
https://www.youtube.com/watch?v=ww1UsGZV8fQ
https://www.youtube.com/watch?v=Ldlz4V-UCFw

and yes, read the actual python docs

once you've seen those talks, the docs should be easy to understand

[tall mulch]

Hey everyone I am having trouble executing a multiline doctest. I am using VSCode and WSL started a new WSL window and I am running the command through that terminal window.

def helloDocTest(n):
    """Return the factorial of n, an exact integer >= 0.
    This is a body of text explaining the doc test:
    >>> a = 10
    ... b = 10
    ... c = a + b
    10
    """
    return ""


if __name__ == "__main__":
    import doctest
    doctest.testmod()

This is the error

File "docttest.py", line 4, in __main__.helloDocTest
Failed example:
    a = 10
    b = 10
    c = a + b
Exception raised:
    Traceback (most recent call last):
      File "/usr/lib/python3.8/doctest.py", line 1336, in __run
        exec(compile(example.source, filename, "single",
      File "<doctest __main__.helloDocTest[0]>", line 1
        a = 10
             ^
    SyntaxError: multiple statements found while compiling a single statement
1 items had no tests:
    __main__
**********************************************************************
1 items had failures:
   1 of   1 in __main__.helloDocTest
1 tests in 2 items.
0 passed and 1 failed.
***Test Failed*** 1 failures.

I have searched google no avail looking for some help

[pearl cliff]

each = assignment is a separate statement, it looks like doctest only supports 1 statement per >>> line

i dont use doctest so im not sure how to handle this otherwise, but that's what the error seems to be saying

if you write it like this, will it print each line?

>>> a = 10
>>> b = 10
>>> c = a + b

i see in #editors-ides that you asked about if and for; those work with ... because you are continuing the same statement

[tall mulch]

Ah I see what you mean. yeah that doesn't work. I guess these were really only supposed to use the documented function not to setup that function for use.

[pearl cliff]

you can also do

>>> a = 10 ; b = 10 ; c = a + b

or maybe even this will work:

>>> a = 10 ; \
... b = 10 ; \
... c = a + b

[tall mulch]

Thanks Salt rock Lamp! that works! the first one

man haha those semi colon separators to the rescue haha

[pearl cliff]

out of curiosity what happens if you do this?

>>> a = 10
>>> b = 10
>>> c = a + b

[tall mulch]

it treats each one as a statement and a test

Trying:
    a = 10
Expecting nothing
ok
Trying:
    b = 10
Expecting nothing
ok
Trying:
    c = a + b
Expecting nothing
ok

rather than the whole setup being a single test

[pearl cliff]

ah, i see

[heady axle]

mdg

[frigid basalt]

Does anyone have any advice for timing-based tests?

I have a ratelimiter I know nothing about, all I know is that in one way or another it will eventually block (whether that's sleeping, waiting for an event or queue, making a network request that doesn't return until complete, etc.) and I need to detect whether it did block.

Right now I am essentially running each test twice, timing it the first time, and then using this timer times 3 to then see if the next time takes longer than that.

If it does, and it's not supposed to, I fail the test. Issue is, this still gives spontaneous failures.

If I increase the time waiting, I will be slowing down tests a lot, if I don't I continue to get spontaneous failures. Should I continue changing that number until it works or I am perhaps going about this the wrong way?

[hexed cloak]

https://anyio.readthedocs.io/en/stable/api.html#anyio.wait_all_tasks_blocked

ratelimiters should probably implement statistics

So you run your thing that swamps the rate limiter all waiting on some_event.wait() and start an additional task that waits for wait_all_tasks_blocked() - assert on your ratelimiter statistics and finally call some_event.set()

[frigid basalt]

Ah, good idea I think I can do something with that, thanks.

[hexed cloak]

https://github.com/agronholm/anyio/blob/d88cddce985e8d8a882e5f93a42153346ca67f47/src/anyio/_backends/_asyncio.py#L1865 it's quite simple

[bitter wadi]

src/anyio/_backends/_asyncio.py line 1865

if task._fut_waiter is None or task._fut_waiter.done():  # type: ignore[attr-defined]```

[hexed cloak]

Trio's is nicer of course - it puts a task in a collection and the scheduler wakes them before it calls select/proact

[frigid basalt]

Yeah, I see.

[deep dew]

What is preferred library to use for mocking with pytest? unittest.mock or pytest-mock?

Which one will be better for learning fundamentals?

[hexed cloak]

I prefer with mock.patch( this way you can razer focus your patches to the specific location

pytest-mock patches apply for the whole duration of the test

[deep dew]

I see, thanks. One more question - Does it make sense to mock an object inside of a fixture and pass that fixture to multiple tests?

E.g. mock a class with it's attributes (and return values of methods) and pass it as a fixture later on to different tests checking correctness of that classes methods

[pearl cliff]

that seems useful to me, you just need to be careful that you set the fixture scope correctly

that said, i'm not sure if there's much benefit compared to just creating the patch/mock wherever you specifically need it. maybe a bit less boilerplate. but you can also do the same by writing a context manager that returns an ExitStack with all your mocks pre-defined

[deep dew]

You've lost me on the ExitStack sentence. I just read a write-up about it, but still not sure how it would work in this scenerio. Do you have some example by any chance?

[pearl cliff]

from contextlib import ExitStack, contextmanager
from unittest import mock

@contextmanager
def standard_patches():
    mocks = {}
    with ExitStack() as patches:
        mocks['foo.bar'] = patches.enter_context(mock.patch('foo.bar'))
        mocks['quux.abc'] = patches.enter_context(mock.patch('quux.abc'))
        yield mocks

i've done something like this before

this way you can just do

def test_something():
    with standard_patches() as mocks:
        ...

[deep dew]

Oh, I see, that makes a lot of sense.

Thanks!

[hexed cloak]

Just stack all the withs in one big with statement and let black figure out how to format it

[pearl cliff]

only in 3.9 with parenthesized withs 😛

and that isn't easily reusable of course

[royal orbit]

And, alas, iirc black doesn't know how to format those yet 🥲

[dusky dew]

black now does!!!!

https://black.vercel.app/?version=stable&state=_Td6WFoAAATm1rRGAgAhARYAAAB0L-Wj4ABqAE1dAD2IimZxl1N_WmufDs0Lm7p9NofBzVAOj1xmNYaakgVNT5o5tRL_ur6GNi-5baxiK6bZQBI2PcSwVwc3vkzzJxRLHcSey2MPpommqGpjAAAAAFtGoSen79tQAAFpaxXd2U4ftvN9AQAAAAAEWVo=

you can thank @tulip kestrel for whatever hacks they wrote to make this work :)

lol apparently the support got added in 21.11b0

[royal orbit]

Oh, sweet! https://github.com/psf/black/issues/664 is still open though, and black preserves but doesn't add parenthised with 😭

[hexed cloak]

But you can still do one big with statement in very old pythons

[pearl cliff]

but do you want to

i think this is how black formats them:

with foo(
    a, b, c
), bar(
    x, y, z
):
    ...

i guess that isnt bad

[hexed cloak]

Yeah I like it like that tbh

Parenthesised with is obviously nicer

[deep dew]

Hey all, can someone please find a flaw in my thinking? I'm trying to mock configparser.ConfigParser.__getitem__, to feed different config dictionaries to test Exception handling in establishConnection class method. Unfortunately, when looping through config['postgresql'], instead of calling MagicMock object, ConfigParser.__getitem__ is called

# test_TransactionRepo.py
#! python3

PROJECT_ROOT = pathlib.Path(__file__).parents[1].resolve()
sys.path.append(str(PROJECT_ROOT))

import pytest, sys, pathlib
from unittest.mock import patch
import FinanceApp.FinanceApp
from FinanceApp.FinanceApp import TransactionRepo

class TestTransactionRepoConstructor():

    postgresConfig = {
        'host': 'localhost',
        'database': 'financeapp',
        'user': 'zaizu',
        'password': 'admin',
        'port': '5433'
    }

    @patch('FinanceApp.FinanceApp.configparser.ConfigParser.__getitem__', side_effect=postgresConfig)
    def test_mock(self, mockConfig):
        
        with pytest.raises(Exception):
            with TransactionRepo.establishConnection() as repo:
                pass

# FinanceApp.py
class TransactionRepo:
    @classmethod
    @contextmanager
    def establishConnection(cls) -> Generator[TransactionRepo, None, None]:
        """Read .ini file and load database config parameters and database column name mappings.
        Create ContextManager handling connection with the database.

        Raises:
            Exception: raised in case of unexpected config file formatting

        Yields:
            Generator[TransactionRepo, None, None]: ????
        """

        config = configparser.ConfigParser()
        # Preserve case-sensitivity of keys/values
        config.optionxform = lambda option: option
        # Ensure that DECIMAL postgres data type is cast by default to Float and not Decimal
        DEC2FLOAT = psycopg2.extensions.new_type(
            psycopg2.extensions.DECIMAL.values,
            'DEC2FLOAT',
            lambda value, curs: float(value) if value is not None else None)
        psycopg2.extensions.register_type(DEC2FLOAT)

        # Read config file
        configFilePath = pathlib.Path(__file__).parents[1].joinpath('config','postgresConfig.ini')
        config.read(configFilePath)

        # Read separate config sections and extract key-value pairs
        postgresConfig = {}
        try:
            for key in config['postgresql']:
                postgresConfig[key] = config['postgresql'][key]

            # Check if all necessary keywords for DB connection are present
            if not {'host', 'database', 'user', 'password', 'port'} <= set(postgresConfig):
                raise Exception

        except:
            raise Exception('Unexpected postgresConfig.ini formatting') 

        a = psycopg2.connect(**postgresConfig)
        try:
            with a as connection:
                with connection.cursor() as cursor:
                    yield cls(connection, cursor)
        finally:
            connection.close()

[hexed cloak]

Use a tmp_path and pass a custom config file

[sharp magnet]

where take project

how write unit test?

[deep dew]

Yes, this would be a better way, but could you tell me why my Mock object is not passed into establishConnection() and does not replace __getitem__ method?

[pearl cliff]

what if you use return_value instead of side_effect? that said, there seem to be some issues around mocking __getitem__ https://stackoverflow.com/q/18905026/2954547, so maybe you would need to patch ConfigParser itself. instead of replacing it with a mock, you can replace it with a dict that contains some static config

but i am also +1 on using an actual real config file for the tests if that's also an option

[unborn bane]

hi, im having problem with the testing discovery with pytest

idk if someone hes come with the same probkem

[pearl cliff]

show your file layout and describe how you are running pytest

[ebon raven]

why is unit testing important

[ember maple]

@ebon raven when done well it can help to sample ad demonstrate errors in programs and understanding, also it is a help to demonstrate that the building blocks of a larger system operate as expected

in metaphors - unit tests do have the power to prevent a software engineer from accidentally turning a solid foundation into quicksand, unfortunately that power is utilized far less than i would hope

[ebon raven]

thanks

[mellow geyser]

In addition to what was said above, it can help document the behavior of the code and enable to refactor with confidence.
Often times, you may inherit a large and old code base that you need to update. Without tests, you wouldn't really have any basis for understanding what is expected and what is not. And having a set of tests enables you to update that code base without violating any of the expectations and ensuring you won't break things

[ebon raven]

so it is like a guide line to walk through a code safely and execute safely.

[ember maple]

@ebon raven its also importan tot have the tests walk trough paths that will fail (accidentially having something succedd that ought not to can be a dire situation if it means commiting incorrect data afterwards

[ebon raven]

i learned coding for 1 year now Python JavaScript and C++(begginer in this one) but i never knew about algorithms or data structures or SOLID concept or testing iam totally ignorant

[dawn frost]

dw, lots of people don't test

ironic since C is the language most in need of testing

[ember maple]

c is a language in need of annihilation before more collateral's

unfortunately currently i don't think Linus will fancy something else just yet

[deep dew]

I'm at my wit's end when it comes to patching. The way this method works is completely elusive to me. It seemed my previous problem was with how i refered in patch decorator to patched function, Somehow it got to work with __getitem__, but now I'm not able to patch ConfigParser even tho I'm repeating exactly same steps.

wrongKeys: dict = {'postgresql': {'wrongKey1': 'wrongValue1',
                                        'wrongKey2': 'wrongValue2',
                                        'wrongKey3': 'wrongValue3',
                                        'wrongKey4': 'wrongValue4',
                                        'wrongKey5': 'wrongValue5'}
}
wrongConfig = configparser.ConfigParser()
wrongConfig.read_dict(wrongKeys)

@patch('FinanceApp.FinanceApp.configparser.ConfigParser', return_value=wrongConfig)
def test_wrongConfigs(config):
    with pytest.raises(Exception):
        with TransactionRepo.establishConnection() as repo:
            pass

This is my testcase - when debugging, it successfully mocks ConfigParser object in my source file, but when called on config, it creates ConfigParser object:

# configparser.ConfigParser
# >>> <MagicMock name='ConfigParser' id='140022834502368'>
config = configparser.ConfigParser()
# configparser.ConfigParser
# >>> <MagicMock name='ConfigParser' id='140022834502368'>
# config
# >>> <configparser.ConfigParser object at 0x7f599b4fb430>```

Does anyone has any explanation to this? My head is bleeding from banging against the wall

[lime temple]

config

[frigid basalt]

Do you create the config before patching it? I think you need to patch the config variable itself?

[pearl cliff]

your module is actually called FinanceApp.FinanceApp?

what is your file layout?

you have files FinanceApp/FinanceApp.py? and FinanceApp/__init__.py?

i have a feeling you might be misunderstanding how python modules/packages work

and yes, bluenix makes a good point @deep dew . if the config = configparser.ConfigParser() happens at the module top-level, this code has already been executed by the time the patch is put into place

you can either patch the config variable directly (instead of patching config.ConfigParser), or you can move config = configparser.ConfigParser() out from the top level of the module

[tidal rock]

ive been having trouble figuring out the best way to write some tests for a class. it's convertible to/from bytes. for one set of tests i want to:

1. create randomized instances of it.
2. apply some random changes to them.
3. apply the equivalent set of changes to a bytearray based on the same random data used to create the instance.
4. convert both to bytes and check if they're equal (i.e. the changes have the intended outcome).

are there some standard practices for tests like these? would this be considered fuzzing? and specifically covering points 2 and 3: is there some ideal method for generating test inputs that are supposed to be used to transform or operate on other inputs?

[deep dew]

My directory structure is following:

FinanceApp
├── config
│   ├── postgresConfig.ini
│   └── tableMaps.ini
├── FinanceApp
│   ├── DB_setup.sql
│   ├── FinanceApp.py
│   ├── __init__.py
│   └── poop.py
├── README.md
└── tests
    ├── __init__.py
    ├── pytest.ini
    ├── tests_context.py
    └── test_TransactionRepo.py

[pearl cliff]

ok, so you do have FinanceApp/FinanceApp.py, that i at least needed to confirm

[deep dew]

I was able to run another test patching pathlib.Path and it works fine, therefore I assume this is not an issue

[pearl cliff]

(might i ask... why? might want to just put this in FinanceApp/__init__.py or similar)

[pearl cliff]

do you understand the last point, what bluenix said? about the config object already existing before patching

[deep dew]

Give me a second, I'll respond to all your messages, just need few minutes

[pearl cliff]

you are describing "property-based testing", which is practically similar to fuzzing but has different goals. there is the python library Hypothesis for this purpose, which generates random inputs for you

!pypi hypothesis

[bitter wadi]

hypothesis v6.40.0

A library for property-based testing

[pearl cliff]

applying random transformations to the inputs is a slightly different task, but same objectives and similar techniques apply

[frigid basalt]

Won't you just reimplement the class at that point?

[deep dew]

My config variable is defined inside of a class method, which is wrapped by my test case function ( I believe?) To my understanding, what happens, is, I'm replacing configparser.ConfigParser object with Mock object for this class method, and there I create variable config which references to this object. The code is given here:
https://discordapp.com/channels/267624335836053506/463035728335732738/957577984187981826

[hexed cloak]

You should pass the config file path in as an argument to your classmethod

[deep dew]

This is what I have done later on: ```py
class TestEstablishConnection:
@pytest.fixture
def wrongConfigs(self, tmp_path: pathlib.Path) -> tuple[pathlib.Path, pathlib.Path]:

    wrongConfigPath = tmp_path.joinpath('wrongConfig.ini')
    wrongKeys: str= '''[postgresql]
                        wrongKey1=wrongValue1
                        wrongKey2=wrongValue2
                        wrongKey3=wrongValue3
                        wrongKey4=wrongValue4
                        wrongKey5=wrongValue5'''
    wrongConfig = configparser.ConfigParser()
    wrongConfig.read_string(wrongKeys)
    with open(wrongConfigPath, 'w') as configFile:
        wrongConfig.write(configFile)

    emptyConfigPath = tmp_path.joinpath('emptyConfig.ini')
    emptyKeys: str = ''
    emptyConfig = configparser.ConfigParser()
    emptyConfig.read_string(emptyKeys)
    with open(emptyConfigPath, 'w') as configFile:
        emptyConfig.write(configFile)

    return wrongConfigPath, emptyConfigPath

def test_wrongConfigs(self, wrongConfigs):
    with patch('FinanceApp.FinanceApp.pathlib.Path.joinpath', side_effect=wrongConfigs):
        with pytest.raises(Exception):
            with TransactionRepo.establishConnection() as repo:
                pass
        with pytest.raises(Exception):
            with TransactionRepo.establishConnection() as repo:
                pass

But I'd like to understand whats the problem with the previous approach

[hexed cloak]

You should actually assert which Exception you get

And you shouldn't use a fixture when it's only used in one test

[deep dew]

I see, will not do it again

[deep dew]

For now I use Exception as I had no clue what Exception Type should I use. I handle these checks like that:

        if not config:
            raise Exception('Config file is empty or does not exist')

        # Read separate config sections and extract key-value pairs
        postgresConfig = {}
        try:
            for key in config['postgresql'].keys():
                postgresConfig[key] = config['postgresql'][key]

            # Check if all necessary keywords for DB connection are present
            if not {'host', 'database', 'user', 'password', 'port'} <= set(postgresConfig):
                raise Exception
        except:
            raise Exception('Unexpected postgresConfig.ini formatting') 

Obviously, as you most probably surmised by now, I have no clue if this is acceptable way.

[hexed cloak]

You should avoid that bare except

[deep dew]

Is there a simple way I can check what kind of Exception will be raised by a given method/function?

coz by 'bare' you mean not handling specific Exceptions?

[pearl cliff]

no, python doesn't have this. either the author puts it in the docs, or you have to read the source code. (most languages do not have static checks for this, and in the few that do it's controversial and doesn't always work well)

[hexed cloak]

this one except:

[deep dew]

How can I mock a variable if it's not an attribute? In all stackoverflow posts it was always said to mock a function when given following code `a = something()'

[pearl cliff]

you can't. i thought it was a top-level variable in FinanceApp.py, not inside a method/function

[deep dew]

So I presume global variables in FinanceApp.py are objects of FinanceApp module and can be found with patch.object()?

[pearl cliff]

yes, that's correct

[hexed cloak]

!paste

[bitter wadi]

Pasting large amounts of code

If your code is too long to fit in a codeblock in discord, you can paste your code here:
https://paste.pythondiscord.com/

After pasting your code, save it by clicking the floppy disk icon in the top right, or by typing ctrl + S. After doing that, the URL should change. Copy the URL and post it here so others can see it.

[hexed cloak]

@deep dew I'd probably write your code like this https://paste.pythondiscord.com/gejuqugudo.py

[hexed cloak]

woops I got the wrong function in the register_type call https://paste.pythondiscord.com/amohefahog.py

should have been psycopg2.extensions.register_type(_dec2float_type(), conn)

if you use the one argument version of register_type it mutates global state - you should always use the 2 arg version and pass the connection or cursor you want to apply the type to

[deep dew]

Thank you, a lot of new concepts to unpack for me, but I recognize how much modular it is

[deep dew]

Noted!

[proper wind]

How do you do this?

[hexed cloak]

!rule 8

[bitter wadi]

Rules

8. Do not help with ongoing exams. When helping with homework, help people learn how to do the assignment without doing it for them.

[hexed cloak]

@proper wind also this isn't the right channel for this

also you should at least try to type in the exam question....

[tidal rock]

can hypothesis do something like "generate tuples of length N containing items of type T then feed them into this function" where N and T are dependent on other test inputs?

[pearl cliff]

yeah, you can do pretty complicated custom stuff with the composite strategy. check out the "what you can generate and how" page

[royal orbit]

st.builds(
    this_function,
    st.lists(st.from_type(T), min_size=N, max_size=N).map(tuple),
    # `st.tuples(*([st.from_type(T)] * N))` would work, but it's ugly
)

and https://hypothesis.readthedocs.io/en/latest/data.html

[tidal rock]

@pearl cliff @royal orbit thanks ill look into it further

[pearl cliff]

what's the use case for st.builds anyway? is it for times when you can't use @given()? or is it just convenience so you don't have to use st.composite?

i always assumed that hypothesis tracked one "session" per instance of given, so i assumed that invoking strategies outside of given wouldn't shrink correctly. is that true?

[river magnet]

In pytest is there a way to parametrize fixtures? I have fixtures that need to be tested in pairs but can't find a way to do it

[frigid basalt]

Yeah just parametrize the name of it

[royal orbit]

It's a convenient way to construct instances of... anything you can call. It's also got some helpful magic for type-based inference if you don't want to spell out everything.
st.composite lets you express dependencies between things, but is overkill if you don't actually need to do that.

[full meadow]

how do i save pytest fails to a file

is that in the command line?

[hexed cloak]

Like a xunit log?

[hexed cloak]

https://docs.pytest.org/en/latest/how-to/output.html#creating-junitxml-format-files

[pearl cliff]

so st.builds is actually a strategy then? i maybe misread the docs, i thought it just invoked the function

oh i think i remember this now... yeah, duh

ignore me 😆

i even tried using it once

[royal orbit]

Yep, the st. means it's a strategy 😉

[dawn frost]

i'm a little confused, do i put pytest in my requirements.txt file? how does this work?

because my package doesn't need it, but the tests do

[royal orbit]

Depends on the project, but I'd usually put it in either tox.ini or requirements-dev.txt

[dawn frost]

requirements-dev.txt is an actual official thing?

[oblique shale]

requirements.txt is not even a official thing, it is just a very commonly used file name

files that use this format are often called “pip requirements.txt files”, since requirements.txt is usually what these files are named (although, that is not a requirement).
source: pip docs (https://pip.pypa.io/en/latest/reference/requirements-file-format/)

[pearl cliff]

i've been thinking lately about using names like these

pip-requirements-run.txt
pip-requirements-dev.txt
pip-requirements-test.txt
pip-constraints.txt

so they are all sorted alphabetically in your file tree

[maiden pawn]

I think my reflexes are too stuck to requirements/constraints in order to change the namings.
and they are written to be used in CI anyway... I can propose a different naming though
How about

requirements-run.txt
requirements-dev.txt
requirements-test.txt
requirements-constraints.txt

it should be less a shock than pip prefixed namings

[hexed cloak]

constraints.txt plz

[pearl cliff]

i don't like having related files "scattered" across the top level directory

heck i don't see why these files even need to be at top level

pip-requirements/{run,test,dev,constraints}.txt

[hexed cloak]

I don't make the rules

[pearl cliff]

there are no rules!!!

[stuck oasis]

so guys i have been left with a school group project and told to do basic unit testing

i dont know much about the code and the assignment is due in about 6 hours

i really need some help with some basic unit testing please someone save my ass

[hexed cloak]

Is the code already completed?

[stuck oasis]

yeah its all done

its like some basic bot prevention system

[hexed cloak]

How do you run it?

[stuck oasis]

so it just runs in the background of a website hosted on AWS

[hexed cloak]

Is there a function you call?

[stuck oasis]

and when we run some basic scalper bot it will detect it based on a simple algorithm that check number of requests

i think its check_bot

[stuck oasis]

is there a way i can show you the code without showing everyone else?

[hexed cloak]

No

[stuck oasis]

ok no worries

so what do you think we can do im stressing here

[hexed cloak]

So make a test that calls that function

[stuck oasis]

ok and what shall i have in the test

[hexed cloak]

Just call the function and see what it does:

assert check_bot() is None

[stuck oasis]

so shall i make like a new class to do this?

or just scroll to the bottom to do it

this is basically the main detection function that needs to be tested @hexed cloak

[hexed cloak]

Just put it in a test file. How many files make up the existing code and what's the directory layout?

[stuck oasis]

this is all the stuff and the flask_gpu.py contains the main stuff

[hexed cloak]

So put your tests in test_flask_gpu.py

@stuck oasis what happens when you run your test?

[stuck oasis]

give me a second please ❤️

[hexed cloak]

?

[stuck oasis]

sorry i just got caught up with something

[hexed cloak]

What happens when you run your test?

[deep dew]

Hello everybody, guess who's back 😉

Is there a cleaner way how I can pass second argument into my pytest.mark.parametrize decorator?
Creating a separate variable as a class attribute seems somehow wrong (?), but the decorator is verbose now, and will be even more if I add additional fixture call outs. The createConfigFile function is outside of TestEstablishConnection as it's called in other testClasses as well.

@pytest.fixture
def createConfigFile(tmp_path: pathlib.Path, request) -> pathlib.Path:    
    configPath = tmp_path.joinpath(request.param[0])
    configContent = request.param[1]
    config = CaseSensitiveConfigParser()
    config.read_string(configContent)
    with open(configPath, 'w') as configFile:
        config.write(configFile)

    return configPath

class TestEstablishConnection:
    # Arrange
    @pytest.mark.parametrize(
        'createConfigFile', 
        [('wrongConfig.ini','''[postgresql]
                                wrongKey1=wrongValue1
                                wrongKey2=wrongValue2
                                wrongKey3=wrongValue3
                                wrongKey4=wrongValue4
                                wrongKey5=wrongValue5'''),
        ('emptyConfig.ini', '')],
        indirect=True
    )
    def test_wrongConfigs(self, createConfigFile):
        # Assert
        with patch('FinanceApp.FinanceApp.pathlib.Path.joinpath', return_value=createConfigFile):
            with pytest.raises(InvalidConfigError):
                # Act
                with TransactionRepo.establishConnection() as repo:
                    pass

[ebon raven]

any good documentation i could learn unit testing from for Django?

[empty dust]

https://docs.djangoproject.com/en/4.0/topics/testing/
https://docs.djangoproject.com/en/4.0/intro/tutorial05/

[ebon raven]

thx ♥️

[ebon raven]

why do we use reverse() in django testing?

what does it do specificaly

[lament pelican]

hi guys,
is the job of sofrware engineer tester good? and what is it mainly about, will i get to write code?

[maiden pawn]

It gives you URL address of endpoint by its nickname

[maiden pawn]

It depends on the company. Do they prefer manual or automated testing

[mellow geyser]

it depends.
Testing can be manual or automated.
In the case of manual, you won't write code.
In the case or automated, you may help writing code to validate paths and features written by others. But that won't be writing new features

It's also not as challenging nor interesting as writing the code for an actual application

it's also seen as a lower status and the pay would be lower in general

[lament pelican]

i see

[maiden pawn]

Good testing can be written only in the process of writing new features I think

Same person who implements features, should write tests

That is the best kind of testing

Which is quite enjoying

So called TDD approach is related to it

[lament pelican]

i have an interview for this role next week and having second thoughts

[mellow geyser]

If you aren't stuck, I would avoid tester

(also called QA/QE)

[maiden pawn]

I can agree. It looks a bit of a dead end career
It will not give you room to grow into anything else
QA in itself looks like the last career step in its short ladder

[maiden pawn]

Just to be clear, if u get a regular software Dev role... Automated testing is always part of its job. Half of the code is roughly written for tests.

That is normal and great

[delicate basin]

I have favourable and invafourable opinions of the so called TDD practice. So far in my learning experience and research, I've found out that taking it too far might not be a good idea. We are prone to making mistakes, but that doesn't mean we should suppress our good ideas when we have them. I think you have to balance TDD with good forethought and design principles, and throughout the development you have to pay attention to what design you want to end up with. That being said, the practice of writing tests before the code seems like a good way to check your work, and it doesn't limit your creativity in itself.

TL;DR I like unit testing before writing production code, but I do not think it should 100% guide the project structure, or you could end up with something crazy that won't work.

[maiden pawn]

The point of the TDD to make you forced thinking in advance about good enough structure to be tested.
if you wish to do more than that, you need to think at a higher abstraction level of the project planning

Even in this case, TDD is still needed to make the structure well tested.

I think this is because TDD and higher level planning solve a different kind of problems to project planning

TDD makes to insure that lowest layer of implemnetation is good enough

but we still can apply a higher level of thinking at the same time to do it even better. And still applying TDD while we implement the higher planned structure

So, TDD forever! Long live the queen!

Clean architecture can be always well tested.

May be the first iteration of implementation can be not well. That's why TDD involves re factorization as one of steps. Nothing stops you from making in advance better architecture even in the first attempt though

[delicate basin]

💯

[dawn apex]

hi

i have a question

[maiden pawn]

and I am the swarm, your point?

just kidding. Don't ask to ask, just ask.

[maiden pawn]

https://pythondiscord.com/pages/resources/guides/asking-good-questions/ read here for more information how to ask good questions

[dawn frost]

def test_abstract_base_inheritance():
    """
    Checks inheritance matches:
    ANY
    │
    └───BYTES
    │
    └───INTEGRAL
        │
        └───INT
        └───UINT
    """
    # INT and UINT inherit from INTEGRAL
    assert issubclass(atomics.INT, atomics.INTEGRAL)
    assert issubclass(atomics.UINT, atomics.INTEGRAL)
    
    # INT and UINT do not inherit from BYTES
    assert not issubclass(atomics.INT, atomics.BYTES)
    assert not issubclass(atomics.UINT, atomics.BYTES)
    
    # INT and UINT are not otherwise related
    assert atomics.INT is not atomics.UINT
    assert not issubclass(atomics.INT, atomics.UINT)
    assert not issubclass(atomics.UINT, atomics.INT)
    
    # INTEGRAL and BYTES inherit from ANY
    assert issubclass(atomics.INTEGRAL, atomics.ANY)
    assert issubclass(atomics.BYTES, atomics.ANY)
    
    # INTEGRAL and BYTES are not otherwise related
    assert atomics.INTEGRAL is not atomics.BYTES
    assert not issubclass(atomics.INTEGRAL, atomics.BYTES)
    assert not issubclass(atomics.BYTES, atomics.INTEGRAL)

so i wanna test this inheritance heirarchy, but i'm not sure if i'm going about this in the right way

is this fine? is there some other way i should be testing this? is there a reason i shouldn't test this?

oooh i could check MRO

[dawn frost]

(or __bases__)

ig this wasn't really a great #unit-testing questoin

[glass pivot]

Why do you even need this?

I've never seen anyone testing inheritance before, since it's mostly irrelevant and can change, what's important are really just the functionalities

[dawn frost]

i want it for the type hints, and i wanna rewrite this in Cython/C, which makes me worried that i'll forget something

so this is more of a regression test for when i make the switch

[glass pivot]

type hints? Wouldn't a simple type-checker run after you made the switch enough

[dawn frost]

hmm... maybe?

[pearl cliff]

perhaps we should be doing things like testing for subclass membership in our test suites

[dawn frost]

in that case my excuse is procrastination from writing the harder unit tests 😂

[dawn frost]

i like this train of thought better 🙂

[pearl cliff]

ive been a big fan of writing test "groups", methods that make a bunch of related and standardized assertions about something that you need to test in several distinct places

such as openapi conformance tests for some http endpoint

of course at some point you end up having to test your tests 😛 but if kept under control i think you could definitely have a test like assert_framework_class_is_set_up_correctly(MyNewClass)

not sure if that is relevant here of course

[dawn frost]

i just started writing tests today, so idk either lol

[glass pivot]

I don't know, I'd say it's usually pretty hard to break inheritance like this and unless you have some weird use case where you're dynamically creating classes and you need to ensure they're created properly, these tests will add a lot of clutter into your test code with little benefit

[dawn frost]

maybe, maybe not if i'm implementing this in a completely different language

inheritance with the C-API can go wrong

also i don't trust myself

[glass pivot]

well, you can obviously just run the test once you made the switch to confirm everything is alright, but do you need to have this test run each time after that? Even with the code in C, once implemented properly, this probably won't go wrong, and if it will, it's probably going to get caught with good enough general test coverage

[dawn frost]

i don't need it each time, but if i have it, why not keep it 🤷

[glass pivot]

that's fair, you could say it reduces some clutter in the test code, but if you don't mind that then sure

But I'd stick with issubclass checks here, checking if the class is in the mro manually is just the same with more complexity

[dawn frost]

lemme write it rq, and then see if you still think that

def test_abstract_inheritance():
    """
    Checks inheritance matches:

    ANY
    │
    └───BYTES
    │
    └───INTEGRAL
        │
        └───INT
        └───UINT
    """
    # check all types are unique
    atypes = [atomics.ANY, atomics.BYTES, atomics.INTEGRAL, atomics.INT, atomics.UINT]
    assert len(set(atypes)) == len(atypes)

    # check types follow inheritance tree
    for _ in range(len(atypes)):
        a = atypes.pop(0)
        assert set(a.mro()).isdisjoint(atypes)

hmm ok not quite right :/

i still think it's much less verbose than issubclass

[glass pivot]

that doesn't verify that say INT is actually a subtype of INTEGRAL though, only that there's no unexpected inheritance

with this logic, they might as well be completely unrelated types

[dawn frost]

yes, i see now :/

does pytest allow you to call a function in your test, and put your assertions in there?

or do your assertions have to be in the top level function?

[glass pivot]

I'm not sure, you can always just try it and see

I think it should be fine to have assertion in a different function though, they'd raise an asserion error which will propagate into the test func

you really shouldn't be testing the test here though, if this condition failed, it doesn't mean the inheritence is wrong, it means the test is poorly defined ```py
assert len(set(atypes)) == len(atypes)

[dawn frost]

Hmm yeah that’s fair

[hollow forum]

Quick question: I have received a assignment during job interview to web scrape a .html file. I have completed that part However, it also say "the preferred UnitTest framework is NUnit. We expect that your code is unit tested.".

Now does anyone know how to Nunit test a Web scrapped project?

[mellow geyser]

the same way than any other piece of code for which you want to write tests

which I know may sound generic, but that's pretty much it.

[hollow forum]

I have 0 experience with testing!

Also, what conditions should i test my code? I have no imagination skills here

[hexed cloak]

Aim for 100% coverage first then worry about different conditions

[mellow geyser]

You don't need any imagination here. I would recommend to look for books as it's not a topic which can be taught in 2-3 discord messages. There are also a few pinned resources to this channel.
At its core, it's about validating the behavior of your code, not just positive but also negative cases.
So look at your classes and methods and how they should be used. Then based on that, write tests to validate they do indeed do what they are supposed to do.
Looking at the test coverage is a great way to guide yourself in terms of what is being tested and what should be tested. However while 100% would be ideal, I would recommend to stay pragmatic rather than dogmatic about it.
And if you find yourself having troubles to write tests because it gets complicated, it can be a code smell that your class should be refactored. Code should be testable.
Eventually, look also into mock objects.

[fierce elk]

To reiterate a point, write tests to validate that classes/functions do what they are supposed to do, but also that they don't do what they are not supposed to do. (So many tests I see concentrate on the first, but don't touch the second. Many of the tests I see are also ones I write :/ )

[pearl cliff]

i kind of disagree that testing doesn't require imagination. but it's a specific kind of imagination that comes from experience and from reasoning carefully about your own program.

i would also suggest avoiding mocks until you have some basic proficiency with testing

in order to come up with test cases, ask yourself these two questions:

1. how do i know if my program is working correctly?
2. how do i know if my program is not working correctly?

[fierce elk]

I got stumped by a question in an interview once about testing some string manipulation function. The answer was "empty string" and "null"…while I knew this, it just wasn't something that popped into my head. That is definitely a form of imagination - imagining what things are required to sufficiently test some interface. (In hindsight I put the interview thing down to pressure, but it still irks me that it wasn't something that just sprang to mind.)

[pearl cliff]

this is one reason why Hypothesis (and property-based testing in general) is so great

it's a whole class of imagination that you don't need to have

[mellow geyser]

I guess it depends how one defines "imagination" or "creativity", or the actual english term we should use.
To me, imagination is more akin to art. Whereas CS and testing is more akin to engineering where you do require some form of creativity to explore the search/solution space, but it's not the same as devising an art piece. No different from an architect/engineer devising a bridge or a house (ignoring the aesthetics part).

[pearl cliff]

i suppose. i think imagination is one part of creativity

[pearl cliff]

using hypothesis, is there some generally-accepted way to generate strategies that depend on fixtures which are established only for the lifetime of the test method?

normally i write tests like this:

class TestMyThing(TestCase):
    @hypothesis.given(a=st.foos(), b=st.bars())
    def test_my_thing(self, a, b):
        ...

but what if i want st.foos() to depend on something defined in TestMyThing.setUp()?

can i still use given somehow? or do i need to forego that and use some lower-level interface?

one use case would be parameterizing a strategy according to something in a database, e.g. st.lists(st.sampled_from(values_from_database))

[royal orbit]

Use @given(data=st.data()) and then xs = data.draw(st.lists(st.sampled_from(values_from_database))) inside your test method.

[pearl cliff]

oooooooooh

so that's how you get access to draw

[royal orbit]

This is exactly the same as if you could pass it to given, except that you can use data which isn't defined at decorate-time.

Yep!

[pearl cliff]

dare i ask, what's the type of st.data()? SearchStrategy[???]?

https://hypothesis.readthedocs.io/en/latest/data.html#interactive-draw found this

thank you for the pointer to data

[royal orbit]

SearchStrategy[Callable[[SearchStrategy[T]], T]]

[pearl cliff]

nice

[royal orbit]

Approximately, anyway, there's a protocol in there for the attribute access that I'd like to elide.

[pearl cliff]

i appreciate having DrawFun and SearchStrategy

DataStrategy maybe

[pearl cliff]

oh no... hypothesis doesn't support async test methods

sad

[wind breach]

So what is the general stand on TDD on this server and in this community? I have heard that it's really good but obviously also the opposite, I am not quite sure what my stand is on that topic, because on the one hand I really think it's useful, but I feel like it's very difficult to get into.

[pearl cliff]

some people here really like it

i think strict adherence is silly

but you might learn some things about testing in general by trying to follow it for a while

i certainly think that writing tests early in the development cycle is a good habit

[delicate basin]

I tried it myself and I think that's it's a good practice, but it shouldn't be a philosophy so to speak. My current stance on unit testing is that you want to force yourself to write testable code so that you can write actually serious tests when it becomes important.

I don't think TDD in itself results in a test suite that's useful as a test suite.

But, that could be the result of a misunderstanding.

[pearl cliff]

i think your experience and opinion is shared by a lot of people @delicate basin

[delicate basin]

That's a relief. Not because I want to be the same as everybody else, but because I don't want to spread wrong information because of inexperience with it. 😅

[royal orbit]

This is well supported by pytest-asyncio or pytest-trio!

[pearl cliff]

i am stuck using unittest on an existing project

i noticed that it's possible to set a custom "executor", but i wasn't sure if i could run async test methods that way

for now i was able to de-async-ify this test method

and after like 10 hours of debugging i finally got an "OK" output 😆

[royal orbit]

https://hypothesis.readthedocs.io/en/latest/details.html#custom-function-execution

Or just write a "run with $backend" decorator and apply it in between your test and given.

Also yes, custom executors can absolutely run async code, an executor is just a function which takes the thing to run + args, and runs it.

[pearl cliff]

i think the problem with this is that i'd have to wrap it in asyncio.run(), which could be pretty bad due to setting up multiple event loops

that or i start an event loop in a separate thread and send coroutines to run there... yikes

i'll have to see how the pytest plugins work

i'm not brave enough for that tonight

[pearl cliff]

this also seems relevant https://github.com/python/cpython/blob/cc400585fab02994255f21ae8183d5f147236815/Lib/unittest/async_case.py#L76-L155

[pearl cliff]

@royal orbit one more ping, sorry 🙂 i just wanted to let you know that hypothesis helped me write tests that i never would have been able to write by hand, by covering a huge amount of edge cases.

my work all day yesterday literally depended on it. i truly do not know what i would have done without it.

[cinder night]

[pytest]
the pytest suggest we should install our developing package to our computer. What is the benefit of it? Thank you.
Website: https://docs.pytest.org/en/6.2.x/goodpractices.html

[frigid basalt]

By installing your package, it makes it possible for your tests to import it

[cinder night]

don't we can also import them with "import" statement, without "install" it?

[frigid basalt]

That depends on where your tests are and how you run it ☝️

[cinder night]

do you mean we can run the test everywhere after install it?

[frigid basalt]

Yes! It is the same as installing a package and being able to import it everywhere

Don't forget the virtual environment though

[royal orbit]

For me, the most important reason to install your package when testing it is that this is how everyone else will use it!
If you test without installing, how do you know that the installed version works? It might be missing key files!

(I have literally had this happen to me)

[pearl cliff]

big +1

especially when dealing with non-python data files

i think src/ layout is mandatory for this reason if you are developing a library or application that is meant to be distributed to other users

[maiden pawn]

Sigh

To implement last feature in the legacy project

I need to apply their 850 code lines of crappy tests to check 250 code lines of functionality

[fierce flare]

Don't forget that virtual env: https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/#creating-a-virtual-environment

[jolly tangle]

Does anyone know about pytest and is this the right place

to ask

[carmine sage]

@royal orbit we block that site because it's often used in a way that is unfriendly to beginners

[royal orbit]

@jolly tangle yes, this is the place to ask 🙂
(tip for next time: if you'd included your question at the same time, I might have been able to answer it immediately!)

[jolly tangle]

@royal orbit I asked on stackoverflow so if i don't get an answer can I ask you here at later time?

[whole orchid]

Could someone help with this question? https://stackoverflow.com/questions/71751883/vscode-python-test-discovery-not-loading-properly-because-of-imports Thanks

[fading needle]

hey guys, would someone perhaps know why would playwright/pytest tests randomly fail when run together, eventho succeeding each time when running solo. can i make something to prevent this from happening?

[frigid basalt]

That would depend on the test - only knowing that they fail when running together doesn't help sorry.

[pearl cliff]

fail how? what does "randomly" mean?

[trail pine]

!e

# Ontario has the following area codes:
# 905, 807, 705, 647, 613, 519, 416, 343, 289 and 226

area_codes = [905, 807, 705, 647, 613, 519, 416, 343, 289, 226]
phone_num = ""

for i in range(8):
    if i == 0:
        # pick a random area code
        phone_num += str(random.choice(area_codes))
    elif i == 1:
        # phone number does not start with 0
        phone_num += str(random.randint(1, 9))
    else:
        phone_num += str(random.randint(0, 9))
        
    area_code = (phone_num[ : 3])
    phone_num1 = (phone_num[4 : 7])
    phone_num2 = (phone_num[8 : ])
print(f"Your new random phone number is:  ({area_code})-{phone_num1}-{phone_num2}")```

[bitter wadi]

@trail pine :white_check_mark: Your eval job has completed with return code 0.

Your new random phone number is:  (705)-717-39

[trail pine]

oops im sorry

[jolly tangle]

I am going to link my stack overflow question. I am getting an error in pytest
https://stackoverflow.com/questions/71759859/i-am-running-pytest-and-getting-an-error-werkzeug-utils-importstringerror-imp

You can answer here if you want. If you want I can post the exact question here.

Thanks

[lapis steeple]

how I can use Factory_by on a model in which I have two attribute which are in FK to the same model?

[lunar wind]

this might be a silly question, but is it possible to use python's builtin doctest module to test a single rst file but break things up so each separate code block is it's own test ... so essentially the rest becomes a test suite ?

or perhaps something that can do that with pytest ?

so something like this, https://gist.github.com/rockyburt/1e0b430c20ae26206ba6041cab0b1cb9 , comes back as having run 3 tests

[blissful eagle]

Hi guys! I find this lines hard to read, but I don't know how to rewrite them.

How would you do it if it's you refactoring this?

@pytest.mark.parametrize('arrays, expected', [(case[0], case[1]) for case in smallest_difference_test_cases]) def test_smallest_difference(arrays, expected): assert smallest_difference(arrays[0], arrays[1]) == expected

[pearl cliff]

just needs some whitespace:

@pytest.mark.parametrize(
    ('arrays', 'expected'),
    [case[:2] for case in smallest_difference_test_cases],
)
def test_smallest_difference(arrays, expected):
    assert smallest_difference(arrays[0], arrays[1]) == expected

how much you want to stretch it vertically depends on how wide your editor window is

@blissful eagle ☝️

[hexed cloak]

What's the type of smallest_difference_test_cases tuple[tuple[Array, Array], Array]]?

I'd probably [(left, right, expected) for (left, right), expected in smallest_difference_test_cases]

[blissful eagle]

Thank you guys!

[blissful eagle]

It's this
smallest_difference_test_cases = [ [( [-1, 5, 10, 20, 28, 3], [26, 134, 135, 15, 17] ), [28, 26] ], [( [-1, 5, 10, 20, 3], [26, 134, 135, 15, 17] ), [20, 17] ] @hexed cloak

[cerulean rune]

yo how do i write a very basic unit test?

[pearl cliff]

smallest_difference_test_cases = [
    (
        ([-1, 5, 10, 20, 28, 3], [26, 134, 135, 15, 17]),
        [28, 26]
    ),
    (
        ([-1, 5, 10, 20, 3], [26, 134, 135, 15, 17]),
        [20, 17]
    ),
]

@pytest.mark.parametrize(
    ('inputs', 'expected'), smallest_difference_test_cases
)
def test_smallest_difference(inputs, expected):
    assert smallest_difference(*inputs) == expected

[blissful eagle]

Woah this is much cleaner. Thank you! Will this still hold even if I have 30 test cases and each array inputs contains 40 elements?

[pearl cliff]

the two inputs each have 40 elements? yes it should be fine

instead of / in addition to writing out 30 test cases, consider using property-based testing

https://hypothesis.works/

[blissful eagle]

Got it. I'll give it a read

[crimson veldt]

Hi I am new here

Can anyone make discord spam bot

[fresh tinsel]

@crimson veldt We won't help you with that here.

!rule 5

[bitter wadi]

Rules

5. Do not provide or request help on projects that may break laws, breach terms of services, or are malicious or inappropriate.

[blazing violet]

Is there a respx style library for Sockets? I can't seem to get the mock library to behave with nested functions.

[tidal rock]

is there a way to tell hypothesis that a set of inputs are supposed to fail?

or should i just do my own try/except to check?

[pearl cliff]

fail in what sense?

[pulsar yacht]

Hello there 👋
Any tips on performance-degradation testing? (I'm using pytest)
The main idea is to notice once tests performance start to degrade, e.g. today x test runs in y seconds, another day same test run way slower. I want to notice that, see some % of how slower it got and etc.

[tidal rock]

in the sense that hypothesis will report the inputs that dont raise some kind of exception as falsifying examples. if they raise some kind of exception theyre considered to 'succeed'. so something like:

@given(inputs_that_are_supposed_to_succeed())
def ensure_success(test_input):
  # do stuff that ISNT supposed to raise exception

@given(inputs_that_supposed_to_fail(), should_fail=True)
def ensure_failures(test_input):
  # do stuff that IS supposed to raise exception

wondering if there's an argument or extra decorator that can add the should_fail behaviour

[pearl cliff]

I think you would have to manage that with the test suite itself, since "failure" either means raising an assertion error, or raising some other uncaught exception

If the failure conditions are complicated, you might have to put both sets of inputs into the same test method and check which case you are in using if

although one problem with that technique is that you could end up testing one code path a lot more than the other

maybe splitting the conditions with assume is better

it might help to know the specific details of what you're trying to test, and with what inputs

@given(inputs_that_are_supposed_to_succeed())
def ensure_success(test_input):
  ...

@given(inputs_that_supposed_to_fail())
def ensure_failures(test_input):
  with pytest.raises():
    ...

basically it's not the job of hypothesis to handle this part, but you do have to consider how much to split up your assertions into different methods

[pearl cliff]

i like how the character is kind of goat-like. unexpected genre crossover

[empty dust]

You need to use fixtures to access the execution time of each test (see https://stackoverflow.com/a/61439935 )

Make sure not to fail a test based on a variable execution time limit, instead you can use something like https://pypi.org/project/pytest-timeout/ to set constant timeouts for each test

[digital sigil]

Hi guys! I have to unit-test a sendgrid api implementation. Any ideas? what should I mock?

[static grove]

that indent lol

[maiden pawn]

I have a function that sends requests.post to outside
Which way do you think the most efficient to check that everything is sent correctly in Django / Pytest framework?

I would usually create dummy endpoint to check it, but it is not good in the terms of having endpoint serving not-app functionality

https://requests-mock.readthedocs.io/en/latest/mocker.html this looks cool

[empty dust]

I use https://pypi.org/project/pytest-httpx/ since it works well with async requests

[maiden pawn]

I discovered the legacy project I have, already uses requests-mock. so the choice is obvious

[broken shale]

hello i need some help with my code, can some watch and explain why i always get these error?

[hexed cloak]

pytest-recording?

[pearl cliff]

what's the advantage of this versus plain vcr?

[hexed cloak]

It's got a mark decorator?

[pearl cliff]

i found the pypi page, looks like it's a lot of nice convenience stuff

yeah, makes sense

[hexed cloak]

Yeah was discussing this with a colleague recently and they were like hey VCR seems unmaintained

Checks maintenaners....

Wait hang on 🤔

[maiden pawn]

nice to know this thing. I liked vcr. thanks.
The question was about a diffferent thing though, I just needed to have mocked request of requests library
and to scan its request.headers / response.body of the already formed requests. requests.mock library does it fine

[hexed cloak]

Already formed requests?

[maiden pawn]

yeah, I mean we send the requests.post, get the response of the mocked sending
and in response we scan what we sent in the initial request and what received

I was checking outgoing headers

[hexed cloak]

What's the SUT code look like?

[maiden pawn]

stuff like this

def test_my_auth_request_headers_addition():
    url = 'https://example.com'
    with requests_mock.Mocker() as m:
        m.post(url, text='resp')
        resp = requests.post(
            url=url,
            json={'123': 123},
            auth=SignedAuth(secret='456'),
            )
    assert 'SIGNATURE_256' in resp.request.headers
    assert resp.request._request.headers['SIGNATURE_256'] \
        == "kQMGNaVA2YEgGDDVClIwGQJ3kknw58oOQqNa+qh0jKs="

except instead of requests.post, some abstracted thing

[hexed cloak]

I mean the SUT: system under test

[maiden pawn]

not familiar with termin, do you mean to ask that it is Django Backend application with some Message Queue Celery stuff?
Although technically we tested just requests library

[hexed cloak]

I mean the source code of the thing you're testing

[maiden pawn]

Oh I get it

feeling myself dummy

class Webhook(models.Model):
    # Different django model fields

    def send(self, payload):
        return requests.post(
            url=self.target,
            json=payload,
            timeout=DEFAULT_TIMEOUT,
            auth=SignedAuth(
                secret=self.secret
            )
        )

[maiden pawn]

the thing above

[hexed cloak]

What's SignedAuth?

[maiden pawn]

class SignedAuth(requests.auth.AuthBase):
    # http://docs.python-requests.org/en/master/user/authentication/#new-forms-of-authentication
    def __init__(self, secret):
        # we must specify all the possible auth credentials here,
        self.secret = secret

    def __call__(self, r: requests.Request):
        # Implement my authentication
        if self.secret is not None:
            r.headers['SIGNATURE_256'] = hash_payload(
                data=r.body,
                secret=self.secret,
            )
        return r

Basically it adds HEADER with... hash validation of the request body

[hexed cloak]

And what calls send?

[maiden pawn]

Abstracted thing executing it in Celery task. Not really important

[hexed cloak]

Well maybe

[maiden pawn]

Anyway, I already did it

It has tests covering all aspects

of the requested functionality

[hexed cloak]

I'd probably test this only via the celery task function

The AuthBase and use of requests are all implementation details

The "as a service I expect to run a celery task and write the following bytes to the TCP socket resolved from HOST" is probably the only level I'd bother testing at

[maiden pawn]

interesting position.

I am just not familiar with code project enough to do it from celery task)

I would probably need to dig more to do that

[hexed cloak]

Then you can refactor it to use httpx or some such

VCR is nice because it abstracts http requests in general

[maiden pawn]

requests-mock is already used in the project. I did not need to install dependencies

[hexed cloak]

Ah well you're stuck with it then ;)

[maiden pawn]

exactly)
https://qph.cf2.quoracdn.net/main-qimg-b816c65ef034756adf217f31cc6ef7ea-lq

[hexed cloak]

The "as a service I expect to run a celery task and make the following http request somehow" is the next best

The "as a service I expect to run a celery task and make the following http request with requests by KR" is what you're doing currently

[maiden pawn]

as the way I tested it, it is a unit test. Testing as a celery... can I do it without having running worker? 🤔 i saw the mention of pytest-celery library.

it would be cool to check it

I don't want to have it as integration test

integration tests are horribly run in docker-compose Robot testing framework there

I wish to keep it as unit test in pytest if possible. Because it is way faster to do 😉 To develop and to run.

[hexed cloak]

pytest-celery is just my shim to load the pytest plugin from celery - you probably don't need it

[maiden pawn]

YOUR shim? xD

the project has it already

[pearl cliff]

hah! that's a new one

[hexed cloak]

https://github.com/celery/pytest-celery/blob/main/pyproject.toml#L8

[bitter wadi]

pyproject.toml line 8

author = "Thomas Grainger"```

[frigid basalt]

I mean, to be fair you have 1.2 K repositories so it's quite hard to keep track of what you've made 😅

[pearl cliff]

I realized recently that GitHub does not let you search for repositories specifically created by a user, as opposed to forked

[radiant mason]

hey, I have a test that doesn't end, I suspect it is an infinite loop somewhere inside, does anybody have any tips?

PS: I let the test run for one hour and it didn't end

[empty dust]

Show your code?

[radiant mason]

I modified some of the code of the asks library (https://github.com/theelous3/asks)
right now test_anyio.py is the one failing...

[frigid basalt]

Which test?

Run pytest with the highest verbosity

[radiant mason]

first one fails after a while
second one just blocks up

[empty dust]

Is that link your modified code or the unmodified library?

[radiant mason]

unmodified lib

[empty dust]

Do you know if your modifications are making the test fail? Does it pass for you when unmodified?

[deep dew]

If I'm testing a lot of methods which load different files and process them, does it make sense to create test-files which will be used as the input data to test tgese methods? Initially I was creating them by fixtures with tmp_path, but it starts to get messy the more test files I need to generate

[hexed cloak]

Messy?

[deep dew]

Haha, I guess I'm incorrect. Let's start with the fact that I'm parametrizing fixtures generating my test input files, and this feels, well, not very readable e.g.:

    @pytest.fixture
    def createTempConfigFile(self, tmp_path: pathlib.Path, request) -> pathlib.Path:
        """Create a temporary .ini file containing mock values used for tests"""

        configPath = tmp_path.joinpath(request.param[0])
        configContent = request.param[1]
        config = CaseSensitiveConfigParser()
        config.read_string(configContent)
        with open(configPath, "w") as configFile:
            config.write(configFile)

        return configPath

    # Arrange
    @pytest.mark.parametrize(
        "createTempConfigFile",
        [
            (
                "wrongConfig.ini",
                """[postgresql]
                        wrongKey1=wrongValue1
                        wrongKey2=wrongValue2
                        wrongKey3=wrongValue3
                        wrongKey4=wrongValue4
                        wrongKey5=wrongValue5""",
            ),
            ("emptyConfig.ini", ""),
        ],
        indirect=True,
    )
    def test_establishConnection(self, createTempConfigFile: pathlib.Path) -> None:
        with patch(
            "FinanceApp.FinanceApp.pathlib.Path.joinpath",
            return_value=createTempConfigFile,
        ):
            # Assert
            with pytest.raises(InvalidConfigError):
                # Act
                with TransactionRepo.establishConnection() as repo:
                    pass

I could also create a file in each test case instead of a fixture, but then i have a lot of repeating code.

• when my input file gets verbose, it gets wierd (?) to keep it inside of a code? E.g. .csv

[hexed cloak]

I wouldn't bother with indirect here

Just make createTempConfigFile a plain function rather than a fixture

[deep dew]

So, what would be the use case for parametrized fixtures vs just plain functions? Why would I use param. fixtures?

[hexed cloak]

You can still use parametrize but no need for indirect

[deep dew]

Right, but in case my createTempConfigFile() fixture is used in yet another method, i need to use indirect, right? As my fixture setups will be different for different test cases

[hexed cloak]

No you just call it like a regular function with your param

You also don't need to do all that read string stuff

Just do (tmp_path / config_path).write_text(config_content, encoding="utf8")

[deep dew]

sigh, a little bit confused. I can't call my fixture directly, so you mean to just set it as a plain function and call it directly in test cases?

[deep dew]

Good hint, thank you.

[peak summit]

hey guys
I'm trying to implement unit test into my flask app. I ran into an error of "AttributeError: 'function' object has no attribute 'test_client'". I'm actually having the same code as the tutor, but I can't find where lies the problem in my code

my test.py file looks like this:

import unittest
from app import create_app

class FlaskTestCase(unittest.TestCase):
    
    def test_index(self):
        with create_app.test_client() as test_client:
            response = test_client.post('/')
            assert response.status_code == 200

if __name__ == '__main__':
    unittest.main()

my app.py file:

from website import create_app


if __name__ == "__main__":
    app = create_app()
    app.run(debug=True)

test.py file is right next to app.py

should I add this "test_client" feature somewhere in my code? Tutorial does not explain where is this from, the guy just wrote it, so I followed, but it seems that is has to come from somewhere. Does anyone know the solution to this? Other posts advised to delete or restore "self" attribute in test_main function, I also tried to change post method to get, but that did not resolve the problem

[glass pivot]

It should probably be create_app().test_client

Since the error tells you that create_app is a function, and it doesn't make sense to access some test_client function or variable on a simple function, instead you want to call it and access it on the result of that func

Though I don't directly have experience with testing Flask things, but this should be the issue

[peak summit]

woah, it worked
though now I only get "AssertionError" and I don't know what it means
so idea why assertion error was raised

I checked what is the status code of this request and it gives me 302
when I changed it to assert response.status_code == 302 the test ran with success

so that it something already 🙂
I think I will have to go further from that

@glass pivot thank you so much!

[untold ore]

I would just change the test to

...
response = test_client.post('')
...

Since you get a redirect status code (302 Found), which is redirecting from '/' to '' :-)

[ornate spear]

hi can anybody suggest me the book for python programming

[peak summit]

I have a WTF form (login form), which has set minimum password length to 6 digits
my test that looks like this:

    def test_logging_in(self):
        with create_app().test_client() as test_client:
             response = test_client.post('/login', data=dict(username="[e-mail]", password="123"), follow_redirects = True)
             assert response.status_code == 200

passes with success, even though I put password shorter than allowed. Why is that? Is there any error in my test? I tried to find something there, but since I'm only a couple of hours in testing, I could not retrieve it. Where do I make a mistake?

[empty dust]

what does the code for your form look like?

[peak summit]

class LoginForm(FlaskForm):
    email = StringField('EMAIL', validators=[DataRequired()])
    password = PasswordField('PASSWORD', validators=[DataRequired(), Length(min=6)])
    submit = SubmitField('LOGIN')

[empty dust]

what about the code handling your POST?

[peak summit]

@auth.route("/login", methods=['GET', 'POST'])
def login():
    form = LoginForm()
    if current_user.is_authenticated:
        return redirect(url_for('views.home'))
    if form.validate_on_submit():
        user = User.query.filter_by(email=form.email.data).first()
        if user and bcrypt.check_password_hash(user.password, form.password.data):
            login_user(user)
            flash(f'Logged in {current_user.id}!', category='success')
            next_page = request.args.get('next')
            return redirect(next_page) if next_page else redirect(url_for('views.home'))
        else:
            flash('Wrong credentials, try again', category='error')
            return redirect(url_for('login'))
    return render_template("login.html", form=form)

[empty dust]

If your validation fails, you return render_template("login.html", form=form)
By default render_template has a status code of 200, and redirect has a status code of 302

You need to return a tuple like shown here https://flask.palletsprojects.com/en/2.1.x/quickstart/#redirects-and-errors to change the error code

[peak summit]

aaahhh, I see
no matter what wrong credentials I put with it, it will be returned to the login page again and that is in the end 200 status code, right?

[empty dust]

Yep

[peak summit]

so in order to check if this test works, I could add something that asserts an element that is displayed only in my home page

if test can't log in, it won't be able to find it and therefore it will fail

[empty dust]

No, there should be some way of telling that the data in the request was invalid. Either return 400 if validation fails, or 200 with an errorCode inside the body (depends on preference).

[peak summit]

alright, I get it now, thanks!

but what I could do to actually log test in?
I'm following a guy in tutorial and I have the same snippet as him

we just have different credentials

if I put correct credentials it's not logging in

[cyan flicker]

How could I create a unit test for a function like this? I've created a fixture for JiraWrapper.update, but I'd like to be able to confirm that everything is created correctly. Only thing I can think of is creating a variable _debug=False and call the function with true so it returns the updates dictionary.

    def _RunUpdate(self, missingFields: list, updates: dict) -> None:
        ticket = self.ticket
        reporter = ticket.fields.reporter.name
        if missingFields:
            if not self._HasMissingFieldLabel():
                updates['labels'] = self._AddMissingFieldLabel()
                updates['assignee'] = {'name': reporter}
                # Only add a comment if it's a new ticket or field comment flag is set
                if self.requiredFieldsComment or (self.newPriorityComment and self.newTicket):
                    self._Comment(missingFields=missingFields,
                                  reporter=reporter)
        if updates:
            log.info(f"Adding following updates to {ticket.key}: {updates}")
            JiraWrapper.update(ticket=ticket, **updates)

[empty dust]

I'd recommend debugging and seeing how the request is being received and processed to make sure everything is working as expected

[peak summit]

okay okay, thank you!

[violet wind]

Hi, currently writing unit test for my API integration, is there an easy way I can save a return value of api_connection.get_crypto_quotes from my live code so that i can mock this call out in a unit test and return a realistic response?

api_result = self.api_connection.get_crypto_quotes(COINS.BITCOIN, str_start_time, str_end_time)
return api_result[-1]._raw['ap']

[violet wind]

For anybody searching for this in the future: I ended up constructing the mock objects by using untitest.mock and unittest.PropertyMock to build up an object that had the same signature as my expected api response, I then used patch in order to mock the call to the definition of self.api_connection in order to return the mocked version of the object

def mock_get_quotes_now(self):
        mock_quote_now_data = pd.read_json(path_or_buf=self.MOCK_QUOTES_NOW_DIR).T.to_dict()
        result = []
        for _, quote in mock_quote_now_data.items():
            mock_response = Mock()
            type(mock_response)._raw =  PropertyMock(return_value=quote)
            result.append(mock_response)
        return result

#In the test

mock_service = Mock(spec=package.RealClass)
mock_service.get_crypto_quotes = Mock(return_value=self.mock_get_quotes_now())```

[dry idol]

how to i unittest sending some data to serial port ? should i create virtual serial port to test the logic or or a real device ?

[bitter bison]

!code

[wild granite]

Hi

For django developers

How do u test abstract model

Without inheriting it

[hexed cloak]

Why is this a requirement?

[dusk coyote]

um can anyone test my computer crasher?

[carmine sage]

Please do not post malware

[brave vine]

Is anyone available to help me with a testing question over in #help-mushroom ?

[mighty aurora]

.

[delicate basin]

You don't need to return the dictionary. If your test passed in the dictionary, the test already has it.

[cyan flicker]

Hmm. I'm quite new to unit testing, as in this is the first project I'm adding it to. So I'm not sure I follow.

[delicate basin]

But what you should be testing is that the ticket got updated correctly. The updates dict is an implementation detail.

[cyan flicker]

That's what I'm doing... this is the test I created

@pytest.mark.parametrize("missing_fields, updates, labels", [
    ([requiredFields[0]], {}, []),
    ([requiredFields[0]], {}, ["test1"]),
    ([requiredFields[0]], {}, ["test2", missingFieldLabel]),
    ([], {'duedate': '2023-22-4'}, ["test3"]),
    ([], {'duedate': '2023-22-4'}, ["test4", missingFieldLabel]),
    ([requiredFields[:]], {'team': 'test-team'}, ["test5", missingFieldLabel])
])
def test_RunUpdate(missing_fields, updates, labels, mockJiraWrapperResponses):
    ticket = CreateTicket()
    setattr(ticket.fields, fields['labels'], labels)
    reporter = getattr(ticket.fields, fields['reporter'])
    reporter.name = "Bob Smith"
    check = CheckTicket(ticket)

    result_dct = check._RunUpdate(missingFields=missing_fields, updates=updates, _debug=True)

    if missing_fields and missingFieldLabel not in labels:
        expected_result = updates | {"labels": labels + [missingFieldLabel],
                                     "assignee": {'name': reporter.name}}
    elif missing_fields and missingFieldLabel in labels:
        expected_result = updates
    elif not missing_fields and missingFieldLabel in labels:
        expected_result = updates | {"labels": [l for l in labels if l != missingFieldLabel]}
    elif not missing_fields and missingFieldLabel not in labels:
        expected_result = updates
    assert result_dct == expected_result

I'm not actually updating the Jira ticket because that would be slow, and ideally I'm not updating production tickets, or creating new ones each time changes are made.

[delicate basin]

updates and results_dct are the same dict.

[cyan flicker]

No updates | {"labels": [l for l in labels if l != missingFieldLabel]} adds them together.

[delicate basin]

What I'd suggest you actually do is mock JiraWrapper.update

[delicate basin]

Ah, you're right.

[cyan flicker]

I've done that, but this is all it does atm 😢 .

    def mockUpdate(ticket: CreateTicket, **kwargs):
        pass

[delicate basin]

You can use unittest.mock if you want. Should work for this.

Just patch out the call and set the expected call value.

[cyan flicker]

I'd need to think how to approach it. I should be able to also assert in the mockUpdate method all of the pairs I'll get and that they match what Jira expects..

So have the method have a mapping for each key in the update dict

mappings = {'assignee': dict{'name': str}}

Something like that so that it ensure that the datatypes are right 🤔

[delicate basin]

You can pass it the expected result, no?

[cyan flicker]

Sure.. I'm just thinking about how to add more coverage to the test. I only did bare minimum for that test because it was difficult to try and cover everything in the test_RunUpdate method.

So I'd remove the asserts from RunUpdate and move those to the mock method.

[brave vine]

    @patch('app.views.auth.User')
    @patch('app.views.auth.create_access_token')
    def test_user_login(self, token_mock, user_cls_mock):
        user_cls_mock.side_effect = [self.user_mock]
        token_mock.return_value = ['<jwt token>']
        user_cls_mock.query.filter_by.first.return_value = PropertyMock({'email': '[e-mail]'})
        user_cls_mock.check_password.return_value = True
        # When registered and logged in
        p = self.client.post('/auth/login', query_string=self.user_data)
        token_mock.assert_called_once()
        self.user_mock.check_password.assert_called_with('1234')

is testing

@bp.route('/login', methods=['GET', 'POST'])
def login():
    if not request.query_string:
        return {}, 400
    email = request.args.get('email')
    password = request.args.get('password')
    user = User.query.filter_by(email=email).first() // <MagicMock name='User.query.filter_by().first()' id='140148715119040'>
    logger.info(user.email)
    check_pass = user.check_password(password) //<MagicMock name='User.query.filter_by().first().check_password()' id='140148715197536'>
    if not user or not check_pass:
        return 'Wrong username or password', 401
    resp = make_response(redirect(url_for('index.time')), 200)
    token = create_access_token(identity=user.email)
    resp.headers['Authorization'] = 'Bearer %s' % token
    logger.info('Login successful')
    return resp

and leads to this error...

tests/test_auth.py:41 (TestAuth.test_user_login)
__wrapped_mock_method__ = <function NonCallableMock.assert_called_with at 0x7f131f3ef790>
args = (<MagicMock name='mock.check_password' id='139720076236256'>, '1234')
...

My question is how do I mock return values to be able to access email property and why is check_pass not counting as called?

[hexed cloak]

Don't mock database models - just run against a testing database

Also Authorization is a request header not a response header

[brave vine]

How should the back end send the Auth token then?

[astral raft]

I know a bit of using AWS with JS. But now I want to write my AWS code in Python.
I want to follow TDD though. Does anyone know how to write tests to test AWS code?

I'm using pytest

[maiden pawn]

U mean the code like... boto3 stuff that creates AWS objects?

[proper wind]

Hi what is the best book for unit testing?

[astral raft]

Yep

[maiden pawn]

1. Option first: You are writing your business logic isolated from low level code
   The low level code related to AWS is implemented in class inherting from abstract class
   Your business logic code is tied to send actions to this abstract class, which you can easily redeclare as a mock class for testing purposes
   You test your business logic part of code, and you are happy. The tests are fast.

2. Option two: Boto library has special testing means to use their stuff in testing mode. U use them to run the tests without logic separation, or use it to test only the separated class dealing with low level code related to Boto. The tests should be fast.

3. Option three: Your code just raises with your code staging environemnt infrastructure
   against which you are running integration tests that everything is running correctly. The tests are long, but still necesary

[astral raft]

Thank you. Great explanation

[jolly tangle]

Hi I am trying to run a simple pytest and can't get it to work and I think there might be a flaw in conda. Can someone help fix this?

Here is my github https://github.com/NML240/flaskblog2/tree/master .
I accidentally called the side branch master I just have not had a chance to change it.
I also added an empty __init__.py file in my unit folder.

app/tests/functional/test_login.py

# Here is the code I am trying.

assert b'this should come back true'

# I even tried the code below and got the same error though this code is not currently running.

def test_number():

    assert b'this should come back true'

app/tests/unit/test_models

# Here is the code I am trying.

assert 1 == 1

# I even tried the code below and got the same error though this code is not currently running.

def test_number():

    assert 1 == 1

I am getting the error still when I type pytest -m python .
https://pastebin.com/pyD49LGR

When I type conda list I get

conda list:
https://pastebin.com/TZKLk42E

Any help would be greatly appreciated

[royal orbit]

The error message says that your source code contains null bytes - probably you copy-pasted some actual binary data, which contains the invisible invalid character(s). Try:

1. Deleting and manually re-typing the bytestring, or opening the file in a null-aware editor like notepad++
2. Check whether the exact example you posted here has the same effect

[jolly tangle]

@royal orbit

I forgot to add a few things related to the previous post

For some reason the error message is https://pastebin.com/PCNh2eiD and when I click on ........\anaconda3\envs\flaskblog2\lib\importlib_init_.py:

the file opens up https://pastebin.com/9kbLPK8z and I get the error Import "_frozen_importlib" could not be resolved

Should I still just run my flaskblog folder in notepad++ ?

I am confused exactly what to do from your instructions. If I open the code in notepad++ will it automatically fix it in visual studio code? Then I run the code in vsc?

[delicate basin]

How on Earth do people learn TDD? This doesn't seem feasible to self teach well, it's so difficult.

It's been a month or so and I have 1 project that didn't get completely wrecked because of it, and it's because I oversimplified it and had to drop it because it's not useful.

The time spent just researching my confusion on specific things is sufficient to have coded the behavior I wanted to test multiple times over.

I heard TDD isn't supposed to be significantly slower, because it saves debugging time. I actually have some doubts that's true of any project under 300 lines, maybe more.

[proven elm]

Do a big project that requires it

It seems hard to justify for small personal projects

[royal orbit]

It won't automatically fix it for you, but notepad++ will show you "invisible" (non-printing) characters by default.
Or you can try a vs code extension like https://marketplace.visualstudio.com/items?itemName=ShaneRay.InvisibleCharacterVisualizer
This won't fix the problem for you, but should let you see it.

[delicate basin]

Why?

Is this just some observed thing that it isn't useful for personal projects?

[proven elm]

Well if it slows your coding down significantly i will just code then add tests retroactively if it is a personal project....for corp mega project stick to TDD when possible

[delicate basin]

🤔

For learning I'm more concerned first with the actual practice itself.

As long as the time is reasonable.

My issue so far has been that I actually drive myself into a corner, e.g. I'm trying to write a solver for the Codingame Spring Challenge, and now I can't add new functionality without breaking tests, because every iteration requires a behavior change.

I have no idea at what point I messed it up, so I learned nothing useful.

[proven elm]

You learned you can make stuff moar complex than they need to be...drop some tests then make changes

[mellow geyser]

this would point to a code design and architecture smell.
See for instance design patterns or the https://en.wikipedia.org/wiki/Open–closed_principle

One of the goal is to put yourself in a position where you can update your code to support additional behaviors with the least amount of moves. If you find yourself having to change a lot of things, it would point to the fact that you have a lot of tight coupling and interactions.

[royal orbit]

IMHO the lesson here is that "strict" TDD is not actually more productive than sketching out a workable design and then co-developing the implementation and tests.
But different people prefer different ways of working, and it depends a lot on the project and the team 🤔

[knotty shale]

Hi Guys, I'm trying to setup the unittest for a project, but I don't know how to patch the library correctly.

Suppose I have the following 3 files:

################################### lib.py
import sqlite3

def init_database():
    print("connecting to database")
    con = sqlite3.connect('example.db')
    return con
db = init_database()

def calc_sum(a, b):
    return a + b

################################### main.py
import lib

def main():
    print(lib.calc_sum(1, 2))

if __name__ == "__main__":
    main()

################################### test_main.py
import unittest
import mock
from unittest.mock import MagicMock
from unittest.mock import patch

def dummy(*args, **kwargs):
    print("patched dummy")
    return {}


path1 = 'main.lib.init_database'
path2 = 'lib.init_database'
with mock.patch(path1, side_effect=dummy):
    with mock.patch(path2, side_effect=dummy):
        import main

When I run it with python -m unittest, I was hoping for the mock functions to make lib.py not have to connect to the example database.
i.e. I was hoping the output to look like
patched dummy

The actual results were looking something like:
connecting to database

I'm kinda stumped, even after googling/stackoverflow and reading the docs

any help would be appreciated thanks!

[delicate basin]

Where's the assertion?

[delicate basin]

Is the problem here that I tried to change the required behavior in the first place?

Not just extend it, but change it altogether.

Isn't that something important to be able to do in incremental design though? Part of the reason I decided to use TDD for this is that I didn't decide on a solution up front and I want to try a couple and improve gradually.

[knotty shale]

To be more specific, at this stage, I'm just trying to setup the lib.py not to invoke the database connection. I'm not even at the stage of writing the unit tests yet.

[delicate basin]

Then... don't invoke init_database()?

It isn't clear what you want to do.

@knotty shale

Maybe we can move to a help channel.

Nvmnd, you're saying this is setup so you can test it.

One solution would be to pass in the mock object to init_database.

[knotty shale]

that's what I was trying to do with:

path2 = 'lib.init_database'
with mock.patch(path2, side_effect=dummy):
    import main

but I couldn't get the patch to successfully override the init_database function.

[delicate basin]

Why do you need to patch this?

I'm not seeing a scenario where your test cases would care about whether the database was opened or not - ever.

[knotty shale]

In this toy example, as the library is imported, it automatically connects to a local sqlite file (the actual code actual connects to an AWS database).

As you would imagine, I would like to override this behaviour, and not get the library to connect to AWS everytime the lib.py is loaded for unittesting

[delicate basin]

I think? you need to import it before patching it.

[knotty shale]

Hmm.. in this case, it wouldn't work then, as importing the lib auto-connects to the database.

[hexed cloak]

You need to remove the import side effect

[knotty shale]

Ah thanks, @hexed cloak. I had to do a google, but it seems like functions shouldn't be called when they are import, only when they are called.

[hexed cloak]

?

the way mock.patch works is by importing the target and replacing an attribute

So it's impossible to replace a function before it's called if it gets called at import

[jolly tangle]

@royal orbit Is there anything for visual studio code for Invisible Character Visualizer instead of visual studio?

[royal orbit]

Maybe, but your google search is as good as mine here sorry!

[faint anvil]

Is it possible to have a function that runs before each test func in unittest.TestCase? ```py
class TestSomething(unittest.TestCase):
def setUp(self):
self.foo = X()

# is there something like this?
def runBeforeEveryTest(self):
    self.foo.flush()

def test_feature(self):
    self.foo.write(...)
    ...

[agile kestrel]

I believe that's setUp

https://docs.python.org/3/library/unittest.html#unittest.TestCase.setUp

[maiden pawn]

!e

import unittest

class TestSomething(unittest.TestCase):
    def setUp(self):
        print('setting up')

    def test_abc(self):
        print('test1')

    def test_abc2(self):
        print('test2')

if __name__=="__main__":
    unittest.main()

and i confirm it

[bitter wadi]

@maiden pawn :white_check_mark: Your eval job has completed with return code 0.

001 | setting up
002 | test1
003 | .setting up
004 | test2
005 | .
006 | ----------------------------------------------------------------------
007 | Ran 2 tests in 0.000s
008 | 
009 | OK

[faint anvil]

lol, you're right, I've always been treating setUp as setUpClass and needlessly ran tons of things multiple times, thanks

[maiden pawn]

drop unittest, use pytest

it allows quite precise reusage of even multiple setups

with different reusage rules. to reuse for every function, for module, for testing session, whatever

[jolly tangle]

@royal orbit Thank you a ton I am pretty sure I managed to fix it by opening the code in notepad++. I noticed part of pycache file and folder had a path with a space in it. Basically one of my folders had flask code instead it should have been flaskcode. I didn't even know that would cause a error but I decided to test it. Again thank you I could have never solved this without your help.

[royal orbit]

Glad I could help!

[delicate basin]

To follow up on my earlier outburst: I learned 2 things.

• You have to design the tests with the intention that the behavior they test may not be modified (only deleted)
  I was testing my whole system through 1 function, and I fixed the design by making a separate function for each concern.

• You don't realize how much famiality with the language matters until you put it to the test (pun intended)
  I rewrote in Python, in like half the time I spent writing in Go, most of the system's functionality, with way, way, way more tests, and a smarter pathing algorithm.

[slow hill]

Hi folks, is there any guide that you know of about comprehensive test cases? e.g. tips for ensuring that you've covered the necessary corner cases for your problem, etc... So far, I see mostly help materials wrt the technical aspect of implementing tests in certain frameworks, but not these "how to write comprehensive tests in general" type of help.

[maiden pawn]

The general book about testing with telling books for further directions

And at the same time.... writing tests is writing Architecture for your application

for better understanding about what is going on and how tests fit the architecture of your application
better to read

The book mentions stuff to continue reading anyway

• Head First Design patterns book could be helpful as an easier... introduction to the architecturing stuff

[slow hill]

Cool, thanks @maiden pawn , I have Clean Code by Uncle Bob too, and have seen his talks. I do not have either of those books; appreciate the recommendation.

But you're right, I'm sitting here writing tests, and honestly it just has to do with thinking about what the application requirements are really. But people who specify requirements do not always consider all the edge cases of course.

[fierce elk]

At which point you should push back to the requirements people :P

[royal orbit]

This is (part of) why I love property-based testing; you can handle "what should my code do" and "what possible inputs are there" in separate places, and Hypothesis helps a lot with edge cases for the latter.

https://increment.com/testing/in-praise-of-property-based-testing/

[kind meadow]

When using pytest-cov and running pytest --cov test/test_something.py, all my tests run rather than only the one in the module I specify. If I remove --cov then it works as expected (only tests in that file run).

How do I still run with coverage but have it only run tests in the files I specify?

[frigid basalt]

Looking at the documentation, shouldn't you do pytest test/test_something.py --cov=test/test_something.py?

[kind meadow]

Sort of yeah. Thanks

When I do --cov test/test_something.py it parses it equivalent to --cov=test/test_something.py so as far as pytest is concerned, I'm asking it to test everything.

What I actually needed to do was pytest --cov= test/test_something.py

Use --cov= to not do any source filtering and record everything.

[jolly tangle]

When running pytest in flask while using conda I am getting an error werkzeug.utils.ImportStringError: import_string() . Also there is a weird part of the error where it says

failed for 'flask_t d missing init.py in a package; package

or module path not included in sys.path; - duplicated package

or module name taking precedence in sys.path;

https://testdriven.io/blog/flask-pytest/ I also added an __init__.py file to the tests folder. In total I have 3 __init__.py files in the tests folder.

Here is the error I am getting I don't know how to fix this. I have to assume I have the proper __init__.py files.

To see the full error message click on the pastebin.

https://pastebin.com/NjTdf4yH

Here is the relevant code

flaskblog2/app/run.py

# import __init__.py in the userinfo folder from app 
import create_app app = create_app() 
if __name__ == '__main__':      
    app.run(debug=True)     

app/models.py

def __init__ (self ,username: str, hashed_password: str, email: str, confirmation_email: bool, reset_email_password: bool):
        self.username = username
        # needs to be changed
        self.hashed_password = hashed_password
        self.email = email
        self.confirmation_email = confirmation_email
        self.reset_email_password_= reset_email_password

app/tests/functional/test_login/test_login_page.py

from app import create_app
def test_login_page():
    """ 
    GIVEN a Flask application configured for testing
    WHEN the '/login' page requested (GET) 
    THEN check that the response is valid 
    """

 
    # what is 'flask_test.cfg'?
    app = create_app('flask_test.cfg')
    # Use the test_client to check the route which is a get request
    with app.test_client() as test_client: 
        response = test_client.post('/login')
        """
        In Python, the assert statement is used to continue the execute if the given condition evaluates to True. 
        If the assert condition evaluates to False, then it raises the AssertionError exception with the specified error message.
        """
       
        assert response.status_code == 405
        
        # checking username 
        assert b'Flask User management Example' in response.data

app/tests/unit/test_models/test_models.py

from app.models import User 

def test_new_user():
    ''' 
    Given a User model
    When a new user is being logged in
    Check the User database columns
    '''
   
   # username  hashed_password  emai  confirmation_email  reset_email_password 
    user = User('aiohrgihrtg', 'jotpjgjbgt','[e-mail]', True, False)
    # Just for testing ?
    assert user.username == 'aiohrgihrtg'

How do I fix this?

I can upload my entire github if anyone wants

Thanks

[lofty wren]

Hi folks! I'm getting started with pytest and playwright for work.
I have a dumb question. I want to set some variables at the beginning of the test run and refer to them. I am doing that at the beginning of conftest.py. At some point, I saw an example of setting variables inside the pytest namespace, i.e. pytest.username = "myuser". Is that a standard practice, or am I better off just using a global of my own?

[royal orbit]

Mutating other people's modules is generally a bad idea, so I'd recommend using a global of your own.

[spark zenith]

maybe show us the example you found. pytest does a looooooooot of magic

[wary moat]

so I did manual unit testing for years with pretty simplistic c code. Now, I'm developing unit tests for python and it's almost an entirely different world, especially from the OOP perspective. What are the best resources that you've all used to practice?

[frigid basalt]

The error is somewhat correct I guess? It's trying to import flask_test but you mention no such file.

[maiden pawn]

original_function = obj.query.names_to_path
obj.query.names_to_path = MagicMock()
obj.query.add_ordering(*field_names)
obj.query.names_to_path = original_function

how correctly to replace this construction with something like next thing?

with patch.object(obj.query, 'names_to_path'):
    obj.query.add_ordering(*field_names)

Nvm, fixed.

[muted furnace]

anyone could explain, is unittesting necessary?

[maiden pawn]

if you are writing anything in Python of more than 10 code lines? Then answer is yes. Absolutely necessary and must have to have. and pytest will serve better than default unittest

[hexed cloak]

I've seen a few test codebases that do pytest.something = whatever

I'm not sure where that "pattern" came from

[jolly tangle]

@frigid basalt if you look at this link they don't show you need to import flask_test .
Here is the link https://testdriven.io/blog/flask-pytest/ and how do I import flask_test ?

My structure of my app is similar to this https://github.com/CoreyMSchafer/code_snippets/tree/master/Python/Flask_Blog/11-Blueprints

I just realized the tutorial has a gitlab linked let me check it I will let you know if it does not work.

[digital sigil]

hi, at work we are sending emails through sendgrid. should i test its implementation? and how? i dont know if i have to mock its response or what

[frigid basalt]

Ah, no I don't think you need to import flask_test. What I did was read the error out for you: for some reason there's some import of flask_test during testing causing it to fail.