okay!😅
#unit-testing
stable tinsel
self.__test = 1
private
that i want is change the value of self.__test
outside TestVar
a = t.get_test()
a = 6
i want this, make self.__test value is 6
nor 1
stable tinsel
python 3.8
spice stag
okay then, go for it
stable tinsel
-.-
visual sentinel
Let's say you have 11 edge cases to test a couple of lines regex function.
Would you really make 11 unittest methods? Is it bad practice to just stick them all in a list, and iterate over it in one method?
marsh raft
do whatever seems easiest to understand
write it one way or the other, show it to someone who knows python; ask them if it's clear
i.e., have a code review 🙂
visual sentinel
Ah I don't know anyone, but thanks for tips.
marsh raft
well there's this thing called "discord"
visual sentinel
haha. ok 🙂
tawdry dew
class TestVar:
def __init__(self):
self.test = None
def get_test(self):
return self.test
def set_test(self, value):
self.test = value
if __name__ == '__main__':
t = TestVar()
print(t.get_test())
# None
t.set_test(6)
print(t.get_test())
# 6
rugged schooner
Hi, i am a newcomer to python, having some trouble unit testing my functions,
Namely a function foo() that takes as argument the request lib, i would expect that in my unit tests, i would be able to replace requests with a unittest.mock function as to not perform real requests. Is this the common way of doing testing utilizing dependency injection in python?
marsh raft
not sure
I don't know what you mean by foo() that takes as argument the request lib
rule of thumb: if your function takes an argument that's some complex, hard-to-mock thing like a requests session, or even a file handle, then don't make that function have any complex logic; insead move the logic into a smaller function that takes simple python data types like strings, numbers, dicts, and lists
unit-test that function
then the outer function, you only need to see it run a couple of times to be sure it works
weak spoke
hi, me again
remember workman? we have an update
aliases, they shorten commands to ```workman````
rugged schooner
Hey all, is there a better way to dinamically set a mocks return value based on input args, than using this clunky func?
def mock_side_effect(*args, **kwargs):
if args[0] == 'foo':
return 1
elif args[0] == 'bar':
return 2
def test():
mock = Mock()
mock.side_effect = mock_side_effect
Basically i am looking for something like
def test():
mock = Mock()
mock.called_with('foo').return(1)
mock.called_with('bar').return(2)
verbal timber
Table Printer
Write a function named printTable() that takes a list of lists of strings
and displays it in a well-organized table with each column right-justified.
Assume that all the inner lists will contain the same number of strings.
For example, the value could look like this:
tableData = [['apples', 'oranges', 'cherries', 'banana'],
['Alice', 'Bob', 'Carol', 'David'],
['dogs', 'cats', 'moose', 'goose']]
Manipulating Strings 143
Your printTable() function would print the following:
apples Alice dogs
oranges Bob cats
cherries Carol moose
banana David goose
Hint: Your code will first have to find the longest string in each of the
inner lists so that the whole column can be wide enough to fit all the strings.
You can store the maximum width of each column as a list of integers. The
printTable() function can begin with colWidths = [0] * len(tableData), which
will create a list containing the same number of 0 values as the number
of inner lists in tableData. That way, colWidths[0] can store the width of the
longest string in tableData[0], colWidths[1] can store the width of the longest string in tableData[1], and so on. You can then find the largest value in
the colWidths list to find out what integer width to pass to the rjust() string
method.
help me solve this
proper wind
https://bpaste.net/OT2A is anyone here able to help me figure out why I am getting an “int is not iterable” error on my ATM program that I am beginning to work on?
kind meadow
That is off topic for this channel. This channel is for testing.
Ask in a help channel, which can be claimed at the top
proper wind
ok, thank you very much
tender vapor
hi, i am unble to test the structure that google translate api returns, does anyone have an experiance with that? https://gitlab.com/RhinoFlip/google-api-translate
long story short, Mock() is not iterable, 'property' cannot be iterable. the api returns some structure which has languages property, which is iterable, and each language in there has property language_code. I need the mock that would consider all of it.
kind meadow
You are free to modify the mock
You can define the methods for it to make it iterable
Subclass the mock if you prefer
autumn egret
This might have been asked before, but what would be a good way of writing a unit test that tests that code still works as expected after a python2 to python3 code conversion? Meaning I've converted some scripts to work with python3 and now I have to test that all of the scripts I've converted still function as normal
marsh raft
ordinary unit tests should do that
nothing special about porting from python2 to python3 afaict
either your code works, and the tests pass; or it doesn't, and they fail
autumn egret
Thanks for the help!
inland sleet
Im getting an error in pytest that says
PytestCollectionWarning: cannot collect test class 'TestAccount' because it has a __init__ constructor (from:/my/test/dir)
class TestAccount:
-- Docs: https://docs.pytest.org/en/latest/warnings.html
Ive read the docs but im a bit confused, what am I doing wrong?
My TestAccount has a __init__ method but its needed to initialize a pytest fixture im using
visual sentinel
Can't seem to mock datetime.date.today()
https://paste.pythondiscord.com/recupalajo.py
TypeError: can't set attributes of built-in/extension type 'datetime.date'
I understand this is because it's written in C/is immutable.
Is there a way I can do this?
marsh raft
try the freezegun library
inland sleet
@marsh raft hope im not bugging you but just wanted to see if you have any feedback regarding my post above (related to cannot collect test class 'TestAccount')
Thanks in advance,
tight geyser
because it has a __init__ constructor
maybe remove the init constructor then
visual sentinel
Thanks offby1.
hybrid solar
how do you make python exit the program?
whole pawn
exit() exists
hybrid solar
is there a command that restarts the entire program?
hybrid solar
is there?
verbal mesa
You should not use exit() or quit() really. Better is sys.exit or structure your code so that it will exit when complete
hybrid solar
what about a command to restart the program?
verbal mesa
Do you want to restart when the program ends, or do you want to start from the beginning instead of ending?
For the latter option you can achieve this with a loop, a very simple example would be
def my_program():
...
for _ in range(3): # loop 3 times
my_program()
while True: # loop forever
my_program()```For the former you'd need some other program to monitor the one you want to run, when it ends you have it start your program again
proper wind
Hey guys i'm interested in taking on the ISTQB certificate in the near future and i'm doing some excersises and would appreciate if someone could correct me if wrongly answered? What is confusing are the bulletins stating "There is no need to test month-day and leap year combinations"
My answers below:
1a. The test conditions are
C1: 1 ≤ month ≤ 12
C2: 1 ≤ day ≤ 31
C3: 0000 ≤ year ≤9999
1b. I don't get the concept of this one?
1.c Not all requirements, what about february, 28 days?
1d. In terms of features, they mean black-box testing?
tired ermine
wow, the bulletins are completely dumb. not only not accounting for febuary or such, but also year in 0000-9999?
That is asking for typos
so yeah, c:
all months with less than 28 days
and all years that are in future/way back in past
b and d - dunno
classic stupid textbook examples
proper wind
hahha
lone otter
Do you guys test with sqlite or do you use the same db you use in production? Just gathering thoughts (I use sqlite in test and postgres in prod)
obtuse sequoia
i use sqlite for light testing
fiery cove
I use Postgres for testing and in production. I've never used sqlite
balmy shell
☝️ +1 for using postgres in test and prod, especially if you have any queries or functions that are specifically suited for postgres
fervent radish
Posted a few pytest questions in #help-burrito
ancient spear
@woven vortex We do not allow advertising in the server, you can read about it here
!rules 6
bitter wadiBOT
kind meadow
I'm trying to make unittest.mock.patch.multiple pass the mocks to the decorated function as positional arguments rather than keyword arguments.
Has anyone heard of this being done before?
This is the code responsible for it https://github.com/python/cpython/blob/3.8/Lib/unittest/mock.py#L1305
But it isn't as simple as setting the attribute_name to None
The way patch.multiple works is it has one root patcher and the rest of the patchers are added to the root patcher as a sequence in the addtional_patchers attribute.
The patcher returns the additional mock objects as a dictionary if they're present https://github.com/python/cpython/blob/3.8/Lib/unittest/mock.py#L1495
The return on line 1499 is what line 1304 receives
Notice on 1496 it enters the context for each additional patcher. If I manually remove the attribute_name from the additional patchers, then this operation will return just the mock. Therefore, the update on 1498 will fail cause it expects an iterable of tuples.
Even if that did work, decoration_helper would have to extend the extra_args list rather than append since there could be multiple mocks returned.
So with that background, what's a good way to accomplish what I want?
I don't mind creating a utility function to wrap patch.multiple
One idea I have is to ditch patch.multiple and somehow decorate the function myself with multiple patches.
kind meadow
Yup that works. Don't know why I had to try to go for the the more complicated route
def autospec(target, *attributes: str, **kwargs) -> unittest.mock._patch:
"""Patch multiple `attributes` of a `target` with autospecced mocks and `spec_set` as True."""
# Caller's kwargs should take priority and overwrite the defaults.
kwargs = {'spec_set': True, 'autospec': True, **kwargs}
# Import the target if it's a string.
# This is to support both object and string targets like patch.multiple.
if type(target) is str:
target = unittest.mock._importer(target)
def decorator(func):
for attribute in attributes:
patcher = unittest.mock.patch.object(target, attribute, **kwargs)
func = patcher(func)
return func
return decorator
little magnet
can someone help me, my pip isnt working as im trying to get pyinstaller to convert py into exe but it doesnt work
gray mortar
Googled how to read the first line of a text file and this works perfectly, I just don't understand how it works
with open('notes.txt',"r") as f:
wordlist = [r.split()[0] for r in f]```
where does ``r`` come from?How is that related to software testing
Are you sure you don't want to instead ask via #❓｜how-to-get-help
plush vale
I'm confused about assertLogs https://docs.python.org/3/library/unittest.html#unittest.TestCase.assertLogs
it fails the test case whenever the context exits and no log was made
is there no way to use this to assert that no log was made?
in my case I'd like to write a test case runner which is parametrized by n_logs: int to check whether the expected amount of logs was made at warning level and worse
but it seems like there's no way to have this as the test will autofail if no logs are made (so having n_logs=0 is pointless)
looks like there actually is no way to do that https://bugs.python.org/issue39385
that's odd, why add assertNoLogs when it could just not autofail and let you fail the case based on the log recorder it returns
kind meadow
Mock the logger and assert no calls are made :D
Well no you'd need extra logic
Nvm
I'm a bit suspicious of why you need to test logs like this
plush vale
I have a deco that wraps a getter and if the getter's bound instance isn't ready yet it will abort the getter, log a warning and return None
I of course don't need to test this via the logs, but I'm confused about assertLogs's design
kind meadow
Yes it's strange
Maybe something with mocking will work if you really want to have a variable count
Like checking length of mock_calls
proud nebula
If you are doing pytest-style tests for smaller projects, check out my new project hammett: https://github.com/boxed/hammett It's still a bit early but it's a pytest-compatible test runner that has WAY lower overhead (proof: https://github.com/boxed/test-benchmarks)
rich eagle
Hi guys is it possible in selenium to make driver.get method like this :
driver.get(x)
?
split field
@proud nebula - is the plan to eventually have close to 100% feature parity with pytest, or will it always support a subset?
proud nebula
@split field Always a subset of pytest. Part of the reason pytest is slow is precisely because of the features. But I am working on some new features that will be unique to hammett that can make tests much nicer to work with and also even faster because I will be able to know which tests to run, while pytest just runs everything. This is also an explicit goal because I need way more speed for mutation testing (I am also the author of mutmut). This is why I started with hammett actually but since then I have increased my hubris a bit :P
proud nebula
Worth noting is that most people don't even use 1% of the features of pytest. It's hugely complicated. I know because I made hammett after giving up on fixing pytest. I have contributed some fixes though, but it's super hard to fix something that complex after a thousand small performance regressions have been introduced.
I also think there needs to be more of a re evaluation of how we test in python. Considering how huge python is it's crazy that there really only is unittest and pytest (and maybe nose2) and they don't have different ideas really.
winter dragon
Hey folks, does anyone have any best practices when it comes to testing with deep learning (e.g PyTorch), while keeping them fast?
kind meadow
Can you elaborate on what you mean by re-evaluating testing? Do you mean testing methodologies? In what way?
proud nebula
@kind meadow I mean, we should try different ideas to try to find something better. We should also look at what other languages/frameworks have and see what good ideas we can find there. For example I have learned that some PHP test runners have a way to mark a test as only giving coverage to a specific function, instead of polluting the coverage data over everything it touches indirectly.
In rust tests are written in the same file as the functions that are being tested. This means functions and their tests, that clearly belong together, are also together. This is very nice because then if I change function X I can run the relevant tests easily: just run the tests for the file!
In pytest, unittest, nose/nose2, there is one big pile of tests over here, and one big pile of code over there, and who knows what relates to what!
plush vale
Following the paradigm of structuring test packages such that they replicate the source code's structure, i.e. core/module.py is tested in tests/core/test_module.py, does a test for a function defined in a pkg's __init__.py go into the respective test package __init__.py?
i.e. core/__init__.py will be tested in tests/core/__init__.py
because that breaks the convention of prepending the modules with test_
but I don't think I could handle test___init__.py
marsh raft
I wouldn't be quite that dogmatic. I'd call it test_init.py and let the humans figure it out 🙂
@plush vale 
plush vale
yeah, that's an option
I may end up doing that, put the test in the __init__.py initally but people probably wouldn't expect a test in there
marsh raft
honestly I wouldn't either, mostly because I wouldn't expect your regular __init__.py to contain anything complex enough to need testing
there's no law against it, but generally people only put imports in there
remember, the contents of that file run when you import the package that it defines, and it's evil to do anything fancy at import time
[[I imagine there are some cases where that's the only possibility, but those cases are rare]]
proper wind
im testing a program to joke a friend. anyone can help? it only runs on windows
you can run it on a vm if you want
plush vale
@marsh raft It does a bunch of imports and provides 2 utility functions operating on the imported classes
there's no complicated logic that'd really require testing, but if it's the only thing separating you from 100% coverage, might as well
@proper wind I'm not sure what your program does but please make sure it doesn't violate rule 5 before you discuss it
!rule 5
bitter wadiBOT
marsh raft
@plush vale ah, it's slightly unusual to define functions in __init__.py too. Why not define them in regular modules, and then (if you want) import and re-export them from init?
proper wind
it doesn't break rules
plush vale
It's also an option, but I don't think it warrants a separate module
it just makes a lot of sense to have them defined there, in my opinion
marsh raft
now you have to balance two unpleasant options -- extra probably-not-needed module, vs slightly-unconventional init
plush vale
ah, you're right
marsh raft
I think this sort of issue is more important than most people think. Sure, your program will run fine regardless of how you choose here; but your choice will affect readability, and that is important.
plush vale
no I agree with you
I like having these kinds of discussions, because you never consider everything
it helps you see how others approach these kinds of situations, and reconsider own views
not aimed at "you" personally, but an abstract you, lol
proud nebula
In hammett I have the pattern of module__tests.py so they end up lexiographically next to the file that it tests, which is super nice. And for __init__.py, you'd have __init__tests.py so it also ends up next to the file.
Also, in hammett I can't just let humans figure out because I have features where hammett keeps track of file changes and reruns the corresponding tests module. Then you do need to match it exactly.
plush vale
right, but using a test_ prefix also maintains lexicographical order, right? and if you define something that is not a test, it will be clearly separated
so how does hammett figure out that the test is named __init__tests.py and not __init____tests.py?
(difference in amount of underscores)
it's the consistency vs allure question that originally sparked my thoughts on this
oh, do you mean that in hammett the pattern is to define the test module next to the tested module?
not in a separate package
proud nebula
Well, it has lexiographic order where tests are far removed from the thing they are testing. I don't think that's a good idea!
exactly yes
for __init__ specifically I think I'll just implement it as supporting both __init__tests.py and __init____tests.py (and with 3 _ too) by just checking prefix and suffix and not caring if they intersect
With the current master of hammett, it will only run the corresponding module if you change the module. It does depend on what your testing philosophy is, and I have an idea of how to get the awesome speed of doing that tight coupling between module and its test module, and the completeness of the entire suite. Watch this space :P
(it will run the tests if you change the tests too obviously)
ember maple
Aloha
hollow basin
Is there a way to have a warning in a unit test instead of an assertion? I have some code that isn't guaranteed to output anything, but if it doesn't, I want to know.
So I don't want that test failing to be treated the same way.
ember maple
In pytest just use a python warning
hollow basin
hmm thanks
proud nebula
Ah. I don't think I support that in hammett. I'll put it on my todo
ember maple
@proud nebula i hope to move things like assertion rewrites, exception details, warning/loggin/stdio capture into a shareable and cythonizable library
split field
@proud nebula I gave hammett a (very) quick try. A lot of the tests for my current project failed with it. Lack of the monkeypatch fixture, of the caplog fixture, and of the match() method on ExceptionInfo appeared to be responsible for most of the failures.
proud nebula
@ember maple I personally am starting to believe assertion rewriting might be just a misfeature. Try hammett and see how it acts with complex asserts. It does not have assertion rewriting and (imo) has better exception feedback for many cases and I have just baaaarely started working on hammett while pytest has got what? decades? behind it.
@split field Seems like pretty straight forward things to add. I think I looked at the monkeypatch thing and thought that I should just lift the pytest code straight off, but then I got hesitant about licenses and attribution. It should be ok, but I always get an icky feeling when I have to even think about that stuff...
ember maple
@proud nebula assertion rewriting is not a missfeature ^^
proud nebula
Well, the UX of what it enables isn't. But the methodology to get to it, I'm not sure
ember maple
also assertion rewriting is not that old, originally it was reinterpretation, which triggered bad stuff
it shold be a builtin to the language, but it isnt
proud nebula
oh, interesting! I am doing reinterpretation in hammett
yea, it can be a problem if you have side effecty stuff in the assert line
but the cost for the feature is big
and you pay that cost when tests succeed!
ember maple
reinterpretation was coming with caveats and lots of breakages for people, it was a misfeature that triggered the introduction of assertion rewriting
but these days i have a number of suites that need neither
ideall case for me would be a feature library that can reinterpret, rewrite and nil
@proud nebula so for me the ideal case is to have assertion annotation features handled by a library and eventually PEP-ify it to have it as a language feature
proud nebula
what would it look like in a library? I can see how the language can do something but not library...
the reinterpret I do is that I go back to the stack frame of the exception, load the AST of the relevant assertion line and re-execute the left and right side of the assert
it's suuuuper fast and you have zero cost in the success case
ember maple
the library would provide ast reexecute, ast rewrite, and annotated assertions
annotated assertions, because if your assertion helper can give you the annotation, you dont need rewrite or reinterpret
proud nebula
Imo it's more reasonable to have the runner re-run the test with assertion rewriting in the fail case if it deems the reexecute isn't safe. It doesn't matter hugely that the test is slow in the fail case I think.
ember maple
that still need a provider for rewriting
proud nebula
sure, absolutely
ember maple
the library is supposed to provide the different modes of assertion annotation,
aka annotated eception, reinterpretation and rewrite
the test framework than then orchestrate it
proud nebula
I'm just saying that assertion rewriting on all tests all the time in the success case is not good for performance reasons.
hmm.. yea well, better if CPython is changed to store all the intermediates as a local variable __assert_parts
(a list)
ember maple
that should be a anotation on the assertionerror
proud nebula
by annotation you mean a member?
there are advantages to magic local variables, for debugging and crash reporting but yea, they should be handling the AssertionError specially in that case too I guess
ember maple
its for debugging clarity - pytest assertion rewriting currently creates a number of non-accessible variable names prefixed with @pytest
i would like to see the assertion epxression running in a new namespace thats catching all the intermediates, then passing that unnamed variable to the assertion
@proud nebula btw, pytest assertion rewriting has no repeat cost on unchanged files (so if the tests dont change, then the cached precompiled pyc for the test is used
(its import system integrated)
proud nebula
ah, didn't know about the pyc thing... I'm not 100% sure I accept that performance hit anyway but I'd have to try it in hammett where other things are fast to form a evidence based opinion :P
I would love this to be changed in CPython but I just am extremely disillusioned about getting any change in there
ember maple
if its a working library wit a example and used/prepared for mutliple test frameworks, writing a PEP is not that bad
o have other things i want to get into python that need a wroking example first as well
(like package repositories for reuse)
proud nebula
oh, you have a link?
ember maple
thats unimplemented, there is only https://discuss.python.org/t/proposal-sharing-distrbution-installations-in-general/2524
proud nebula
I have this change waiting forever and it's super trivial: https://github.com/python/cpython/pull/9655 I don't see how I'd get something really useful into cpython :P
ah sorry, I misread above
Improvements to pip are very welcome. This part of pythons eco system is super bad.
That was a big part of my latest rant (https://kodare.net/2020/05/19/python-is-slow-does-not-have-to-be.html)
getting back to assertions.. I don't see how you can really implement this nicely in a library.. it seems I'm not understanding what you mean
ember maple
well, test frameworks obviousl have to do a little glue to orchestrate, but the bulk is rewriting, module caching, reinterpreting and some facade work
so putting that into a library is natural
ember maple
@proud nebula from m pov the major part of rewriting/reinterpreting is support code to implement the features/results, the minor part is integration/explanation
proud nebula
true enough. If the stdlib kept intermediates in the AssertionError everything else would just be a tiny bit of code in the test runner
ember maple
yes, the base idea for the assertion support would be to have the big chunks availiable to other users
also - sanitized assertion reinterpretation if advisable (aka identify mocks/properties and dont walk them)
drifting basin
Anyone here familiar with behave? If so, how are ya'll handling organizing the steps directory? It is a mess in my current project. When I took the project over, it was believed that there should be a 1-to-1 mapping between .feature files and the steps. But as I was actually learning it, it doesn't really seem like a scalable way to handle it honestly.
spare junco
Hi everybody. I do not know if this is the correct chat for my doubt. I'm reading aa bunch of files and while iteartes over each line I print a line (as figure shows) but for a strange reason my script get frezee at the middle of the print. Is this possible? I'm using jupyter on Linux.
The red line indicates the lines that is printed on a single print statment. I cannot find the error. Some help please.
spare junco
I think I found a solution. Flush on the input file.
proud nebula
@drifting basin I have been under the impression that BDD is snake oil. It's more code to do less just to produce reports no one reads that you aren't even sure aren't lying. Scaling and isolating parts of tests in a code base is hard enough as is.
drifting basin
I have no qualms with it personally, but I'm not the one who gets to make these decisions, so how my thoughts are irrelevant honestly.
Testing is testing at the end of the day.
ember maple
i consider "bdd" a worthwile addition to keep the plubing described and larger acceptance tests comprehensible,
but i would like to see better modeling tools to express systems under test (black box testing applications with multiple areas and hundrets of use-cases)
slim burrow
@spare junco what is that program?
thick sinew
!e ```py
for t in range(int(input())):
a, b = list(map(int, input().split()))
arr = list(map(int, input().split()))
count = 0
for i in range(len(arr)):
if arr[i] == b:
ok = True
have_num = [arr[n] for n in range(i, min(i + b, len(arr)))]
for j in range(len(have_num) - 1):
if have_num[j] - 1 == have_num[j + 1]:
continue
else:
ok = False
break
if ok:
count += 1
print(f'Case #{t + 1}: {count}')
You are not allowed to use that command here. Please use the #bot-commands channel instead.
ember maple
ah finally - setuptools_scm 4 is off my plate, time to set up COT
ember maple
hmm, anyone aware of a fuser aliek api that checks the processes of the current user?
or lsof for that matter
ember maple
@proud nebula i started workin on https://github.com/cogs-of-testing/cot-tmpdir - not sure if i can get the extracted pieces to a working state today
ember maple
@proud nebula btw, are you aware of the cost of pathlib vs string paths in detail?
proud nebula
Haven't looked at it really. I'm assuming it's more expensive or equal. Probably f-strings are faster than join too but looks icky if you care about windows.
ember maple
gm
ember maple
@proud nebula im contemplating going down the string route and only have Path & co at the outside facade if requested
proud nebula
In [3]: %timeit Path('foo') / 'bar' / 'baz'
18.7 µs ± 486 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
In [4]: %timeit join('foo', 'bar', 'baz')
2.64 µs ± 40.8 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
of course those aren't totally equivalent, since you do get a Path object at the end for the first one
In [5]: %timeit Path(join('foo', 'bar', 'baz'))
12 µs ± 668 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
I guess because path gets three components and thus can do something fancy, while Path does operator overloading and just gets one string twice plus the original one and creates an object every time
those are some pretty damn small numbers though, so not sure it'd matter unless you're crawling a big filesystem or something
ember maple
the make_numbered_dir handling is a bit expensive atm, but i think i can get rid of most of the expense
hmm
actually
i have some ideas, i wont need to drop path, but i will end up with some fun
but more of the details after my workday
proud nebula
Yea, if you're going to the filesystem then the use of Path seems like it'd be hard to even measure... but that's also the logic that lead to pytest being dog slow :P
ember maple
@proud nebula pytest being dog-slow is pretty much part of how it grew features and extensibility
proud nebula
Partly.. but partly it's because of py path and weird stuff that now become hard compatibility breaks
And mostly not looking to see what the performance is, that's the core
ember maple
@proud nebula py.path will go away remiving it will be part of the larger Node.from_parent refactoring line
unfortunately the timeline is somewhat between 1 and 5 years with lots of uncertainity
proud nebula
I just don't have that kind of patience :P To me it seems like it's easier and more worth while to start over and try new fresh ideas
I think we can make radical improvements to how good testing is by rethinking some basics.
ember maple
@proud nebula thats why i startet cot - its so much easier to start new when you have stuff like tmpdir lifecycles, assertion integration - collection basics, sorted out in small effective libs that are easy to use and to replace
proud nebula
Agreed.
ember maple
one of the problems pytest has is that it grew and grew and grew and grew
and thats also sometihng that will invariably happen to any tool that doesnt take care of that ^^
proud nebula
Some stuff probably can't be clean libraries and does need vertical integration to be really good though. Like test collection, that is already fundamentally different in hammett compared to pytest.
I guess one could squint and say pytest and hammett are compatible with this... but only if pytest throws data on the ground
ember maple
@proud nebula there are lots ofthings one can manage together, like properties, safe attribute getting, special casing mock librariers, correct/safe mro walking
why does every test collector have to reinvent not breaking over global mock objects, same goes for various other scanning activities - for test packages/directory trees, importation of test files based on configuration
its trivial to reinvent many of those, but typically its incorrect and then has to catcha few cludges
proud nebula
A stronger focus on clean APIs will really be nice yea... I have been burned by pytests lack of one when doing mutmut.
ember maple
clean apis are hard for things that take many iterations to emerge tho
fixtures where vastly different in the beginning
but some things are also just plain gone bad transmorgriafictions
@proud nebula behind every really strange api there is likely a major refactoring that was skipped for a kludge instead since else some things would have taken months/years instead of the days that where availiable
ember maple
@proud nebula im still against starting fresh, instead i want to compartmentalize, so remixing becomes an actual option
else the new how stuff is just doomed to repeat mistakes that others ironed out years before, but did so strangely due to legacy constraints
proud nebula
Sure, I understand. Maybe it's more important to salvage the test suite though. Because that should not carry too much baggage.
In the last few years I have started to see the problem of just refactoring, after we developed (what is now) iommi, and porting views to that is always faster by just throwing away the old and doing it in the nice new thing. One can look at the old GUI, but looking at the code just slows you down.
My experience trying to track down and fix performance issues in pytest felt the same way: a gigantic castle of complexity that wasn't tractable itself and then you had the threat of breaking backwards compatibility with the plugins hanging over you too.
I am not smart enough to deal with that. I have a small brain, a short attention span, and as my colleague says (with fondness) I'm too angry :P
I gave up on refactoring Cosmic Ray and MutPy, and here we are with mutmut today which is just way better.
hollow basin
Here's an interesting problem I'm now facing
I'm working on a project in which I subclassed a number of classes I wrote for another library
Now I want to make sure that they not only perform the added functionality, but that they are Liskov-complete, or whatever you want to call that concept.
I'm wondering if there's an easy way automatically run tests written in unittest, applied towards a subclass.
proud nebula
Sure. Monkey patch the module to put your class there and then run the tests.
cedar aurora
Hi, recently i have been trying to implement a few automated tests for my code base
import unittest
from src.utils.games.slotmachine import Slots
class TestSlotmachine(unittest.TestCase):
def play_round(self) -> None:
slots = Slots()
slots.play_round()
self.assertIsNotNone(slots.win)
self.assertIs(slots.win, int)
if __name__ == '__main__':
unittest.main()
this is supposed to test a class located in the src directory
project
|-- src
| |-- utils
| |-- games
| |-- slotmachine.py
|-- tests
|- test.py
but when running it this will show up and no tests will be ran at all, help as to how this can be solved or why this is happening would be appreciated.
$ python -m unittest tests/slot_machine.py
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK
keen crystal
@cedar aurora Test methods need to be named test_*
Try calling it test_play_round(self) instead.
cedar aurora
Thanks, that works indeed.
jade depot
not sure if this is the right area to post this, however im running into an issue with import modules on raspbian. when i try import my twython module i receive the error cannot import twython from twython. it wont let me upgrade either. sorry if this isnt the right chat to post
kind meadow
No, this is not the right channel. See #❓｜how-to-get-help
ember maple
@proud nebula i can feel that sentiment - at certain levels of complexity noone is able to refactor, hence im breaking it up over time, slowly moving it to smaller replacable things that are comprehensible and can be used by completely different remixers
@proud nebula its foe example why i introduced Node.from_parent in pytest - its a initial step that when layering a construction protocol on top, will allow to refactor the constructors of various node-types
proud nebula
hmm.. actually, are you extracting and purifying useful parts or are you extracting them while refactoring pytest to use those parts?
ember maple
@proud nebula refactoring pytest while extracting
the firs thing i take out is the tmpdir lifecycle management, then i make pytest use it
then something like exception repressentation or assertion rewriting
@proud nebula that serves multiple practical needs a) if it has users, whatever i take out is at least practical (i have no desire to do cleanromm architecture research, i want to take usefull bits out of pytest and then get other testrunners to use/integrate with them while they are also being used by pytest
in essence enabling remixing, where you can start with all the common usefull bits, but replace those you need different/cant reuse
proud nebula
I'm just afraid you'll still end up with something that is a big gnarly and too pytest specific
But I guess we'll find out :P
I'm also worrying that you'll spend too much time on it, compared to making a clean break where you can move much faster and make bigger improvements.
ember maple
my goal is to make pytest better, not to thow another bus-factor test framework with new standards into the game
from my pov the best way to do that is to distill good things out of pytest and making them accessible for others
@proud nebula one key part of why i want to rope you into it is that you make test tools/framework with very specific goals that are in a sweet spot at odds with how pytest traditionally moved, so if my stuff can be used effectively by your stuff and pytest, i likely didn't fail ^^
ember maple
and then its just a question of getting ward, behave and some others in there as well
proud nebula
huh, never heard about ward before
ember maple
it got some interesting things, and some that i rather dislike
proud nebula
That there's no smooth transition from unittest or pytest seems like a biggie
Using _ for the function name seems to not be popular with PyCharm..
ember maple
yeah, it needs more/different support
proud nebula
Doesn't seem like it really has an idea of improving the paradigm of one big random pile of tests :( I guess I'm the only person who sees the problem
ember maple
what dou you mean by that, i have some ideas around structure/reuse/configration of partial testuites
proud nebula
in hammett if you change foo.py then it knows to run the tests in foo__tests.py, and not be stupid about it and run all tests
running all tests for all changes always is not a scalable idea
ember maple
yeah, i work more with integration/system tests, so its much harder to tell and i'd rather have coverage inform what tests to run on code change
there is some pytest plugins that do something like that, but its rather experimental i would say
proud nebula
I wrote one :P
coverage only prunes the tree, it's still going to catch way too much
like, take the most extreme example of putting all the python projects in the world in one repo including the standard library. Now change the implementation of str.split()... using coverage you'd run pretty much all tests
the "solution" to module isolation is right now to break everything into micro-repos, which is terrible in another way
ember maple
@proud nebula the trick here thats missing from that is granularity identification
if a unit changes, you ideally test only that unit, but it may be necessary to also test things layerd on top
imagine hcanging a class, its uniitests pass, but by accident a subclass that wasnt propperly isolated breaks, and its test break
if you pull in the tests too narrow, you miss that
proud nebula
yea.. there's a difference between "these are the tests for this module" and "these are the tests that might break if you change this module"
the plan for hammett is to run the first category and then if that passes run the second
ember maple
now thats a plan i like
proud nebula
In the watching scenario we'd run the first category until completion always, but the second category we'd abort if the code changes on disk and start category 1 again
I would like some good nomenclature for this! It's like "the tests" and the "just make sure tests"... that's not good names :P
"testing" and "make sure I didn't break anything by mistake testing"? heh
We also contemplating supplying an "order" of tests, where we'd run category 1, and then we skip the tests for modules that are "before" because they'd be logically unaffected.. but it's unclear right now how to differentiate between "depends on" and "run after" in a way that isn't extremely verbose and annoying
It'd be nice if we had a tool to analyze dependencies that was super quick (presumably incremental) so we could just extract that from the code
ember maple
for me thats a later item tho ^^
atm my topics are tmpdirs, exception repressentaion/details, assertions and then the pytest internal nodetree structure (which will be followed up by cot.gathering)
(imagine a testrunner mode than already knows the filenames, and spawns multple processes to do the different deeds as quick as possible + triggers recompile of pyc/assertion annotaded pyc on file safe, so test running would have optimal time and collection of integration tests would happen while the unittests run)
proud nebula
Yea, we are approaching the problem from different angles but clearly we agree on the goal every time we redefine it after a new idea comes up.
Multi-process is also something I'd really like to tackle in hammett.. but I don't want to end up like pytest-xdist where the advantage is swallowed ten times over by the overhead for small test suites
Seems like Darren (the Ward guy) would be nice to have in this conversion also...
ember maple
Good idea, i just didn't think of him because unlike you he doesn't interact with pytest
proud nebula
I fired off an email to him
ember maple
neat, am i cc?
proud nebula
No.. I mentioned you, said I was working on hammett and invited him here, that's pretty much it.
proud nebula
I'll try to keep the idea of cc in mind. To me cc is something assholes in corporations do :P
proper wind
I need help but I'm not sure if this is the right place
I have a python program that when I make it into exe file it does nothing
proud nebula
It's not the right place. I don't really know which channel is suitable. Maybe ask in #python-discussion
lofty tangle
Hello! Thanks for pointing me here @proud nebula
ember maple
@lofty tangle hi, are you behind ward?
lofty tangle
I am
ember maple
lovely, im one of the pytest maintainers, and im currently trying to bootstrap a project/organization that provides test runners/frameworks with key parts of infrastructure for certain details (like tmpdir managment, assertion rewriting/reinterpretation) and so on
Was discord down for everyone or just me
halcyon zodiac
They seem to be having some issues today
ember maple
I see, thanks
PSA I won't work on cot this or next weekend as I have a international move upcoming
lofty tangle
Sounds interesting @ember maple -- would certainly be beneficial. It's something I've had in mind while working on Ward but I've been guilty of not extracting libraries/functionality where I should have
ember maple
I want to use this as opportunity to simplify pytest as well
ember maple
@lofty tangle so one thing I want to do is to reach out to maintainers of test runners and starring to share / collaborate on common gnarly infrastructure like assertion, temporary directions, robust introspection
ember maple
Seems like today is a bad stability day for discord
proud nebula
And I'm the author of hammett and mutmut. 👋
I started hammett as a pytest compatible (subset!) runner because of a frustration with pytest speed. I've landed some PRs but it's just not enough to make a debt so I started fresh.
The reason I was frustrated was because of mutmut which lays the performance issues of pytest bare and shows that pytest was a barrier for improvement.
I also have a little benchmark suite that I want to expand to cover ward in addition to pytest, nose, and hammett.
cobalt ore
what's a typical ordering for continuous integration and local tests?
i mean - are they typically written in tandem, or would one be written prior to the other? Are local tests moved up to CI?
i don't typically test much - and am trying to build up a habit, as well as implement some for a project
ember maple
Without more context this does not make sense
Typically ci runs the same tests as local
cobalt ore
ok, i thought that sometimes the ci tests were heavier than local for some reason
so, things that took hours might be put onto it as well, idk... all good
lofty tangle
I think you might be confusing CI with "integration tests"
cobalt ore
what's the difference
ember maple
It's common to do local test running on a subset of larger integration suites and letting ci run all of them
cobalt ore
an integration suite is just a set of tests?
would a tests dir be an integration suite?
ember maple
That is dependent on personal choices and the used tools
cobalt ore
say pytest
ember maple
Pytest can do both
cobalt ore
both integration suites and continuous integration?
because i'm still not sure what the difference between those is
ember maple
No
Pytest is a test runner
A test runner is something a continuous integration system would invoke
cobalt ore
ok, what's an integration suite
ember maple
A integration suite is a fuzzy term, i would take it as a test suite specifically for integration tests
cobalt ore
what's a test suite
ember maple
Im going to stop here, I'm not going to give a terminology intro on mobile
cobalt ore
that's fine
lofty tangle
@proud nebula That's cool. Where did you find the biggest performance improvement over pytest? It'd be nice to see Ward added to the benchmark! One thing I'd note from my own benchmarking a while back is that Ward has quite verbose output by default, which slows things, and so I run it using a specific output mode that makes it closer to the pytest output.
cobalt ore
seems to be what i said earlier
ie - a dir with test in it, the ./tests dir could be considered a test suite here
lofty tangle
@ember maple Nice, it'd be good to see movement on this front. I'm happy to help with input/thoughts based on my experiences developing Ward so far, and possibly some code. I'm not contributing a huge amount of time to OSS at the moment however.
ember maple
Me neither, corona mitigation, upcoming move and a toddler
But I'm starting to collect interested people and putting first bits in place
kind meadow
Any suggestions for how to test a conditional like this: if a() and b(): ...
I find that trying to assert calls gets ugly
Due to the short-circuit nature of and
marsh raft
write a test that invokes all four combinations of a and b being true or false
ensure that only the true, true version runs the code
kind meadow
I do that already and I'm not a fan of how it turned out
The test code looks confusing
marsh raft
it's possible you don't really need the test
if a() and b() are themselves tested, just make them both return true, so you can exercise the code controlled by that if
kind meadow
Yes, they are already tested. I'm mocking their return values here.
I do think I need some test for this. It is important the function I'm testing returns None if both aren't true.
This is what I came up with earlier and it just seems like a lot of code to test a simple and... ```py
subtests = (
(False, False, False),
(False, True, False),
(True, False, False),
(True, True, True),
)
for a_return, b_return, expected in subtests:
a_mock.reset_mock()
b_mock.reset_mock()
with self.subTest():
a_mock.return_value = user_return
b_mock.return_value = time_return
actual = my_func(...)
self.assertIs(actual, expected)
a_mock.assert_called_once_with(...)
if a_return:
b_mock.assert_called_once_with(...)
well like I said: skip the subtests
keep at most two of them
I wouldn't bother testing that a and b are called, frankly
unless they have side effects, do you even care?
what you care is: my_func got the right answer
with all that called_once stuff, you're "coupling" the test to the code, which means if you later decide to rewrite the innards of my_func, you'll have to rewrite the tests, too
if this is a toy project, of course, none of that matters
kind meadow
I don't know, should I care?
The way i see it, if I don't assert these calls, then my test doesn't guarantee that the function gets its result in the correct manner
Really exaggerated example, but if I just check the return value, someone could write def foo(): return True and it will pass the test all the same.
I don't want tests to be coupled heavily to implementation details, but I also don't want tests to be too "lax"
marsh raft
why do you care about "the manner" in which your function gets its result, as long as that result is correct?
Really exaggerated example, but if I just check the return value, someone could write
def foo(): return Trueand it will pass the test all the same.
@kind meadow how likely is that to happen, and how much effort is it worth to you to detect that?
kind meadow
Because I am not sure if tests passing is circumstantial otherwise, and having 1000s of test inputs isn't feasible
marsh raft
"test passing is circumstantial"?
Do you mean you want your unit tests to absolutely guarantee your code is perfect?
because they can't do that, you know
kind meadow
No, but I wanted them to be accurate to what extent they could be
But based on your attitude towards this, it sounds like the wrong approach
kind meadow
Yes
marsh raft
well, go crazy and test everything!
either I'm right, and at some point you'll say "nah that wasn't really worth it", or I'm wrong and you'll have bulletproof code.
either way, nothing lost
kind meadow
Well I am already questioning if it's worth it
Some tests are complex and seem silly
And I don't know if there's a way to improve them
Or to just stop doing them like that
marsh raft
there's no right answer of course
try it a buncha ways.
You know what I do -- I pay attention to code coverage. If my tests seem reasonably sane, and the coverage is high, I say that's good enough.
(I also have "integration" and "system" tests at work; those cover different stuff)
kind meadow
It's easy to tell myself to just test everything, cause then I avoid having to make decisions. Otherwise, I don't have the intuition on how far to go with them.
kind meadow
I assume that means designing your code to be testable from the beginning? Or is it the same as test driven development, where you write tests first?
marsh raft
oh I guess it's the same as tdd
I'm sure the zealots would argue and say "oh there's a big difference", but .... I was thinking of writing the tests first.
and then "do the simplest thing that could possibly work" to make the tests pass, then refactor, &c &c
it's tough to actually do this in the real world, but once in a while when you're building something from scratch, you can do it
kind meadow
I haven't. It seems scary
I don't normally plan out an entire feature before I build it
Which I feel I'd have to do if I start with tests
I usually work incrementally and frequently make design changes until I land on something that makes sense
marsh raft
there is that, yes.
you don't really know what you want until you've built something, and then see that it's not quite what you wanted 🙂
kind meadow
Or I cannot foresee every issue my initial idea will run into until I'm implementing that specific aspect of it and then it hits me.
But I should try it. People use it successfully, why can't I?
proud nebula
@kind meadow if you want a really solid test suite you might want to look at mutation testing. Install mutmut. In my opinion mocks are rarely a good idea, you want to test the behavior of your code, not the behavior of the mocks. It's easy to make a mistake on this front. And if you use too many mocks you aren't spending time writing tests for the external behavior of your functions/libs.
TDD is amazing if you know exactly what you want to build. This is rare but it does happen.
kind meadow
Yes, I'd like to get into other types of testing but so far our coverage is really poor even for unit tests so we're trying to work on that first.
proud nebula
@lofty tangle yes, more verbose and complicated error output slows things down. I have already surpassed pytest in how much I output and how much I process what I output. I still win the benchmark against pytest by a big margin though :)
@kind meadow my two cents: mocking that much won't give you a lot of coverage per line of test. Try to get the easiest coverage hits you can and don't try to hit 100% on every function before moving on.
Then when you have the easiest wins, look for critical parts and try to get closer to full coverage. If you really care, for example security and access control: do mutation testing on those parts.
Most people don't really have that mission critical code so it's worth 100% coverage imo.
kind meadow
By coverage, do you mean the normal definition of it or the coverage reports you get from tools? Cause I feel the latter can be misleading and I honestly haven't looked at it until now while rewriting these tests (but surprise, I have 100% anyway). You can write poor or flat out incorrect tests and manage to get 100% coverage from my experience
My problem is more so trying to get out of the habit/mentality of coupling tests too hard to the code being tested, as discussed above.
Like, I'm overly paranoid or something
proud nebula
Hmm... Yes ok. Well then if you are writing tests while blind it makes perfect sense to become paranoid.
You need to be writing tests with your eyes open so you know if you're making progress. Seems like you want mutation testing.
Or maybe you need more YOLO in your life heh. Not sure which.
lofty tangle
@proud nebula that's nice. I haven't focused too much on performance in Ward since the impact of framework overhead becomes relatively minimal as test suites grow in size. Would still be cool to see how it fares though.
proud nebula
I've also thought that. I don't anymore because very often people run just one test while developing and if the framework overhead is like half a second like pytest that's really bad.
proud nebula
I'm guessing you probably have decent performance just by being a relatively new project... will be interesting to see
proud nebula
@lofty tangle welp, my guess was wrong. Ward does better than pytest, but way worse than nose2 and hammett. https://github.com/boxed/test-benchmarks
You should look at the many folders test... you can probably improve that quite easily. No way you should be slower than pytest.
lofty tangle
@proud nebula That's interesting, I cloned the test-benchmarks repo to check it and can't reproduce. Here's what I get:
it seems like Ward isn't doing anything? maybe it's erroring or something
just weird that it's taking 28-30ms in all cases
nvm, i see the problem - i made a PR 😄
lofty tangle
I suspect the many folders test is partly due to my liberal use of Pathlib, but I haven't investigated. This benchmark really doesn't come close to representing real-world usage and impact though. A single passing test running in 190ms (on my machine) is not cause for concern IMO.
I think this benchmark could easily be misunderstood as the time taken for "each" test, when it's more a measure of how long the framework itself takes to start up.
In more realistic scenarios -- every day test suites -- I would suspect the difference between the frameworks to be almost negligible.
proud nebula
All true. Lies, damned lies, and benchmarks applies.
But I do think some of that is pretty valid. Like the parameterized tests.. that could have some real impact.
Not sure how this combines with something like hypothesis but I'd guess any framework overhead is not going to be good. You just multiply the overhead too many times. That's the same problem I had with pytest and mutmut: you can't just shrug off 200ms because it happens "once" when you in fact need to run those 200ms two thousand times.
Mutation testing is the reason I am taking such a hard stance here. If you aren't doing it, you aren't feeling the same pain I am. And the more you don't care about these performance issues the more they grow and grow, until mutation testing is infeasible and that's not good.
I notice that ward actually beats hammett for single passing test. Probably because hammett has started doing bigger things like taking the timestamps of all files and storing them in a database. Work that is useless for the benchmark but in real life can be a big gain in perceived speed.
I like ward though.. I had written out ward-style tests on a notepad once, just shelved the idea because I want to have a smooth way to move from pytest to hammett
cobalt ore
how to people typical test functions that use matplotlib?
is this it? https://pypi.org/project/pytest-mpl/
it has the name pytest, but i'm not sure if this is what most people use or not
proud nebula
That seems weird to me. What is the point of testing matplotlib itself? I would just test up to the parameters that would then be sent to matplotlib.
proper wind
Reproducing V&V documents? I've hooked pytest into my HIL pipeline automating report generation. I suppose at scale you would want to also unittest your plot look?
proud nebula
It doesn't make sense to test that matplotlib works. It's slow for no benefit. Testing is a trade off. If you just want more testing you can just randomly test stuff like 1+1=2 or that str.split() works. It's just not a good use of electricity.
cobalt ore
@proud nebula i'm not sure - haven't done much of it and it was just something that i'd been considering for some work. I don't know what best / most common practice is with this sort of thing
i guess you're saying - assume that matplotlib will plot things right and just test that the data transformations which are passed to it are correct, ?
that's how i interpret your statement at least, i might be wrong
plush vale
@cobalt ore the paradigm that I've been trying to adapt is that your test cases should be entirely isolated and always only depend on specifically on the function or object that you are trying to make assertions about
so if you have a function that does some simple logic and then calls some matplot endpoint, the test case for it should only be concerned with whether it passes the expected parameters to matplot
everything else should be mocked
I don't have matplot around but let me give an example with pandas
cobalt ore
the paradigm that I've been trying to adapt is that your test cases should be entirely isolated and always only depend on specifically on the function or object that you are trying to make assertions about
right... so this involves making a certain set of assumptions for each test i guess ?
plush vale
let's say that I have a function which contains some internal logic that determines whether we want to load birds.csv or elephants.csv, then passes the filename to pandas and returns the dataframe
so what can I do to test this without relying on or testing pandas directly? I can patch the function it's supposed to be calling and then make assertions about the mock
>>> import pandas
>>> from unittest.mock import Mock, patch
>>>
>>> with patch("pandas.read_csv", Mock()) as mocked_function:
... pandas.read_csv("birds.csv")
...
<Mock name='mock()' id='140419066201760'>
>>>
>>> mocked_function.assert_called_once_with("elephants.csv")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/kwzrd/.pyenv/versions/3.8.2/lib/python3.8/unittest/mock.py", line 925, in assert_called_once_with
return self.assert_called_with(*args, **kwargs)
File "/home/kwzrd/.pyenv/versions/3.8.2/lib/python3.8/unittest/mock.py", line 913, in assert_called_with
raise AssertionError(_error_message()) from cause
AssertionError: expected call not found.
Expected: mock('elephants.csv')
Actual: mock('birds.csv')
everything else should be mocked
i'm not too sure what mocked means in the context of the example you gave here... I am reading the rest now and will not interrupt 🙂
plush vale
this never actually calls pandas at all, and I'm in full control of everything that the tested function interacts with
this will not raise:
>>> mocked_function.assert_called_once_with("birds.csv")
so the test case passes
cobalt ore
i don't get it
here the logic is all independent of pandas then i guess?
i mean - what if the logic was group sizes or something?
you'd make a mock function for .groupby ?
plush vale
it depends on what exactly you are trying to test
cobalt ore
well say when grouped on weight the average is over 20KG it's probably elephant not birds or whatever
plush vale
if you want to test that your function will pass the correct parameters to groupby, then you can do that by injecting some object that will have a pretended groupby method and that will tell you what it was called with
cobalt ore
my point is - if i have a function that uses pandas, i don't see how i remove pandas from a test
but that's not what the function does
i mean - testing that the function passes the right args
if the function is def determine_animal_by_group_size or whatever
idk, i'm too tired for this atm i think
plush vale
let me give a maybe better example
cobalt ore
i didn't really follow the last one
plush vale
I will create an actual function this time, it will be a simple wrapper
>>> def add_column(dataframe: pandas.DataFrame, column_name: str) -> None:
... dataframe[column_name] = None
so if I pass in a df and a colum name, it will add that column and fill it with nulls
cobalt ore
right
plush vale
it's an in-place operation so it doesn't return anything
so now if I want to test that this works, there are two approaches
I can make my own df, pass it in, and then check that it has the column as expected
that's the simplest way you can do it, and in this case it's probably fine, but it will depend on pandas and that's probably not what we want
we don't want to test that pandas works, we want to test that our function works
so what can we do to make this test our code only?
a possible approach is to use the MagicMock class that unittest provides
>>> from unittest.mock import MagicMock
>>>
>>> my_mock = MagicMock()
>>>
>>> add_column(my_mock, "desired_column")
>>>
>>> my_mock.__setitem__.assert_called_once_with("desired_column", None)
instead of passing in a dataframe, I pass in my own object, which will let the function call its __setitem__ and then it will remember what it was called with
this way, I can then make assertions about what it would have called the dataframe with, without having to use pandas at all
does that make more sense?
this is what happens when I assert that it was called with a different column name:
>>> my_mock.__setitem__.assert_called_once_with("some_other_column", None)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/kwzrd/.pyenv/versions/3.8.2/lib/python3.8/unittest/mock.py", line 925, in assert_called_once_with
return self.assert_called_with(*args, **kwargs)
File "/home/kwzrd/.pyenv/versions/3.8.2/lib/python3.8/unittest/mock.py", line 913, in assert_called_with
raise AssertionError(_error_message()) from cause
AssertionError: expected call not found.
Expected: __setitem__('some_other_column', None)
Actual: __setitem__('desired_column', None)
it fails
cobalt ore
__setitem__ this is what pandas uses then?
plush vale
__setitem__ is what gets called when you use the [] operator
so the dataframe implements that
cobalt ore
ok, so one needs to shadow all these methods for testing?
plush vale
but yeah that was a bad example again lol, should have used a function with an explicit name
cobalt ore
i'm aware of dunder things but never need them, so don't use them
plush vale
well, the good thing about MagicMock is that you dont explicitly need it to tell to mock the __setitem__, it will do that automatically
cobalt ore
also i'd use pytest, not unittest, i guess the logic is the same tho
plush vale
so the unittest framework is equipped to help you with such situations
I'm sure pytest provides similar classes, but I've never used it myself
cobalt ore
yeah, i don't get the logic / workflow / approach so it doesn't really matter atm for me anyway, pytest is what they'd end up in for me tho
it seems that, for a simple function which uses a few pandas methods, the above approach could be quite involved
i don't understand how removing pandas makes anything easier
either you remove it as you have - and assume that pandas functionality won't change, or you leave it in, and assume that pandas functionality is going to operate correctly
plush vale
yeah, there are definitely situations where it'd be acceptable to pass in an actual dataframe, and then make assertions about it
unless you are writing tests for pandas specifically, it shouldn't be your concern to test it
we have to work with the assumption that pandas works
cobalt ore
i will try and get an example
plush vale
it appears that testing can be a quite polarizing topic so I'm sure there are people who would disagree with my approach
in many cases, if something is difficult to test, that may be a sign that it's not designed well in the first place
that is also something to keep in mind
if you find it difficult to write tests for, it may be difficult to reason about in general
cobalt ore
def w_counts(dm, weight_group="source_file", value="value"):
columns = ["var", value, "w_count", weight_group]
dfwa = pd.DataFrame(columns=columns)
for grp in dm[weight_group].unique():
dff = dm[dm[weight_group].eq(grp)]
xtab = (
pd.crosstab(dff["var"], dff[value], dff["weight"], aggfunc=sum)
.stack()
.reset_index()
.rename(columns={0: "w_count"})
)
xtab[weight_group] = grp
dfwa = pd.concat([dfwa, xtab], axis="rows", sort=True)
return dfwa
if you have something like this
proud nebula
I agree with what kwzrd says except I don't like mocks. I'd have one function that does the data transformation, and you test that, and one function that calls the data transformation and then matplotlib and don't test that
@cobalt ore and try hammett.. it's (mostly) pytest compatible and faster.. because I wrote it :P
cobalt ore
@proud nebula 😛 thanks, not sure i'd be able to get people to shift from pytest. But, i do get space to try pretty much what i'd like, so i'll have a look when i finally get the point of it all
@plush vale how'd you approach a function along the lines of what i gave there? Seems that mocking everything would be a nightmare?
proud nebula
yea, it's a bit early days still, so keep the tests working for both pytest and hammett
pandas is pretty quick right? Just make a small test with a bunch of prime numbers
cobalt ore
pandas is pretty quick right? Just make a small test with a bunch of prime numbers
what do you mean? pandas is quick i agree, but i'm not sure what you mean - quick as in run time? or time to code?
and in the case of making a test with prime numbers - presumably you'd not be mocking things with __dunder__ stuff (as you've outlined you're not a big fan of mocks).
hopefully i'm not boarding on any test based flame wars without realising
proud nebula
runtime
cobalt ore
ok, i don't really follow still. Would you just create a mock data frame / csv , store it as a json string or whatever within test_w_counts() and ensure that the output equals what you expect?
proud nebula
it does depend on how critical it is obviously.. are you sending humans on top of a rocket that depends on this math? then mocks, mutation testing, property based testing, lots of example tests, static typing, whatever it takes :P
plush vale
this doesnt look like its making calls to any untestable / difficult to test endpoint, it will just build a dataframe based on the passed parameters, right?
proud nebula
that sounds about right... most bang for the buck. Mocks are good when the dependencies are super slow.
plush vale
so you are already in full control of what should come out
proud nebula
yea
cobalt ore
i've no idea - it makes a dataframe sure
plush vale
it'd probably be perfectly fine to just pass in parameters, retrieve the returned dataframe, and compare it to your own
proud nebula
one example calculation seems sufficient
cobalt ore
but you had a much smaller example and mocked it earlier
plush vale
or just make basic assertions such as, does it have the columns that im expecting? is it the expected shape?
cobalt ore
so , to me, seeing that i would assume the example i've posted here to be more complex
plush vale
there arent really things you can make assertions about when generating matplot graphs
cobalt ore
i think we've moved onto more generally testing things, and using pandas as example as that's what i typically use
i'm happy to leave the graph checking ha
plush vale
yea, i just used pandas because it's what I had around
and specifically started with the read_csv function to show how we can mock it to test our functions without necessarily having the csv file there
cobalt ore
no pandas is good - i use it
and, continuing the example that i posted, would a suitable test be
from module import w_counts
def test_w_counts():
data_in = <some data to read in>
data_out = <expected result>
data_test = w_counts(data_in)
assert data_test.equals(data_out)
something like this? obv filling in the < ... >
cobalt ore
right ok, i can get this i think ha
if i was to instead mock everything from that example function though, i'd be lost
plush vale
no, you dont have to
cobalt ore
Why not? I'm happy not to ofc, but what's to say i shouldn't mock everything here?
plush vale
but imagine that w_counts also builds a graph using matplotlib as it performs its internal logic
we can't really test what the graph looks like, rigth? maybe we dont even care
cobalt ore
sure - but the graph here is separate - i'm not sure how you determine when to mock / when not to
plush vale
so we could patch the matplot function it calls with some mock, which will just not do anything, and maybe remember which parameters it was called with
my immediate response to that would be, "when it's convenient"
not a very good answer admittedly
I'd have to think more about it
cobalt ore
no worries, just wondered if there was some rule of thumb or whatever
plush vale
ok, you know when I'd use a mock?
if w_counts depended on some config object
but the config object doesnt get passed with the parameters
the module imports it
so you aren't in control
to have 100% control over the test case, you'd want to patch in your own mocked config, set up specifically for that one function
cobalt ore
hrm, how would #unit-testing message be augmented to make you change from not mocking to mocking.
some config object
what's this? i use config files, never config objects... to my knowledge at least
plush vale
if the function is pure, i.e. doesnt depend on anything but the parameters you pass in, you dont really need mocking generally, I think
well that'd be the same thing
the config file will be represented by some object at runtime lol, like a dict
thats what i meant
cobalt ore
oh, ok
plush vale
example time
cobalt ore
def w_counts(dm, weight_group="source_file", value="value"):
columns = config.columns['weighted']
dfwa = pd.DataFrame(columns=columns)
for grp in dm[weight_group].unique():
dff = dm[dm[weight_group].eq(grp)]
xtab = (
pd.crosstab(dff["var"], dff[value], dff["weight"], aggfunc=sum)
.stack()
.reset_index()
.rename(columns={0: "w_count"})
)
xtab[weight_group] = grp
dfwa = pd.concat([dfwa, xtab], axis="rows", sort=True)
return dfwa
this?
(i've changed the columns var to use a config )
plush vale
yeah, exactly
so when you call this function
you aren't 100% in control of what the outcome should be, right?
cobalt ore
hrm... so
from module import w_counts
def test_w_counts():
data_in = <some data to read in>
data_out = <expected result>
data_test = w_counts(data_in)
assert data_test.equals(data_out)
would change to what
you aren't 100% in control of what the outcome should be, right?
i'm not sure - i guess i'm not 100% in control of the function in that there's something 'injected' into it's scope via an external file? Idk if i'm using words that make sense lol
cobalt ore
why's it difficult? the test was simple before
difficult because of the config that's used?
plush vale
it contains too much syntax noise that is unnecessary for the purposes of an example
cobalt ore
ah ok 👍
plush vale
lets just work with
cobalt ore
it was an example of a typical function i guess, but it's not an important example
plush vale
import config
def adder(a, b):
"""Add `a` and `b` and multiply by configured constant."""
return (a + b) * config.multiplier
so this isn't pure in the sense that it doesn't depend on the parameters only
cobalt ore
right, there's the external param
plush vale
if you do adder(1, 2) and know nothing about config, you cannot predict the outcome
so what you could do, is inject your own config
with patch("config", Mock(multiplier=5)):
outcome = adder(1, 2)
assert outcome == (1 + 2) * 5
could you just have
class config:
multiplier = 3
or something 🤔
plush vale
sure
what you can do is
class FakeConfig:
multiplier = 3
with patch("config", FakeConfig):
...
the Mock is just a convenience class provided by the unittest framework
it doesn't necessarily have to be used
the point is that we use some kind of a mock object (regardless of whether it's an instance of unittest.Mock or other)
to be fully in control of everything that the tested function depends on
cobalt ore
hm ok, i think that makes sense... if there are external files / configs then this sort of thing should be done
and this is probably why some people don't like consants.py i guess
plush vale
yeah, maybe, although there's probably more at play there
cobalt ore
idk i got told not to use them with no explanation lol
been wondering what the reason was
plush vale
one of the arguments for functional programming is that it's easier to test
i'd dislike a constants.py module if it contains constants that are only used in one place and need to be imported needlessly
it creates unnecessary indirection / distance between the reference and the definition
but for config values used globally across a project, having them defined in one place together and imported makes a lot of sense
cobalt ore
how would you do it without one and keep it functional? You'd have to repeat yourself constantly i assume, and that's a trade off 🤔
plush vale
well, imagine that instead of having that function import the config value, it could just take it as an argument
the test suddenly becomes trivial
plush vale
then the behaviour becomes more difficult to test
by the way, I'm not saying config shouldnt exist
proud nebula
I wasn't kidding about prime numbers before btw. They are great for calculation tests because they are harder to accidentally combine into the expected output.
cobalt ore
yes that's a good tip @proud nebula , makes sense 🙂
is there a way to get primes in python 🤔 without writing a sieve...
i think what you're saying makes sense though kwzrd
plush vale
yea, thats good to hear - hope I was able to help at least a little bit
I'm still super new to testing so trying to figure out these things
having a conversation about it is always good
cobalt ore
yeah - it can be hard to draw a line
in that - oh i should test z, but that uses y, which uses x, which uses w, which ...
at which point i decide that maybe i'll write tests another day
plush vale
a good approach to those dependency chains is taking it from the very end
e.g. test w first
cobalt ore
but w depends on v
plush vale
if w works, writing the test for x would be easier - you already know that w works
cobalt ore
like when i gave the example earlier
from module import w_counts
def test_w_counts():
data_in = <some data to read in>
data_out = <expected result>
data_test = w_counts(data_in)
assert data_test.equals(data_out)
i didn't know if that was going to be ok or not
just seemed like an approach to me, but there could have been a bunch of responses of "oh you need to check x y z as well" or whatever, i didn't have an intuition on that
plush vale
I generally think that if you cannot have an impenetrable test because you dont have time, resources or the will to test something properly
a basic test is still useful
i.e. call the function, check that it returned a dataframe
that already means the code was executed and doesnt contain an AttributeError for example
cobalt ore
where i find myself wishing i had tests is when i want to refactor
plush vale
of course, you cannot guarantee that it will never fail, but if there is a problem, chance are you will catch it
cobalt ore
like, am i breaking something? i'm not too sure sometimes ha
plush vale
yes, regression testing - super useful
cobalt ore
regression testing relates to refactoring? I mean - if i have three things to do i might create three functions, but then it turns out they can be abstracted into one
at this point, i have no tests, and have to refactor, it can be messy
most of the time i don't have time to refactor things anyway, but when there is, this is an issue
plush vale
I think broadly speaking regression testing is used when you want to make some change to the tested code (i.e. refactor) but dont want its behaviour (as observed from outside) to change
so maybe even things like making the internal computation more efficient
cobalt ore
ah ok, i thought it was when a bug was found
plush vale
I'm not sure what the exact definition is
proud nebula
Testing to prevent regressions. Whatever a regression is :)
cobalt ore
if there's some function f which has multiple parts within it, should there be a single test for that function (with multiple asserts), or should there be a separate test for each part? I'm guessing the former
marsh raft
latter
cobalt ore
@marsh raft oh right 🤔
marsh raft
well, if the parts are themselves functions
cobalt ore
no, assume that f is a single pure function, just there's a fair bit of stuff involved with it
marsh raft
well maybe it shouldn't be a single function
cobalt ore
or at least, a few different things which need to be checked
lament drift
hi i need help i can't "https://www.tiktok.com/tag/hair?lang=TR" open url with selenium and this web site detected selenium
marsh raft
maybe it should call out to some helper functions and combine their results in a reasonably-simple (i.e., easy-to-test) way
cobalt ore
@marsh raft hrm... i'm reading through the sklearn tests atm and there are typically many asserts
marsh raft
@lament drift Unless your question is somehow about software testing, you should get a regular help channel: #❓｜how-to-get-help
cobalt ore
for example - i've not actually looked at the original function this is testing to check
but i guess i'm curious about the multiple asserts
lament drift
oh sorry bro @marsh raft
proud nebula
@cobalt ore you very probably need one call of the function under test per behavior in the function to test it completely. So if there are three ifs you need three calls with different arguments. You can shove those three calls into one test or three, doesn't matter much.
proud nebula
There is a Dogma out there that you should only have one assert per test. This is a stupid dogma (like all dogma). I've merged lots of tests at work with very dramatic improvements to the execution time because I could avoid duplicate work. One assert in a test is a nice thing, but don't sacrifice execution speed on the altar of ideological purity.
ember maple
The definition of assertion is rather unflexibel, there is a number of cases where one would like to do more kinds of checks
I like how subtests in pytest /unittest allow different reporting there
zinc nimbus
can someone help me pls. I am a beginner in python. I try to open a file in cmd but it doesnt work. the result: can't open file 'C:..... [Errno 2] No such file or directory
what can i do ?
thx for helping
long ember
@zinc nimbus if you haven't solved the problem, you'll get a faster answer in a normal help channel. See #❓｜how-to-get-help. This isn't the right channel for your question really
zinc nimbus
@zinc nimbus if you haven't solved the problem, you'll get a faster answer in a normal help channel. See #❓｜how-to-get-help. This isn't the right channel for your question really
@long ember thank you
candid sigil
how do I let a function return nothing? the function needs to print something but if I just use print() it ends with 'None' and if I add return "" there is a whiteline too much
plush vale
@candid sigil in Python functions always return something - if no explicit return statement is reached then the function returns None implicitly
feral cave
Hey, could you recommend me library with which I could test our software by for example fill TextBoxes, click Menu Items (save, load, print), custom C# controls and all that stuff that simple user would do?
Project became much bigger than we thought and its pain in the .... to test it manually.
Is Selenium a thing when it comes to testing software?
proud nebula
Avoid GUI testing as far as you possibly can. Selenium is extremely slow.
feral cave
Avoid GUI testing as far as you possibly can. Selenium is extremely slow.
@proud nebula why is that?
proud nebula
Because it's starting an entire browser. The why isn't terribly interesting. It just is.
halcyon zodiac
!tempban 331876823443177473 10d This server doesn't exist to perpetuate whatever petty drama you have with another user. Stop ghost pinging them, and, should you decide to return after your ban expires, try reading the rules you agreed to when joining the server
bitter wadiBOT
:ok_hand: applied ban to @sage turret until 2020-06-12 13:47 (9 days and 23 hours).
rancid hound
I was trying to download chrome webdriver but it shows ERR_CONNECTION_CLOSED
I'm trying for like 3-4 hrs I thought connection problem at first but it isn't
what could be the reason ?
reef rune
@rancid hound You're downloading the latest right?
rancid hound
I'm downloading the latest stable version
is it issue with chrome coz then I tried with edge and it worked perfectly fine
proper wind
hey guys few weeks ago i started learning python and im trying to create multithreaded tool but something is not working, can anyone help me? here is some of my code
import random
import threading
from concurrent.futures import ThreadPoolExecutor as ttt
try:
bots = input("Please choose threads amount (1-500): ")
if int(bots) > 500:
print("Use less threads please!")
elif int(bots) <= 500:
print("Using", bots, "threads.")
except:
print("Wrong format!")
def printer():
print("Hello World, testing multithreading")
def main():
with ttt(max_workers=500) as runner:
runner.map([printer] * bots)
runner.shutdown(wait=True)```it should spam "Hello World, testing multithreading" but its not doing anything
proper wind
@pearl cliff so how do i occupy? sorry im new
proud nebula
@proper wind map seems to just return an iterable. The docs are super unclear to me.
proper wind
@proud nebula welp im new to python, im trying to make program that uses multithreading and requests together
can u give me link to guide or something?
royal wren
Hi, it’s been a long time since I did QA and very rusty. I’m trying to write some boundary test cases for a state machine, let’s say if a user places an order and once that happens it goes into Pending status. What would be some good boundary cases for this scenario? Also is it possible to do Equivalence Partitioning when there is no max or min value?
proud nebula
Not sure I follow.. what are the boundaries in this scenario? (opens the wikipedia page for Equivalence Partitioning)
Hmm.. to me it sounds like if there is no min or max then all values are valid and there is no boundaries and thus equivalence partitioning doesn't really apply. But I just heard of this concept just now and just skimmed the wikipedia article :P
rigid cave
im coding something called rawscript and it will be designed for beginners
seems like the most appropriate place to post it
rigid cave
i send it before it was finished >:O
rough creek
proud nebula
"It"?
rigid cave
it's supposed to be an object oriented language but it's just a mess
kind meadow
That doesn't sound related to software testing so this isn't the right channel to discuss it.
still cobalt
Any great examples of well-tested libraries using pytest? Maybe they also come with explanation for why they made those tests? 🙂
kind meadow
What? Why are you pinging in random channels? Just be patient and someone will answer your question.
Please, this isn't the right channel.
upbeat galleon
I have a small FastAPI api and I want to write tests for it but it's working really as I would expect and I'm not sure what to do.
# test/test_main.py
from fastapi.testclient import TestClient
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from ..database import Base
from .. import app
from ..utils import get_db
SQLALCHEMY_DATABASE_URL = "sqlite:///./test.db"
engine = create_engine(
SQLALCHEMY_DATABASE_URL, connect_args={"check_same_thread": False}
)
TestingSessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
Base.metadata.create_all(bind=engine)
def override_get_db():
db = TestingSessionLocal()
try:
yield db
finally:
db.close()
app.dependency_overrides[get_db] = override_get_db
client = TestClient(app)
def test_create_user():
r = client.post(
"/users/",
json={"username": "test1", "password": "12345"}
)
assert r.status_code == 200
# main.py
# ...
@app.post("/users/", response_model=schemas.User)
def create_user(user: schemas.UserCreate, db: Session = Depends(get_db)):
db_user = crud.get_user_by_username(db, name=user.username)
if db_user:
raise HTTPException(status_code=400, detail="Username already registered")
else:
return crud.create_user(db=db, user=user)```
The issue is that this is always returning 404 when I run `pytest`. I also tried to manually start the server and then run pytest but it was still a 404The code for test_main is pretty much copy pasted from here https://fastapi.tiangolo.com/advanced/testing-database/
proud oyster
Spyros, I reject all friend requests from Python server save the mods
as for your testing problem, I don't test my databases, I just use Powershell pester to test my APIs against known good data so I'm of little help
upbeat galleon
Oh thanks for replying, I only added because you said you use this at work so I figured you might know something about testing it. The thing is, it has nothing to do with db, it just returns 404, as if the route is wrong
But thanks for taking a look anyway
proud oyster
your route looks wrong
upbeat galleon
It does?
proud oyster
wait, it's JSON post with data correct?
upbeat galleon
Yeah
proud oyster
I do base model then tear apart
I also don't use ORM
for example
from fastapi import APIRouter, HTTPException
from pydantic import BaseModel
from ..modules import calcmodule as calcmodule
router = APIRouter()
class Calc(BaseModel):
op: str
number1: int
number2: int
@router.post("/calc")
def calc(body: Calc):
return calcmodule.perform_operation(body.number1,body.number2,body.op)```Btw I tried to also print r.text and if I recall corectly it would print something like "detail doesn't exist"
The api it self works fine, for example I can test it manually from /docs or with curl but using pytest fails, that's why I am buffled
proud oyster
I'm not familiar with python testing libraries, sorry
upbeat galleon
That's fine, thanks for your time
proud oyster
It "Calc JSON Post / Adding 5+15"{
$testURI = "$($URI)/calc/calc"
$body = @{
op = 'add'
number1 = 5
number2 = 15
}
(Invoke-RestMethod -Method POST -Body ($body | ConvertTo-Json) -Uri $testURI | ConvertTo-Json) | Should -Be (@{result = 20} | ConvertTo-Json)
}```that's my tests
upbeat galleon
I don't even recognise what that is
proud oyster
it's Powershell
upbeat galleon
Ohh right, you mentioned it earlier
I'd like to use pytest/or something so I can integrade it with github and things like that, but doing it manually is my last resort, it shouldn't be too bad since it's not a huge project
proud oyster
my company does C# development
so we are on Windows for some of our stuff so Powershell is good bridge language that works on both platforms
upbeat galleon
I may use C# for a native app for this thing but that's for the future
proud oyster
like if I need to configure Windows Server, Powershell is great (much better then python) but if I'm on Linux, we install Powershell Core
and pwsh works fine
upbeat galleon
Makes sense, I work alone so I can use pretty much whatever I want haha
proud oyster
BTW, Azure Devops supports Powershell
upbeat galleon
Makes sense since it is microsoft
proud oyster
though you would have to spin up application for testing
upbeat galleon
Yeah I did also try to manually start the server but that didn't fix it
I may have missed something
Earlier you said that my routes do look wrong, could you tell me why?
proud oyster
I thought they were get
upbeat galleon
You mean in the test, in the main file, or both?
proud oyster
@app.post("/users", response_model=schemas.User)
def create_user(user: schemas.UserCreate, db: Session = Depends(get_db)):
db_user = crud.get_user_by_username(db, name=user.username)
if db_user:
raise HTTPException(status_code=400, detail="Username already registered")
else:
return crud.create_user(db=db, user=user)```No that didn't fix it
proud oyster
yea, I'm not familiar enough with response_model and crazy class schema stuff you are doing
not to say it's wrong, just don't know enough to say either way
upbeat galleon
{"detail":"Not Found"} that's r.text
proud oyster
this isn't in a router is it?
proud oyster
FastAPI supports router files
upbeat galleon
It's all in main.py, but I do initialise app in a different file and import it to main.py
proud oyster
you want to see my interview application I wrote
upbeat galleon
Ahm sure. Btw all this is public in gihub it case you want to take a look ¯_(ツ)_/¯
astral scaffold
I have a testing related question in #help-broccoli
proper wind
Anybody know where I can learn about pytest_generate_tests in pytest?
In good details
@twilit trellis can u help me please?
low frost
So I have created a simple util that I would like to unit test.
In a fixture is it possible to mock the guild class?
gilded dagger
hey guys
i've got the assets made
i just want to make sure it doesn't need to be like super local and all that
:)
past swan
nice
honest patio
is this the right one for printing random uuids
cinder girder
hey everyone, so I have some code (in a function) that raises an exception but then handles said exception and returns a result. Should this exception be recorded in the function docstring? If it helps, I am using the numpy docstring format.
proud nebula
It's an internal implementation detail so no. (Doesn't seem like the right channel either)
cinder girder
Figured, wasn't sure where to ask, guess it could have been asked in a help channel**
marble hatch
Is there a simple, idiot-proof explanation of how to structure tests for current versions of pytest? I upgraded setuptools and pip and installed arcade, and now the way i had my unit tests structured no longer works
My main goal is to have a place I can import test helpers from
I found a stack overflow post that listed a plugin, which I tried but it is reportedly broken
I also tried a conftest.py hack to add a tests/helpers directcory to path
but that seems to have not worked either
I previously had a util.py under tests in my project root
now that seems to no longer be supported
the helpers I wrote can't just be passed into the tests as fixtures, as some of them generate values for parametrizing tests
marble hatch
According to @obtuse mauve , adding an __init__.py to the tests folder fixes the issue even if I'm not sure how it worked exactly. Ty! #help-donut message
proud nebula
This is just normal python. If you don't have a dunder init it's not a python module.
bleak dove
Can anyone recommend some good examples of learning python unit testing?
I got some basic practice and watched a few videos of it
But I am hoping I can expand on my knowledge by seeing it how it is properly applied
I checked out this discord server's bot's unittests, but are other there any others?
marble hatch
@proud nebula the thing is, i don't think it is. pytest used to recognize files directly under tests as importable from a tests module
now it doesn't by default
and it's also mysteriously broken again
and i don't know why
marsh raft
Can anyone recommend some good examples of learning python unit testing?
@bleak dove google for "python koans"
proud nebula
Well then maybe pytest was broken before. More likely is that you are confused. Humans are often mistaken, computers not so much.
marble hatch
I'm definitely confused, but there isn't a clear description of how/why pytest treats certain things as modules
that's why i'm confused
pytest or something that influences how modules are found changed at some point
but i can't figure out what
i know it used to work perfectly before i upgraded some packages to get arcade to run.
so
i tried cloning from github in a fresh directory
and made two venvs. one with default setuptools + pip, another upgraded to the latest
it appears pytest works perfectly under both
and i do not understand why
marble hatch
@obtuse mauve do you have any suggestions as to why this is breaking in the old repo but not in a freshly cloned one?
marble hatch
removing & recreating the venv, removing .pytest_cache, as well as clearing all *.pyc (with find . -name \*.pyc -delete) seems to have fixed it
im still not clear on why things suddenly broke
marble hatch
Ok, this issue seems to crop up after i change the contents of certain source files
it doesn't appear to have to do with syntax
i just changed a source file and things were broken
changing it back doesn't alter things
i can still run the actual program itself
finite kestrel
does anyone have suggestions about how to improve my project, WinMod?
pearl cliff
@finite kestrel did you post it in #303934982764625920 ?
mighty cave
someone help me make a key detection system for the dji tello drone go ahead with w go back with s
and go sideways with a and d
kind meadow
Wrong channel. That's not related to software testing. See #❓｜how-to-get-help to claim a channel. I also suggest you make your question more specific
proper wind
Hey
brisk steppe
Hello Guys, i m working on a script to automate a game, but it's not working, can someone maybe help me? Here is my code:
from pynput.mouse import Button, Controller
from pynput.keyboard import Key, Controller
import time
print("Process has started")
mouse = Controller()
while True:
print('The current pointer position is {0}'.format(
mouse.position))
# Set pointer position
mouse.position = (960, 540)
print("position check")
# Press and release
mouse.press(Button.left)
print("mouse press check")
time.sleep(15)
Controller.release(Button.left)
mouse.type("Hello")
print("type check")
time.sleep(2)
Unless you're automating it for the purposes of software testing, this isn't the right channel. Consider claiming a help channel instead #❓｜how-to-get-help
sturdy osprey
Hello, i need help with my project, when execute the program, the buttoms are pressed. Thanks https://github.com/AlexanderMelian/Customer-Management-Pizza
balmy prawn
kind meadow
That isn't related to software testing.
alpine vine
@kind meadow I'm trying to tweak it from the os side. Which channel?
kind meadow
Just claim an available help channel
alpine vine
I'm confused. Any specific channel? Is topical chat/help a help channel?
kind meadow
mystic viper
@alpine vine #❓｜how-to-get-help. Tl;dr: take any channel from the Python Help: Available category, and write anything(preferably the question) to claim it.
alpine vine
👍
proper wind
having errors with scrappy can someone help me please?
for results in response.xpath("//a[@class='mt_a']"):
yield {
"Country": results.xpath("//a[@class='mt_a']/text()").extract_first()
}```this is a web crawler and im trying to store a list of countries however this only iterates through the first iteration (USA)
iv'e tried multiple stuff yet it wont work
kind meadow
How is that related to software testing?
proper wind
oh my bad, where am i supposed to be posting it
kind meadow
Claim an available help channel #❓｜how-to-get-help
lament crypt
!e
bitter wadiBOT
You are not allowed to use that command here. Please use the #bot-commands channel instead.
light cedar
documentation, couldn't find a channel for it so here I go. I'm using sphinx to document my code. I've got a .. automethod:: configuration(config) to force a annotation-function, but is there a way to make it more clear that for instance https://flask.palletsprojects.com/en/1.1.x/api/#flask.Flask.route is a decorator? other than adding example-code in the description on the function to highlight that it's a decorator. Can't find anything in sphinx about it, so I'm starting to get worried that there's no such thing.
tldr; can i get @configuration or @route definition in the docs graphical layout, instead of just configuration or route to enhance the detail that it's a decorator function?
proud nebula
I don't understand why this channel gets random unrelated questions...
gritty crane
im getting issues trying to get av to install for a dev project, and it seems that it has to do within the pip3 install configs itself and im not sure how to fix it
but im not sure if its becuase the repo is using an older version of av or if its something that i can upgrade post repo install
grim glacier
Does anyone have any recommendations for large python repos that have a good structure/integration of pylint?
proud nebula
None of those questions are related to testing
long ember
Does anyone have a link to a pretty general read about testing? I have a vague idea bout unit testing and integration testing, but I would like to read about best practices and stuff a bit more in detail
torn mirage
I'm doing some scraping and keep running into unnoticed erroneous behaviour on some of the more obscure but needed html patterns, what do you think would be a good way to create tests for this? I'm not sure if checking the exact parsed result would be good as almost every change is likely to influence that in some way
small kayak
Is there a way to detect if a function is recursive? I write Python curriculum with coding challenges, and I'd like to be able to verify that a student's solution is indeed recursive.
I'm not super familiar with Python's introspection capabilities, or if such a unit test exists.
ebon bough
perhaps count the calls in the stack trace
small kayak
class CallCount:
def __init__(self, func):
self.func = func
self.count = 0
def __call__(self, *args, **kwargs):
self.count += 1
return self.func(*args, **kwargs)
the problem with this is that I don't know how to access the self.count attribute once the recursive function terminates. I could print the current call count out, but then I assume I'd have to rig up a way to read that output from stdout?
proper wind
Hi, I've completed the classes for the Article class for the summer Jam and I've tried to run the tests but I get this error ``` test_description = test.shortDescription().rstrip(".!?")
AttributeError: 'NoneType' object has no attribute 'rstrip'
I don't get it
Its nothing to do with my code but the test files that have been provided
mystic viper
~~Don't think this is the right channel for this, you might want to just claim a generic help channel.~~ah, I see
As for the problem - can't say anything without the code; the exception just means that you tried to do something.rstrip() and something turned out to be Null.
ah, it's for the Jam?
proper wind
yes
I'm not looking for the exercise solution because I;ve done them, I've never done testing in python cos I'm just learning the language
so I don't understand the behaviour
mystic viper
Hmm. Does this manifest when running the provided test suite https://github.com/python-discord/summer-code-jam-2020-qualifier#test-suite?
proper wind
yes
mystic viper
Shouldn't it tell you which test is broken, then, or something? What's the full exception?
proper wind
yup that is it
y_var = int(input("On a scale of 1 to 10 how smart are you?")) if int is false then print ("False Answer")
Right
Is that correct
Guys?
torn mirage
can you add in print(test) to that method?
proper wind
?
I thought it was a mistake on my part that I should have named a method short_description but when I traced the code, it didn't matter
y_var = int(input("On a scale of 1 to 10 how smart are you?")) if int is false then print ("False Answer")
Right
Is that correct
Guys?
torn mirage
DJBrett that doesn't seem related to testing, please take a help channel; instructions on how to do that in #❓｜how-to-get-help
proper wind
@torn mirage I did and it came back with the same error
torn mirage
what did it print?
proper wind
Python Discord Summer Code Jam 2020: Qualifier Test Results
====================================================================================================
Date: 2020-07-03 00:47:08 UTC
====================================================================================================
test_qualifier (unittest.loader._FailedTest)
----------------------------------------------------------------------------------------------------
test_qualifier (unittest.loader._FailedTest)
Traceback (most recent call last):
File "run_tests.py", line 229, in <module>
main()
File "run_tests.py", line 225, in main
runner.run(test_suite)
File "run_tests.py", line 71, in run
test(result)
File "/usr/lib/python3.8/unittest/suite.py", line 84, in __call__
return self.run(*args, **kwds)
File "/usr/lib/python3.8/unittest/suite.py", line 122, in run
test(result)
File "/usr/lib/python3.8/unittest/case.py", line 736, in __call__
return self.run(*args, **kwds)
File "/usr/lib/python3.8/unittest/case.py", line 695, in run
result.stopTest(self)
File "run_tests.py", line 199, in stopTest
test_description = test.shortDescription().rstrip(".!?")
AttributeError: 'NoneType' object has no attribute 'rstrip'
same as before
torn mirage
Assume you put it at the start of the method?
proper wind
"""Finalize the test phase of an individual test method."""
print(test)
test_description = test.shortDescription().rstrip(".!?")
self.results[self.current_testclass.name][test_description] = not self.failure_output
self.stream.write_test_outcome(test_description, self.failure_output)
I did
import java.util.Scanner;
public class EM215_U03_As02
{
public static void main(String [] args)
{
int age1 = 0, age2 = 0;
Scanner scan = new Scanner (System.in);
System.out.println ("Enter the first person's age:");
age1 = scan.nextInt ();
System.out.println ("Enter the second person's age:");
age2 = scan.nextInt ();
System.out.println(" You are " + Math.abs(age1-age2) + ("years apart."));
}
}
Is this right?
torn mirage
Don't raelly see what could be going wrong there when I look at the tests at github and have no idea which testcase it's failing at
proper wind
I really don't have a clue tbh, I was hoping that it gives me a simple output and then I can take it from there
torn mirage
try printing test.__class__
proper wind
@torn mirage I get the same error as above
torn mirage
yeah the error should stay, is there any output before that?
proper wind
Nope, its the same stack trace
torn mirage
Above the traceback, the test_qualifier (unittest.loader._FailedTest) should've chagned
proper wind
Python Discord Summer Code Jam 2020: Qualifier Test Results
====================================================================================================
Date: 2020-07-03 01:07:31 UTC
====================================================================================================
test_qualifier (unittest.loader._FailedTest)
----------------------------------------------------------------------------------------------------
test_qualifier (unittest.loader._FailedTest)
Testing for something
Traceback (most recent call last):
File "run_tests.py", line 230, in <module>
main()
File "run_tests.py", line 226, in main
runner.run(test_suite)
File "run_tests.py", line 71, in run
test(result)
File "/usr/lib/python3.8/unittest/suite.py", line 84, in __call__
return self.run(*args, **kwds)
File "/usr/lib/python3.8/unittest/suite.py", line 122, in run
test(result)
File "/usr/lib/python3.8/unittest/case.py", line 736, in __call__
return self.run(*args, **kwds)
File "/usr/lib/python3.8/unittest/case.py", line 695, in run
result.stopTest(self)
File "run_tests.py", line 200, in stopTest
test_description = test.shortDescription().rstrip(".!?")
AttributeError: 'NoneType' object has no attribute 'rstrip'
@torn mirage yes it did, my bad, I also added the statement 'Testing for something' to see if it is actually working
torn mirage
Is that with the __class__?
proper wind
yes, where it says test_qualifier (unittest.loader._FailedTest) I get that when I do print(test.__class__)
sorry forgot to save my code its actually <class 'unittest.loader._FailedTest'>
@torn mirage
torn mirage
I'm not really familiar with unittest to know how it works in the back but does one of the tests fail?
proper wind
I don't know, it just gives the above output, nothing about tests failing or anything, if I have that then I'll have a better idea that there is something wrong with my code. Its okay, thanks for your help, got work in the morning so thats me done
torn mirage
try replacing the line with some random string
I guess it is a problem with the suite
<@&267628507062992896> Not sure what exactly is happening above but looks like a problem with the tests when the _FailedTest gets passed to the custom stopTest from somewhere
stable juniper
interesting
hmm
did you remove any doc strings from the tests?
unittest seems to be saying that there is no docstring for a test
torn mirage
Do the test cases get turned into failed tests when they fail? Something wrong with the docstring was my conclusion at the start but it's getting the field test instead of the defined test cases
stable juniper
i'm not particularly sure. i'm not super familiar with unittest
perhaps @kind meadow might know why this is happening.
shut hedge
@proper wind hey, Manny, I was asleep. This shouldn't happen, but I probably did not account for something in the custom test runner. I'll DM you later.
proper wind
really sorry guys, it was spelling mistake on my part, I called my file qualifer.py instead of qualifier.py really sorry again, I woke up and looked at it again and noticed the spelling mistake straight away
upbeat kayak
replacement = Mock()
replacement.__str__.return_value = 'Replacement{}'.format(num)```why does this give me AttributeError: 'method-wrapper' object has no attribute 'return_value'
torn mirage
Try using a MagicMock
In [10]: a = MagicMock()
In [11]: a.__str__.return_value = "string"
In [12]: str(a)
Out[12]: 'string'
ty, that worked
proven lake
Given the state of pytest, what does unittest have pytest? Trying to make an argument that we should change my company's test framework to pytest. Being more readable is pretty evident. Curious if there were some pros that make unittest worth keeping.
marsh raft
I'm not aware of any; where I work we use pytest exclusively
@river pilot might have a clue
river pilot
@proven lake pytest has way more power
the only advantage of unittest is that you don't have to install another package, but that's no real advantage
i guess another advantage of unittest is that because it is simpler, you are forced to write tests in a simple way, but that could mean repetitive tests
marsh raft
👍
proven lake
👍 Thanks!
proud nebula
@proven lake pytest has a huge overhead. Unittest does not. But you can use my test runner hammett and get the 80% best parts of pytest with the speed of unittest ;)