🔒 Help with generators

[past crown]

@whole spindle

Python help channel opened

Remember to:

• Ask your Python question, not if you can ask or if there's an expert who can help.
• Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
• Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.

[whole spindle]

So I am trying to make a webcrawler (for university) that will look at all the links on a page and the access those links and so on and so forth. Here is what I have: https://github.com/braxten509/proj4/blob/master/webcrawler.py

However, the code will print each link like 20 times and I am unsure why it's doing that. Any help would be much appreciated!

It is all in the count_links function btw

[vernal shoal]

        if link in links_counted.keys():
            links_counted[link] += 1
            print(f"Processed: {link}")
```I'm guessing it's because you're printing it even when it's already been processed

also I suggest using a queue instead of a list + generator

[whole spindle]

Oh cool!

How do I make a queue?

[vernal shoal]

https://docs.python.org/3/library/collections.html#collections.deque

[whole spindle]

Ok ill look at that!

[whole spindle]

Okay, so I want to modify the "queue" or "list" during its iteration so that it will keep cycling through newly added links. Queue doesn't seem capable of doing that, so what is or how could I?

[vernal shoal]

just pop something from the queue when you need it, and push something to the queue when you have it

don't use a for-loop to iterate through it, use a while loop to repeat that process until the queue is empty

[whole spindle]

oooh okay

let me try that

[whole spindle]

it seems to work, thank you!

!close

[past crown]

Python help channel closed

This help channel has been closed and it's no longer possible to send messages here. If your question wasn't answered, feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.