Trying to parse a recipe website using goquery, having a bit of an issue getting nested elements

[echo nacelle]

So I'm trying to get the ingredients for a recipe (trying goquery, used colly before to no success either). The HTML is the following way:

<div class="recipe-ingredients">
  <ul>
    <li>1 ½ lb chicken breast tenders</li>
    <li><strong>Marinade</strong>
      <ul>
        <li>1 egg</li>
        <li>1 tablespoon smoked paprika</li>
        <li>2 teaspoons garlic powder</li>
        <li>2 teaspoons onion powder</li>
        <li>Pinch of sea salt &amp; pepper</li>
        <li>1 tablespoon maple syrup</li>
      </ul>
    </li>
    <li>1 tablespoon arrowroot (or corn starch)</li>
    <li>1 ½ cups corn flakes (crushed)</li>
  </ul>
</div>

This is what I have right now:

doc.Find(".recipe-ingredients").Each(func(i int, selection *goquery.Selection) {
        //selection.Find("ul").First().
    })

As it shows, the second ingredient Marinade has sub-ingredients. I'd like to store these ingredients on the side. As in:

{
  Name: Marinade,
  Ingredients: ["1 egg", "1 tablespoon...", ...]
}
{
  Name: MainRecipe
  Ingredients: ["1 ½ lb chicken...",...]
}

When I was trying with colly, here I would now use selection and search the li elements, but I couldn't find a way to just search the first layer of children, so I would have to run over all of the elements.

Is there a way for me to tell it to search all the li elements that aren't inside another li element?

[austere mortar]

<div class="ingredients"> does not match .Find(".recipe-ingredients")

[echo nacelle]

Yeah, I copy pasted and forgot to adapt the code when writing it down. In my case the class is recipe-ingredients and I do get inside that find. It is once inside that idk how is best to proceed

[austere mortar]

You already have selection.Find("ul").First(). If that doesn't return an element, then you're at top level. If it does, it's a sub-recipe. 🤷

[echo nacelle]

Yeah but I do have to parse each li. If I do

selection.Find("li").Each()

Then I'll first get to the Marinade level, and from here I can detect if there's a ul and get the sub-ingredients for the Marinade. But on the next iteration, Instead

instead of going from Marinade to 1 Tablespoon arrowroot it goes to 1 egg, which I have already parsed when doing Marinade

I don't know if I explain myself well enough

I'd want something like selection.Find("li").Each() inside the ul, but only at that level. If there's a ul and li inside of it, to ignore those

or another way to achieve the same

[austere mortar]

Show the code?

[echo nacelle]

Right now I don't have much more code, because I can't seem to find a way to achieve what I want.

When doing the equivalent selection.Find("li").Each() with colly, this is each iteration:

1 ½ lb chicken breast tenders
Marinade
1 egg // FROM MARINADE
1 tablespoon smoked paprika // FROM MARINADE
 2 teaspoons garlic powder // FROM MARINADE
2 teaspoons onion powder // FROM MARINADE
Pinch of sea salt & pepper // FROM MARINADE
1 tablespoon maple syrup // FROM MARINADE
1 tablespoon arrowroot (or corn starch)
1 ½ cups corn flakes (crushed)

Meanwhile I'd rather get this:

1 ½ lb chicken breast tenders
Marinade
1 tablespoon arrowroot (or corn starch)
1 ½ cups corn flakes (crushed)

What I have now with goquery is this, but I don't ever get to the println inside the second Find:

doc.Find(".recipe-ingredients").Each(func(i int, selection *goquery.Selection) {
        // other code
        selection.Find("ul").Each(func(i int, selection *goquery.Selection) {
            println(selection.Text()) // never gets here
        })
    })

Tried it this way too:

doc.Find(".recipe-ingredients").Each(func(i int, selection *goquery.Selection) {
        // other code
        selection.ChildrenFiltered("ul").Each(func(i int, selection *goquery.Selection) {
            println(selection.Text()) // never gets here
        })
    })

[echo nacelle]

Nevermind this worked now, maybe I can do this way. I'll try it in a bit

[austere mortar]

Yeah, okay. 🤷 Neverminding, then 🙂

I was trying to get a minimum example going in a playground, but it refused to load goquery for me. 😆

[echo nacelle]

This way I have the same problem I describe here, though. I go through each li even the one inside marinade

doc.Find(".recipe-ingredients").Each(func(i int, selection *goquery.Selection) {
        //other code
        selection.ChildrenFiltered("ul").Each(func(i int, selection *goquery.Selection) {
            selection.ChildrenFiltered("li").Each(func(i int, selection *goquery.Selection) {
                println(selection.Text())
            })
        })
    })

I'll check more later

[austere mortar]

Anyway, what I was thinking is that you just check the if there are any sublists, and use that to know if you're at the top level, or in an under-recipe.

Something to the tune of https://goplay.tools/snippet/VJdappWhjQx

Though that code is obviously very quick-and-dirty, just illustrates what I mean.

You'd need to accumulate the data in a struct and blah blah blah, all the surrounding logic, obviously.

[echo nacelle]

This does help a lot, now I'm having trouble with a few other things but that does solve the initial issue. I'm trying to fix the few issues now! Thanks!

[echo nacelle]

https://goplay.tools/snippet/EetINjm9PFP

Your code helped a lot. I've tried making it recursive in case there are more internal ones but don't manage to do it. In any case, it's a huge improvement from before with your hlep

[austere mortar]

It is my experience that recursion is very rarely needed and almost always makes things more complicated than they need to be.

[echo nacelle]

No idea how to do that then

Is not like I know how to do it with recursion either, but in my head that should work

[austere mortar]

You can do it with a regular loop and a cursor variable.