fickle rivet
I'm looking to submit an arbitrary number of prompts and then save each output as a pair to the input.
I should be able to write myself a script to do this using the API, but I just wanted to check that no one else had done something similar first. A cursory look through the extensions and PRs didn't seem to suggest anything.
Thanks!
soft pewter
not so far as i'm aware, but other people have asked about it
fickle rivet
That was near instantaneous, thank you!
soft pewter
if you do a search, you might find something, but unfortunately 'batch' is also a training term
someone may have built something. others want multiplexed queries too
i'd expect something to be added to the distro in time
a contribution would obviously be most welcome
fickle rivet
Yes, that was my first issue. I tried table, X,Y (which did return something, but for running the same prompt over different models), but mostly it just returned poor results.
Thank you for your help - I'll take a further look around.
wet gorge
openai supports batched input for prompts, but I have not added support for them in the openai extension yet, but it's moving to the top of my list. I'm also interested is anything you find, I've seen recently that the lower level tensors for generation are multidimensional already, so it could be efficient, but I have a feeling it would just use more vram and end up not being possible ... and you would just need to loop over the prompts anyways.