Cheap Article Generation

[worthy lintel]

We are building an AI model to take our video content, use whisper to transcribe it, and present the information in written form, as well as optimize it for our social media posts.

However, we are currently facing issues with the API costs being significant when using GPT-4, but the prompts being sub-par when using GPT-3.5-Turbo.

We take a very large summary of the video transcript, and then create a structure for the written content and publish it.

Any proposals to improve it? Any success yourself?

[hexed blaze]

You could try using llama-2, it's an open source LLM that levels with GPT-3.5's performance and it's kind of "free", it can summarize long paragraphs, but the maximum context length is 4096 tokens though

[true heron]

I don't know if it will fit your case, but you could try working in smaller segments, i.e., summarizing segment by segment. This is how I already do it with a web chat GPT version and a VoxScript plugin.

you may reduce costs that way

also you need a good prompt for the summarizing process - what exactly you want to extract from the transcription

[worthy lintel]

That's already done

The issue mainly comes from generating the article

See the last 3

very generic, hyperbolaic

typical GPT style