#wait for GPT-4 API too long

49 messages · Page 1 of 1 (latest)

solar jasper
#

I summitted to access GPT-4 months ago, why I do not receive any response.
Please help. I need an accuracy model

modest marlin
#

GPT4 is for choosen/lucky ones. Lots of demand and most can use only gpt 3.5 turbo model. I heard that 3.5 is getting better and after all recent development in AI I would count more lighter models getting better.

hearty charm
#

Not sure about OpenAi, but i think i seen GPT-4 onboarding at Azure is paused right now.

#

Gotta wait for the next batch of H100s or something more exotic coming online 👐

granite basin
hearty charm
#

H100s are the lastgen of AI GPUs

#

if you still wanna call it GPU

hearty charm
#

I remember that older models also already required nearly a full DGX (8x A100 GPUs with 80GB Memory each) to just run interference. (Aka using the model)

#

Maybe GPT-4 is so heavy on interference that it requires something special.

flint thunder
#

And how often do you get errors due to overload? With open ai it seems to be quite often, especially in the rush hours.

granite basin
# flint thunder Oh, that's interesting. Say, is the price different from what open ai offers?

Nope, price is exactly the same and I think you get a new deployment for each model so if OpenAI goes down (which is usual now) it won't affect you, that's why I'm looking for GPT4 on Azure
And gpt 3.5 on Azure is really fast

OpenAI, 45 seconds
Azure OpenAI, 20 seconds

https://www.reddit.com/r/OpenAI/comments/1404wsk/azure_openai_is_a_lot_faster_than_openai_in_terms/

reddit
r/OpenAI - Azure OpenAI is A LOT faster than OpenAI in terms of speed

16 votes and 16 comments so far on Reddit

▶ Play video
flint thunder
granite basin
solar jasper
#

Thanks all for your info ❤️

hearty charm
granite basin
#

Also, for some reason I'm able to use more than token limit of gpt-3.5 which is 4000 something.
I wanted to handle the error in case I reach the token limit so I added Paul Grahams essay which is ~5k tokens but I'm not getting any errors

With OpenAI I get token limit reached for the same model

flint thunder
granite basin
# flint thunder You can see the token count in the API call response.

Azure OpenAI is dumb. I guess they don't have checks for the input but input gets cut off at 150 tokens if the total token is more than 4000 so you will never get any error.

This time I directly called their API and not through my code.

{
    "id": "cmpl-7Po2XpCdSCNw1yfMt6TMX5Ieh6yR4",
    "object": "text_completion",
    "created": 1686384909,
    "model": "gpt-35-turbo",
    "choices": [
        {
            "text": " All errors are my own. Explanation of the Vector explanation:\n\nIf you consider the vector between two companies in the same market, the question is not which one has a more appealing initial idea, but which one can get users faster. Those are the two components of the vector. If anything, pre-seed startups are even more constrained by needing to get users manually. They're not only competing against companies in the same market that have been around longer, but against startups three classes behind them. If anyone is interested in a follow-up post on why startups need to get users manually and everything to do with it, let me know.\n\"\"\"\ninfo = extract_information(article)\nfor key in info:\n    print(f\"{key}: {info[key]}\")<|im_sep|>",
            "index": 0,
            "finish_reason": "stop",
            "logprobs": null
        }
    ],
    "usage": {
        "completion_tokens": 150,
        "prompt_tokens": 5236,
        "total_tokens": 5386
    }
}
#

One heads up before moving to Azure @flint thunder
Their content filtering is trash so if you have a user facing app where user enter text, make sure to add error handling for the content filters.

flint thunder
flint thunder
#

@granite basin hey. I was trying to sign up for the Azure open ai, but they seem to require a company email, not a personal one. Since this is a side project unrelated to my work, I don't have a company email for it...
Did you provide a company email to them?

#

Ugh... There are other company related questions. I guess they don't provide access to individuals...

#

Yeah. It's only for enterprises now. Damn...

Azure OpenAI requires registration and is currently only available to approved enterprise customers and partners.
#

That explains why their api is not overloaded I guess.😅

granite basin
warped moon
#

I think Azure now stopped giving gpt4 access same as openai - I was added onto the waitlist for gpt4 (after being approved for oai services) and got this We are writing to confirm that you have been added to the GPT-4 waitlist.  Please note that we are currently paused on onboarding new GPT-4 customers due to the high demand and do not have an estimated time for when we will onboard new customers. We will let you know when we are able to onboard you.

silent fiber
#

Have you tried using stream?

granite basin
granite basin
silent fiber
hearty charm
#

When i gone over 4k Tokens i had to start using stream, normal completion would just timeout 😄

amber bane
#

Does anyone know of a company that has gotten access to Azure GPT-4?

foggy light
brisk merlin
#

When will this wait be over?

granite basin
north sigil
#

Fwiw, I applied for azure openai this afternoon and got access a few hours ago. I also applied/got access to the Microsoft for startups program today, unclear if these things are related. But they're going through them at least.

granite basin
granite basin
north sigil
#

oh, damn, it's separate from the general form you fill out?

granite basin
north sigil
#

got a link to that?

granite basin
#

sure, let me find it

north sigil
#

thanks!

granite basin
north sigil