#For those who ve tried Github Copilot
fathom wigeon
seems a bit better IMO
But I can also ask it to change things more easily than I can copilot
sharp kernel
You should, I was testing ChatGPT on some medium level leetcode problems and some of the solutions were a little iffy. Not sure how copilot performs on those types of questions
But i'd imagine ChatGPT's code explanations are a lot more indepth than copilot
fathom wigeon
I gave it my favorite interview question. The first answer is not really correct, but also matches what took some candidates like 30min haha
vivid tree
the logic seems to be complete ,
fathom wigeon
yeah, it's good with simple stuff.
Starting with
First, explain some ways you might write this function and then pick your favorite design. We'll write the function after.
and then
Okay, write the function using your approach!
does a little better.
radiant notch
actually a LOT better. Copilot can generate pseudocode that is meaningful, but doesn't really work. While ChatGPT generates actual working code, that is often 100% correct
fathom wigeon
to be fair, copilot is decent too. I think the fact that we can give ChatGPT context more easily makes us a bit more biased.
tawdry goblet
to be fair, copilot is decent too. I think the fact that we can give ChatGPT con...
+1 to this. Couldn't agree more. But it's justifiable bias. Objective even.
It can, objectively, do a better job because we can give it more information.
toxic citrus
I think copilot is still better, but just because it is directly plugged to the IDE where it can see more context easily
but having something similar to chatGPT, trained on github, using all its context capabilities with the context the IDE can offer.. it would be amazing
sharp kernel
I’m wondering if it’s possible to make a chatGPT vscode extension.
toxic citrus
copilot maight be updated some day to use it
chatGPT is not a release, it is a test
it might in a near future be availiable on the API
and then other products will be released on top of it
radiant notch
even if copilot is updated, it's WAY easier to use chat interface to ask stuff that you know nothing about. For copilot you'd have to write comments and do some trickery
deft python
final nebula
The Github Labs VSCode extension has a bunch of ways you can explore using Copilot more interactively, fwiw. I think they're both on similar levels, but ChatGPT likely seems more convincing to people because it is designed to be “chatty” so it provides more explanations. Unfortunately, as StackOverflow has seen, the answers do a better job at seeming helpful than actually being helpful. StackOverflow has actually (temporarily) banned the use of ChatGPT for answers at all (kind of hard to police, but that's the rule).
hollow cedar
I haven't tested copilot yet, but I tested code-davinci-002 (which I believe copilot is based on) and it's significantly worse at handling domain specific languages like emacs lisp compared to chatgpt. It can handle simple prompts, but give it anything more complex and it outputs nonsense or repeats the prompt. Whereas chatgpt can output coherent answers to complex queries and explain the code in plain language, even though the code may not run correctly.
granite bone
I think currently they both have their strengths but chatGPT has got several legs up for now, copilot has more context for a given project, and inline completions are near instant, but chatGPT allows you to talk about a problem, brainstorm solutions, have the AI suggest some code to solve it, wherin not only does it normally generate decent code to start with, but its nicely commented, and then comes with a description as to how it works, and the best part about it, is even if its off, or its not quite right, you can just tell it exactly what it got wrong and then it fixes it.
when copilot is off about something, its generally easier to just keep writing something yourself until it catches on, rather than spending the time figureing out how to reword your comments to coax out the right sort of solution out of it
hollow cedar
gpt-j is the open source alternative to gpt-3 and is supposed to be better at coding? Has anyone tested and compared them?
vital mist
I've tried GPT-J-6B and found it was awful, it kept pretending to be someone else - one run said it was happy to be talking "in this forum" (sample must have been from a chat board) and my last run said it was somebody working in a nail palour. When I asked for help with things it would bring back relevant stuff, like a few paragraphs, but then following paragraphs would often be broken or incorrectly inferred. Any long chat history would also slow it down massively on my 16GB 3080 (around 65% in GPU ram, the rest of the model in system ram), and in order for it to respond quickly, it would have to be ~80 chars per response - not helpful for coding. Additionally when doing regular queries, it would make up a lot of rubbish, such as incorrectly telling me when Blackpool Tower was built and its history. I can imagine hosting it online would give better performance, but the data accuracy was abysmal. ChatGPT may be proprietary but its still miles better than this open source variant 😦
sharp kernel
looks like it blew up massively on TikTok judging by the discussions channel messages now lol