#implementing system for chatgpt

30 messages · Page 1 of 1 (latest)

onyx ivy
#

Hey, cs student here, I have to make a final year project.
My idea is that I make a chess game with stockfish as opponent. Chatgpt will engage with the player for stuff like threats or plans or blunders.

Now, for eg- player plays nf3. Stockfish will provide lines to chatgpt or chatgpt can directly access lines of opponent(stockfish) Suppose there is a line where nf3 will lead to blunder of the queen, then chatgpt will tell the player that queen might be taken in future or this move will blunder your queen

The problem is : how will chatgpt be able to identify the line where queen is taken?
Chatgpt says that I have to create a system where I could translate the threats and then prompt chatgpt to respond

I just want chatgpt to know the threat and tell the player that the move is bad?

How does stockfish analysis work? Can it tell chatgpt that a certain line could lead to loss queen?

Please help me out with this.

frigid epoch
#

Sf can't convey in natural language what's good or bad about a position

#

And what can a move lead to

#

What SF can give you is the strongest move to play and how that affects the game state (score wise)

#

Say you have position X

#

The best move is queen B4 because otherwise you leave a queen hanging

#

SF will simply say

#

Best move queen B4 score Y

#

It will not communicate anything about a piece being left hanging or anything else

#

That being said for threats you can do so something like lichess does

#

"if we were to give our opponent a free move what would they do"

#

To get that you can just feed SF the same position but with an inversed STM and see what it wants to play

thin tendon
#

first thing to do would be to check how much chatgpt can infer from SF's PV

#

if not much then it will be very hard to make use of it [chatgpt]

frigid epoch
#

Yeah you can feed it the response and hope it Infers on its own the whys

#

But with no system to check if it's spouting bullshit it's meh

#

But I guess anyone that uses chatgpt expects potentially bullshit answers

onyx ivy
#

you are right, there is no way to tell chatgpt that this move is good because of said reason. I would have to rely on chatgpt's capabilities to comment on the position because I can't convey the logic behind the moves.

#

At this point I can only make a chess game where player plays against chatgpt which uses stockfish moves. Which is super lame

midnight temple
#

Maybe you can extract the PV, play it on the board and use eval UCI command to understand why the resulting position is better

#

You can even play every single move separately, collect PV from each of them, play the PVs on the board and then call eval to understand why move a was chosen instead of move b or move c.

#

But the explanation would be too crude anyway, I guess

onyx ivy
#

surprisingly, chatgpt is not too shabby in chess especially on commenting on chess positions. Although the commentry is not even close to stockfish level, it can still fool amateur players like myself that the position he recommended is good and will spit out some comments too!

#

Maybe I can fool the project examiner that this is ballsdeep level depth analysis and hopefully chatgpt won't recommend illegal moves!!!

midnight temple
#

Well the best thing you can get ChatGPT to do is something like "move a is the best option in this position because move b blunders the knight after ...moves... and move c leads to a position where opponent has a strong passed pawn". Though this needs an immense amount of prompt engineering and maybe even tuning.

Suppose you run a multipv search on your position:

1) e2e4 e7e5 g1f3 b8c6
2) d2d4 d7d5 g1f3 e7e6
3) e2e4 c7c5 g1f3 b8c6
...

Now you can play each of these PV lines on the board:

position fen [FEN] moves [PV Moves]

and then use eval command to get results of different evaluation terms such as mobility, passed pawns, space, king safety etc:

#

You can craft a detailed prompt using all of these information (though I'm not sure if the displayed terms are in cp scale, I've not checked SF src).

frigid epoch
#

Yeah using hce might help