Code archives/Algorithms/Tic tac toe that learns
This code has been declared by its author to be Public Domain code.
Download source code
| |||||
| This is a fun tic tac toe game that I made that has VERY basic controls and graphics. This basic program slowly learns from its mistakes. The better you are at tic-tac-toe the better it will become after a few hundred games against you :) If you make tons of mistakes or tie a lot then it will end up the same way. Enough about that here's how it works... The first time you play it, it saves your game after moving almost completely randomly... The second time you play it, it will do the same but this time if you are playing the opposite side you were on the first time it may try and pull the same trick on you, however there is a 2 in 10 chance it will try a new move that it doesnt know will make it win... ... 1000 plays later it has figured out all of your moves and how to defend them and it is hard for you to beat... or its at least a little challenge :) thats how it works so here it is... Very sorry for the extremely simple graphics and controls but that was not the point of this project... controls are 1 2 3 q w e a s d for each square on the board other things to know sometimes it goes first sometimes you go first there is a delay of 400 ms before it moves the bigger the number above the board the better its chances of winning p.s. sorry its not commented but that shouldnt be too much of a problem because it is relatively simple and I just explained it IMPORTANT: make this program its own folder with nothing else in it so that it doesnt fill your other folders with junk :) | |||||
; ID: 2370
; Author: Nate the Great
; Date: 2008-12-10 02:54:40
; Title: Tic tac toe that learns
; Description: Learns... slowly but surely
SeedRnd(MilliSecs())
Graphics 160,120,0,2
Dim mov(9)
Dim grid(3,3)
Dim moved(9)
Global membankcount = 0, test = 0
Type game
Field m[9],winner
End Type
cnt = 0
While ext <> 1
fileopn$ = "membank" + cnt + ".mem"
cnt = cnt + 1
file = OpenFile(fileopn$)
If Not file = 0 Then
g.game = New game
g\m[1] = ReadInt(file)
g\m[2] = ReadInt(file)
g\m[3] = ReadInt(file)
g\m[4] = ReadInt(file)
g\m[5] = ReadInt(file)
g\m[6] = ReadInt(file)
g\m[7] = ReadInt(file)
g\m[8] = ReadInt(file)
g\m[9] = ReadInt(file)
g\winner = ReadInt(file)
membankcount = membankcount + 1
CloseFile(file)
file = OpenFile(fileopn$)
g.game = New game
g\m[7] = ReadInt(file)
g\m[4] = ReadInt(file)
g\m[1] = ReadInt(file)
g\m[8] = ReadInt(file)
g\m[5] = ReadInt(file)
g\m[2] = ReadInt(file)
g\m[9] = ReadInt(file)
g\m[6] = ReadInt(file)
g\m[3] = ReadInt(file)
g\winner = ReadInt(file)
CloseFile(file)
file = OpenFile(fileopn$)
g.game = New game
g\m[9] = ReadInt(file)
g\m[8] = ReadInt(file)
g\m[7] = ReadInt(file)
g\m[6] = ReadInt(file)
g\m[5] = ReadInt(file)
g\m[4] = ReadInt(file)
g\m[3] = ReadInt(file)
g\m[2] = ReadInt(file)
g\m[1] = ReadInt(file)
g\winner = ReadInt(file)
CloseFile(file)
file = OpenFile(fileopn$)
g.game = New game
g\m[3] = ReadInt(file)
g\m[6] = ReadInt(file)
g\m[9] = ReadInt(file)
g\m[2] = ReadInt(file)
g\m[5] = ReadInt(file)
g\m[8] = ReadInt(file)
g\m[1] = ReadInt(file)
g\m[4] = ReadInt(file)
g\m[7] = ReadInt(file)
g\winner = ReadInt(file)
g.game = New game
g\m[3] = ReadInt(file)
g\m[2] = ReadInt(file)
g\m[1] = ReadInt(file)
g\m[6] = ReadInt(file)
g\m[5] = ReadInt(file)
g\m[4] = ReadInt(file)
g\m[9] = ReadInt(file)
g\m[8] = ReadInt(file)
g\m[7] = ReadInt(file)
g\winner = ReadInt(file)
CloseFile(file)
file = OpenFile(fileopn$)
g.game = New game
g\m[9] = ReadInt(file)
g\m[6] = ReadInt(file)
g\m[3] = ReadInt(file)
g\m[8] = ReadInt(file)
g\m[5] = ReadInt(file)
g\m[2] = ReadInt(file)
g\m[7] = ReadInt(file)
g\m[4] = ReadInt(file)
g\m[1] = ReadInt(file)
g\winner = ReadInt(file)
CloseFile(file)
file = OpenFile(fileopn$)
g.game = New game
g\m[7] = ReadInt(file)
g\m[8] = ReadInt(file)
g\m[9] = ReadInt(file)
g\m[4] = ReadInt(file)
g\m[5] = ReadInt(file)
g\m[6] = ReadInt(file)
g\m[1] = ReadInt(file)
g\m[2] = ReadInt(file)
g\m[3] = ReadInt(file)
g\winner = ReadInt(file)
CloseFile(file)
file = OpenFile(fileopn$)
g.game = New game
g\m[1] = ReadInt(file)
g\m[4] = ReadInt(file)
g\m[7] = ReadInt(file)
g\m[2] = ReadInt(file)
g\m[5] = ReadInt(file)
g\m[8] = ReadInt(file)
g\m[3] = ReadInt(file)
g\m[6] = ReadInt(file)
g\m[9] = ReadInt(file)
g\winner = ReadInt(file)
CloseFile(file)
Else
ext = 1
EndIf
Wend
tmprndnum = Rnd(100)
If tmprndnum > 50 Then
tmprndnum = 1
Else
tmprndnum = 0
EndIf
Global move = tmprndnum,num
Global start = move
SetBuffer BackBuffer()
Cls
AppTitle("Tic-Tac-Toe")
Text 1,1, "Tic-Tac-Toe"
;Text 1,20,"Press a key."
Flip
;WaitKey()
;Delay 1000
FlushKeys()
Cls
Text 1,1, "Tic-Tac-Toe"
Line 14,40,14,95
Line 34,40,34,95
Line 1,54,54,54
Line 1,74,54,74
Flip
ext = 0
While Not ext
Cls
If KeyDown(1) Then ext = True
Text 1,1, "Tic-Tac-Toe"
;If move = 0 Then
If move = 0 Then
Goto wngplc2
.wngplc
num = num - 1
.wngplc2
WaitKey()
num = num + 1
If KeyHit(2) Then
If moved(1) = 0 Then
mov(num) = 1
moved(1) = 1
Else
FlushKeys()
Goto wngplc
EndIf
ElseIf KeyHit(3)
If moved(2) = 0 Then
mov(num) = 2
moved(2) = 1
Else
FlushKeys()
Goto wngplc
EndIf
ElseIf KeyHit(4)
If moved(3) = 0 Then
mov(num) = 3
moved(3) = 1
Else
FlushKeys()
Goto wngplc
EndIf
ElseIf KeyHit(16)
If moved(4) = 0 Then
mov(num) = 4
moved(4) = 1
Else
FlushKeys()
Goto wngplc
EndIf
ElseIf KeyHit(17)
If moved(5) = 0 Then
mov(num) = 5
moved(5) = 1
Else
FlushKeys()
Goto wngplc
EndIf
ElseIf KeyHit(18)
If moved(6) = 0 Then
mov(num) = 6
moved(6) = 1
Else
FlushKeys()
Goto wngplc
EndIf
ElseIf KeyHit(30)
If moved(7) = 0 Then
mov(num) = 7
moved(7) = 1
Else
FlushKeys()
Goto wngplc
EndIf
ElseIf KeyHit(31)
If moved(8) = 0 Then
mov(num) = 8
moved(8) = 1
Else
FlushKeys()
Goto wngplc
EndIf
ElseIf KeyHit(32)
If moved(9) = 0 Then
mov(num) = 9
moved(9) = 1
Else
FlushKeys()
Goto wngplc
EndIf
EndIf
Else
Delay 400
Goto wngplc4
.wngplc3
num = num - 1
.wngplc4
match = 0
smmove = 0
count = 0
For g.game = Each game
count = count + 1
same = 0
If g\winner = 1 And start = 1 And num > 0 Then
same = 1
For a = 1 To num
If g\m[a] <> mov(a) Then same = 0
Next
ElseIf g\winner = 2 And start = 0 And num > 0 Then
same = 1
For a = 1 To num
If g\m[a] <> mov(a) Then same = 0
Next
EndIf
If num = 0 And g\winner = 1 Then
same = 1
EndIf
If same = 1 Then
chnce = Rnd(0,10)
If chnce > 5 Then
smmove = g\m[num+1]
test = True
EndIf
Else
Delete g.game
EndIf
Next
If smmove = 0 Then
tmppic = Rnd(1,9)
Else
tmppic = smmove
EndIf
num = num + 1
If tmppic = 1 Then
If moved(1) = 0 Then
mov(num) = 1
moved(1) = 1
Else
FlushKeys()
Goto wngplc3
EndIf
ElseIf tmppic = 2 Then
If moved(2) = 0 Then
mov(num) = 2
moved(2) = 1
Else
FlushKeys()
Goto wngplc3
EndIf
ElseIf tmppic = 3 Then
If moved(3) = 0 Then
mov(num) = 3
moved(3) = 1
Else
FlushKeys()
Goto wngplc3
EndIf
ElseIf tmppic = 4 Then
If moved(4) = 0 Then
mov(num) = 4
moved(4) = 1
Else
FlushKeys()
Goto wngplc3
EndIf
ElseIf tmppic = 5 Then
If moved(5) = 0 Then
mov(num) = 5
moved(5) = 1
Else
FlushKeys()
Goto wngplc3
EndIf
ElseIf tmppic = 6 Then
If moved(6) = 0 Then
mov(num) = 6
moved(6) = 1
Else
FlushKeys()
Goto wngplc3
EndIf
ElseIf tmppic = 7 Then
If moved(7) = 0 Then
mov(num) = 7
moved(7) = 1
Else
FlushKeys()
Goto wngplc3
EndIf
ElseIf tmppic = 8 Then
If moved(8) = 0 Then
mov(num) = 8
moved(8) = 1
Else
FlushKeys()
Goto wngplc3
EndIf
ElseIf tmppic = 9 Then
If moved(9) = 0 Then
mov(num) = 9
moved(9) = 1
Else
FlushKeys()
Goto wngplc3
EndIf
EndIf
EndIf
For a = 1 To 9
If a = 1 Or a = 3 Or a = 5 Or a = 7 Or a = 9 Then
Select mov(a)
Case 1
Text 1,40,"X"
Case 2
Text 20,40,"X"
Case 3
Text 40,40,"X"
Case 4
Text 1,60,"X"
Case 5
Text 20,60,"X"
Case 6
Text 40,60,"X"
Case 7
Text 1,80,"X"
Case 8
Text 20,80,"X"
Case 9
Text 40,80,"X"
End Select
Else
Select mov(a)
Case 1
Text 1,40,"O"
Case 2
Text 20,40,"O"
Case 3
Text 40,40,"O"
Case 4
Text 1,60,"O"
Case 5
Text 20,60,"O"
Case 6
Text 40,60,"O"
Case 7
Text 1,80,"O"
Case 8
Text 20,80,"O"
Case 9
Text 40,80,"O"
End Select
EndIf
Next
Line 14,40,14,95
Line 34,40,34,95
Line 1,54,54,54
Line 1,74,54,74
move = Not move
If checkwin() Then ext = 1
If num = 9 Then ext = 1
Text 1,20,count
Flip
Wend
winner = checkwin()
Cls
If winner = 1
If start = 0
Text 1,1,"Player Wins"
Else
Text 1,1,"Computer Wins"
EndIf
ElseIf winner = 2
If start = 1
Text 1,1,"Player Wins"
Else
Text 1,1,"Computer Wins"
EndIf
EndIf
fileopn$ = "membank" + membankcount + ".mem"
filetmp = WriteFile(fileopn$)
For a = 1 To 9
WriteInt(filetmp,mov(a))
Next
WriteInt(filetmp,winner)
Delay 1000
Flip
Delay 1000
End
Function checkwin()
win = 0
For x = 1 To 3
For y = 1 To 3
grid(x,y) = 0
Next
Next
For a = 1 To 9
If mov(a) > 0 Then
If a = 1 Or a = 3 Or a = 5 Or a = 7 Or a = 9 Then
If mov(a) > 3 And mov(a) < 7
grid(2,mov(a)-3) = 1
ElseIf mov(a) > 6
grid(3,mov(a)-6) = 1
ElseIf mov(a) < 4
grid(1,mov(a)) = 1
EndIf
Else
If mov(a) > 3 And mov(a) < 7
grid(2,mov(a)-3) = 2
ElseIf mov(a) > 6
grid(3,mov(a)-6) = 2
ElseIf mov(a) < 4
grid(1,mov(a)) = 2
EndIf
EndIf
EndIf
Next
For a = 1 To 3
If grid(a,1) = 1 And grid(a,2) = 1 And grid(a,3) = 1 Then win = 1
If grid(a,1) = 2 And grid(a,2) = 2 And grid(a,3) = 2 Then win = 2
If grid(1,a) = 1 And grid(2,a) = 1 And grid(3,a) = 1 Then win = 1
If grid(1,a) = 2 And grid(2,a) = 2 And grid(3,a) = 2 Then win = 2
Next
If grid(1,1) = 1 And grid(2,2) = 1 And grid(3,3) = 1 Then win = 1
If grid(1,3) = 1 And grid(2,2) = 1 And grid(3,1) = 1 Then win = 1
If grid(1,1) = 2 And grid(2,2) = 2 And grid(3,3) = 2 Then win = 2
If grid(1,3) = 2 And grid(2,2) = 2 And grid(3,1) = 2 Then win = 2
Return win
End Function |
Comments
| ||
| some things I forgot to mention in my first post... here is how it "learns" there are 3 stages after bout 100 plays = 1st stage it now generally knows how to get 3 in a row but its not perfect and it is vulnerable after bout 500 plays = 2nd stage it now can block some/most attacks depending on how well you play after bout 1000 plays = 3rd stage it is now pretty good and it keeps you alert please post if you get different results hmm... I guess it is too good lol after playing a long time I realized I was playing some of the same games over and over. I assume this is because i have been the only one playing it and it has become like me in the way it plays... therefore it is like playing yourself... I recommend having many people play it so that this doesn't happen as it did to me |
| ||
| You need to include the save file. |
| ||
| it doesnt really need a save file... it senses whether there is already a file and if there isnt it creates one. |
| ||
| moved a closefile. Tht fixed the problem. |
| ||
| oh sorry I will update that thanks! |
| ||
| hmmm... are you useing blitz plus or b3d? that may be the problem |
| ||
| B3D. The closefile was outside of the if statement, so even though it didn't exist, that closefile would still execute. |
| ||
| Interesting, but you must keep in mind that tic-tac-toe is really a set game. If the first move is an "X" in one corner, it will always be a tie or X's win if the X player knows how to move, no matter what O does (ever seen those "Beat The Bird" games at carnivals, that is how they work). Still impressive, and I would love to see something like this with checkers! |
| ||
| H. T. U. that is exactly what this program takes advantage of! however it could be applied to games like checkers and chess possibly if it is modified |
| ||
| Cool - I'll have to look at this to see how it works. It seems like a form of machine learning. Have you considered modifying it so that two computers could play against each other? It would quickly become invincible - or crash. :-) I'm very interested in neural networks, machine learning, and other AI. This is cool! On a related topic, has anyone created a neural network with Blitz?.. |
| ||
| Have you considered modifying it so that two computers could play against each other? It would quickly become invincible - or crash. :-) yep and it doesnt learn very fast pleying against an opponent that knows just as much as itself :) On a related topic, has anyone created a neural network with Blitz?.. as far as I know the answer is no :( although it would be interesting |
| ||
| Showing my age here but I had a flashback to http://www.haar.clara.co.uk/Lights/merlin.html |
| ||
| What's key here, of course, is the code for HOW the computer 'learns', and it's "understanding" of how to play. It's well done here, for tic-tac-toe, and a very clear example, but would need sdome heavy work for something like chess and other gameplay where the potential for strategc moves increases and the releveance of moves varies by greater amounts to more possible favours such as defensive play, offensive play, and the various "values" that can be ascribed to defending or attacking a particular piece. I prefer to go with chess AI where the difference between a "good AI level" to a "poor AI level" is the numebr of iterations of possible moves checked for 'better overall values' and consideration of (depth of numbers of ) counter-moves. Essentially, the AI would spend more time 'thinking' on a move. Allowing a chess program to 'learn' as in the above Tic-Tac-Toe method, would be somewhat specific to circumstances, popponent and opening strategies, which are MUCH more versatile than Tic-Tac-Tow, and where the generalised gameplay strategy is sufficient. |
| ||
| It is possible to use learning AI's in games: - The learning AI was one of the main selling points for Fear (the "can't beat it the 3rd time" line is because the AI has learned enough to be nigh unbeatable the 3rd time through). - Although I haven't heard it mentioned before, C&C Tiberium Wars seems to have a learning AI that gets reset after every patch - I've noticed that the brutal AI especially is pathetic for the first couple games after a patch, but them it regains its strength. If you want to make a learning AI for like chess or something, try to figure out a way for it to access a data file on some website so you can have lots of computers providing input and thereby getting it to learn much faster. |
Code Archives Forum
   |