Mar-06-2021, 09:16 PM
I'm trying to create an AI using a Neural Network a Genetic Algorithm to learn how to play tetris, but it looks like something is wrong because, even after 20 generations, i can't see any improvement.
The code of the neural network is this:
All the board where 0 means block free and 1 means the block is already occupied. The board is 22x10 so those are 220 inputs already.
7 input are for the piece the AI is actually using since you can have 7 different pieces.
4 are for the rotation of the piece
2 are for the X coordinate and Y coordinate.
The output are 5
The genetic algorithm parameters are like this:
This is not my fitness function, i tried with a lot of different ones and found this one inside a blog where another tetris project was discussed and that was the Fitness function the guy used.
This is the genetic algorithm implementation:
[1]: https://codemyroad.wordpress.com/2013/04...ct-player/
The code of the neural network is this:
import numpy import os os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3' # or any {'0', '1', '2'} from keras.models import Sequential from keras.layers import Dense import configuration class NeuralNetwork(object): def __init__(self, weights): # define the keras model self.model = Sequential() self.model.add(Dense(configuration.NEURONS_HIDDEN_1, input_dim=configuration.INPUT, activation='relu', use_bias=False)) self.model.add(Dense(configuration.OUTPUT, activation='softmax', use_bias=False)) x = weights[0:configuration.INPUT * configuration.NEURONS_HIDDEN_1].reshape(configuration.INPUT,configuration.NEURONS_HIDDEN_1) self.model.layers[0].set_weights([x,]) x = weights[configuration.INPUT * configuration.NEURONS_HIDDEN_1 :].reshape(configuration.NEURONS_HIDDEN_1, configuration.OUTPUT) self.model.layers[1].set_weights([x,]) def update_parameters(self, vision): """Update all the input values of the Neural Network.""" self.__input_values = vision.reshape(-1, configuration.INPUT) def get_action(self): predictions = self.model.predict(self.__input_values) return numpy.argmax(numpy.array(predictions))Where the configuration for those parameters are those one:
# NEURAL NETWORK INPUT = 234 NEURONS_HIDDEN_1 = 144 OUTPUT = 5 NUMBER_WEIGHTS = INPUT * NEURONS_HIDDEN_1 + NEURONS_HIDDEN_1 * OUTPUTInput are:
All the board where 0 means block free and 1 means the block is already occupied. The board is 22x10 so those are 220 inputs already.
7 input are for the piece the AI is actually using since you can have 7 different pieces.
4 are for the rotation of the piece
2 are for the X coordinate and Y coordinate.
The output are 5
action = self.get_action_from_nn() if action == 0: self.shape.move_piece(1) if action == 1: self.shape.move_piece(-1) if action == 2: self.shape.drop() fast_piece_multiplier = configuration.points['fast_piece_multiplier'] if action == 3: self.shape.rotate()The last output is "doing nothing"
The genetic algorithm parameters are like this:
# GENETIC ALGORITHM NUMBER_OF_POPULATION = 1000 NUMBER_OF_GENERATION = 200 NUMBER_PARENTS_CROSSOVER = 50 MUTATION_PERCENTAGE = 0.05The fitness function is this one:
alfa = -0.510066 beta = 0.760666 charlie = -0.35663 delta = -0.184483 score = alfa * (self.aggregate_height()) + beta * self.total_cleared + charlie * self.holes() + delta * self.bumpiness() return scoreWhere
`self.aggregate_height()`calculate the total height of each column inside the board,
`self.total_cleared`are how many rows the AI cleared,
`self.holes()`calculate how many holes are inside the board and
`self.bumpiness()`calculate the difference in height between each pair of columns.
This is not my fitness function, i tried with a lot of different ones and found this one inside a blog where another tetris project was discussed and that was the Fitness function the guy used.
This is the genetic algorithm implementation:
import multiprocessing import random from multiprocessing import Process import numpy import configuration from game import Game def thread_function(procnum, tetris_ai, return_dict_scores, return_dict_shapes): game = Game(tetris_ai) return_dict_scores[procnum] = game.end_game_score() #return_dict_shapes[procnum] = game.end_game_shapes() def calculate_fitness(population): """Calculate the fitness value for the entire population of the generation.""" # First we create all_fit, an empty array, at the start. Then we proceed to start the chromosome x and we will # calculate his fit_value. Then we will insert, inside the all_fit array, all the fit_values for each chromosome # of the population and return the array. max_points is used to return all the apple positions of the game with # the best snake so that we can watch again the game later all_fit = [] pieces_backup = [] max_points = 0 manager = multiprocessing.Manager() return_dict_scores = manager.dict() return_dict_shapes = manager.dict() index = -1 for j in range(1): processes = [] for i in range(configuration.NUMBER_OF_POPULATION): index += 1 p = Process(target=thread_function, args=(index, population[index], return_dict_scores, return_dict_shapes,)) p.start() processes.append(p) for p in processes: p.join() for value in return_dict_scores.values(): if value > max_points: pieces_backup = None max_points = value all_fit.append(value) return all_fit, pieces_backup def select_best_individuals(population, fitness): """Select X number of best parents based on their fitness score.""" # Create an empty array of the size of number_parents_crossover and the shape of the weights # after that we need to create an array with x number of the best parents, where x is NUMBER_PARENTS_CROSSOVER # inside config file. Then we search for the fittest parents inside the fitness array created by the # calculate_fitness function. Numpy.where return (array([], dtype=int64),) that satisfy the query, so we # take only the first element of the array and then it's value (the index inside fitness array). After we have # the index of the element we just need to take all the weights of that chromosome and insert them as a new # parent. Finally we change the fitness value of the fitness value of that chromosome inside the fitness # array in order to have all different parents and not only the fittest parents = numpy.empty((configuration.NUMBER_PARENTS_CROSSOVER, population.shape[1])) for parent_num in range(configuration.NUMBER_PARENTS_CROSSOVER): index_fittest = numpy.where(fitness == numpy.max(fitness)) index_fittest = index_fittest[0][0] parents[parent_num, :] = population[index_fittest, :] fitness[index_fittest] = -999999999999 return parents def crossover(parents, offspring_size): """Create a crossover of the best parents.""" # First we start by creating and empty array with the size equal to offspring_size we want. The type of the # array is [ [Index, Weights[]] ]. We select 2 random parents and then mix their weights based on a probability offspring = numpy.empty(offspring_size) for offspring_index in range(offspring_size[0]): while True: index_parent_1 = random.randint(0, parents.shape[0] - 1) index_parent_2 = random.randint(0, parents.shape[0] - 1) if index_parent_1 != index_parent_2: for weight_index in range(offspring_size[1]): if numpy.random.uniform(0, 1) < 0.5: offspring[offspring_index, weight_index] = parents[index_parent_1, weight_index] else: offspring[offspring_index, weight_index] = parents[index_parent_2, weight_index] break return offspring def mutation(offspring_crossover): """Mutating the offsprings generated from crossover to maintain variation in the population.""" # We mutate each genes of a chromosome based on a probability, but the range of the weights must be -1 and 1 for offspring_index in range(offspring_crossover.shape[0]): for index in range(offspring_crossover.shape[1]): if numpy.random.random() < configuration.MUTATION_PERCENTAGE: value = numpy.random.choice(numpy.arange(-1, 1, step=0.01), size=1) offspring_crossover[offspring_index, index] += value if offspring_crossover[offspring_index, index] < -1: offspring_crossover[offspring_index, index] = -1 elif offspring_crossover[offspring_index, index] > 1: offspring_crossover[offspring_index, index] = -1 return offspring_crossoverCredit: [Tetris Project][1]
[1]: https://codemyroad.wordpress.com/2013/04...ct-player/