#Two player battlesheep how to#
#Two player battlesheep download#

The following shows one GA-CCRi employee’s view of a game in progress with another employee.

We collected data from 83 real, two-person games. We hosted the app on Heroku, encouraged our co-workers at GA-CCRi to play,and saved logs of the games that were played using the add-on papertrail. We modified phoenix-battleship to save logs of ship locations and player moves and we made slight configuration changes for the sizes of ships and generated data. To generate the sample data, we began with the open source phoenix-battleship, which was written in elixir using the phoenix framework. This helps to minimize overestimation of Q-values. Double Q-learning essentially uses two Q-learners: one to pick the action and another to assign the Q-value. It also helps the model take into account past moves and positions, providing a richer source of training. This helps the model avoid converging to a local minima because the model won’t be getting information from a sequence of moves in a single game. Experience replay is when games are stored in a cyclical memory buffer so that we can train batches of moves and we can sample from games that were already played. To help with faster training and model stability, more advanced deep Q-learning methods use techniques such as experience replay and double Q-learning. Playing Battleship, we found that starting at 80% and decaying to 10% worked well. One basic method is to start with completely random choices some percentage of the time and to then slowly decay to a smaller percentage as the model learns. Instead, there are various exploration-exploitation methods designed to balance the ‘exploring’ of the state space in order to gain and access information on a wider range of actions and Q-values versus ‘exploiting’ what the model has already learned. In practice, when we are training the Q-learner, we do not always pick the action that has the maximum Q-value as the next move during the self-play phase. The Q represents the “quality” of some move given a specific state the following pseudo-code outlines the algorithm: The simplest way to use an agent trained from Q-learning is to pick the action that has the maximum Q-value. Here, our neural network acts as a function approximator for a function Q, where Q(state, action) returns a long-term value of the action given the current state. Perhaps the most common technical approach is Q-learning. The typical setup involves an environment, an agent, states, and rewards.

#Two player battlesheep how to#

The learning agent learns by interacting with the environment and then figures out how to best map states to actions.

There are various technical approaches to deep reinforcement learning, where the idea is to learn a policy that maximizes long-term reward represented numerically.

Our experiments focused on how to use the data collected from human players to refine the agent’s ability to play (and win!) against human players.

Over time, the automated Battleship-playing agent did better and better, developing strategies to improve its play from game to game.Ĭurrent progress has been made to establish a framework for (1) playing Battleship from random (or user-defined) ship placement (2) deep reinforcement learning from a Deep Q-learner trained from self-play on games starting from randomly positioned ships and (3) collecting data from two-player Battleship games.

#Two player battlesheep download#

Ten years after that initial release, Milton Bradley released a computerized version, and now there are numerous online versions that you can play and open source versions that you can download and run yourself.Īt GA-CCRi, we recently built on an open source version to train deep learning neural networks with data from GA-CCRi employees playing Battleship against each other. According to the Wikipedia page for the game Battleship, the Milton Bradley board game has been around since 1967, but it has roots in games dating back to the early 20th century.