Update documentation to support Python 3.6.

PiperOrigin-RevId: 234139600

README.md

@@ -34,6 +34,10 @@ For completeness, we also provide an implementation of DQN
 For additional details, please see our
 [documentation](https://github.com/google/dopamine/tree/master/docs).
 
+We provide a set of [Colaboratory
+notebooks](https://github.com/google/dopamine/tree/master/dopamine/colab)
+which demonstrate how to use Dopamine.
+
 This is not an official Google product.
 
 ## What's new
@@ -76,78 +80,59 @@ for additional details.
 Finally, these instructions are for Python 2.7. While Dopamine is Python 3
 compatible, there may be some additional steps needed during installation.
 
-#### Ubuntu
-
-First set up the virtual environment:
+First install [Anaconda](https://docs.anaconda.com/anaconda/install/), which
+we will use as the environment manager, then proceed below.
 
 ```
-sudo apt-get update && sudo apt-get install virtualenv
-virtualenv --python=python2.7 dopamine-env
-source dopamine-env/bin/activate
+conda create --name dopamine-env python=3.6
+conda activate dopamine-env
 ```
 
 This will create a directory called `dopamine-env` in which your virtual
 environment lives. The last command activates the environment.
 
-Then, install the dependencies to Dopamine. If you don't have access to a
-GPU, then replace `tensorflow-gpu` with `tensorflow` in the line below
-(see [Tensorflow instructions](https://www.tensorflow.org/install/install_linux)
-for details).
-
-```
-sudo apt-get update && sudo apt-get install cmake zlib1g-dev
-pip install absl-py atari-py gin-config gym opencv-python tensorflow-gpu
-```
-
-During installation, you may safely ignore the following error message:
-*tensorflow 1.10.1 has requirement numpy<=1.14.5,>=1.13.3, but you'll have
-numpy 1.15.1 which is incompatible*.
-
-Finally, download the Dopamine source, e.g.
+Install the dependencies below, based on your operating system, and then
+finally download the Dopamine source, e.g.
 
 ```
 git clone https://github.com/google/dopamine.git
 ```
 
-#### Mac OS X
+#### Ubuntu
 
-First set up the virtual environment:
+If you don't have access to a GPU, then replace `tensorflow-gpu` with
+`tensorflow` in the line below (see [Tensorflow
+instructions](https://www.tensorflow.org/install/install_linux) for details).
 
 ```
-pip install virtualenv
-virtualenv --python=python2.7 dopamine-env
-source dopamine-env/bin/activate
+sudo apt-get update && sudo apt-get install cmake zlib1g-dev
+pip install absl-py atari-py gin-config gym opencv-python tensorflow-gpu
 ```
 
-This will create a directory called `dopamine-env` in which your virtual
-environment lives. The last command activates the environment.
-
-Then, install the dependencies to Dopamine:
+#### Mac OS X
 
 ```
 brew install cmake zlib
 pip install absl-py atari-py gin-config gym opencv-python tensorflow
 ```
 
-During installation, you may safely ignore the following error message:
-*tensorflow 1.10.1 has requirement numpy<=1.14.5,>=1.13.3, but you'll have
-numpy 1.15.1 which is incompatible*.
-
-Finally, download the Dopamine source, e.g.
-
-```
-git clone https://github.com/google/dopamine.git
-```
-
-#### Running tests
+### Running tests
 
 You can test whether the installation was successful by running the following:
 
 ```
+cd dopamine
 export PYTHONPATH=${PYTHONPATH}:.
 python tests/dopamine/atari_init_test.py
 ```
 
+If you want to run some of the other tests you will need to `pip install mock`.
+
+
+### Training agents
+
+#### Atari games
+
 The entry point to the standard Atari 2600 experiment is
 [`dopamine/discrete_domains/train.py`](https://github.com/google/dopamine/blob/master/dopamine/discrete_domains/train.py).
 To run the basic DQN agent,

setup.py

@@ -43,7 +43,7 @@
 
 setup(
     name='dopamine_rl',
-    version='2.0.0',
+    version='2.0.1',
     include_package_data=True,
     packages=find_packages(exclude=['docs']),  # Required
     package_data={'testdata': ['testdata/*.gin']},

tests/dopamine/agents/implicit_quantile/implicit_quantile_agent_test.py

@@ -137,16 +137,16 @@ def test_q_value_computation(self):
                                     feed_dict)
 
       q_values_arr = np.ones([agent.num_actions]) * q_value
-      for i in xrange(agent.num_actions):
+      for i in range(agent.num_actions):
         self.assertEqual(q_values[i], q_values_arr[i])
       self.assertEqual(q_argmax, 0)
 
       q_values_target = sess.run(agent._replay_net_target_q_values, feed_dict)
 
       batch_size = agent._replay.batch_size
 
-      for i in xrange(batch_size):
-        for j in xrange(agent.num_actions):
+      for i in range(batch_size):
+        for j in range(agent.num_actions):
           self.assertEqual(q_values_target[i][j], q_values[j])
 
   def test_replay_quantile_value_computation(self):
@@ -162,12 +162,12 @@ def test_replay_quantile_value_computation(self):
           agent.num_tau_samples, batch_size, agent.num_actions])
       replay_target_quantile_vals = replay_target_quantile_vals.reshape([
           agent.num_tau_prime_samples, batch_size, agent.num_actions])
-      for i in xrange(agent.num_tau_samples):
-        for j in xrange(agent._replay.batch_size):
+      for i in range(agent.num_tau_samples):
+        for j in range(agent._replay.batch_size):
           self.assertEqual(replay_quantile_vals[i][j][0], agent.num_actions)
 
-      for i in xrange(agent.num_tau_prime_samples):
-        for j in xrange(agent._replay.batch_size):
+      for i in range(agent.num_tau_prime_samples):
+        for j in range(agent._replay.batch_size):
           self.assertEqual(replay_target_quantile_vals[i][j][0],
                            agent.num_actions)