Version 0.9 (#307)

* fix #250

* travis only tests 1.1 and 1

* removed inferred_in_latest

* removed all of the old deprecated generative stuff

* removed ddn code

* before removing programatic deprecation macros

* tests pass

* before switching back to master

* initial steps

* tests pass

* started

* got rid of errors, switched to distribution initialstate (#308)

* DDNOut -> Val

* brought back DDNOut

* tests pass

* working on docs

* working on docs

* cleaned up example

* a bit more cleanup

* finished documentation to fix #280

* added deprecation case for when initialstate_distribution is implemented

* Changed emphasis of explit/generative explanation

* Update README.md

* fixed typo

* Update docs/src/def_solver.md

Co-authored-by: Jayesh K. Gupta <mail@rejuvyesh.com>

* Update runtests.jl

* moved available() and add_registry() to deprecated.jl

* Update def_pomdp.md

Co-authored-by: Jayesh K. Gupta <mail@rejuvyesh.com>

.travis.yml

@@ -1,7 +1,7 @@
 language: julia
 
 julia:
-  - 1.0
+  - 1.1
   - 1
 os:
   - linux
@@ -20,7 +20,7 @@ jobs:
     - os: windows
   include:
     - stage: "Documentation"
-      julia: 1.0
+      julia: 1
       os: linux
       script:
         - julia --project=docs/ -e 'using Pkg; Pkg.instantiate();

Project.toml

@@ -1,22 +1,21 @@
 name = "POMDPs"
 uuid = "a93abf59-7444-517b-a68a-c42f96afdd7d"
-version = "0.8.4"
+version = "0.9.0"
 
 [deps]
 Distributions = "31c24e10-a181-5473-b8eb-7969acd0382f"
-LibGit2 = "76f85450-5226-5b5a-8eaa-529ad045b433"
 LightGraphs = "093fc24a-ae57-5d10-9952-331d41423f4d"
-Logging = "56ddb016-857b-54e1-b83d-db4d58db5568"
 NamedTupleTools = "d9ec5142-1e00-5aa0-9d6a-321866360f50"
+POMDPLinter = "f3bd98c0-eb40-45e2-9eb1-f2763262d755"
 Pkg = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"
 Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
 Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"
 
 [compat]
-Distributions = "0.17,0.18,0.19,0.20,0.21,0.22, 0.23"
+Distributions = "0.17, 0.18, 0.19, 0.20, 0.21, 0.22, 0.23"
 LightGraphs = "1"
 NamedTupleTools = "0.10, 0.11, 0.12, 0.13"
-julia = "1"
+julia = "1.1"
 
 [extras]
 Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

README.md

@@ -11,13 +11,19 @@
 
 This package provides a core interface for working with [Markov decision processes (MDPs)](https://en.wikipedia.org/wiki/Markov_decision_process) and [partially observable Markov decision processes (POMDPs)](https://en.wikipedia.org/wiki/Partially_observable_Markov_decision_process). For examples, please see [POMDPExamples](https://github.com/JuliaPOMDP/POMDPExamples.jl), [QuickPOMDPs](https://github.com/JuliaPOMDP/QuickPOMDPs.jl), and the [Gallery](https://github.com/JuliaPOMDP/POMDPGallery.jl).
 
+---
+
+**NOTE**: We are currently in the process of upgrading solvers to work with POMDPs v0.9. This process is expected to take several weeks. For compatibility with all solvers, please install POMDPs v0.8 with `pkg"add POMDPs@v0.8"`.
+
+---
+
 Our goal is to provide a common programming vocabulary for:
 
 1. Expressing problems as MDPs and POMDPs. 
 2. Writing solver software.
 3. Running simulations efficiently.
 
-There are [nested interfaces for expressing and interacting with (PO)MDPs](http://juliapomdp.github.io/POMDPs.jl/stable/def_pomdp): When the *[explicit](http://juliapomdp.github.io/POMDPs.jl/stable/explicit)* interface is used, the transition and observation probabilities are explicitly defined using api [functions](http://juliapomdp.github.io/POMDPs.jl/stable/explicit/#functional-form-explicit-pomdp); when the *[generative](http://juliapomdp.github.io/POMDPs.jl/stable/generative)* interface is used, only a single step simulator (e.g. (s', o, r) = G(s,a)) needs to be defined. Problems may also be defined with probability [tables](http://juliapomdp.github.io/POMDPs.jl/stable/explicit/#tabular-form-explicit-pomdp), or with the simplified [QuickPOMDPs interfaces](https://github.com/JuliaPOMDP/QuickPOMDPs.jl).
+There are [several ways to define and interact with (PO)MDPs](http://juliapomdp.github.io/POMDPs.jl/stable/def_pomdp): transition and observation distributions and rewards can be defined with explicit probability distributions or implicitly with a function that samples from the distribution, or all of the dynamics can be defined in a single step simulator function: (s', o, r) = G(s,a). Problems may also be defined with probability [tables](https://github.com/JuliaPOMDP/POMDPExamples.jl/blob/master/notebooks/Defining-a-tabular-POMDP.ipynb), and the [QuickPOMDPs interfaces](https://github.com/JuliaPOMDP/QuickPOMDPs.jl) make defining simple problems easier.
 
 **Python** can be used to define and solve MDPs and POMDPs via the QuickPOMDPs or tabular interfaces and [pyjulia](https://github.com/JuliaPy/pyjulia) (Example: [tiger.py](https://github.com/JuliaPOMDP/QuickPOMDPs.jl/blob/master/examples/tiger.py)).
 

docs/make.jl

@@ -19,18 +19,19 @@ makedocs(
 
         "Defining (PO)MDP Models" => [
             "def_pomdp.md",
-            "ddns.md",
-            "basic_properties.md",
-            "explicit.md",
-            "generative.md",
+            "static.md",
             "interfaces.md",
-            "requirements.md",
+            "dynamics.md",
            ],
 
 
-        "Writing Solvers and Updaters" => [
+        "Writing Solvers" => [
             "def_solver.md",
-            "specifying_requirements.md",
+            "offline_solver.md",
+            "online_solver.md"
+           ],
+
+        "Writing Belief Updaters" => [
             "def_updater.md"
            ],
 

docs/src/api.md

@@ -1,24 +1,6 @@
 # API Documentation
 
-Documentation for the `POMDPs.jl` user interface. You can get help for any type or
-function in the module by typing `?` in the Julia REPL followed by the name of
-type or function. For example:
-
-```julia
-julia> using POMDPs
-julia> ?
-help?> reward
-search: reward
-
-  reward{S,A,O}(pomdp::POMDP{S,A,O}, state::S, action::A, statep::S)
-
-  Returns the immediate reward for the s-a-s triple
-
-  reward{S,A,O}(pomdp::POMDP{S,A,O}, state::S, action::A)
-
-  Returns the immediate reward for the s-a pair
-
-```
+Docstrings for POMDPs.jl interface members can be [accessed through Julia's built-in documentation system](https://docs.julialang.org/en/v1/manual/documentation/index.html#Accessing-Documentation-1) or in the list below.
 
 ```@meta
 CurrentModule = POMDPs
@@ -30,7 +12,6 @@ CurrentModule = POMDPs
 Pages = ["api.md"]
 ```
 
-
 ## Index
 
 ```@index
@@ -50,40 +31,26 @@ Updater
 
 ## Model Functions
 
-### [Explicit](@id explicit_api)
-
-These functions return *distributions*.
+### Dynamics
 
 ```@docs
 transition
 observation
-initialstate_distribution
 reward
-```
-
-### [Generative](@id generative_api)
-
-These functions should return *states*, *observations*, and/or *rewards*.
-
-!!! note
-
-    `gen` in POMDPs.jl v0.8 corresponds to the `generate_` functions in previous versions
-
-```@docs
-@gen
 gen
-initialstate
-initialobs
+@gen
 ```
 
-### [Common](@id common_api)
+### Static Properties
 
 ```@docs
 states
 actions
 observations
 isterminal
 discount
+initialstate
+initialobs
 stateindex
 actionindex
 obsindex
@@ -92,29 +59,16 @@ convert_a
 convert_o
 ```
 
-## Distribution/Space Functions
+### Distributions and Spaces
 
 ```@docs
 rand
 pdf
 mode
 mean
-dimensions
 support
 ```
 
-## Dynamic decision networks
-
-```@docs
-DDNStructure
-DDNNode
-DDNOut
-DistributionDDNNode
-FunctionDDNNode
-ConstantDDNNode
-GenericDDNNode
-```
-
 ## Belief Functions
 
 ```@docs
@@ -152,26 +106,8 @@ actiontype
 obstype
 ```
 
-### Requirements Specification
-```@docs
-check_requirements
-show_requirements
-get_requirements
-requirements_info
-@POMDP_require
-@POMDP_requirements
-@requirements_info
-@get_requirements
-@show_requirements
-@warn_requirements
-@req
-@subreq
-implemented
-```
-
 ### Utility Tools
 
 ```@docs
 add_registry
-available
 ```

docs/src/concepts.md

@@ -46,7 +46,7 @@ represented by a concrete subtype of the [`POMDP`](@ref) abstract type,
 explicit definition is often not required), and `O` is defined by
 implementing [`observation`](@ref) if the [*explicit*](@ref defining_pomdps) interface is used or [`gen`](@ref) if the [*generative*](@ref defining_pomdps) interface is used.
 
-POMDPs.jl can also be extended to accommodate POMDP-like problem classes with expanded [dynamic decision networks](@ref Dynamic-Decision-Networks), such as constrained or factored POMDPs, and it contains functions for defining optional problem behavior
+POMDPs.jl contains additional functions for defining optional problem behavior
 such as a [discount factor](@ref Discount-Factor) or a set of [terminal states](@ref Terminal-States).
 
 More information can be found in the [Defining POMDPs](@ref defining_pomdps) section.

docs/src/def_pomdp.md

@@ -7,7 +7,7 @@ Since POMDPs.jl was designed with performance and flexibility as first prioritie
 - [QuickPOMDPs.jl](https://github.com/JuliaPOMDP/QuickPOMDPs.jl) provides structures for concisely defining simple POMDPs without object-oriented programming.
 - [POMDPExamples.jl](https://github.com/JuliaPOMDP/POMDPExamples.jl) provides tutorials for defining problems. 
 - [The Tabular(PO)MDP model](https://github.com/JuliaPOMDP/POMDPExamples.jl/blob/master/notebooks/Defining-a-tabular-POMDP.ipynb) from [POMDPModels.jl](https://github.com/JuliaPOMDP/POMDPModels.jl) allows users to define POMDPs with matrices for the transitions, observations and rewards.
-- The [`gen` function](@ref generative_doc) is the easiest way to wrap a pre-existing simulator from another project or written in another programming language so that it can be used with POMDPs.jl solvers and simulators. See also [RLInterface.jl](https://github.com/JuliaPOMDP/RLInterface.jl) for an even higher level interface for simulators where the state is not accessible.
+- The [`gen`](@ref) function is the easiest way to wrap a pre-existing simulator from another project or written in another programming language so that it can be used with POMDPs.jl solvers and simulators.
 
 ## Overview
 
@@ -16,36 +16,27 @@ Custom POMDP problems are defined by implementing the functions specified by the
 
 !!! note
 
-    The main generative and explicit interfaces use an object-oriented programming paradigm and require familiarity with Julia. For users new to Julia, [QuickPOMDPs](https://github.com/JuliaPOMDP/QuickPOMDPs.jl) usually requires less knowledge of the language and no object-oriented programming.
+    The main POMDPs.jl interface uses an object-oriented programming paradigm and require familiarity with Julia. For users new to Julia, [QuickPOMDPs](https://github.com/JuliaPOMDP/QuickPOMDPs.jl) usually requires less knowledge of the language and no object-oriented programming.
 
-There are two ways of specifying the state dynamics and observation behavior of a POMDP. The problem definition may include a mixture of *explicit* definitions of probability distributions, or *generative* definitions that simulate states and observations without explicitly defining the distributions. In scientific papers explicit definitions are often written as ``T(s' | s, a)`` for transitions and ``O(o | s, a, s')`` for observations, while a generative definition might be expressed as ``s', o, r = G(s, a)`` (or ``s', r = G(s,a)`` for an MDP).
-
-Accordingly, the POMDPs.jl model API is grouped into three sections:
-1. The [*explicit*](@ref explicit_api) interface containing *functions that explicitly define distributions for DDN nodes.*
-2. The [*generative*](@ref generative_api) interface containing *functions that return sampled states and observations for DDN nodes.*
-3. [*Common*](@ref common_api) functions used in both.
+In this guide, the interface is divided into two sections: functions that define static properties of the problem, and functions that describe the dynamics - how the states, observations and rewards change over time. There are two ways of specifying the dynamic behavior of a POMDP. The problem definition may include a mixture of *explicit* definitions of probability distributions, or *generative* definitions that simulate states and observations without explicitly defining the distributions. In scientific papers explicit definitions are often written as ``T(s' | s, a)`` for transitions and ``O(o | s, a, s')`` for observations, while a generative definition might be expressed as ``s', o, r = G(s, a)`` (or ``s', r = G(s,a)`` for an MDP).
 
 ## What do I need to implement?
 
-Because of the wide variety or problems and solvers that POMDPs.jl interfaces with, the question of which functions from the interface need to be implemented does not have a short answer for all cases. In general, a problem will be defined by implementing a combination of functions from the generative, explicit, and common parts of the interface.
+Because of the wide variety or problems and solvers that POMDPs.jl interfaces with, the question of which functions from the interface need to be implemented does not have a short answer for all cases. In general, a problem will be defined by implementing a combination of functions.
 
 Specifically, a problem writer will need to define
 - Explicit or generative definitions for 
-    - the state transition model ([DDN](@ref Dynamic-Decision-Networks) node `:sp`),
-    - the reward function ([DDN](@ref Dynamic-Decision-Networks) node `:r`), and
-    - the observation model ([DDN](@ref Dynamic-Decision-Networks) node `:o`, for POMDPs only).
+    - the state transition model,
+    - the reward function, and
+    - the observation model.
 - Functions to define some other properties of the problem such as the state, action, and observation spaces, which states are terminal, etc.
 
-!!! note
-
-    Since an explicit definition for a DDN node contains all of the information required for a generative definition, POMDPs.jl will automatically synthesize the generative functions for that node at runtime if an explicit model is available. Thus, there is never a need for both generative and explicit definitions of a node, and it is usually best to avoid redundant definitions because it is easy for them to become inconsistent.
-
 The precise answer for which functions need to be implemented depends on two factors: problem complexity and which solver will be used.
 In particular, 2 questions should be asked:
 1. Is it difficult or impossible to specify a probability distribution explicitly?
 2. What solvers will be used to solve this, and what are their requirements?
 
-If the answer to (1) is yes, then a generative definition should be used. Question (2) should be answered by reading about the solvers and trying to run them, or through the [requirements](@ref requirements) interface if it has been defined for the solver.
+If the answer to (1) is yes, then a generative definition should be used. Question (2) should be answered by reading about the solvers and trying to run them. Some solvers have specified their requirements using the [POMDPLinter package](https://github.com/JuliaPOMDP/POMDPLinter.jl), however, these requirements are written separately from the solver code, and often the best way is to write a simple prototype problem and running the solver until all `MethodError`s have been fixed.
 
 !!! note
 
@@ -55,9 +46,6 @@ If the answer to (1) is yes, then a generative definition should be used. Questi
 
 The following pages provide more details on specific parts of the interface:
 
-- [Dynamic Decision Networks](@ref)
-- [Explicit DDN node definitions](@ref explicit_doc)
-- [Generative DDN node definitions](@ref generative_doc)
-- [Basic Properties (common part of the api)](@ref basic)
+- [Static Properties](@ref static)
 - [Spaces and Distributions](@ref)
-- [Requirements](@ref requirements)
+- [Dynamics](@ref dynamics)