Client server architecture
The toolbox has been implemented in Java, and it is therefore recommended to implement and test models and algorithms directly in Java files. In case you want to perform constrained planning in another programming language, then it is possible to communicate with the toolbox based on a client-server architecture. The toolbox acts as a server, and external processes can connect to the server in order to request solutions to planning problems. A high-level overview of the architecture is shown below. An example implementation of the client-server communication is provided in the python folder, which provides a full implementation of a Python client.
The toolbox server can be started by running the class executables.Server or the Server.jar file that is included in the toolbox. More information is available here.
A request is represented by a string, and the server responds to a request by sending a response string. Below we give an overview of the requests that can be handled by the server.
- dumpDefaultDomain_domainName_numAgents_numDecisions - Writes a problem instance to an XML file for the given number of agents and decisions. The instance is stored in the file javaInstance.xml. The domainName should be one of the following options: advertising, cbm, maze, tclFixedLimit, tclMultiLevel, webad. The response of the server is an empty string. Example request: dumpDefaultDomain_advertising_3_10
- writeCMDPSolution_filename - Writes the last computed CMDPSolution to a file with the name filename. The response is an empty string. Example request: writeCMDPSolution_solution.out
- writeCPOMDPSolution_filename - Writes the last computed CPOMDPSolution to a file with the name filename. The response is an empty string. Example request: writeCPOMDPSolution_solution.out
- readCMDPSolution_filename - Reads a CMDPSolution from the file with the name filename. The response is an empty string. Example request: readCMDPSolution_solution.out
- readCPOMDPSolution_filename - Reads a CPOMDPSolution from the file with the name filename. The response is an empty string. Example request: readCPOMDPSolution_solution.out
Solve requests can be used to compute a solution for a given problem instance using an algorithm. The server does not send the solution back to the client. Instead, it keeps the solution in memory, such that is can be used afterwards when receiving simulation requests.
- solveDefaultDomain_domainName_numAgents_numDecisions_algName - Loads a problem instance and computes a solution using the specified algorithm. The domainName should be one of the following options: advertising, cbm, maze, tclFixedLimit, tclMultiLevel, webad. The algName should be one of the following options: cmdp, colgen, deterministicpreallocation, dynamicrelaxation|alpha|beta, cgcp, calp, ccpomcp|k|iter|alphaScalar. The response of the server is the expected reward of the computed solution, or EXCEPTION. Example request: solveDefaultDomain_advertising_3_10
- solveXMLDomainMDP_algName - Loads an MDP problem instance from pythonInstance.xml and computes a solution using the specified algorithm. The algName should be one of the following options: cmdp, colgen, deterministicpreallocation, dynamicrelaxation|alpha|beta. For dynamic relaxation the tolerance alpha and the relaxation factor beta are part of the algorithm name. The response of the server is the expected reward of the computed solution, or EXCEPTION. Example requests: solveXMLDomainMDP_cmdp, solveXMLDomainMDP_dynamicrelaxation|0.05|2
- solveXMLDomainPOMDP_algName - Loads a POMDP problem instance from pythonInstance.xml and computes a solution using the specified algorithm. The algName should be one of the following options: cgcp, calp, ccpomcp|k|iter|alphaScalar. The response of the server is the expected reward of the computed solution, or EXCEPTION. Example request: solveXMLDomainPOMDP_calp
The client can perform evaluation of the solution by running simulation runs. The client never receives the solution to the problem explicitly. Instead, it needs to send requests to the server in order to get the actions to be executed by the agents. In addition, the client needs to communicate information back to the server, depending on the outcome of the actions executed.
- getActionsCMDP_t_s - After solving an MDP problem instance, this request can be used to obtain the actions to be executed in joint state s at time t. The joint state is simply a list containing the individual states of the agents. The response of the server is a list containing actions. Example request: getActionsCMDP_2_[3, 4]
- getActionsCPOMDP_t - After solving a POMDP problem instance, this request can be used to obtain the actions to be executed by the agents at time t. The server performs belief updates internally, and therefore the beliefs of the agents are not explicitly used in this request. Instead, the server receives observations in a separate request, as discussed below, which ensures that actions depend on the current time step t and the observations received in the past. Example request: getActionsCPOMDP_3
- actionObservations_actions_observations - During POMDP simulations the agents execute actions and they receive observations. In order to enable the server to update its internal data structures, the client needs to inform the server about the observations received. Example request: actionObservations_[2, 3]_[4, 6]
The client can send server requests to close the connection. It is also possible to shut down the server process completely.
- disconnect - This request must be sent when client wants to close the connection. The response of the server is an empty string.
- shutdown - This request kills the server process. The response of the server is an empty string.