Simulacia POMDP - Revision history

Robot at 00:41, 5 July 2016

2016-07-05T00:41:52Z

2016-07-05T00:41:05Z

2016-07-05T00:35:59Z

2016-07-05T00:25:17Z

2016-07-05T00:18:40Z

2016-07-04T23:24:17Z

2016-07-04T22:56:12Z

2016-07-04T21:32:33Z

2016-07-04T20:15:39Z

2016-07-04T20:15:21Z

‎Parameters of the example

@@ Line 18: / Line 18: @@
 In POMDP agent does not have certain knowledge about state, only belief. In this example, we have two-state belief that can be held by one variable. If '''belief''' = 0, agent is certain that he is in state x2, if '''belief''' is 1, he is certainly in state x1.
 Moreover, action '''u3''' can potentially lead to state transition with probability 0.8.
 '''z1''' and '''z2''' are sensing probabilities, robot can sense before executing any action, thus improving its belief.

@@ Line 18: / Line 18: @@
 In POMDP agent does not have certain knowledge about state, only belief. In this example, we have two-state belief that can be held by one variable. If '''belief''' = 0, agent is certain that he is in state x2, if '''belief''' is 1, he is certainly in state x1.
 Moreover, action '''u3''' can potentially lead to state transition with probability 0.8.
 For more information about used algorithm, see [http://dai.fmph.uniba.sk/courses/airob/sl/airob9.pdf]

← Older revision		Revision as of 00:35, 5 July 2016
Line 23:		Line 23:

	== Source code ==		== Source code ==
−	Zipped folder containing the [[Media:~~pomdp_source~~.zip\|project source code]].	+	Zipped folder containing the [[Media:Pomdp.zip\|project source code]].

@@ Line 38: / Line 38: @@
 We can see that prediction error is worse near kinks in our value function.
 But prediction error rates do not change that much with changing horizon length, except from horizon 1, where robot is not allowed any sensing action.
 [[Image:Pomdp_errors.png‎ | 900px]]
 [[Image:Rewards.png‎ | 900px]]

@@ Line 13: / Line 13: @@
 [[Image:Pomdp_example_cut.png‎|700px]]
-The actions ''u1'' and ''u2'' are terminal actions. <br>
+The actions ''u1'' and ''u2'' are terminal actions with immediate payoffs. <br>
-The action ''u3'' is a sensing action that potentially leads to a state transition. <br>
+The action ''u3'' is a sensing action that potentially leads to a state transition.It costs -1. <br>
 The horizon is finite and γ=1.
 == Results ==
 [[Image:Horizon10.png‎|700px]]
 Results of simulation with horizon=10
 [[Image:pomdp_graph.png‎ | 1100px]]
 [[Image:Pomdp_errors.png‎ | 900px]]

← Older revision		Revision as of 23:24, 4 July 2016
Line 24:		Line 24:
	Results of simulation with horizon=10		Results of simulation with horizon=10
	[[Image:pomdp_graph.png‎ \| 1100px]]		[[Image:pomdp_graph.png‎ \| 1100px]]
		+
		+	[[Image:Pomdp_errors.png‎ \| 900px]]

← Older revision		Revision as of 21:32, 4 July 2016
Line 12:		Line 12:

	[[Image:Pomdp_example_cut.png‎\|700px]]		[[Image:Pomdp_example_cut.png‎\|700px]]
		+
		+	The actions ''u1'' and ''u2'' are terminal actions. <br>
		+	The action ''u3'' is a sensing action that potentially leads to a state transition. <br>
		+	The horizon is finite and γ=1.
		+
		+
		+
		+	== Results ==
		+
		+
		+	Results of simulation with horizon=10
		+	[[Image:pomdp_graph.png‎ \| 1100px]]

← Older revision		Revision as of 20:15, 4 July 2016
Line 11:		Line 11:
	The simulated scenario consists of an robot agent with two possible states and three actions, two of which are terminal actions (can be taken only once). Each terminal action has some associated reward, depending on the state of the agent.		The simulated scenario consists of an robot agent with two possible states and three actions, two of which are terminal actions (can be taken only once). Each terminal action has some associated reward, depending on the state of the agent.

−	[[Image:Pomdp_example_cut.png‎\|~~400px~~]]	+	[[Image:Pomdp_example_cut.png‎\|700px]]