-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathsoccer.html
163 lines (147 loc) · 7.88 KB
/
soccer.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
<!DOCTYPE HTML>
<!--
Massively by HTML5 UP
html5up.net | @ajlkn
Free for personal and commercial use under the CCA 3.0 license (html5up.net/license)
-->
<html>
<head>
<title>RL Robot Soccer Project</title>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no" />
<link rel="stylesheet" href="assets/css/main.css" />
<noscript><link rel="stylesheet" href="assets/css/noscript.css" /></noscript>
</head>
<body class="is-preload">
<!-- Wrapper -->
<div id="wrapper">
<!-- Header -->
<header id="header">
<a href="index.html" class="logo">RL Soccer</a>
</header>
<!-- Nav -->
<nav id="nav">
<ul class="links">
<li><a href="index.html">Projects</a></li>
<li class="active"><a href="about.html">RL Soccer</a></li>
</ul>
<ul class="icons">
<!-- <li><a href="#" class="icon brands fa-twitter"><span class="label">Twitter</span></a></li>
<li><a href="#" class="icon brands fa-facebook-f"><span class="label">Facebook</span></a></li>
<li><a href="#" class="icon brands fa-instagram"><span class="label">Instagram</span></a></li> -->
<li><a href="#" class="icon brands fa-github"><span class="label">GitHub</span></a></li>
</ul>
</nav>
<!-- Main -->
<div id="main">
<!-- Post -->
<section class="post">
<header class="major">
<h1>Reinforcement Learning for Adversarial Robot Soccer</h1>
<p>
Applying RL algorithms to teach two teams of turtlebots how to play soccer--by letting two teams compete against each other.
To replicate the results, please visit the GitHub <a href="https://github.com/muye1202/Adversarial_RL_RoboSoccer" target="_blank">repo!</a>
</p>
</header>
<div class="video">
<video width="1280" height="720" autoplay muted loop>
<source src="images/successful_defender.mp4" type="video/mp4">
</video>
</div>
<h2>Training Pipeline</h2>
<header class="image fit">
<img src="images/soccer_pipeline.png" alt="" />
</header>
<p>
The training pipeline for individual soccer player is straight-forward. I built a ROS2 physics simulator that simulates the interaction between the
robot and the soccer, and the robot against its opponent or teammates. The DDPG RL algorithm is updated by sampling collected experience tuple
composed of actions, rewards, and previous plus current states.
</p>
<h3>Training Attacker</h3>
<div class="video">
<video width="1280" height="720" autoplay muted loop>
<source src="images/naive_attacker.mp4" type="video/mp4">
</video>
</div>
<p>
I trained the naive attacker first and then proceed to train a defender, and then continue this cycle multiple times
to make both the attacker and defender more sophisticated. The naive attacker's goal is simple--dribble the ball through the
opening at the other end of the field. I gave the robot reward for each step towards the goal without taking it outside the field,
and punish the agent when the ball is out of field or taken to a wrong direction. The robot is given different starting positions
at each reset so it will know how to attack no matter its current positions.
</p>
<h3>Training the Defender</h3>
<p>
After training the attacker, I proceed to train a defender that can tackle the ball from the attacker. The attacker is controlled
by the trained model, while the defender's control network is being trained from scratch.
</p>
<h3>ROS2 Simulator</h3>
<p>
To visualize the training process and evaluate the trained model, I created this physics simulator based on ROS2 that features
basic ball physics (velocity decay included). The turtlebot robot models receive speed, direction, kicking power, and kicking direction.
</p>
</section>
</div>
<!-- Footer -->
<footer id="footer">
<section>
<form method="post" action="#">
<div class="fields">
<div class="field">
<label for="name">Name</label>
<input type="text" name="name" id="name" />
</div>
<div class="field">
<label for="email">Email</label>
<input type="text" name="email" id="email" />
</div>
<div class="field">
<label for="message">Message</label>
<textarea name="message" id="message" rows="3"></textarea>
</div>
</div>
<ul class="actions">
<li><input type="submit" value="Send Message" /></li>
</ul>
</form>
</section>
<section class="split contact">
<section class="alt">
<h3>Address</h3>
<p>2145 Sheridan Rd<br />
Evanston, IL 60208</p>
</section>
<section>
<h3>Phone</h3>
<p><a href="#">(734) 510-1501</a></p>
</section>
<section>
<h3>Email</h3>
<p><a href="#">muyejia2023@u.northwestern.edu</a></p>
</section>
<section>
<h3>Social</h3>
<ul class="icons alt">
<!-- <li><a href="#" class="icon brands alt fa-twitter"><span class="label">Twitter</span></a></li>
<li><a href="#" class="icon brands alt fa-facebook-f"><span class="label">Facebook</span></a></li>
<li><a href="#" class="icon brands alt fa-instagram"><span class="label">Instagram</span></a></li> -->
<li><a href="https://github.com/muye1202" class="icon brands alt fa-github"><span class="label">GitHub</span></a></li>
</ul>
</section>
</section>
</footer>
<!-- Copyright -->
<div id="copyright">
<ul><li>© Untitled</li><li>Design: <a href="https://html5up.net">HTML5 UP</a></li></ul>
</div>
</div>
<!-- Scripts -->
<script src="assets/js/jquery.min.js"></script>
<script src="assets/js/jquery.scrollex.min.js"></script>
<script src="assets/js/jquery.scrolly.min.js"></script>
<script src="assets/js/browser.min.js"></script>
<script src="assets/js/breakpoints.min.js"></script>
<script src="assets/js/util.js"></script>
<script src="assets/js/main.js"></script>
</body>
</html>