1
00:00:00,000 --> 00:00:00,894


2
00:00:00,894 --> 00:00:01,788
Hi, everyone.

3
00:00:01,788 --> 00:00:04,150
Thank you so much,
Kelly, for the intro,

4
00:00:04,150 --> 00:00:07,792
and thank you everyone for
attending today's panel.

5
00:00:07,792 --> 00:00:09,200
Hi.

6
00:00:09,200 --> 00:00:11,570
Today, my talk will be
on brains and thoughts

7
00:00:11,570 --> 00:00:13,390
and making the
two work together.

8
00:00:13,390 --> 00:00:16,005
So before I start, my
good friend, L-E, here,

9
00:00:16,005 --> 00:00:18,260
would like to say a few words.

10
00:00:18,260 --> 00:00:18,926
[VIDEO PLAYBACK]

11
00:00:18,926 --> 00:00:41,976


12
00:00:41,976 --> 00:00:42,972
[END VIDEO PLAYBACK]

13
00:00:42,972 --> 00:00:45,700
[LAUGHS] So some
brief background.

14
00:00:45,700 --> 00:00:48,550
I entered Mount Holyoke College
as a neuroscience and behavior

15
00:00:48,550 --> 00:00:51,670
major and later realized towards
the end of my sophomore year

16
00:00:51,670 --> 00:00:55,930
that I also wanted to pursue
a computer science major.

17
00:00:55,930 --> 00:00:58,130
So since I was a
sophomore at that point,

18
00:00:58,130 --> 00:01:01,450
I really had to scramble
to find an internship that

19
00:01:01,450 --> 00:01:05,260
could incorporate some
knowledge from both fields.

20
00:01:05,260 --> 00:01:07,570
And more importantly,
I really wanted

21
00:01:07,570 --> 00:01:10,180
to gain new experience
in programming

22
00:01:10,180 --> 00:01:14,050
and possibly learn new languages
to balance my skill sets.

23
00:01:14,050 --> 00:01:17,740
So I worked previous summers
at the Yale Child Study Center,

24
00:01:17,740 --> 00:01:19,750
and while researching
many of their labs,

25
00:01:19,750 --> 00:01:24,180
I happened to stumble upon the
Technology and Innovation Lab.

26
00:01:24,180 --> 00:01:27,250
And something that's really
common for STEM majors

27
00:01:27,250 --> 00:01:30,720
is that if you're not a
client through a RISE program

28
00:01:30,720 --> 00:01:33,910
or an undergraduate summer
program for research,

29
00:01:33,910 --> 00:01:37,650
most of the time, you were
simply reaching out to a lab.

30
00:01:37,650 --> 00:01:41,210
Either a lab
director, a post-doc,

31
00:01:41,210 --> 00:01:44,530
or a principal investigator.

32
00:01:44,530 --> 00:01:48,940
And usually you send an email,
send a resume, a cover letter,

33
00:01:48,940 --> 00:01:51,850
and you link them to
your LinkedIn account.

34
00:01:51,850 --> 00:01:55,480
And so before you
start applying to labs,

35
00:01:55,480 --> 00:01:57,960
I think it's really
important to know clearly

36
00:01:57,960 --> 00:02:00,700
what skill sets you wish
to stretch before you apply

37
00:02:00,700 --> 00:02:04,879
to these labs, and
you let them know

38
00:02:04,879 --> 00:02:07,970
what you're passionate about
and what you're eager to learn.

39
00:02:07,970 --> 00:02:10,860
And I think that really makes
up for any lack of experience

40
00:02:10,860 --> 00:02:12,080
in the field.

41
00:02:12,080 --> 00:02:16,570
I believe that while I did lack
a lot of technical experience,

42
00:02:16,570 --> 00:02:21,460
my willingness to learn
a lot of new languages

43
00:02:21,460 --> 00:02:28,712
and understand how to program
really got labs to consider me.

44
00:02:28,712 --> 00:02:30,322
And I think, while
it's certainly

45
00:02:30,322 --> 00:02:32,530
important to build upon
already existing connections,

46
00:02:32,530 --> 00:02:35,214
it's always an
option to consider.

47
00:02:35,214 --> 00:02:38,980
Yale Child Study Center
was a really good entity

48
00:02:38,980 --> 00:02:41,125
with a strong community
of researchers,

49
00:02:41,125 --> 00:02:44,080
and they are all more or less
knew one another in the Center.

50
00:02:44,080 --> 00:02:47,440
So I was really able to build
upon the various connections

51
00:02:47,440 --> 00:02:52,370
and managed to find Technology
Innovation Lab through that.

52
00:02:52,370 --> 00:02:56,410
So overall, there's no official
application or interview,

53
00:02:56,410 --> 00:02:59,910
except for maybe a formal
psych interview sometimes.

54
00:02:59,910 --> 00:03:03,370
So it's important to
maintain flexibility

55
00:03:03,370 --> 00:03:07,500
when you're applying to
a research experience

56
00:03:07,500 --> 00:03:13,630
at a lab that isn't offered
through a RISE program.

57
00:03:13,630 --> 00:03:16,155
And so it was also
really hard to emphasize

58
00:03:16,155 --> 00:03:19,330
various aspects of your resume.

59
00:03:19,330 --> 00:03:21,640
So for example,
while I didn't have

60
00:03:21,640 --> 00:03:23,950
a lot of experience
in technology,

61
00:03:23,950 --> 00:03:28,670
I had a lot of experience
working with children.

62
00:03:28,670 --> 00:03:30,220
I had a lot of
tutoring experience,

63
00:03:30,220 --> 00:03:31,870
and I also had a
lot of knowledge

64
00:03:31,870 --> 00:03:33,400
in scientific writing.

65
00:03:33,400 --> 00:03:36,160
And so it was all about making
certain skill sets appeal

66
00:03:36,160 --> 00:03:38,770
to the particular
lab's mission statement

67
00:03:38,770 --> 00:03:40,910
and their overall goals.

68
00:03:40,910 --> 00:03:44,050
So that was how I
found my internship.

69
00:03:44,050 --> 00:03:49,630
My projects, well,
L-E, what you saw,

70
00:03:49,630 --> 00:03:51,630
was my main project this summer.

71
00:03:51,630 --> 00:03:54,820
I also worked briefly with
ongoing projects working with

72
00:03:54,820 --> 00:03:58,360
[? Keypon ?] and [? Sparrow. ?]
I can talk to people about that

73
00:03:58,360 --> 00:04:00,510
later if they're interested.

74
00:04:00,510 --> 00:04:04,720
But L-E was my main
project, and it was just

75
00:04:04,720 --> 00:04:07,590
getting started when I entered
the lab, so it was really

76
00:04:07,590 --> 00:04:10,450
a great experience to see
L-E change and evolve,

77
00:04:10,450 --> 00:04:13,780
working out the bugs,
developing the protocol

78
00:04:13,780 --> 00:04:16,430
and the experimental design
throughout the summer.

79
00:04:16,430 --> 00:04:20,560
So to go over, briefly, what
I did, both of my coding

80
00:04:20,560 --> 00:04:23,602
was in Visual Studio, which I
had no prior experience working

81
00:04:23,602 --> 00:04:25,370
in.

82
00:04:25,370 --> 00:04:28,830
And then it was also
working with an Arduino

83
00:04:28,830 --> 00:04:35,300
board, which is a pretty
standard microcontroller.

84
00:04:35,300 --> 00:04:39,260
We interfaced it with an
Arduino servosensor here,

85
00:04:39,260 --> 00:04:43,260
and that was able to attach
through multiple servomotors

86
00:04:43,260 --> 00:04:46,017
that would allow us to
control the facial movements

87
00:04:46,017 --> 00:04:47,330
of the robot.

88
00:04:47,330 --> 00:04:51,340
So Visual Basic was mostly to
call the servomotor functions

89
00:04:51,340 --> 00:04:53,305
and coordinate the
movements of the robot's

90
00:04:53,305 --> 00:04:57,470
face and other behaviors.

91
00:04:57,470 --> 00:04:59,980
So this is what it overall
looked like from the back

92
00:04:59,980 --> 00:05:03,041
after we put it together.

93
00:05:03,041 --> 00:05:05,230
And we also created
a GUI from which

94
00:05:05,230 --> 00:05:08,080
we could tele-operate
the robot or remotely

95
00:05:08,080 --> 00:05:11,540
control from an operating room.

96
00:05:11,540 --> 00:05:17,810
And so as you can see, there
are faces, various actions

97
00:05:17,810 --> 00:05:21,995
and body positions,
conversational basics, scripts,

98
00:05:21,995 --> 00:05:22,900
and so forth.

99
00:05:22,900 --> 00:05:25,585
We also wanted to personalize
it as much as we could.

100
00:05:25,585 --> 00:05:26,940
So we would have a name box.

101
00:05:26,940 --> 00:05:29,666
We'd enter the child's name,
so when the child came in,

102
00:05:29,666 --> 00:05:31,780
the robot would be
like, hi, Jeffrey,

103
00:05:31,780 --> 00:05:35,070
and the child would be like,
whoa, how do you know me?

104
00:05:35,070 --> 00:05:38,540
So that was really exciting
to see the child react.

105
00:05:38,540 --> 00:05:41,190
So after implementing all
these behaviors and gestures,

106
00:05:41,190 --> 00:05:44,310
giving it a voice,
giving it a personality.

107
00:05:44,310 --> 00:05:47,182
Getting to work
with it, we moved on

108
00:05:47,182 --> 00:05:48,640
to the more behavioral
neuroscience

109
00:05:48,640 --> 00:05:51,660
aspect of the project,
which was introducing them

110
00:05:51,660 --> 00:05:54,185
to ASD therapies.

111
00:05:54,185 --> 00:05:58,120
And the most prevalent therapy
is pivotal response treatment,

112
00:05:58,120 --> 00:05:59,940
and it's considered
an effective therapy

113
00:05:59,940 --> 00:06:02,685
that's more personalized
for the child.

114
00:06:02,685 --> 00:06:05,630


115
00:06:05,630 --> 00:06:09,090
It focuses on the child's
interests and works

116
00:06:09,090 --> 00:06:14,430
with the child in a way
that's much less stressful.

117
00:06:14,430 --> 00:06:19,116
A lot of ASD treatments,
known behavioral therapies,

118
00:06:19,116 --> 00:06:20,490
are known to be
stressful and not

119
00:06:20,490 --> 00:06:23,710
necessarily effective
for children with ASD.

120
00:06:23,710 --> 00:06:28,830
And so we try to build upon
pivotal response training.

121
00:06:28,830 --> 00:06:32,340
There's also a lot of ongoing
research regarding the learning

122
00:06:32,340 --> 00:06:34,650
by teaching paradigm,
which was established

123
00:06:34,650 --> 00:06:37,200
by a psychiatrist
named William Glasser.

124
00:06:37,200 --> 00:06:42,030
And he established that 90%
of what we retain the most

125
00:06:42,030 --> 00:06:43,740
is through what we teach.

126
00:06:43,740 --> 00:06:47,040
And so there's a learning
period associated with that.

127
00:06:47,040 --> 00:06:49,800
There's also a lot of
emerging research based

128
00:06:49,800 --> 00:06:52,890
on the benefits of
socially-assistive robots used

129
00:06:52,890 --> 00:06:55,140
in ASD therapy.

130
00:06:55,140 --> 00:06:58,830
They're known to reduce stress
and pressure often associated

131
00:06:58,830 --> 00:07:00,820
with interacting with people.

132
00:07:00,820 --> 00:07:03,220
So we believe that
socially-assitive robots

133
00:07:03,220 --> 00:07:05,685
would be an effective
way to help children

134
00:07:05,685 --> 00:07:09,645
with ASD feel more comfortable
learning in academic settings,

135
00:07:09,645 --> 00:07:13,050
if they were to engage
with a robot beforehand

136
00:07:13,050 --> 00:07:17,650
or to learn how to
interact with them.

137
00:07:17,650 --> 00:07:19,890
So I'm just going to
go over this really

138
00:07:19,890 --> 00:07:21,500
quickly because of lack of time.

139
00:07:21,500 --> 00:07:23,970
We recruited about
six to 10 participants

140
00:07:23,970 --> 00:07:25,145
for our pilot study.

141
00:07:25,145 --> 00:07:27,330
They were children ranged
from five to eight,

142
00:07:27,330 --> 00:07:33,070
all varying intensities on
the autism spectrum disorder.

143
00:07:33,070 --> 00:07:37,510
And we allotted about a
20-minute session for the child

144
00:07:37,510 --> 00:07:39,720
to interact freely
with the robot,

145
00:07:39,720 --> 00:07:42,590
to get a general idea of what
the child was interested in.

146
00:07:42,590 --> 00:07:45,340
And from that, after
that 20-minute session,

147
00:07:45,340 --> 00:07:47,610
we would develop a
simple script in which

148
00:07:47,610 --> 00:07:53,720
the child could ask
questions and encouraged

149
00:07:53,720 --> 00:07:56,300
the child to teach the
robot a concept that they

150
00:07:56,300 --> 00:07:57,870
were passionate about.

151
00:07:57,870 --> 00:08:00,620
And we formulated
the same script

152
00:08:00,620 --> 00:08:03,350
that we have with the robot
with a human confederate,

153
00:08:03,350 --> 00:08:06,440
and after that, we
compared and measured,

154
00:08:06,440 --> 00:08:09,710
through video coding,
the following points.

155
00:08:09,710 --> 00:08:11,240
We looked at
percentage and duration

156
00:08:11,240 --> 00:08:14,410
of responses from the child
when the robot initiated

157
00:08:14,410 --> 00:08:17,785
a communication versus when
the human confederate initiates

158
00:08:17,785 --> 00:08:20,860
communication and when
the facilitator initiates

159
00:08:20,860 --> 00:08:22,250
communication.

160
00:08:22,250 --> 00:08:25,085
The lag time of the child's
responses to the robot.

161
00:08:25,085 --> 00:08:29,010
The percentage and duration of
child-initiated communication,

162
00:08:29,010 --> 00:08:32,470
number of conversational turns,
and quality of conversation.

163
00:08:32,470 --> 00:08:35,850
And this was done
through video coding.

164
00:08:35,850 --> 00:08:39,299
Unfortunately, video coding is
a very long and tedious process,

165
00:08:39,299 --> 00:08:42,360
and I did not intern at the
Technology Innovation Lab

166
00:08:42,360 --> 00:08:45,940
long enough to obtain
statistical data.

167
00:08:45,940 --> 00:08:48,588
But we did receive--

168
00:08:48,588 --> 00:08:50,340
well, L-E received
a lot of love.

169
00:08:50,340 --> 00:08:53,280
There were a lot
of parent comments.

170
00:08:53,280 --> 00:08:55,880
Parents who sat in
the observation room

171
00:08:55,880 --> 00:08:57,690
said they were really excited.

172
00:08:57,690 --> 00:09:00,860
Their child had never been
so engaged and passionate

173
00:09:00,860 --> 00:09:06,410
and have not been
so talkative before.

174
00:09:06,410 --> 00:09:10,480
With L-E, a lot of children
demanded revisits with L-E

175
00:09:10,480 --> 00:09:14,430
after meeting him and
having the two sessions.

176
00:09:14,430 --> 00:09:16,465
So those were all good--

177
00:09:16,465 --> 00:09:18,690
I think that's the
best data of all,

178
00:09:18,690 --> 00:09:22,230
seeing people react well to L-E.

179
00:09:22,230 --> 00:09:25,500
And then for future
directions, we

180
00:09:25,500 --> 00:09:27,950
plan on making L-E
more autonomous.

181
00:09:27,950 --> 00:09:30,900
Before I left, I added a little
bit of speech recognition,

182
00:09:30,900 --> 00:09:32,330
but it was very buggy.

183
00:09:32,330 --> 00:09:36,510
So getting rid of the
bugs, adding a camera

184
00:09:36,510 --> 00:09:38,486
for face detection, et cetera.

185
00:09:38,486 --> 00:09:43,280
So moving from
tele-operating to autonomy.

186
00:09:43,280 --> 00:09:45,990
And then we also
want to introduce L-E

187
00:09:45,990 --> 00:09:47,620
to a lot of local schools.

188
00:09:47,620 --> 00:09:50,790
They planned on piloting
them in in-school tutoring

189
00:09:50,790 --> 00:09:54,300
programs at local schools.

190
00:09:54,300 --> 00:09:55,680
OK.

191
00:09:55,680 --> 00:09:57,680
Before I end, I just
wanted to show you

192
00:09:57,680 --> 00:10:00,470
that when we first
started, L-E was not

193
00:10:00,470 --> 00:10:02,800
a very attractive robot.

194
00:10:02,800 --> 00:10:05,030
L-E was [INAUDIBLE].

195
00:10:05,030 --> 00:10:09,336
And so there was a lot
of growing and evolving.

196
00:10:09,336 --> 00:10:11,520
So while L-E grew, I also grew.

197
00:10:11,520 --> 00:10:12,640
I learned lot of things.

198
00:10:12,640 --> 00:10:14,894
I made a lot of mistakes.

199
00:10:14,894 --> 00:10:17,060
So if you don't get a
mechanical engineering student

200
00:10:17,060 --> 00:10:20,504
to help you with the 3D
printer, the robot's face

201
00:10:20,504 --> 00:10:21,234
will be orange.

202
00:10:21,234 --> 00:10:24,450
[LAUGHS] But we worked with it.

203
00:10:24,450 --> 00:10:25,465
The kids were fine.

204
00:10:25,465 --> 00:10:27,400
They liked orange.

205
00:10:27,400 --> 00:10:32,810
We just told them L-E was
a little under the weather.

206
00:10:32,810 --> 00:10:34,460
So we worked with that.

207
00:10:34,460 --> 00:10:36,950
Other times, things
just don't work out,

208
00:10:36,950 --> 00:10:39,990
and the robot looks creepy.

209
00:10:39,990 --> 00:10:42,025
Like uncanny valley,
there are no eyebrows,

210
00:10:42,025 --> 00:10:43,640
it just looks really creepy.

211
00:10:43,640 --> 00:10:47,730
But ultimately, L-E ended up
being quite a cute, adorable,

212
00:10:47,730 --> 00:10:48,830
and charming robot.

213
00:10:48,830 --> 00:10:51,350
The tie, the little
bowtie, we also

214
00:10:51,350 --> 00:10:52,886
got a bunch of different kinds.

215
00:10:52,886 --> 00:10:54,635
We made a lot of little bowties.

216
00:10:54,635 --> 00:10:56,860
So L-E's got a lot of
different personalities

217
00:10:56,860 --> 00:11:01,550
to understand what
the child likes.

218
00:11:01,550 --> 00:11:03,990
For this project, I'd like
to thank my mentor, the PI,

219
00:11:03,990 --> 00:11:07,820
and I met a lot of awesome
and brilliant professors

220
00:11:07,820 --> 00:11:10,910
and researchers that
I had the pleasure

221
00:11:10,910 --> 00:11:12,710
of working with this summer.

222
00:11:12,710 --> 00:11:15,760
So questions?

223
00:11:15,760 --> 00:11:17,910