CS224N | Winter2019 | Assignment 3 Solution

Jun 18, 2020 · 2 mins read

CS224n, 2019W - Assignment3 Solution

HW3: Dependency parsing and neural network foundations
you can find material: code handout

1. Machine Learning & Neural Networks (8 points)

i.

Momentum is working as acceleration reflecting the previous gradients tendency.
Hight variance is well known to raising overfitting problem.

ii.

Parameters that previous gradients are small and not volatile get larger updates.

(b) Dropout

i.

$\gamma = \frac{1}{1-P_{drop}}$

ii.

Drop out is one of the regularizations, which restrict an overfitting. Randomly setting units zero is for make units stronger on abrupt absence of other units.

And evaluation is made from train, to model being improved. But if we use drop out also at evaluation, it does re-trained with same circumstances, so that can’t do well in test.

2. Neural Transition-Based Dependency Parsing (42 points)

(a)

Stack Buffer New dependency Transition
[ROOT] [I, parsed, this, sentence, correctly]   Initial
[ROOT] [I, parsed, this, sentence, correctly]   SHIFT
[ROOT, I] [parsed, this, sentence, correctly]   SHIFT
[ROOT, I, parsed] [this, sentence, correctly] parsed$\rightarrow$ LEFT-ARC
[ROOT, parsed] [this, sentence, correctly]   SHIFT
[ROOT, parsed, this] [sentence, correctly]   SHIFT
[ROOT, parsed, this, sentence] [correctly] sentence$\rightarrow$this LEFT-ARC
[ROOT, parsed, sentence] [correctly] parsed$\rightarrow$sentence RIGHT-ARC
[ROOT, parsed] [correctly]   SHIFT
[ROOT, parsed, correctly] [] parsed$\rightarrow$correctly RIGHT-ARC
[ROOT, parsed] [] ROOT$\rightarrow$parsed RIGHT-ARC
[ROOT] []

(b)

2n
n steps for each of buffer and stack

Whole scripts are here.

(c), (d), (e)

Tips: problem 2-d needs shallow copy, not deep copy and TA advised not to use del function, here is good explaination remove vs del

(f)

i.

• Error type: Verb Phrase Attachment Error
• Incorrect dependency: wedding → fearing
• Correct dependency: heading → fearing

ii.

• Error type: Coordination Attachment Error
• Incorrect dependency: rescue → and
• Correct dependency: rescue → rush

iii.

• Error type: Prepositional Phrase Attachment Error
• Incorrect dependency: named → Midland
• Correct dependency: guy → Midland

iv.

• Error type: Modifier Attachment Error
• Incorrect dependency: element → most
• Correct dependency: crucial → most