# CS224N | Winter2019 | Assignment 3 Solution

Jun 18, 2020 · 2 mins read

# CS224n, 2019W - Assignment3 Solution

HW3: Dependency parsing and neural network foundations
you can find material: code handout

## 1. Machine Learning & Neural Networks (8 points)

### (a) Adam Optimizer

#### i.

Momentum is working as acceleration reflecting the previous gradients tendency.
Hight variance is well known to raising overfitting problem.

#### ii.

Parameters that previous gradients are small and not volatile get larger updates.
This helps model to handle with sparse gradients(merits of AdaGrad) and also non-stationary objectives(merits of RMSProp)

### (b) Dropout

#### i.

$\gamma = \frac{1}{1-P_{drop}}$

#### ii.

Drop out is one of the regularizations, which restrict an overfitting. Randomly setting units zero is for make units stronger on abrupt absence of other units.

And evaluation is made from train, to model being improved. But if we use drop out also at evaluation, it does re-trained with same circumstances, so that can’t do well in test.

## 2. Neural Transition-Based Dependency Parsing (42 points)

### (a)

Stack Buffer New dependency Transition
[ROOT] [I, parsed, this, sentence, correctly]   Initial
[ROOT] [I, parsed, this, sentence, correctly]   SHIFT
[ROOT, I] [parsed, this, sentence, correctly]   SHIFT
[ROOT, I, parsed] [this, sentence, correctly] parsed$\rightarrow$ LEFT-ARC
[ROOT, parsed] [this, sentence, correctly]   SHIFT
[ROOT, parsed, this] [sentence, correctly]   SHIFT
[ROOT, parsed, this, sentence] [correctly] sentence$\rightarrow$this LEFT-ARC
[ROOT, parsed, sentence] [correctly] parsed$\rightarrow$sentence RIGHT-ARC
[ROOT, parsed] [correctly]   SHIFT
[ROOT, parsed, correctly] [] parsed$\rightarrow$correctly RIGHT-ARC
[ROOT, parsed] [] ROOT$\rightarrow$parsed RIGHT-ARC
[ROOT] []

### (b)

2n
n steps for each of buffer and stack

Whole scripts are here.

### (c), (d), (e)

Tips: problem 2-d needs shallow copy, not deep copy and TA advised not to use del function, here is good explaination remove vs del

### (f)

#### i.

• Error type: Verb Phrase Attachment Error
• Incorrect dependency: wedding → fearing
• Correct dependency: heading → fearing

#### ii.

• Error type: Coordination Attachment Error
• Incorrect dependency: rescue → and
• Correct dependency: rescue → rush

#### iii.

• Error type: Prepositional Phrase Attachment Error
• Incorrect dependency: named → Midland
• Correct dependency: guy → Midland

#### iv.

• Error type: Modifier Attachment Error
• Incorrect dependency: element → most
• Correct dependency: crucial → most