cs224n,

CS224N | Winter2019 | Assignment 3 Solution

Follow Jun 18, 2020 · 2 mins read
CS224N | Winter2019 | Assignment 3 Solution
Share this

CS224n, 2019W - Assignment3 Solution

HW3: Dependency parsing and neural network foundations
you can find material: code handout

Table of contents

1. Machine Learning & Neural Networks (8 points)

(a) Adam Optimizer

i.

Momentum is working as acceleration reflecting the previous gradients tendency.
Hight variance is well known to raising overfitting problem.

ii.

Parameters that previous gradients are small and not volatile get larger updates.
This helps model to handle with sparse gradients(merits of AdaGrad) and also non-stationary objectives(merits of RMSProp)

(b) Dropout

i.

ii.

Drop out is one of the regularizations, which restrict an overfitting. Randomly setting units zero is for make units stronger on abrupt absence of other units.

And evaluation is made from train, to model being improved. But if we use drop out also at evaluation, it does re-trained with same circumstances, so that can’t do well in test.

2. Neural Transition-Based Dependency Parsing (42 points)

(a)

Stack Buffer New dependency Transition
[ROOT] [I, parsed, this, sentence, correctly]   Initial
[ROOT] [I, parsed, this, sentence, correctly]   SHIFT
[ROOT, I] [parsed, this, sentence, correctly]   SHIFT
[ROOT, I, parsed] [this, sentence, correctly] parsed$\rightarrow$ LEFT-ARC
[ROOT, parsed] [this, sentence, correctly]   SHIFT
[ROOT, parsed, this] [sentence, correctly]   SHIFT
[ROOT, parsed, this, sentence] [correctly] sentence$\rightarrow$this LEFT-ARC
[ROOT, parsed, sentence] [correctly] parsed$\rightarrow$sentence RIGHT-ARC
[ROOT, parsed] [correctly]   SHIFT
[ROOT, parsed, correctly] [] parsed$\rightarrow$correctly RIGHT-ARC
[ROOT, parsed] [] ROOT$\rightarrow$parsed RIGHT-ARC
[ROOT] []    

(b)

2n
n steps for each of buffer and stack

Whole scripts are here.


(c), (d), (e)

Tips: problem 2-d needs shallow copy, not deep copy and TA advised not to use del function, here is good explaination remove vs del

(f)

i.

  • Error type: Verb Phrase Attachment Error
  • Incorrect dependency: wedding → fearing
  • Correct dependency: heading → fearing

ii.

  • Error type: Coordination Attachment Error
  • Incorrect dependency: rescue → and
  • Correct dependency: rescue → rush

iii.

  • Error type: Prepositional Phrase Attachment Error
  • Incorrect dependency: named → Midland
  • Correct dependency: guy → Midland

iv.

  • Error type: Modifier Attachment Error
  • Incorrect dependency: element → most
  • Correct dependency: crucial → most