We have 3 of 70 papers
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
In this note, we show that the problem of checking, whether a sequence of bits is ordered, is not doable by 1-head 1-layer transformers but is doable by a 2-head 1-layer transformer. Unlike similar previous results, our results assume the model where transformers have an output MLP.
It is shown that the minimal number of layers a transformer needs to compute PARITY is two, and the open problem asking whether a one-layer transformer can compute PARITY is solved by showing that average sensitivity of a one-layer transformer grows slower than that of PARITY.
It is proved that sufficiently parameterized RGNNs contain sparse subnetworks that maintain 1-RWL expressivity and derive a lower bound on the probability that a random pruning yields such a subnetwork.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
