这位与字节跳动张一鸣、美团王兴同乡的福建龙岩人,在胡润百富榜上早已是百亿身家,却始终保持着神秘色彩。他不接受采访、不参加论坛,极少在公开场合露面,唯有通过一份份上市公司公告,外界才能窥见其庞大的矿业帝国版图。
Professional development is only one part of growth,更多细节参见heLLoword翻译
But MXU utilization tells the real story. Even with block=128, flash attention’s MXU utilization is only ~20% vs standard’s ~94%. Flash has two matmuls per tile: Q_tile @ K_tile.T = (128, 64) @ (64, 128) and weights @ V_tile = (128, 128) @ (128, 64). Both have inner dimension ≤ d=64 or block=128, so the systolic pipeline runs for at most 128 steps through a 128-wide array. Standard attention’s weights @ V is (512, 512) @ (512, 64) — the inner dimension is 512, giving the pipeline 512 steps of useful work. That single large matmul is what drives standard’s ~94% utilization.,这一点在谷歌中也有详细论述
重视不等于现在就要部署运用,成熟度还需要耐心等待但看懂趋势,不等于马上冲进去。
В КСИР выступили с жестким обращением к США и Израилю22:46