Definition of "ReLU"

If we had the textbook from a hundred years in the future that contains all the simple ideas that actually work, that take so long to identify in practice — the ReLUs instead of sigmoids — for those of us who’ve been following AI for, like, more than the last couple of years and know what I’m taking about there.

2023 June 29th, “AI will kill all of us | Eliezer Yudkowsky interview” (16:12–16:28 from the start), in The David Pakman Show‎, spoken by Eliezer Yudkowsky, via YouTube