Compute-in-Memory Attention: A Time-Domain Analog Softmax Circuit with RC-Tunable Temperature
Softmax is a key operation in Transformer attention, but its exponentiation and normalization add significant overhead in compute-in-memory (CIM) accelerators, especially when analog attention scores must first be converted to the digital domain. This work presents a tunable-temperature analog softmax circuit in Global...
Ankur Singh, Ashish Gautam, Shruti R. Kulkarni et al.
· 0 citations