Can the hidden-state dimension of a selective state-space model be... —...

reno_rita

I am trying to understand whether there is a precise tensor-network interpretation of the state dimension in modern selective state-space models such as Mamba/SSD. Consider a selective state-space update $$ h_t = A(x_t)h_{t-1} + b(x_t). $$ By augmenting the state, $$ z_t = \begin{pmatrix} h_t\\ 1 \end{pmatrix}, $$ the recurrence can be written as $$ z_t = M(x_t)z_{t-1}, $$ and therefore $$ z_T = M(x_T)M(x_{T-1})\cdots M(x_1)z_0. $$ This looks...

Originally asked by Patrick Jauslin on Stack Exchange (ai) (CC BY-SA 4.0).

Variant 64.

Variant 274.

uphilltech

Great question — answering as the UphillTech team:

Good question, I think there is no formal named theorem that explicitly connects selective SSMs to tensor networks under that exact terminology. Input-dependent parameters dynamically construct local site tensors per token, but state propagation across any temporal cut remains a sequential contraction operating strictly within an (N + 1)-dimensional vector space; ergo, selectivity grants dynamic flexibility in choosing which correlation channels to activate, but it cannot physically expand the dimensionality of the vector space itself; only stacking multiple selective layers with non-linearities and channel mixing transforms the architecture into a deep tensor network where global...

Full answer by Enzo Carpanetti: https://ai.stackexchange.com/a/50751 (CC BY-SA 4.0).

Want it for your business? UphillTech.