The quantity you might be looking for is signal-front delay, which is the delay of the beginning of a signal passing through a linear system. It is simply the largest value $\tau_{sf}$ (in samples) for which
$$h[n]=0,\qquad n<\tau_{sf}\tag{1}$$
is satisfied, where $h[n]$ is the system's impulse response.
If an input, or change in the input, starts at $n_0$, then the system's response to it begins at $n_0+\tau_{sf}$.
If the impulse response has a long period of pre-ringing, the signal-front delay is generally much smaller than the actual propagation delay. In that case, definition $(1)$ can be modified as
$$\big|h[n]\big|<\delta,\qquad n<\tau_{sf}\tag{2}$$
where $\delta$ is a positive constant which is small compared to the maximum value of $h[n]$.
Depending on the nature of the impulse response, another meaningful definition of signal delay introduced by a linear system could be the center of gravity of the impulse response:
$$\tau_{gr}=\frac{\displaystyle\sum_n nh[n]}{\displaystyle\sum_n h[n]}=\frac{jH'(0)}{H(0)}\tag{3}$$
where $H(\omega)$ is the system's frequency response, and $H'(\omega)$ is its derivative w.r.t. to $\omega$. Note that $\tau_{gr}$ is simply the group delay evaluated at $\omega=0$.
Using the center of gravity as an estimate for signal delay is only useful for systems with a lowpass characteristic. For other systems (such as highpass or bandpass filters), the denominator of $(3)$ could be zero or very close to zero. For that reason, for general systems one would exchange $h[n]$ in $(3)$ for either its magnitude or its square.
Note that unlike phase delay and group delay, neither signal-front delay (including its modified form $(2)$) nor the center of gravity are functions of frequency.