Lower Bound of Divergence

Question

I want to prove that a lower bound of the Divergence between two probability distributions $p$ and $q$ defined on the set $\mathcal{U}$ can be expressed by defining a subset $\mathcal{S}\subset\mathcal{U}$ with the following expression:

\begin{equation} D(p||q)\geq d_2(p(\mathcal{S})||q(\mathcal{S}))\text, \end{equation}

where $p(\mathcal{S})=\sum_{u\in \mathcal{S}}p(u)$, similarly for $q(\mathcal{S})$, and $d_2(\alpha||\beta)=\alpha\log \frac{\alpha}{\beta}+(1-\alpha)\log \frac{1-\alpha}{1-\beta}$

I suppose that this could be proven with the data processing theorem for divergence but I don't know how to arrange terms to find the previous expression.

Answer 1

I found the solution. The thing is that, assuming that the inequality given by the data processing theorem for divergence:

$$D(p||q)\geq D(\tilde{p}||\tilde{q})$$

with $\tilde{p}(v)=\sum_{u}P(u)W(v|u)$ is true for all valid probability kernels $W(v|u) (\sum_v W(v|u)=1 \forall u \in \mathcal{U},, W(u,v) \geq 0 \forall u,v)$, we could define a kernel in such a way that $\tilde{p}$ and $\tilde{q}$ are Bernoulli distributions: \begin{equation} W(v|u)={0 if u \in \mathcal{S}, v=0 or u \notin \mathcal{S} ; 1 if u \notin \mathcal{S}, v=0 or u \in \mathcal{S}} \end{equation}

Now $D(\tilde{p}||\tilde{q})=\sum_v\tilde{P}\log\frac{tilde{P}}{tilde{Q}}=p(\mathcal{S})\log\frac{p(\mathcal{S})}{q(\mathcal{S})}+(1-p(\mathcal{S}))\log\frac{1-p(\mathcal{S})}{1-q(\mathcal{S})}:= d_2(p(\mathcal{S})||q(\mathcal{S}))$