hitori_bocchi

hitori_bocchi

屏幕截图(1377).png 草？

hitori_bocchi

#9「江ノ島エスカー」
夏休み中にメンバーから一向に誘われず様子がおかしくなっていくひとりを見て、面々は江の島へ遊びに行くことに……
https://bocchi.rocks/story/

12/3㈯ 24:00～
TOKYO MXほかにて放送開始

URLはこちら
https://youtube.com/watch?v=iI05dQocBtI

#ぼっち・ざ・ろっく

hitori_bocchi

@admin 期待引入LaTex引擎，这样就方便写blog了

hitori_bocchi

||◤放送開始まで4時間◢||
TVアニメ「ぼっち・ざ・ろっく！」
第10話「アフターダーク」
物語は学校……文化祭!?

TOKYO MX、BS11、群馬テレビ、とちぎテレビ
にて、本日24時より放送開始！

ABEMAは地上波同時配信
https://abema.app/wpz8

#ぼっち・ざ・ろっく

hitori_bocchi

@nomana 最近科研太忙了，等放假了再更

hitori_bocchi

@Hero 好诶！（凑够8个字

hitori_bocchi

TVアニメ「ぼっち・ざ・ろっく！」
第11話「十二進法の夕景」

TOKYO MX、BS11、群馬テレビ、とちぎテレビ
にて、本日24時より放送開始！

◆MRT：25時28分～
◆MBS：27時08分～

ABEMAは地上波同時配信
https://abema.app/B2Ef

#ぼっち・ざ・ろっく

hitori_bocchi

屏幕截图(1377).png 草？

hitori_bocchi

@hashhash 可以加贵校碧蓝档案群155199376聊天吹水

hitori_bocchi

@hashhash 阿罗娜可爱!

hitori_bocchi

@nomana 最近科研太忙了，等放假了再更

hitori_bocchi

@nomana 谢谢~

hitori_bocchi

我来美国之前和我爸大吵了一架，但是还是来了（
还觉得我最近花钱太少了，让我别省着

hitori_bocchi

今天就更到这里，去看碧蓝档案3周年fes了（

hitori_bocchi

It turns out that, for the purposes of constructing lower bound benchmarks for functional estimation, it often suffices to use one-dimensional parametric submodels. A common choice of submodel for nonparametric $P\mathcal{P}$ is, for some mean-zero function $\mathcal{Z} \rightarrow \mathbb{R}$ ,

p_{\epsilon}(z)=d \mathbb{P}(z)\{1+\epsilon h(z)\}

where $∥h∥∞≤M<∞\|h\|_{\infty} \leq M<\infty$ and $ϵ<1/M\epsilon<1 / M$ so that $pϵ(z)≥0p_{\epsilon}(z) \geq 0$ . Note for this submodel the score function is $∂∂ϵlog⁡pϵ(z)∣ϵ=0=∂∂ϵlog⁡{1+ϵh(z)}∣ϵ=0=h(z)\left.\frac{\partial}{\partial \epsilon} \log p_{\epsilon}(z)\right|_{\epsilon=0}=\left.\frac{\partial}{\partial \epsilon} \log \{1+\epsilon h(z)\}\right|_{\epsilon=0}=h(z)$ . Therefore the Cramer-Rao lower bound for some $PϵP_{\epsilon}$ in the example one-dimensional submodel $Pϵ\mathcal{P}_{\epsilon}$ above is given by

\frac{\psi^{\prime}\left(P_{\epsilon}\right)^{2}}{\operatorname{var}_{P_{\epsilon}}\left\{s_{\epsilon}(Z)\right\}}=\frac{\left\{\left.\frac{\partial}{\partial \epsilon} \psi\left(P_{\epsilon}\right)\right|_{\epsilon=0}\right\}^{2}}{\mathbb{E}_{P_{\epsilon}}\left\{h(Z)^{2}\right\}}.

Comment: Why one-dimensional submodel? 详细的说明见 Michael Kosorok "Introduction to Empirical Processes and Semiparametric Inference" Chap. 18。

还需要说明的一点是为什么我们选择了 $pϵ(z)=dP(z){1+ϵh(z)}p_{\epsilon}(z)=d \mathbb{P}(z)\{1+\epsilon h(z)\}$ 作为submodel（以下内容改写自Mark van der Laan的 STAT C245B Survival Analysis and Causality 的课程材料）。

We want to define a type of differentiability of $ψ:P→Rq\psi: \mathcal{P} \rightarrow \mathbb{R}^{q}$ , where $ψ\psi$ is the target parameter.

We could use the definition of a directional derivative in direction $h$ :

\psi(\mathbb{P})(h)=\left.\frac{d}{d \epsilon} \psi(\mathbb{P}+\epsilon h)\right|_{\epsilon=0}

However, $P+ϵh\mathbb{P}+\epsilon h$ might not be a path through $P\mathcal{P}$ , and thus ill defined. We need to define a derivative along paths that are submodels of $P\mathcal{P}$ .

Let $P\mathcal{P}$ be nonparametric. We define a class of paths such that:

p_{\epsilon}(z)=d \mathbb{P}(z)\{1+\epsilon h(z)\}

Two key assumptions necessary for it to be a proper submodel are as follows:

$h$ is uniformly bounded
$EPh(z)=0\mathbb{E}_{P} h(z)=0$

For $ϵ∈(−δ,δ)\epsilon \in(-\delta, \delta)$ with $δ=1∥h∥∞\delta=\frac{1}{\|h\|_{\infty}}$ , this is a submodel.

To see why, first note that for the paths to be a proper density, we need:

$\mathbb{P}(z) \{1+\epsilon h(z)\} \geqslant 0$

Sketch proof:

Let $h (z)$ be uniformly bounded and $h(z)=∥h∥∞h(z)=\|h\|_{\infty}$ . If $ϵ⩽∣δ∣,{1+ϵh(z)}⩾0\epsilon \leqslant|\delta|, \{1+\epsilon h(z)\} \geqslant 0$ . Therefore, for $ϵ\epsilon$ sufficiently small and $h$ uniformly bounded, $\mathbb{P}(z) \{1+\epsilon h(z)\} \geqslant 0$ .

$∫{1+ϵh(z)}dP(z)=1\int \{1+\epsilon h(z)\} d \mathbb{P}(z)=1$

Sketch proof:

Note that $∫{1+ϵh(z)}dP(z)=∫dP(z)+ϵ∫h(z)dP(z)=1\int\{1+\epsilon h(z)\} d \mathbb{P}(z)=\int d \mathbb{P}(z)+\epsilon \int h(z) d \mathbb{P}(z)=1$ since $p$ is a proper density and $∫h(z)dP(z)=EPh(z)=0\int h(z) d \mathbb{P}(z)=\mathbb{E}_{P} h(z)=0$ by assumption.

Now consider the score of this submodel.

\begin{aligned} \left.\frac{\delta}{\delta \epsilon} \log \frac{d P_{\epsilon}}{d \mathbb{P}}\right|_{\epsilon=0} & =\left.\frac{\delta}{\delta \epsilon} \log \{1+\epsilon h(z)\}\right|_{\epsilon=0} \\ & =\left.\frac{h(z)}{1+\epsilon h(z)}\right|_{\epsilon=0} \\ & =h(z). \end{aligned}

"Since any lower bound for the submodel $Pϵ\mathcal{P}_{\epsilon}$ is also a lower bound for $P\mathcal{P}$ , the best and most informative is the greatest such lower bound. Can we say anything about the best such lower bound for generic functionals and/or submodels?"

2.2 Pathwise Differentiability

Recall the Cramer-Rao bound

\frac{\left\{\left.\frac{\partial}{\partial \epsilon} \psi\left(P_{\epsilon}\right)\right|_{\epsilon=0}\right\}^{2}}{\mathbb{E}_{P_{\epsilon}}\left\{s_{\epsilon}(Z)^{2}\right\}}

for submodel $Pϵ\mathcal{P}_{\epsilon}$ described in the previous subsection. To find the best such lower bound, we would like to optimize the above over all $PϵP_{\epsilon}$ in some submodels. It is not a priori clear how generally this can be accomplished, since different functionals $ψ\psi$ could yield very different numerators. Therefore let us first consider what we can say about the derivative in the numerator, for a large class of pathwise differentiable functionals.

Namely, suppose the functional $ψ:P↦R\psi: \mathcal{P} \mapsto \mathbb{R}$ is smooth, as a map from distributions to the reals, in the sense that it admits a kind of distributional Taylor expansion

\psi(\bar{P})-\psi(P)=\int \varphi(z ; \bar{P}) d(\bar{P}-P)(z)+R_{2}(\bar{P}, P)

for distributions $Pˉ\bar{P}$ and $P$ , often called a von Mises expansion, where $φ(z;P)\varphi(z ; P)$ is a mean-

zero, finite-variance function satisfying $∫φ(z;P)dP(z)=0\int \varphi(z ; P) d P(z)=0$ and $∫φ(z;P)2dP(z)<∞\int \varphi(z ; P)^{2} d P(z)<\infty$ , and $R2(Pˉ,P)R_{2}(\bar{P}, P)$ is a second-order remainder term (which means it only depends on products or squares of differences between $Pˉ\bar{P}$ and $P)$ .

Intuitively, the von Mises expansion above is just an infinite-dimensional or distributional analog of a Taylor expansion, with $φ(z;Q)\varphi(z ; Q)$ acting as a usual derivative term; it describes how the functional $ψ\psi$ changes locally when the distribution changes from $P$ to $Pˉ\bar{P}$ . For example, when $\in\{1, \ldots, k\}$ is discrete and so $Pˉ\bar{P}$ and $P$ have $k$ countable components, the von Mises expansion reduces to a standard multivariate Taylor expansion with

R_{2}(\bar{P}, P)=\psi\left(\bar{p}_{1}, \ldots, \bar{p}_{k}\right)-\psi\left(p_{1}, \ldots, p_{k}\right)-\left.\sum_{j} \frac{\partial}{\partial t_{j}} \psi\left(t_{1}, \ldots, t_{k}\right)\right|_{t=\bar{p}}\left(\bar{p}_{j}-p_{j}\right).

hitori_bocchi

不打算更directional derivative & pathwise derivative for functional和更多的半参理论的东西了，实在是太多了，够写一本书的。这篇文章本来也是一个偏实用的指南，还是不想太偏离主旨...

hitori_bocchi

半参后面的理论确实有点复杂，会涉及一些泛函的东西，我不打算写的过于理论，更多还是intuition吧

hitori_bocchi

@hitori_bocchi

hitori_bocchi 发布的最佳帖子

hitori_bocchi 发布的最新帖子

2.2 Pathwise Differentiability