Consider a linear model $Y=X \beta+\epsilon$, where $Y$ and $\epsilon$ are $(n \times 1)$ with $\epsilon \sim N_{n}\left(0, \sigma^{2} I\right)$, $\beta$ is $(p \times 1)$, and $X$ is $(n \times p)$ of full $\operatorname{rank} p<n$. Let $\gamma$ and $\delta$ be sub-vectors of $\beta$. What is meant by orthogonality between $\gamma$ and $\delta$ ?

Now suppose

$$Y_{i}=\beta_{0}+\beta_{1} x_{i}+\beta_{2} x_{i}^{2}+\beta_{3} P_{3}\left(x_{i}\right)+\epsilon_{i} \quad(i=1, \ldots, n),$$

where $\epsilon_{1}, \ldots, \epsilon_{n}$ are independent $N\left(0, \sigma^{2}\right)$ random variables, $x_{1}, \ldots, x_{n}$ are real-valued known explanatory variables, and $P_{3}(x)$ is a cubic polynomial chosen so that $\beta_{3}$ is orthogonal to $\left(\beta_{0}, \beta_{1}, \beta_{2}\right)^{\mathrm{T}}$ and $\beta_{1}$ is orthogonal to $\left(\beta_{0}, \beta_{2}\right)^{\mathrm{T}}$.

Let $\widetilde{\beta}=\left(\beta_{0}, \beta_{2}, \beta_{1}, \beta_{3}\right)^{\mathrm{T}}$. Describe the matrix $\widetilde{X}$such that $Y=\widetilde{X} \widetilde{\beta}+\epsilon$. Show that $\widetilde{X}^{\mathrm{T}} \widetilde{X}$is block diagonal. Assuming further that this matrix is non-singular, show that the least-squares estimators of $\beta_{1}$ and $\beta_{3}$ are, respectively,

$$\widehat{\beta}_{1}=\frac{\sum_{i=1}^{n} x_{i} Y_{i}}{\sum_{i=1}^{n} x_{i}^{2}} \quad \text { and } \quad \widehat{\beta}_{3}=\frac{\sum_{i=1}^{n} P_{3}\left(x_{i}\right) Y_{i}}{\sum_{i=1}^{n} P_{3}\left(x_{i}\right)^{2}}$$