LEMMA: the diagonals of a square are perpendicuar bisectors of each other.
Proof: d(AB) = d(BC) since ABCD is a square. So, BD is a perpendicular bisector of AC by the perpendicular bisector theorem. Analogously, it is easy to show that AC is a perpendicular bisector of BD by the perpendicular bisector theorem. QED
Since ABCD is a square, then the angle BCD and the angle ABC are right angles. So, (d(BD))^2 = (d(BC))^2 + (d(CD))^2 by Pythagorean theorem and (d(AC))^2 = (d(AB))^2 + (d(BC))^2. Since ABCD is a square, d(AB) = d(BC) = d(CD) = d(DA). So, (d(BD))^2 = (d(AC))^2 which implies d(BD) = d(AC).
Thus, d(PA) = d(PB) = d(PC) = d(PD) since d(BD) = d(AC) and since the diagonals are bisectors of each other. So, the circle with center at point P with radius = d(AP) is tangent to the four sides of the rhombus.