Expanding the determinant, we obtain \begin{align*} \begin{vmatrix} \sec^2 x & 1 & 1 \\ \cos^2 x & \cos^2 x & \csc^2 x \\ 1 & \cos^2 x & \cot^2 x \end{vmatrix} &= \sec^2 x \begin{vmatrix} \cos^2 x & \csc^2 x \\ \cos^2 x & \cot^2 x \end{vmatrix} - \begin{vmatrix} \cos^2 x & \csc^2 x \\ 1 & \cot^2 x \end{vmatrix} + \begin{vmatrix} \cos^2 x & \cos^2 x \\ 1 & \cos^2 x \end{vmatrix} \\ &= \sec^2 x (\cos^2 x \cot^2 x - \csc^2 x \cos^2 x) - (\cos^2 x \cot^2 x - \csc^2 x) + (\cos^4 x - \cos^2 x) \\ &= \frac{1}{\cos^2 x} \left( \cos^2 x \cdot \frac{\cos^2 x}{\sin^2 x} - \frac{1}{\sin^2 x} \cdot \cos^2 x \right) - \left( \cos^2 x \cdot \frac{\cos^2 x}{\sin^2 x} - \frac{1}{\sin^2 x} \right) + (\cos^4 x - \cos^2 x) \\ &= \frac{\cos^2 x - 1}{\sin^2 x} - \frac{\cos^2 x}{\sin^2 x} (\cos^2 x - 1) + \cos^4 x - \cos^2 x \\ &= \frac{-\sin^2 x}{\sin^2 x} - \frac{\cos^2 x}{\sin^2 x} (-\sin^2 x) + \cos^4 x - \cos^2 x \\ &= -1 + \cos^2 x + \cos^4 x - \cos^2 x \\ &= \cos^4 x.\end{align*}The range of $ \cos^4 x $ is $ [0,1] $. However, if $ \cos^4 x = 0, $ then $ \cos x = 0, $ which means $ \sec x $ is not defined. And if $ \cos^4 x = 1, $ then $ \cos^2 x =1, $ so $ \sin^2 x = 0, $ which means $ \csc x $ is not defined. Therefore, the set of all possible values of the determinant is $ \boxed{(0,1)} $.