Badma Maksimov

We deal with the problem of orbital stability of the periodic solutions of a Hamiltonian system with two degrees of freedom. We assume that third- or sixth-order resonance takes place in this system and solve the orbital stability problem in a special case of degeneracy, when it is necessary to take into account the terms up to the sixth order of the Hamiltonian expansion in the neighborhood of the periodic solution. By using methods of normal forms and KAM theory we obtain sufficient conditions of orbital stability and instability in the form of inequalities with respect to coefficients of the Hamiltonian normal form calculated up to terms of the sixth order. We also show that the above-mentioned problem of orbital stability is equivalent to the problem of Lyapunov stability of an equilibrium position of a reduced system.
We apply the above results in the problem of the orbital stability of the pendulum oscillations of a heavy rigid body with a fixed point in the case when its principal moment of inertia $A$, $B$, and $C$ satisfy the equality $A = C = 4B$.