Although I wasn't able to go into the Horse Guard Building due to it being closed around the time we walked by. I would highly suggest stopping at the cavalry museum however, if the opportunity presents itself. You can also watch the changing of the horse guards & I hear that you can get closer and it is more interesting than that of the changing of the guards at Buckingham Palace.
The building was originally used for the headquarters of the commander in chief of the British army. You also can view the stables with the 7 euro entrance fee. If you don't feel like spending you can still see the building, guards, & horses for some immaculate photographs.