What are the differences and what types of rules govern each type of property?

Lenders look most kindly upon borrowers who are purchasing or refinanicng their own home, the place where they are going to actually live, than the other property types. This is because they assume that you will be more invested in your own home and so assume a lower risk.

Most second home purchases are for those who live in a city, and want another home in a resort area of the country that they frequent in the summer or whenever they can. A second home is not a rental. If any rent is collected that home is considered an investment property. Furthermore a second home cannot have anyone living on the property full time. The second home can have a caretaker, but that person must permanently reside in another home. 

An investment property is generally also known as a rental property, where a borrower is earning income by renting out one of their homes.

Why is this important?

The reason is that an investment property purchase requires a minimum down payment of 25%. If you are interested in refinancing an investment property, you will only be allowed to borrow up to 75% of the home's value.

A second home purchase only requires 10% down.

When working with FHA, a purchase of a primary residence only requires a 3.5% down payment. A borrower of a primary residence can also put down 5%, 10%, 15%, and 20%.

The greater the down payment the less your loan will cost in extra closing cost/lender fees. A down payment of less than 20% will also add a monthly fee to your mortgage payment. 5% down will have a higher fee both in terms of closing costs and extra monthly mortgage payment than 10% down, and 10% down will have higher fees on both the closing costs and the mortgage than 15% down, etc. When a borrower can put 20% down this will be the most beneficial as there is not an extra per month mortgage payment added to your loan when you can put down 20%.

When refinancing a primary residence with the goal of taking out some cash from your equity, a borrower can take cash out of their equity up to a maximum of 80%. On an investment property and second home, the maximum is 75%

Finally, for those who wish to put 10% or 15% down, there are programs availalbe to borrowers with great credit, at least over 700 points, to avoid paying the extra mortgage insurance premium with their regular monthly mortgage payment. This "LPMI" program, which is described elsewhere on the site, allows a borrower to pay a one time fee instead of a regular monthly fee. Your one time fee will be related to your credit score and the amount of the down payment. The higher the score and the larger the down payment, the smaller the fee. For example a 700 score and 10% down will have a higher fee than a 740 score and 15% down.