Bottles can be used to hold potions, fairies, Poes, Bugs, etc...  EXTREMELY useful in the later levels of the game when enemies deal more damage and blue fire is needed multiple times.

Bottle #1:
Win Talon's chicken game at Lon Lon Ranch after waking Talon at Hyrule Castle.  He will give you a bottle with some Lon Lon Milk in it.  To make it easier, first move all the regular chickens in a corner, then talk to Talon and  start his game.

Bottle #2:
Gather all the chickens at Kakariko Village and put them in the Chicken Woman's pen. Talk to her to get the bottle.

Bottle #3:
Found by diving in Lake Hylia after gaining the Silver Scale to dive deeper.  Has a note in it from Princess Ruto.  Give the note to the Zoran King to use the bottle for something else.

Bottle #4:
Given to you from the Poe trader at the entrance to the Market as adult Link.  Bring him 10 Big Poe's and he'll give you the bottle.