Suppose you and your friends live in a house where there is no furniture (no, you are not that poor, its just the culture of your house). So, every important paper you guys have in your life, you put them on the floor. There is person who sweeps your house every morning, asks each one of you before starts sweeping, which of these papers became irrelevant to all of you, so that they can sweep those off.
One morning, and it happens to be a new year morning, you took your ID card (of previous year, which just been expired last night) and thought- "gosh! One year has been passed! It seems like just yesterday". Your friend beside you took a peek and said- "look how funny you're looking in that picture! Let me take a look". And you know they will take that ID card and make fun of you for god knows how many days (happens every year); in the meantime, when the sweeper sweeps the floor, they will ask if you guys need that card any more. You surely don't need that, but your friend will say yes (as long as they are not done making fun of you). Thus an expired ID card will remain in the floor for days.
"What? An expire ID card?? What's next? The expired passport? Yesterday's newspaper? In this manner we won't have any place left put our important papers." - you thought. So you put your ID card in a special box and gave that to your friends. That special box has a special meaning to the sweeper. The sweeper knows, whoever has something in that special box, they don't need that thing. So, if the owner says the thing inside is irrelevant, sweeper empties that box and sweeps the thing away without a second thought! What an inventive way to keep the house clean!
Ok. Analogy ended. Consider the house as java runtime environment; the sweeper is the garbage collector (sounds about right, right?); you guys are objects; the papers on the floor are other objects that you own; and the special box? Well the special box is the thing the whole story is about; its the weak reference! You have something, your friends want that thing from you, but you know they really don't need it, you give them that thing inside a WeakReference. This way when the garbage collector comes knocking, they can't claim that thing, and if you don't need it anymore, then garbage collector will get rid of that keeping the memory clean.
What happens if you don't give your friends that thing inside a WeakReference? Well they could keep that thing indefinitely from garbage collecting and your house will look messy. It will take more time to find empty place for new things. This is called memory leak. Oh, now you fear it!