Most WordPress themes contain option to set Favicon for your WordPress website or blog. But if you don’t have any experience to design and add favicon for your site follow my steps.

Favicon is a simple square image.
Common favicon Sizes16×16 pixels and 32×32 pixels.

You can use 2 types of image format for favicon. PNG and ico.
Modern browser will support .PNG as a favicon. But old browsers not support .png files as a favicon. So use .ico file format for website favicon.
Use Favicon Generator to create favicons or use ICO Format Photoshop plugin.

So How to Add this Favicon in Your WordPress Blog or Website

Upload favicon file in your Website or Blog installation directory.
Go to Appearance -> Editor.
Next open header.php file

Add these lines inside header.php file.
Note: If your header.php file already have these lines just replace favicon path.
<link rel=”icon” href=”” type=”image/x-icon” />
<link rel=”shortcut icon” href=”” type=”image/x-icon” />

Replace with your website name.

If you need simple way, just use any one from these two free plugins.
Favicon Generator
All in One Favicon