Jekyll is not easy to install on Windows. Here is a work-in-progress howto on installing Jekyll on Windows, based on the documentation put together by Julian Thilo.
First, install Ruby 2.0 using the installer downloadable here, following exactly the instructions here.
After installing Ruby, download and install the Ruby Development Kit. Extract the content of the archive to a directory such as C:\RubyDevKit, and then run the initialize script followed by the installation:
> cd C:\RubyDevKit
> ruby dk.rb init
> ruby dk.rb install
When running the initialize, if you encounter the error reported here, apply the recommended solution to change the encoding to 1252 so the script can work:
> chcp 1252
> ruby dk.rb init
> ruby dk.rb install
Now you can install the Jekyll gem using RubyGems:
> gem install jekyll
This might take a while due to the number of dependencies.