User-agent: * 
Disallow: /site/articles/
Disallow: /site/pdf/
Disallow: /site/directory/