Creating robots is not so difficult, and there are a number of books that describe a general algorithm for this (a simple Google search will include a number of algorithms).
Its essence from .NET perspecitve is recursively:
- HttpWebRequest/HttpWebResponse WebClient. , WCF Web API CodePlex, , / REST, ( - )
- Html Agility Pack, fizzler Html Agility Pack. Html Agility Pack HTML- HTML XPath ( ). , fizzler CSS, jQuery.
HTML , , , .