Get a webpage using HtmlAgilityPack.NETCore

Question

I used HtmlAgilityPackto work with html pages. I used to do this:

HtmlWeb web = new HtmlWeb();
HtmlDocument document = web.Load(url);
var nodes = document.DocumentNode.SelectNodes("necessary node");

but now I need to use HtmlAgilityPack.NETCore where it is HtmlWebmissing. What should I use instead HtmlWebto get the same result?

Answer 1

Use HttpClientas a new way to interact with remote resources via http.

, , async , .Result. , HttpClient , .Net 4.5, :

// instance or static variable
HttpClient client = new HttpClient();

// get answer in non-blocking way
using (var response = await client.GetAsync(url))
{
    using (var content = response.Content)
    {
        // read answer in non-blocking way
        var result = await content.ReadAsStringAsync();
        var document = new HtmlDocument();
        document.LoadHtml(result);
        var nodes = document.DocumentNode.SelectNodes("Your nodes");
        //Some work with page....
    }
}

async/await: Async/Await - by @StephenCleary | 2013 .

Answer 2

Visual Studio netcoreapp1.0. HtmlAgilityPack 1.5.0-5.

:

using HtmlAgilityPack;
using System.Net.Http;
using System.IO;

:

HttpClient hc = new HttpClient(); 
HttpResponseMessage result = await hc.GetAsync($"http://somewebsite.com"); 
Stream stream = await result.Content.ReadAsStreamAsync(); 
HtmlDocument doc = new HtmlDocument(); 
doc.Load(stream); 
HtmlNodeCollection nodes = doc.DocumentNode.SelectNodes("//div[@class='whateverclassyouarelookingfor']");

Answer 3

, . ?

using (HttpClient client = new HttpClient())
{
    using (HttpResponseMessage response = client.GetAsync(url).Result)
    {
        using (HttpContent content = response.Content)
        {
            string result = content.ReadAsStringAsync().Result;
            HtmlDocument document = new HtmlDocument();
            document.LoadHtml(result);
            var nodes = document.DocumentNode.SelectNodes("Your nodes");
            //Some work with page....
        }
    }
}

Answer 4

HttpClient .