Remove duplicates in list using Distinct

Question

I'm trying to remove duplicate id's from a list on a browser I'm working on. The list is converted to array and then added to a richtextbox. This is needed for the generic bookmarking system I am working on. The issue is it scrapes duplicates. Even after I add distinct to the code.

string html = WebsCon2.ExecuteJavascriptWithResult("document.getElementsByTagName('html')[0].innerHTML");
var htmlDoc = new HtmlAgilityPack.HtmlDocument();
htmlDoc.LoadHtml(html); var playerIds = new List<string>();
var playerNodes = htmlDoc.DocumentNode.SelectNodes("//a[contains(@href, '/link/profile-view.jsp?user=')]").Distinct();
foreach (var playerNode in playerNodes)
{
    string href = playerNode.Attributes["href"].Value;
    var parts = href.Split(new char[] { '=' }, StringSplitOptions.RemoveEmptyEntries);
    if (parts.Length > 1)
    {
        playerIds.Add(parts[1]);
    }
    string Target = string.Join("", playerIds.ToArray());
    PlayerID.Text = Target;
}

So is there another way I can remove the duplicates?

Selman Genç · Accepted Answer · 2014-12-19 23:43:55Z

3

The problem is that Distinct doesn't compare your nodes by their content. It compares them by reference. If you want distinct nodes by InnerText you can use:

var playerNodes = htmlDoc.DocumentNode
            .SelectNodes("//a[contains(@href, '/link/profile-view.jsp?user=')]")
            .GroupBy(node => node.InnerText)
            .Select(g => g.First());

Or you can use DistinctBy method from MoreLINQ:

var playerNodes = htmlDoc.DocumentNode
            .SelectNodes("//a[contains(@href, '/link/profile-view.jsp?user=')]")
            .DistinctBy(node => node.InnerText);

answered Dec 19, 2014 at 23:43

Selman Genç

102k15 gold badges126 silver badges193 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

msporek · Accepted Answer · 2014-12-19 23:50:17Z

2

Declare the playerIds as a HashSet<string> and then you are sure there will be no duplicates. That's the easy solution to your problem. Check the HashSet constructors to provide a comparer that will allow you to make sure you are case-sensitive/case-insensitive with the distinct strings (depending on your preference).

The Distinct() call you're making you can get rid of. This will not help you doing it the way you call it.

edited Dec 19, 2014 at 23:50

answered Dec 19, 2014 at 23:44

msporek

1,2159 silver badges22 bronze badges

Collectives™ on Stack Overflow

Remove duplicates in list using Distinct

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related