Remove items from array dynamically

Question

I have a huge array of XML nodes, sometimes up to 400k items. I use where-object to to select items based on Type attribute. My problem is that it takes quite some time to walk over these items and it would be easier if I could remove items I have just selected from array, so that array would be smaller for next object. Here's what I'm doing:

$subset = $items | Where-Object {$_.Type -eq "2"}
'prosess ItemType 2'

$subset = $items | Where-Object {$_.Type -eq "4"}
'prosess ItemType 4'

$subset = $items | Where-Object {$_.Type -eq "8"}
'prosess ItemType 8'

Problem is that every time I have to iterate over $items again, instead of iterating over what is remaining from previous where-object

It also doesn't help that where-object is a single thread operation and is executed only on one core

What I was thinking about is something like this, but I'm not sure if it will perform better:

$subset = $items | Where-Object {$_.Type -eq "2"}
$items = $items | Where-Object {$_.Type -ne "2"}
'prosess ItemType 2'

$subset = $items | Where-Object {$_.Type -eq "4"}
$items = $items | Where-Object {$_.Type -ne "4"}
'prosess ItemType 4'

$subset = $items | Where-Object {$_.Type -eq "8"}
'prosess ItemType 8'

Have you considered using a linked-list instead of an array? May be more efficient considering the amount of elements is approaching 400k. Easier to remove elements from the middle.techrepublic.com/article/… — davepmiller
– davepmiller, Commented Oct 19, 2012 at 12:23

Shay Levy · Accepted Answer · 2012-10-19 12:58:26Z

2

Have you tried the Select-Xml cmdlet or the XmlDocument.SelectNodes method? Querying XML via its native methods(using XPath), should perform much faster than Where-Object.

edited Oct 19, 2012 at 12:58

answered Oct 19, 2012 at 12:33

Shay Levy

128k34 gold badges193 silver badges210 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Andrey Marchuk Over a year ago

It is a very good idea, I will try it and report how it goes, thanks

Andrey Marchuk Over a year ago

You were right it takes just 15 seconds with SelectNodes vs 2.5 minutes with Where-Object

Shay Levy Over a year ago

Worth every millisecond of it ;)

Collectives™ on Stack Overflow

Remove items from array dynamically

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related