How to filter a list of lists based on a variable set of conditions with Python?

Question

Question related to this post.

I need help to filter a list of lists depending on a variable set of conditions.

Here is an extract of the list:

ex = [
    # ["ref", "type", "date"]
    [1, 'CB', '2017-12-11'],
    [2, 'CB', '2017-12-01'],
    [3, 'RET', '2017-11-08'],
    [1, 'CB', '2017-11-08'],
    [5, 'RET', '2017-10-10'],
]

I want to apply a final treatment like list(filter(makeCondition, ex)) where makeCondition(myList, **kwargs) function returns a boolean that enables to filter the list.

I have troubles building this function as adviced in this post, as the number of conditions defined in the **kwargs dictionnary is variable.

Examples of conditions sets:

conditions = {"ref": 3}
conditions = {"ref": 1, "type": "CB"}

Here is a start:

def makeConditions(myList, **p):

    # For each key:value in the dictionnary
    for key, value in p.items():

        if key == "ref":
            lambda x: x[0] == value
        elif key == "type":
            lambda x: x[1] == value
        elif key == "date":
            lambda x: x[2] == value

    # return chained conditions...

list(filter(makeConditions, ex))

I don't get the logical idea that various comments attempted to give in the post mentioned above... Do I have to execute the filter() function for each sublist or is it possible to do it for the whole global list? Any advices will be very welcome!

The conditions variable is variable indeed, as shown in the post. The content of the sublists is always constant with the 3 objects: ref, type and date. — wiltomap
– wiltomap, Commented Dec 12, 2017 at 10:07

Serge Ballesta · Accepted Answer · 2017-12-12 12:59:40Z

2

I would simply make a function that returns the conditional:

def makeConditions(**p):
    fieldname = {"ref": 0, "type": 1, "date": 2 }
    def filterfunc(elt):
        for k, v in p.items():
            if elt[fieldname[k]] != v: # if one condition is not met: false
                return False
        return True
    return filterfunc

Then you can use it that way:

>>> list(filter(makeConditions(ref=1), ex))
[[1, 'CB', '2017-12-11'], [1, 'CB', '2017-11-08']]
>>> list(filter(makeConditions(type='CB'), ex))
[[1, 'CB', '2017-12-11'], [2, 'CB', '2017-12-01'], [1, 'CB', '2017-11-08']]
>>> list(filter(makeConditions(type='CB', ref=2), ex))
[[2, 'CB', '2017-12-01']]

answered Dec 12, 2017 at 12:59

Serge Ballesta

150k13 gold badges137 silver badges267 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

wiltomap Over a year ago

It looks good and so much shorter... Thanks @Serge Ballesta! I'll have a closer look to it and come back.

Netwave · Accepted Answer · 2017-12-12 11:30:44Z

0

You was almost there, the idea is to create a list with the functions that checks the conditions you need, once you have them you can just call those functions over the list they have to check and use all function to check if all of them are evaluated to True, note the use of partial so the function in the filter call only takes the data list; check this:

from functools import partial

ex = [
    # ["ref", "type", "date"]
    [1, 'CB', '2017-12-11'],
    [2, 'CB', '2017-12-01'],
    [3, 'RET', '2017-11-08'],
    [1, 'CB', '2017-11-08'],
    [5, 'RET', '2017-10-10'],
]

conditions  = {"ref": 3}
conditions2 = {"ref": 1, "type": "CB"}

def apply(data, *args):
"""
same as map, but takes some data and a variable list of functions instead
it will make all that functions evaluate over that data
"""
  return map(lambda f: f(data), args)


def makeConditions(p, myList):
    # For each key:value in the dictionnary
    def checkvalue(index, val, lst):
      return lst[index] == val
    conds = []
    for key, value in p.items():
        if key == "ref":
            conds.append(partial(checkvalue, 0, value))
        elif key == "type":
            conds.append(partial(checkvalue, 1, value))
        elif key == "date":
            conds.append(partial(checkvalue, 2, value))
    return all(apply(myList, *conds)) # does all the value checks evaluate to true?

#use partial to bind the conditions to the makeConditions function
print(list(filter(partial(makeConditions, conditions), ex)))
#[[3, 'RET', '2017-11-08']]
print(list(filter(partial(makeConditions, conditions2), ex)))
#[[1, 'CB', '2017-12-11'], [1, 'CB', '2017-11-08']]

Here you have a live example

Do I have to execute the filter() function for each sublist or is it possible to do it for the whole global list?

Filter iterates over all the list aplying a function for each of the elements, if the function evaluates to True then the element will remind in the result, so filter works over the whole global list

edited Dec 12, 2017 at 11:30

answered Dec 12, 2017 at 10:15

Netwave

43.2k6 gold badges69 silver badges111 bronze badges

2 Comments

wiltomap Over a year ago

Many thanks @Daniel Sanchez! Several new things to me in your suggestion, anyway it works very well. I have to read the doc about partial() function. I don't get how the checkvalue() function finds the index argument.

Netwave Over a year ago

@wiltomap, partial binds values to functions arguments returning another function with those arguments fixed with those values, so when i call partial(checkvalue, 0, value) im creating a function that will do the same as your lambdas, just to be used afterwards.

Collectives™ on Stack Overflow

How to filter a list of lists based on a variable set of conditions with Python?

2 Answers 2

1 Comment

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related