14

I see this question asked quite often for regular javascript arrays, however none of the answers seems to work if its an array of dates.

I can likely figure this out through trial an error but I do see some benefit to others if I ask.

Basically if you have a javascript array of dates that might have duplicates and need to filter into an array with no duplicates what is the best way to go about that?

I have tried the ES6 solution of Array.from(new Set(arr)) but it just returns the same array.

Also I tried

Array.prototype.unique = function() {
    var a = [];
    for (var i=0, l=this.length; i<l; i++)
        if (a.indexOf(this[i]) === -1)
            a.push(this[i]);
    return a;
}

both came from Unique values in an array

However neither worked, looks like indexOf does not work on date objects.

Here is how my array is generated atm

//this is an array generated from ajax data, 
//its a year over year comparison with a separate year, 
//so to create a reliable date objects I force it to use the same year.
data.map(d => {
   dp = new Date(Date.parse(d.date + '-' + d.year));
   dp.setFullYear(2000);
   return dp;
})

It is about 100 or so different days, but it always ends up with about 350 index's.

Improve this question
5
  • This might help: stackoverflow.com/questions/492994/… Commented Oct 31, 2016 at 16:54
  • 1
    Put each of the dates in a Set first. Commented Oct 31, 2016 at 16:59
  • Definitely the right direction, just need to figure out a way to filter an array using that, would prefer to do it inline without creating other variables, I will play with the idea anyway, thank you @mparnisari Commented Oct 31, 2016 at 17:02
  • If I understand correctly you want a Set of data? developer.mozilla.org/en/docs/Web/JavaScript/Reference/… "Set objects are collections of values, you can iterate its elements in insertion order. A value in the Set may only occur once; it is unique in the Set's collection." Commented Oct 31, 2016 at 17:05
  • @JordanRamstad .map to serialise all of the Date objects into strings -> add them to a Set -> create new array from the Set -> map into Date objects. Alternatively simply filter by maintaining a lookup of all serialised Dates. possibly in a Set. So the filter function will be something like if (Date_as_string in my_lookup) { discard } else { add it to the set and return true} Commented Oct 31, 2016 at 17:07

6 Answers 6

Reset to default
23

ES6 way:

datesArray.filter((date, i, self) => 
  self.findIndex(d => d.getTime() === date.getTime()) === i
)

Thanks to https://stackoverflow.com/a/36744732/3161291

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

12

If you compare two dates via ===, you compare the references of the two date objects. Two objects that represent the same date still are different objects.

Instead, compare the timestamps from Date.prototype.getTime():

function isDateInArray(needle, haystack) {
  for (var i = 0; i < haystack.length; i++) {
    if (needle.getTime() === haystack[i].getTime()) {
      return true;
    }
  }
  return false;
}

var dates = [
  new Date('October 1, 2016 12:00:00 GMT+0000'),
  new Date('October 2, 2016 12:00:00 GMT+0000'),
  new Date('October 3, 2016 12:00:00 GMT+0000'),
  new Date('October 2, 2016 12:00:00 GMT+0000')
];

var uniqueDates = [];
for (var i = 0; i < dates.length; i++) {
  if (!isDateInArray(dates[i], uniqueDates)) {
    uniqueDates.push(dates[i]);
  }
}

console.log(uniqueDates);

Optimization and error handling is up to you.

Improve this answer

2 Comments

This works and the performance is fairly good at about 0.368ms for my dataset.
Please don't do this, unless you only have a small amount of dates, as in the example. The time complexity is quadratic. If you just use a Set of numbers storing the getTime() milliseconds, you can use the has operation which is amortized constant, making the full unique operation linear.
6

You can do a simple filter with a lookup but you need to convert the dates to something that can be compared, since two objects are never the same in JavaScript, unless it's two references to the exact same object.

const dates = [
  new Date(2016, 09, 30, 10, 35, 40, 0),
  new Date(2016, 09, 30, 10, 35, 40, 0), //same
  new Date(2016, 09, 30, 10, 35, 40, 0), //same
  new Date(1995, 07, 15, 03, 15, 05, 0) //different
];


function filterUniqueDates(data) {
  const lookup = new Set();
  
  return data.filter(date => {
     const serialised = date.getTime();
    if (lookup.has(serialised)) {
      return false;
    } else { 
      lookup.add(serialised);
      return true;
    }
  })
}

console.log(filterUniqueDates(dates));

This can be further generalised, if you want to filter anything by just changing how you determine uniqueness

const dates = [
  new Date(2016, 09, 30, 10, 35, 40, 0),
  new Date(2016, 09, 30, 10, 35, 40, 0), //same
  new Date(2016, 09, 30, 10, 35, 40, 0), //same
  new Date(1995, 07, 15, 03, 15, 05, 0) //different
];

const dateSerialisation = date => date.getTime(); // this is the previous logic for dates, but extracted

//as primitives, these can be compared for uniqueness without anything extra
const numbers = [1, 2, 2, 3, 3, 3, 4, 4, 4, 4];
const strings = ["a", "b", "b", "c", "c", "c"];

const people = [
  {name: "Alice", age: 20},
  {name: "Bob", age: 30},
  {name: "Bob", age: 40}, //technically the same
  {name: "Carol", age: 50},
  {name: "Carol", age: 60}, //technically the same
  {name: "Carol", age: 70} //technically the same
]

//let's assume that a person with the same name is the same person regardless of anything else 
const peopleSerialisation = person => person.name;

/* 
 * this now accepts a transformation function that will be used 
 * to find duplicates. The default is an identity function that simply returns the same item.
 */
function filterUnique(data, canonicalize = x => x) { 
  const lookup = new Set();
  
  return data.filter(item => {
     const serialised = canonicalize(item); //use extract the value by which items are considered unique
    
    if (lookup.has(serialised)) {
      return false;
    } else { 
      lookup.add(serialised);
      return true;
    }
  })
}


console.log("dates", filterUnique(dates, dateSerialisation));
console.log("numbers", filterUnique(numbers));
console.log("strings", filterUnique(strings));
console.log("people", filterUnique(people, peopleSerialisation));

This is using ES6 but it's trivial to convert to ES5 compliant code - removing the fat arrow functions, the default parameter and the new Set() here is what you need:

function filterUnique(data, canonicalize) {
  if (!canonicalize) {
    canonicalize = function(x) { return x; }
  }

  var lookup = {};

  return data.filter(function(item) {
     var serialised = canonicalize(item);

    if (lookup.hasOwnProperty(serialised)) {
      return false;
    } else { 
      lookup[serialised] = true;
      return true;
    }
  })
}
Improve this answer

2 Comments

This answer works and looks fairly clean. However TimoSta's does work as well and is also fairly clean. So I took a measurement of the time and TimoSta's has more performance at about 0.368ms compared to this script that takes about 2.97ms on the same data. Not long either way but figured that would be a good way to check. The time difference in the long run is not bad so although I consider TimoSta's the best this is definitely a good resource.
Yeah, I imagine there are ways to make it faster. It's just illustrating the concept here. I'd definitely go with the more functional approach myself, mainly because the .filter is already there, so I don't need to write a separate loop and stuff.
2

The problem with Dates is that the operators === and !== don't work as expected (i.e. they compare pointers instead of actual values).

One solution is to use Underscore's uniq function with a custom transform function to compare the values:

var dates = data.map(d => {
   dp = new Date(Date.parse(d.date + '-' + d.year));
   dp.setFullYear(2000);
   return dp;
})

var unique = _.uniq(dates, false, function (date) {
   return date.getTime();
})
Improve this answer

1 Comment

I would rather avoid adding underscore.js since I have not had a need for it anywhere else in my application yet, so I have not tested this answer. However if it does work I would say this is a fairly clean alternate if already using underscore.js
2

I did mine like this (ES6+), based on some of the other answers:

const uniqueDates = [...new Set(dateInputArray.map(r => r.getTime()))].map((r: number)=>(new Date(r)));

Creates a unique set of dates converted to numbers using getTime(), then map them back to a Date object array afterwards.

Improve this answer

Comments

2

You can use Array.prototype.reduce() combined with Date.prototype.toString():

const dates = [
  new Date(2016, 09, 30, 10, 35, 40, 0), 
  new Date(2016, 09, 30, 10, 35, 40, 0), // same
  new Date(2016, 09, 30, 10, 35, 40, 0), // same
  new Date(1995, 07, 15, 03, 15, 05, 0)  // different
]
const uniqueDates = Object.values(
  dates.reduce((a, c) => (a[c.toString()] = c, a), {})
)

console.log(uniqueDates)

Improve this answer

3 Comments

One should note that this takes advantage of JS's coercion of an object to a string, number, or symbol when used as a key to an object. If the language changes to allow object instances as keys to objects, this method won't work anymore... but hopefully that won't ever happen, as this is damned elegant.
@fleebness I have added an improvement based on your comment.. Thank you
Ah, yep, that'd address that concern pretty nicely.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.