So I want to fetch a large amount of data on Parse, a good solution I found is to make a recursive function: when the data are successfully found, launch another request. The way I'm doing it is pretty simple:
var containedLimit = 1000, // Also tried with 500, 300, and 100
parseUsersWaiting = {
// A lot of Users
},
parseUsers = {}, // Recipt for Users
getPlayers = function (containedIn) {
var count = 0;
if (containedIn == undefined) {
containedIn = [];
for (var i in parseUsersWaiting) {
count++;
if (count > containedLimit) {
break;
}
containedIn.push(parseUsersWaiting[i].id);
delete parseUsersWaiting[i];
}
}
var UserObject = Parse.Object.extend('User'),
UserQuery = new Parse.Query(UserObject);
UserQuery.limit(containedLimit);
UserQuery.containedIn('objectId', containedIn);
UserQuery.find({
success: function (results) {
if (results) {
for (var i in results) {
if (parseUsers[results[i].id] == undefined) {
parseUsers[results[i].id] = results[i];
}
// Other stuff
if (results.length < containedLimit) {
// End of process
} else {
getPlayers();
}
}
} else {
// Looks like an end of process too
}
}, error: function (error) {
console.log(error.message);
getPlayers(containedIn);
}
});
};
Now, here is what I would like to call the "issue": it happen, very frequently, that the "error" callback is launched with this:
Received an error with invalid JSON from Parse: Error: getaddrinfo ENOTFOUND api.parse.com
at errnoException (dns.js:44:10)
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:94:26)
(With code 107, of course) So I searched on the Parse Documentation, and it says the exact same thing: "Received an error with invalid JSON". Yeah.
I'm using the Parse SDK provided by Parse.com (npm install parse)
I also tried to modify a bit the Parse code with replacing the host key by 'hostname' on line 361 of the file parse/node_modules/xmlhttprequest/lib/XMLHttpRequest.js (Parse package version: 1.5.0), and it didn't worked, so I removed my changes.
(By the way, I found a solution talking about using ulimit to change memory usage limit that could solve the problem, but actually, I haven't the necessary rights to execute this command)
