Does using multiple .filter calls in a large array is bad for performance in Javascript?

Question

I wrote this piece of code to filter an array of words. I wrote a filter function for each type of word that I want to filter, and apply it to the array sequentially:

  const wordArray = rawArray.filter(removeNonDomainWords)
                            .filter(removeWordsWithDigits)
                            .filter(removeWordsWithInsideNonWordChars)
                            .filter(removeEmptyWords)
                            .filter(removeSearchTerm, term)
                            .map(word => replaceNonWordCharsFromStartAndEnd(word))

This code repeats throughout the array six times, if I'm not mistaken.

Wouldn't it be more efficient to write one (more complicated, but still easy in my case) filter function that logically combines filter functions to achieve the same result?

I learned about a filter in the context of functional programming, which should make my code shorter and faster. Therefore, I probably did not question what I wrote, thinking: "I am doing FP, this should be good."

Thank!

Answer 1

, , . , , . , , .

, , ( , , , ). , , .

@Redu ( ), ||. .

---

, Array.prototype.filter . API- Java Stream, , , . , , "" .

javascript

rawArray.filter(x)

rawArray , , , . x rawArray.

Java

rawArray.stream().filter(x)

. x . Stream, . , , - - - .

javascript

rawArray.filter(x).filter(y).length

Java

rawArray.stream().filter(x).filter(y).count()

javascript rawArray, x . javascript , y , .

Java rawArray, x, x - true, y , true . .

, , , , , ( - ), , , , , , .

Answer 2

: () .filter, , , , , .

, , , .

, "" , , , , .

const ArrayFilter = (arr, pred) => ({
  filter: pred
    ? (newPred) => ArrayFilter(arr, x => pred(x) && newPred(x))
    : (newPred) => ArrayFilter(arr, newPred),
  execute: (mapper) => mapper 
    ? arr.filter(pred)
    : arr.filter(pred).map(mapper)
});


// Some test data
const oneToFifty = Array.from(Array(50), (_, i) => i);

// ArrayFilter allows for a chained syntax, but only loops over the data
// once and only runs filters when needed
console.log(
  ArrayFilter(oneToFifty)
    .filter(x => x % 3 === 0)
    .filter(x => x < 25)
    .filter(x => x > 10)
    .execute(n => `Result: ${n}`)
);

Hide result

---

, , , , ... 1.000.000 . "" Array#filter .

: execute , .

- , , !

const ArrayFilter = (arr, pred) => ({
  filter: pred
    ? (newPred) => ArrayFilter(arr, x => pred(x) && newPred(x))
    : (newPred) => ArrayFilter(arr, newPred),
  execute: (mapper) => mapper 
    ? arr.filter(pred)
    : arr.filter(pred).map(mapper)
});


// Some test data, with at least one 0
const million = [0].concat(
  Array.from(Array(999999), Math.random)
);

// Filters that keep removing approx
// 100.000 values until 0
const minusNFilters = [
  x => x <= 0.9,
  x => x <= 0.7,
  x => x <= 0.6,
  x => x <= 0.5,
  x => x <= 0.4,
  x => x <= 0.3,
  x => x <= 0.2,
  x => x <= 0.1,
  x => x <= 0.0
];

// Filters that keep removing 50% of the remaining
// items
const fiftyPFilters = [
  x => x <= 1 / 2,
  x => x <= 1 / 4,
  x => x <= 1 / 8,
  x => x <= 1 / 16,
  x => x <= 1 / 32,
  x => x <= 1 / 64,
  x => x <= 1 / 128,
  x => x <= 1 / 256,
  // Final state
  x => x <= 0
];

const map = x => `Result: ${x}`;

const timed = (label, fn) => {
  console.time(label);
  fn();
  console.timeEnd(label);
};

// Some tests
// For these filters, the ArrayFilter tends to
// be just a bit faster
timed(
  "method   : Regular\nsize     : 1000000 items\nfilters  : -n\nms       ",
  () => minusNFilters
    .reduce((xs, f) => xs.filter(f), million)
    .map(map)
);

timed(
  "method   : ArrayFilter\nsize     : 1000000 items\nfilters  : -n\nms       ",
  () => minusNFilters
    .reduce(
      (af, f) => af.filter(f), 
      ArrayFilter(million))
    .execute(map)
);

// For these filters, the ArrayFilter is usually
// slower
timed(
  "method   : Regular\nsize     : 1000000 items\nfilters  : -50%\nms       ",
  () => fiftyPFilters
    .reduce((xs, f) => xs.filter(f), million)
    .map(map)
);

timed(
  "method   : ArrayFilter\nsize     : 1000000 items\nfilters  : -50%\nms       ",
  () => fiftyPFilters
    .reduce(
      (af, f) => af.filter(f), 
      ArrayFilter(million))
    .execute(map)
);

.as-console-wrapper { min-height: 100% }

Hide result