O (log N) == O (1) - Why not?

Question

Whenever I look at algorithms / data structures, I try to replace the parts of log (N) with constants. Oh, I know that log (N) is diverging - but does it matter in real applications?

log (infinity) <100 for all practical purposes.

I'm really curious that the examples in the real world do not stand it.

To clarify:

• I understand O (f (N))
• I'm interested in real-world examples where the behavior of an asymptotic one is more important than the constants of actual performance.
• If log (N) can be replaced by a constant, it can still be replaced by a constant in O (N log N).

This question is intended for (a) entertainment and (b) to collect arguments for use if I run (again) in conflict with project performance.

Answer 1

I think this is a pragmatic approach; O (logN) will never be more than 64. In practice, when terms become “small” like O (logN), you should measure to see if the constant factors win. See also

Using the Ackerman function?

To quote me from comments on another answer:

[Big-Oh] 'Analysis' only matters for factors that are at least O (N). For any smaller coefficient, a big analysis is useless, and you have to measure.

and

"With O (logN) is your input size of matter." That is the whole point of the question. Of course, this is important ... in theory. The question that the OP asks does this really matter in practice? I argue that there is no answer, there is not and never will be, a dataset for which logN will grow as fast as the algorithm always beats up a constant time. Even for the largest practical data set that can be imagined in the lives of our grandchildren, the logN algorithm has a fair chance of beating an algorithm with constant time - you should always measure it.

EDIT

Good conversation:

http://www.infoq.com/presentations/Value-Identity-State-Rich-Hickey

about halfway through, Rich discusses Clojure hash attempts that are clearly O (logN), but the logarithm base is large, and so the trie depth is no more than 6, even if it contains 4 billion values. Here, “6” is still the value of O (logN), but it is an incredibly small value, so you chose this amazing data structure because “I really need O (1)”, this is stupid. This emphasizes that most of the other answers to this question are simply erroneous from the point of view of a pragmatist who wants their algorithm to "work quickly" and "scale well", regardless of what the theory says.

EDIT

see also

http://queue.acm.org/detail.cfm?id=1814327

which says

What is the use of the O (log2 (n)) algorithm if these operations cause page crashes and slow disk operations? For most, the corresponding datasets are O (n) or even O (n ^ 2), which avoids page crashes, there will be circles around it.

(but read the article for context).

Answer 2

The Big O designation will tell you how your algorithm changes with increasing input. O (1) tells you that no matter how much your input increases, the algorithm will always be as fast. O (logn) says the algorithm will be fast, but as your input grows, it will take a little longer.

O (1) and O (logn) makes a big difference when you start combining algorithms.

Take, for example, joins with indexes. If you can make a connection in O (1) instead of O (logn), you will get a huge performance boost. For example, with O (1), you can join any number of times, and you still have O (1). But with O (logn), you need to multiply the number of operations by logn each time.

For large inputs, if you already had an O (n ^ 2) algorithm, you would rather have done an operation that was O (1) inside, rather than O (logn) inside.

Also remember that Big-O nothing can have fixed overhead. Let's say that fixed overhead is 1 million. With O (1), that constant overhead does not increase the number of operations as O (logn) does.

Another point is that everyone thinks of O (logn) representing n elements of a tree's data structure, for example. But it can be anything, including bytes in the file.

Answer 3

This is a common mistake - remember that the Big O note DOES NOT tell you about the absolute performance of the algorithm at a given value, just telling you about the behavior of the algorithm when the input size increases.

When you take it in this context, it becomes clear why the A ~ O (logN) algorithm and the B ~ O (1) algorithm are different:

if I run A at an input of size a and then at an input of size 1,000,000 * a, I can expect the second input to take log (1,000,000) times until the first input

if I run B at an input of size a, then at an input of size 1,000,000 * a, I can expect the second input to take about the same time as the first input

EDIT : Reflecting on my question, I think there is some kind of wisdom in it. Although I would never say that it is correct to say O (logN) == O (1), It IS , it is possible that the O (logN) algorithm can be used according to the O (1) algorithm. This leads to higher absolute performance: just knowing one algorithm is O (1), and the other O (logN) algorithm is NOT , to declare that you should use O (1) over O (logN), of course, it’s possible , given your range of possible inputs, O (logN) may serve you best.

Answer 4

You asked for an example in the real world. I will give you one. Computational biology. One DNA strand encoded in ASCII is located somewhere at the gigabyte level in space. A typical database will obviously have many thousands of such threads.

Now, in the case of the indexing / searching algorithm, the log (n) multiple has a big difference when combined with constants. The reason why? This is one of the applications where the size of your input is astronomical. In addition, input size will continue to grow.

Admittedly, such problems are rare. There are a lot of applications. In these circumstances, though ... it makes a world of difference.

Answer 5

Equality, as you describe it, is a common abuse of notes.

To clarify: we usually write f (x) = O (logN) to denote: "f (x) is O (logN)".

In any case, O(1) means a constant number of steps / time (as the upper limit) for performing an action, regardless of how large the input set is. But for O(logN) number of steps / time is still growing as a function of input size (its logarithm), it just grows very slowly. For most real-world applications, you can be sure that this number of steps will not exceed 100, however I would put a few examples of data sets large enough to mark your expression as dangerous or invalid (packet tracing, environmental measurements, and more a lot of).

Answer 6

For sufficiently small N, O (N ^ N) in practice can be replaced by 1. Not O (1) (by definition), but for N = 2 you can consider it as one operation with 4 parts or constant work.

What if all operations take 1 hour? The difference between O (log N) and O (1) is then large even for small N.

Or if you need to run the algorithm ten million times? Ok, it took 30 minutes, so when I run it a hundred times larger in the dataset, it should still take 30 minutes, because O (logN) is “the same” as O (1) ... eh ... what?

Your statement that “I understand O (f (N))” is clearly false.

Real world applications, oh ... I don’t know .... EVERY USE O () is the designation EVER?

Binary search in a sorted list of 10 million items, for example. This is a very REASON, we use hash tables when the data gets big enough. If you think that O (logN) matches O (1), then why do you ALWAYS use a hash instead of a binary tree?

Answer 7

As many have said, for the real world you need to first look at the constant factors before even worrying about the O (log N) factors.

Then consider what you expect from N. If you have good reason to think that N <10, you can use linear search instead of binary. This is O (N) instead of O (log N), which according to your lights would be significant, but a linear search that moves the found elements to the front can outperform a more complex balanced tree depending on the application.

On the other hand, note that even if the N log cannot exceed 50, the performance factor of 10 is really huge - if you are involved in computing, such a factor can easily make or break your expression. If this is not enough for you, you will often see factors (log N) ^ 2 or (logN) ^ 3 in the algorithms, so even if you think you can ignore one factor (log N), this does not mean you can ignore them more .

Finally, note that the simplex linear programming algorithm has the worst performance O (2 ^ n). However, for practical problems, the worst case never arises; in practice, the simplex algorithm is fast, relatively simple, and therefore very popular.

About 30 years ago, someone developed a polynomial time algorithm for linear programming, but it was not originally practical because the result was too slow.

Currently, there are practical alternative algorithms for linear programming (with a polynomial time case, for what it costs), which in practice can surpass the simplex method. But, depending on the problem, the simplex method is still competitive.

Answer 8

The observation that O(log n) often indistinguishable from O(1) is a good one.

As a familiar example, suppose we wanted to find one element in a sorted array of 1,000,000,000,000 elements:

• with linear search, an average of 500,000,000,000 steps is searched
• with binary search, the search takes an average of 40 steps.

Suppose we add one element to the array that we are looking for, and now we must look for another element:

• with linear search, the search takes an average of 500,000,000,001 steps (indistinguishable changes)
• with binary search, the search takes an average of 40 steps (indiscernible change)

Suppose we doubled the number of elements in the array we are looking for, and now we must look for another element:

• with linear search, an average search takes 1,000,000,000,000 steps (an unusually noticeable change).
• with binary search, the search takes an average of 41 steps (indiscernible change)

As we can see from this example, for all purposes and purposes, the O(log n) algorithm, such as binary search, is often indistinguishable from the O(1) algorithm, such as omniscience.

The upload point is as follows: * we use O(log n) algorithms because they are often indistinguishable from constant time and because they often perform phenomenally better than linear time algorithms.

Obviously, these examples assume reasonable constants. Obviously, these are general observations and do not apply to all cases. Obviously, these points are applicable to the asymptotic end of the curve, and not to the end of n=3 .

But this observation explains why, for example, we use methods such as setting up a query to search for an index, rather than scanning a table - since index search works almost constant time regardless of the size of the data set, while table scanning is slow on large enough data sets . Search by index O(log n) .

Answer 9

You might be interested in Soft-O, which ignores the logarithmic value. Check out this item on Wikipedia.

Answer 10

What do you mean, is it important or not?

If you are faced with the choice of the O(1) and O(lg n) algorithms, then you should not assume that they are equal. You must choose a constant time. Why don't you be?

And if a constant-time algorithm does not exist, then logarithmic time is usually the best you can get. Again, does it matter? You just need to take the fastest that you can find.

Can you give me a situation where you get something by defining two peers? In the best case, this does not matter, and in the worst case you hide some real scalability characteristics. Since usually a constant-time algorithm will be faster than a logarithmic one.

Even if, as you say, lg(n) < 100 for all practical purposes, it is still a factor of 100 on top of other overheads. If I call my function N times, the question begins to arise whether your function works with logarithmic time or a constant, because then the total complexity is equal to O(n lg n) or O(n) .

So instead of asking, it’s “important” that you consider the logarithmic complexity constant in the “real world”, I would ask if that makes sense.

Often you can assume that the logarithmic algorithms are fast enough, but what do you get by considering them constant?

Answer 11

O (logN) * ​​O (logN) * ​​O (logN) is very different. O (1) * O (1) * O (1) is still constant. Also simple O (nlogn) in quicksort style is different from O (n O (1)) = O (n). Try sorting 1,000 and 1,000,000 items. The latter is not 1000 times slower, it is 2000 times, because log (n ^ 2) = 2log (n)

Answer 12

The title of the question is misleading (well-chosen for discussion, remember).

O (log N) == O (1) is obviously wrong (and the poster knows about this). The designation Big O, by definition, considers asymptotic analysis. When you see O (N), N is taken to approach infinity. If N is assigned a constant, it is not Big O.

Please note that these are not just trifles that only theoretical computer scientists need to take care of. All arithmetic used to determine the function O for the algorithm relies on it. When you publish the O function for your algorithm, you can omit a lot of information about this performance.

Big O analysis is cool because it allows you to compare algorithms without linking platform issues (word size, operation instructions, memory speed and disk speed). When N goes to infinity, these problems disappear. But when N 10000, 1000, 100, these problems, along with all the others that we left outside the O function, begin to matter.

To answer the question about the poster: O (log N)! = O (1), and you're right, algorithms with O (1) are sometimes not much better than algorithms with O (log N), depending on the size of the input and that's it those internal constants that were omitted during Big O's analysis.

If you know you're going to scroll through N, use Big O analysis. If you don't, you will need empirical tests.

Answer 13

In theory

Yes, in practical situations, log (n) is limited by a constant, we will say 100. However, replacing log (n) with 100 in situations where it is correct still discards information, making the upper bound of operations that you calculated weaker and less useful. Replacing O (log (n)) with O (1) in your analysis could make your large case n run 100 times worse than you expected based on your small n case. Your theoretical analysis could be more accurate and predict the problem before you build the system.

I would say that the practical goal of analyzing the big conclusion is to try to predict the execution time of your algorithm as early as possible. You can make your analysis simpler by deleting the terms log (n), but then you reduce the predictive ability of the estimate.

Almost

If you read the original articles by Larry Page and Sergey Brin on Google architecture, they talk about using hash tables for everything to make sure that, for example, finding a cached web page requires just one hard drive search. If you used B-tree indexes to search, you may need four or five attempts at the hard drive to search without a cache [*]. A fourfold set of requirements for your repository on your cached web page repository should be taken care of from a business point of view and predictable if you do not expel all O (log (n)) terms.

PS Sorry for using Google as an example, they are similar to Hitler in the computer version of Godwin’s Law .

[*] Assuming that 4KB reads from disk, 100bn web pages in the index, ~ 16 bytes per key in the node B-tree.

Answer 14

Yes, log (N) <100 for most practical purposes, and No, you cannot always replace it with a constant.

For example, this can lead to serious errors in evaluating the performance of your program. If O (N) programmed an array of 1000 elements in 1 ms, you are sure that it will process 10 elements ⁶ in 1 second (or so). If, however, the program is O (N * logN), then it takes ~ 2 sec to process 10 ⁶ elements. This difference can be crucial - for example, you might think that you have enough server power, because you receive 3000 requests per hour, and you think that your server can process up to 3600.

Another example. Imagine that you have a function f () that works in O (logN), and for every iterative function g () that also works in O (logN). Then, if you replace both logs with constants, you think your program is running at a constant time. The reality will be cruel, though - two magazines can give you up to 100 * 100 multipliers.

Answer 15

As others have noted, Big-O tells you how the effectiveness of your problem scales. Believe me - this is important. I came across several algorithms that were just awful and did not meet the requirements of the clients because they were too slow. Understanding the difference and finding a solution to O (1) is many times a huge improvement.

However, of course, this is not the whole story - for example, you may notice that quicksort algorithms will always switch to insertion sorting for small elements (Wikipedia says 8-20) due to the behavior of both algorithms on small data sets.

So, it is a question of understanding what trade-offs you will make, which implies a deep understanding of the problem, architecture and experience in order to understand what to use and how to set up the constants.

No one says that O (1) is always better than O (log N). Nevertheless, I can guarantee that the O (1) algorithm will also scale better, therefore, even if you make incorrect assumptions about how many users will be in the system or the size of the processed data, it does not matter to the algorithm.

Answer 16

The rules for defining Big-O notation are simpler if you don't decide that O (log n) = O (1).

As krzysio said, you can accumulate O (log n) s and then they will have a very noticeable difference. Imagine you are doing a binary search: O (log n) and then imagine that each comparison complexity is O (log n). If you neglect both, you get O (1) instead of O (log ² n). Similarly, you can somehow arrive at O ​​(log ¹⁰ n), and then you will notice a big difference for the not-too-large "n" s.

Answer 17

Suppose that in the whole application one algorithm is 90% of the time when the user is waiting for the most ordinary operation.

Suppose that in real time, the O (1) operation is second in your architecture, and the O (logN) operation is basically .5 seconds * log (N). Well, at that moment I would really like to draw you a graph with an arrow at the intersection of the curve and the line, saying: "It matters right here." You want to use log (N) op for small data sets and O (1) op for large data sets in such a scenario.

Big-O designation and performance optimization is an academic exercise, and not providing real value to the user for operations that are already cheap, but if it is an expensive operation on a critical path, then you put the matter!

Answer 18

For any algorithm that can accept inputs of different sizes N, the number of operations performed is limited from above by some function f (N).

All big-Os say you are a form of this function.

• O (1) means that there exists some number A such that f (N) A for large N.

• O (N) means that there exists some A such that f (N) AN for large N.

• O (N ^ 2) means that there exists some A such that f (N) AN ^ 2 for large N.

• O (log (N)) means that there exists some A such that f (N) AlogN for large N.

Big-O says nothing about how big A is (that is, how fast the algorithm works) or where these functions intersect. He only says that when you compare two algorithms, if their large OOs are different from each other, then there is a value N (which can be small or can be very large) when one algorithm starts to surpass the other.

Answer 19

you are right, in many cases it does not matter for pracitcal purposes. but the key question is "how fast is GROWS N". most of the algorithms that we know take input size, so it grows linearly.

but some algorithms have a value of N obtained in a complex way. if N is the "number of possible lottery combinations for a lottery with X separate numbers", it matters if your algorithm is O (1) or O (logN)

Answer 20

Big-OH ​​tells you that one algorithm is faster than another, given some constant factor. If your input implies a sufficiently small constant coefficient, you can see big gains in productivity by going to linear search, and not to the search vine (n) of some base.

Answer 21

O (log N) can be misleading. Take, for example, operations on Red-Black Trees .
The operations are O (logN), but quite complex, which means many low-level operations.

Answer 22

I do not believe algorithms where you can freely choose between O (1) with a large constant and O (logN) really exists. If at the beginning there are N elements for work, it is simply impossible to do this O (1), the only thing that is possible is to move N to some other part of your code.

What I'm trying to say is that in all the real cases that I know, you have a trade-off between space and time or some kind of preprocessing, such as compiling the data into a more efficient form.

That is, you really don't go O (1), you just move part N to another place. Either you exchange the performance of some part of your code with a certain amount of memory, or you exchange the performance of one part of your algorithm to another. To stay sane, you should always look at the larger image.

I want to say that if you have N elements, they cannot disappear. In other words, you can choose between inefficient algorithms O (n ^ 2) or worse, and O (n.logN): this is a real choice. But you never go O (1).

I am trying to point out that for every problem and condition of the source data there is a "better" algorithm. You can do worse, but never better. With some experience, you can have a good idea of ​​what this complex complexity is. Then, if your general treatment matches this complexity, you know that you have something. You cannot reduce this complexity, but only to move it.

If the problem is O (n), it will not become O (logN) or O (1), you just add some preprocessing so that the overall complexity is unchanged or worse, and maybe a later step to be improved. , , O (N) , O (NLogN), O (1).

? , .. . O (NLogN), O (N).

, , O (1) = O (LogN).

-;-), , O (1) O (LogN) O (LogN) O ( 1). , , - - ( , ..).

Answer 23

, , O (log N), N - . ... , , , , , . , O (logN) 100... , - , ... .