0

Analysis of Algorithms: Kinds of Algorithm Analysis

Preceded by: insertion sort; example; transition to analysis

0 So, (lifts the blackboard) there are different kinds of analyses that people do.
There are different kinds of algorithm analyses. (Writes on the blackboard: Kinds of analysis)
analyses irregular plural of analysis. Other such irregular plurals are theses (pl. of thesis) and crises (plural of crisis).
20.19999999999997 The one we're mostly going to focus on is what's called "worst-case analysis," OK, and this is what we do usually, where we define T(n) to be the maximum time on any input of size n. So it's the maximum input, the maximum it could possibly cost us on an input of size n.

We will mostly focus on the kind of analysis that is called "worst-case analysis." This is the most common kind of analysis. This is the kind of analysis that we usually do. In worst-case analysis, we define T(n) to be the maximum time on any input of size n. So it's the maximum it could possibly cost us to run the algorithm on an input of size n.

this is what we do usually a reference to what is being written on the blackboard

The lecturer repeats important concepts two or three times.

The maximum it could possibly cost us What is "it" here? The action of running the algorithm on an input of size n.

The one one is an indefinite pronoun here, referring to kinds of analyses.

the maximum input He means "the maximum-time input, the input that takes the most time within inputs of the same size."

58.667 So what that does is -- if you look at the fact that sometimes the inputs are better and sometimes they are worse -- we're looking at the worst case of those because that's the way we're gonna be able to make a guarantee: it ALWAYS does something, rather than just sometimes does something, OK -- so we look at the maximum.

If you look at the differences between inputs -- if you take into account that sometimes the inputs are better [and take less time], and sometimes they are worse [and take more time], [you will realize that] we're looking at the worst case and the maximum time because we want to make a guarantee that [within that time] the algorithm will always (on all inputs) do something, rather than only sometimes (on some inputs) do something. That's why we look at the maximum.

what that does is... One of many unfinished sentences that could have been: What that does is it makes it possible for us to give a guarantee.

the fact that An expression that converts a clause into a noun phrase. There are usually better ways to express what you want to say. Two possible strategies are: (1) use the same verb followed by a noun phrase that summarizes the clause; (2) use a verb that can be followed directly by a sentence, such as take into account. Both strategies are illustrated in the paraphrases.

77.66699999999999 Notice that if I didn't have maximum then T(n) in some sense is a relation, not a function, because the time on an input of size n -- well it depends on which input of size n. I can have many different times. But by putting the maximum at it, it turns that relation into a function because there's only one maximum time that it will take. OK?

Note that if I didn't have maximum, T(n) would be a relation, not a function, because the time on an input of size n [could have many different values because] it depends on which input of size n [is given]. But by putting the maximum on it, we turn it (it=T(n)) into a function because there is only one maximum time that it (it=the algorithm) will take.

if I didn't have maximum ...T(n) is a relation The lecturer starts with a counter-to-fact condition but continues in the Present tense. This is sloppy and should never be allowed in writing.

relation... function You need to know these terms to understand what he is saying.

By putting the maximum on it, it turns a relation into a function.. This is also sloppy. The implied subject of putting is we, and it should be the subject of the main sentence: By putting the maximum on it, we turn the relation into a function.

Putting is similar to both a noun and a verb. Like a noun, it can follow a preposition (by putting...). Like a verb, it can have a direct object (putting a maximum on it). Such forms are called gerunds, and the entire phrase is called a gerund phrase.

104.333 Sometimes, we'll talk about average case. So sometimes we'll do this. (Writes on the blackboard.) Here T(n) is then the expected time over all inputs of size n. OK? So, it's the expected time. Now, if I talk about expected time, what else do I need to say here? What does that mean, expected time? (A student says something that the lecturer can't hear.) I'm sorry? Raise your hands so I can see who's [asking the question]. "Expected inputs" -- what does it mean "expected inputs"? (A student says something.) OK, I need .. I need more math. What do I mean by "expected time" here? Math!

Sometimes, we will do average case analysis. So sometimes we will define T(n) to be the expected time of algorithm over all inputs of size n. If I talk about expected time, what else do I need to say? What does "expected time" mean? (Speaking to a student) I'm sorry? Raise your hands so I can see who is talking. (Repeating a student's words) "Expected inputs" - what does it mean, "expected inputs"? I need a more mathematical answer. What do I mean by expected time here?

sometimes we'll do this This is the beginning of the next "section" of the lecture. The previous section started with This is what we do usually We see a contrast between "do usually" and "do sometimes.".

I'm sorry (or Excuse me or I beg your pardon) These apologies, with a specific intonation, mean "I haven't heard what you said, please repeat."

167.8 (Points to a student.) Over here. (A student says something.) You have to take the time of every input and then average them. OK. That's kind of what we mean by "expected time." Good. Not quite. OK -- I mean -- I -- what you say is completely correct except it's not enough -- not quite enough. (Points to a student) Yeah. It's the time of every input times the probability that it will be that input -- which is the way of taking the weighted average, that's exactly right. How do I know what the probability of every input is? What's ... How do I know what the probability that a particular input occurs is? in a given situation? I don't!

(Pointing) Over here. (Repeating after the student; note the "quoting" intonation.) You have to take the time of every input and average them. That's kind of what we mean but not quite. What you say is completely correct except it's not enough. (Pointing) Yes? (Repeating) It's the time of every input times the probability of that input. This is called "the weighted average." But how do I know what the probability of every input or a particular input is? I don't!

You have to take the time of every input and then average them. The lecturer always repeats what the student says so everybody in the lecture room can hear. See also below: It's the time of every input times the probability that it will be that input.

OK, That's kind of what we mean... Good. Not quite The lecturer tries to be positive and encourage the students. Even when he has to say No, he says Not quite instead.

What you say is completely correct except it's not enough Another example of being positive.

221 OK, I have to make an assumption. What's that assumption called? What kind of assumption do I have to make? I need an assumption -- of -- right, of the statistical distribution OK of inputs.

I have to make an assumption. What is that assumption called? What kind of assumption do I need to make? I need an assumption of the statistical distribution of inputs.

make an assumption One of several expressions with the verb make:

suggest - make a suggestion
propose - make a proposal

254.333 Otherwise, expected time doesn't mean anything because I don't know what the probability of something is. In order to do the probability, you need some assumptions.

Otherwise, without such an assumption, expected time doesn't mean anything because I don't know what the probability of each data item is. In order to calculate with probabilities, you need some assumptions.

probability of something a generic pronoun is used instead of a more precise noun
265.333 OK You've got to state those assumptions clearly. So one of the most common assumptions is that all inputs are equally likely. That's called the Uniform Distribution. OK Every input of size n is equally likely, that kind of umm thing.

You've got to state those assumptions clearly. One of the most common assumptions is that all inputs are equally likely. That kind of the statistical distribution is called the Uniform Distribution. Every input of size n is equally likely.

you've got to same as you have to but more emphatic. For even more emphasis, use the full form: you have got to

that kind of thing. The lecturer repeats an important concept, feels a need to do more than simply repeat what has just been said, and adds a phrase that doesn't mean much.

283.66700000000003 But there are other ways that you could make that assumption, and they may not all be umm be true. So this is much more complicated, as you can see. Fortunately, all of you have a strong probability background, and so, so we will not have any trouble addressing these probabilistic issues of dealing with expectations and stuff. OK? So if you don't, -- time to go and say "Gee, maybe I should take that probability class, it's a pre-requisite for this class."
You could make that assumption in some other way. You could assume another statistical distribution, and it may not be true. So, this is much more complicated, as you will see. Fortunately you all have studied probability and have a strong background in probability, and so you will have no trouble addressing probabilistic issues such as expectations and so on. If you don't have a strong background in probability, it is time to say to yourself: "Gee, maybe I should take that probability class, it is a prerequisite for this class."

dealing with expectations and stuff A number of general words in this phrase

deal with X "take some unspecified action with respect to X such that the problem having to do with X will be solved or closer to solving"

they may not all be be true What does it mean for a statistical distribution to be true or not true? Some statistical distributions describe the probabilities of input much better than others: some of them are true, others are false.

much more complicated More complicated than what? He doesn't say. Probably:" more complicated than the worst-case analysis."

and stuff an informal way of saying and so on.

if you don't (have a strong probability background)

Gee! An exclamation that may mean surprise at learning or discovering a new idea.

314.667 OK, the last one I want to mention is the best case analysis, and this, I claim, is bogus. Bogus! No Good! Why is best-case analysis bogus? (Points to a student) Yeah? (Repeats after student) The best case probably doesn't ever happen. (Expressing doubt) Umm Actually, that's interesting, because for the sorting problem the most common things that get sorted are things that are already sorted. Interestingly. So, for example, at least almost sorted.

The last kind of analysis I want to mention is the best case analysis, and this, I claim, is bogus. Why is it bogus? [A student says:] "The best case probably doesn't ever happen." This suggestion points to an interesting fact: it is very common that things that get sorted by a sorting algorithms are things that are already sorted [which is the best case for insertion sort]. So for example (interrupts to correct himself) or things that are almost sorted.

bogus fake, phony, pretending to be something it is not

Umm The intonation clearly suggests that the lecturer disagrees with the student.

Actually, that's interesting, because... Instead of saying "You are wrong" the lecturer points to an interesting fact that contradicts the student's answer. In effect, he turns the student's answer into a question ("Does the best case every happen?") and it turns out that the best case for insertion sort does happen, often.

get sorted... are sorted The first occurrence of sorted describes the action of sorting; the second occurrence describes the condition that results from such an action.

get sorted... are already sorted The word sorted in get sorted refers to the action of sorting. The word sorted in are sorted refers to the condition of being in order. Some things are sorted because they have been sorted. Other things are sorted naturally, before any algorithm sorts them.

358.13300000000004 So, for example one of the most common things that's sorted is check numbers, by banks. They tend to come in in the same order that they are written. OK so they are sorting things that are almost always sorted.
(Starting the sentence again) So for example, one of the most common things that get sorted are check numbers [of checks that have been written by the owner of a checking account]. [The numbers are read off checks as they come into the bank], and they tend to come ordered by number because they tend to come in the same order in which they are written [the account owner], [and the account owner usually writes them in the order of their numbers.] So the banks are sorting things that are almost always sorted.
check numbers In order to understand this example, one needs to be familiar with individual checking accounts.
370.93300000000005 OK -- it's good but... OK, you wanna... (The same student says something.) An upper bound not a lower bound. Yeah, you want to make a guarantee, and so why is this not a guarantee? Um OK So, I mean, you're onto something there, you're onto something, but we need a little more precision here. How can I cheat? (Points to a student) Yeah? (Student says something.) Yeah, yeah, you can cheat!
What you said is good but [it's not quite right]. OK, you want to make another suggestion? (Repeats after student) We want an upper bound, not a lower bound. Yes, you want to make a guarantee, and why is this (best-case estimate) not a guarantee? You are onto something here, but we need more precision. How can I cheat? (In response to a student) Yes, you can cheat.
lower, upper lower is the comparative form of the adjective low; upper is not a comparative form of anything, it's just an adjective that is the opposite of lower.
411 OK. You know, you cheat when you take any slow algorithm you want and just check for some particular input, and if it's that input you say immediately: "Yeah, OK, here's the answer." And it's got a good best case, but it didn't tell you anything about the vast majority of what's going on. OK? So you can cheat, ok, with a slow algorithm that works fast on some input. Doesn't really do much for you. OK? So we don't normally worry about that.
You can always create very good best cases for a slow algorithm by checking for specific inputs and immediately producing the answer. This is cheating. So a good best case doesn't tell you anything about the majority of cases. So, you can cheat by taking a slow algorithm and making it work very fast on a specific input. This does not do anything useful for you. This is why we don't normally consider best cases.

the vast majority of what's going on Usually, the expression the vast majority of is followed by a countable noun in the plural: the vast majority of inputs.

Doesn't really do much for you. There is no subject in this sentence, and the phrase for you is pronounced with the emphasis on the preposition: FOR ya. The lecturer is imitating a very informal manner of speaking.

447.333 OK. So, let's see, what is insertion sort's worst case time?
So, returning to our main topic, what is the worst-case time of insertion sort?
let's see This expression is often used to give the speaker some time to think, especially before answering a question. Here, it is used to signal a change in topic, which is really a return to an earlier topic.
458.333

Grammar topics:

  • Spoken language:
    • simple connections (so, OK)
    • fillers (umm, actually, essentially)
    • pointing phrases (right here, over there, in there)
  • Irregular plural of nouns
    index - indices; analysis - analyses
  • Gerunds and gerund phrases
  • Contrary-to-fact conditionals and the use of would

Expressions:

the fact that; look at (an abstract thing); the same thing; we're done (are you done?); take (time duration);

Vocabulary:

thing; stuff; depend on;

CS terminology :

loop, inner loop, outer loop, loop invariant; array index, indices; algorithm analysis; running time; upper bound; real-time;