Official challenge to the 4th game in the scored LG section Forum

[citrustang]

Our original thread was locked/deleted, so I need to tread lightly. I will only make passing reference to the specifics of the particular issue I am raising, so as not to anger the Mods or the LSAC. As such, I can no longer post a copy of the letter I am sending to the LSAC, since it contains many details that I am prohibited from discussing in the public square. For those of you who recognize or will eventually recognize what I am talking about, please follow a similar code of conduct and refrain from posting anything that might result in bans or the locking/deletion of this thread.
--------------------------------------------------------------------------------
Now that's out of the way, let's talk in general about the fourth game in the scored LG section.

I believe there is a case to be made that the fourth LG contained a critical ambiguity in its use of a single term. I will avoid disclosing the term here. But the term in question can, in common language, be used to describe two different, competing elements from the game.

2 a : an area or division of an activity, subject, or profession
b : the sphere of practical operation outside a base (as a laboratory, office, or factory)

Judging from last night's discussion, many of us interpreted the term incorrectly and suffered the consequences of trying to solve an unworkable game. Some recognized the ambiguity and were able to go back and start over, this time using the correct interpretation, while others were not so lucky. Just to be clear, this was not a case of simply misreading a rule; this was an instance where reading a rule conceivably yielded two distinct meanings and test-takers were forced to choose between two different, seemingly valid, interpretations. The issue reared its ugly head when the term in question failed to appear in the opening paragraph of the LG and test-takers were forced to grapple with what, if appearing in an LR section, would be considered a shift in scope.

Why is ambiguity in LG language an issue?

(excerpt from LSAC handbook found at http://www.lsac.org/pdfs/Policies-web.pdf)

The above is a list of three criteria every LSAT question must meet before being included on an official test. Historically, only individual LR questions have been successfully challenged on the grounds that ambiguous language allowed for more than one correct answer (and thereby breaking at least two of the three rules). So it is with a sober realization of the far-reaching implications of my challenge that I am writing to you all. Ambiguous language in LG is especially problematic when it appears in the opening paragraph or any of the rules. An ambiguity that appears in the setup is bound to affect one's ability to answer all of the corresponding questions for that game. This is why the LSAT writers are so careful to use phrases like "sits immediately to the left of" and "there is exactly one space between them." Clear, unequivocal language is necessary to the functioning of a well-designed LG. Can you imagine a rule that read "Jack sits over there, but Jill sits over there"? It's unclear what "there" is referring to in either instance, and whenever there is uncertainty in the interpretation of an LG rule, there is a problem.

I think that's as far as I'm allowed to proceed, so I will stop there. If none of the above made sense to you, then I am sorry to have wasted your time. However, if while reading my post you thought to yourself, "I completely know what he's talking about and I encountered the same difficulty," then I invite you to write an official challenge to LSAC.

The policy handbook for challenging LSAT questions can be found here: http://www.lsac.org/pdfs/Policies-web.pdf
The general guidlines for a challenge are as follows:

How can I inquire about a test question?

If, while taking the LSAT, you find what you believe to be an error or ambiguity in a test question that affects your response to the question, report it to the test supervisor as soon as you finish the test and write immediately to: Law School Admission Council, Test Development, 662 Penn Street, Box 40, Newtown, PA 18940-0040. You may also contact us by e-mail at LSATTS@LSAC.org. The LSAC document, Policies and Procedures Governing Challenges to Law School Admission Test Questions PDF Icon can be found here.

(http://www.lsac.org/AboutLSAC/faqs-and- ... #questions)

Please write professionally and in a well thought-out manner if you decide to submit a challenge on this issue. LSAC can simply ignore flippant or poorly crafted emails and faxes. If the battle is to be won, it will be won using reason and convincing argumentation. Do your future profession proud, and fight this cause if you deem it justified. I know there are others out there who noticed this, and I hope you will be willing to speak up.

I thank you all for your time and I apologize for the length of this post. Feel free to contact me via PM.

[Kiersten1985]

are you suggesting we all inquire/challenge the question individually or are you trying to set up some petition or something?

4th game was crap, btw.

[citrustang]

The problem with a petition is that only a document containing very specific argumentation will hold any water. I'm concerned posting such a document would be a bannable offense and in violation of my non-disclosure agreement with LSAC. I can't think of a workaround right now, so in the meantime, individual challenges seem like the only option.

[3|ink]

4th game was crap, btw.

Do you mean crap as in 'easy' or crap as in 'poorly written'.

[scruffs mcguff]

I'm pretty sure this game was on a past LSAT I took. Given that it was tested in actual test settings, don't you think all of this has been thought out and worked to where the game meets the standards?

If anyone else can confirm seeing this game before, chime in. Also to avoid angering mods you can PM me if you think you saw it and on what test and maybe we can confirm/deny?

[truffleshuffle]

I know the exact term your talking about, but if you interpreted it the way it was meant then this game was extremely simple. If you were to read it closely now, I'm sure you would see that the correct interpretation was the only possible one. There's a change in tense that I'm pretty sure caused all of this confusion.

[jbill]

I took me about 20 seconds to figure out what the testers meant by the term.

However, a different term was used in the next rule. So by the process of elimination, one could readily figure out what the term meant.

That being said, having to take those steps to figure out what a rule was referring to was an extra and perhaps slightly unfair burden. Yet I do not think it warrants a challenge to the marking of the test itself.

Rather, it'd be best reflected in the curving.

[DaveBear07]

citrustang,

As one who also mis-interpreted the rules, I will be writing as well. If we missed something and LSAC deems that fair then so be it.

However, to the rest of my fellow TLS'ers, I believe it is not too much to ask for a sincere review of the phrasing of the rules, especially since so many people were taken by it.

[citrustang]

I'm pretty sure this game was on a past LSAT I took. Given that it was tested in actual test settings, don't you think all of this has been thought out and worked to where the game meets the standards?

If anyone else can confirm seeing this game before, chime in. Also to avoid angering mods you can PM me if you think you saw it and on what test and maybe we can confirm/deny?

I cannot comment on the process LSAC uses to make sure questions meet appropriate standards. However, I can speak up when I have encountered what I, and many others, have identified as a major flaw in the test. The test-writers and proof-readers are great at what they do: 99.9% of the time the questions are clear, unambiguous, contain only one correct answer, and are reasonably fair and sensitive. We are not dealing with one of those times. We are talking about a decidedly rare, but nonetheless egregious, oversight that introduced ambiguity to a situation where clarity is paramount. An LG with ambiguous rules is not a difficult one, it is a poorly-constructed one. Let's not confuse the two.

[citrustang]

I took me about 20 seconds to figure out what the testers meant by the term.

However, a different term was used in the next rule. So by the process of elimination, one could readily figure out what the term meant.

The same ambiguous term appeared in both rules.

[Jessss22]

Most of the time there is always going to be only one single correct answer, however certain situation arises when words carrying on different meanings, especially since two words were used interchangeably. It is LSACs obligation to be cautious of using such words.The LSAc should not jeopardize our time by forcing us to figure out exactly what is the correct interpretation of a word. Our job for the exam is to set up a game based on specific set of rules, and how could that be if there is ambiguity in one of those rules. That means precious time is wasted on trying to figure out the context of a word and that can produce several correct interpretations.

[whatjusthappened]

I'm definitely on board. That term was COMPLETELY ambiguous. It could EASILY mean two things.

[whatjusthappened]

I'm pretty sure this game was on a past LSAT I took. Given that it was tested in actual test settings, don't you think all of this has been thought out and worked to where the game meets the standards?

If anyone else can confirm seeing this game before, chime in. Also to avoid angering mods you can PM me if you think you saw it and on what test and maybe we can confirm/deny?

When you took it as an experimental, did it have that ambiguous term also?

[Jessss22]

for some fortunate test takers it only cost 20 seconds time, for others however that have a sort of left sided creative brain, it might of took longer than that. So it is not fair to base your opinion as fact for what was manageable for you. At least there is a fair amount of people that can argue there was indeed ambiguity.

[3|ink]

It seemed easier on experimental. I was able to get through at least 3 of them. I would have been perfectly satisfied with that and missing only 3 in this section. It's so annoying because I totally destroyed the other three games. The crucial deduction here was pretty terrible.

Dino game had only two variable sets. That's why it was easier.

[citrustang]

a. good job keeping it non-descript in here.

b. keep it that way

carry on

Thanks, will do!

[whuts4lunch]

"A candidate who has taken the LSAT and whose answer to a question has not been scored as the “credited response” (i.e., has been scored incorrect) is entitled to have the question reviewed pursuant to the following policy if, within 90 days after a score report is sent to the candidate, he or she files with LSAC an initial written inquiry about, or challenge to, the scoring of the question, stating and supporting the reasons why the credited response is not the one and only best answer to the question."

It says that candidates may have questions reviewed within 90 days after receiving a score report. It doesn't say whether a candidate can ask for a review prior to the release of score reports.

[whatjusthappened]

I'm pretty sure this game was on a past LSAT I took. Given that it was tested in actual test settings, don't you think all of this has been thought out and worked to where the game meets the standards?

If anyone else can confirm seeing this game before, chime in. Also to avoid angering mods you can PM me if you think you saw it and on what test and maybe we can confirm/deny?

I cannot comment on the process LSAC uses to make sure questions meet appropriate standards. However, I can speak up when I have encountered what I, and many others, have identified as a major flaw in the test. The test-writers and proof-readers are great at what they do: 99.9% of the time the questions are clear, unambiguous, contain only one correct answer, and are reasonably fair and sensitive. We are not dealing with one of those times. We are talking about a decidedly rare, but nonetheless egregious, oversight that introduced ambiguity to a situation where clarity is paramount. An LG with ambiguous rules is not a difficult one, it is a poorly-constructed one. Let's not confuse the two.[/quote]

TITCR

But the creators of the LSAT spent 10s of thousands of dollars on this test. They are not going to throw out an ENTIRE game. We're not talking about one single poorly constructed LR question here. We're talking about an entire game of six questions. I don't think they will do this, because it would be absolutely catastrophic.

Nonetheless, OP is completely right. This game is complete crap, and the person who created it needs to be fired for incompetence.

[Jessss22]

for-the-record: "Creative" brain types are right-brained.

haha.. thanks for that oops

[whatjusthappened]

Our original thread was locked/deleted, so I need to tread lightly. I will only make passing reference to the specifics of the particular issue I am raising, so as not to anger the Mods or the LSAC. As such, I can no longer post a copy of the letter I am sending to the LSAC, since it contains many details that I am prohibited from discussing in the public square. For those of you who recognize or will eventually recognize what I am talking about, please follow a similar code of conduct and refrain from posting anything that might result in bans or the locking/deletion of this thread.
--------------------------------------------------------------------------------
Now that's out of the way, let's talk in general about the fourth game in the scored LG section.

I believe there is a case to be made that the fourth LG contained a critical ambiguity in its use of a single term. I will avoid disclosing the term here. But the term in question can, in common language, be used to describe two different, competing elements from the game.

2 a : an area or division of an activity, subject, or profession
b : the sphere of practical operation outside a base (as a laboratory, office, or factory)

Judging from last night's discussion, many of us interpreted the term incorrectly and suffered the consequences of trying to solve an unworkable game. Some recognized the ambiguity and were able to go back and start over, this time using the correct interpretation, while others were not so lucky. Just to be clear, this was not a case of simply misreading a rule; this was an instance where reading a rule conceivably yielded two distinct meanings and test-takers were forced to choose between two different, seemingly valid, interpretations. The issue reared its ugly head when the term in question failed to appear in the opening paragraph of the LG and test-takers were forced to grapple with what, if appearing in an LR section, would be considered a shift in scope.

Why is ambiguity in LG language an issue?

(excerpt from LSAC handbook found at http://www.lsac.org/pdfs/Policies-web.pdf)

The above is a list of three criteria every LSAT question must meet before being included on an official test. Historically, only individual LR questions have been successfully challenged on the grounds that ambiguous language allowed for more than one correct answer (and thereby breaking at least two of the three rules). So it is with a sober realization of the far-reaching implications of my challenge that I am writing to you all. Ambiguous language in LG is especially problematic when it appears in the opening paragraph or any of the rules. An ambiguity that appears in the setup is bound to affect one's ability to answer all of the corresponding questions for that game. This is why the LSAT writers are so careful to use phrases like "sits immediately to the left of" and "there is exactly one space between them." Clear, unequivocal language is necessary to the functioning of a well-designed LG. Can you imagine a rule that read "Jack sits over there, but Jill sits over there"? It's unclear what "there" is referring to in either instance, and whenever there is uncertainty in the interpretation of an LG rule, there is a problem.

I think that's as far as I'm allowed to proceed, so I will stop there. If none of the above made sense to you, then I am sorry to have wasted your time. However, if while reading my post you thought to yourself, "I completely know what he's talking about and I encountered the same difficulty," then I invite you to write an official challenge to LSAC.

The policy handbook for challenging LSAT questions can be found here: http://www.lsac.org/pdfs/Policies-web.pdf
The general guidlines for a challenge are as follows:

How can I inquire about a test question?

If, while taking the LSAT, you find what you believe to be an error or ambiguity in a test question that affects your response to the question, report it to the test supervisor as soon as you finish the test and write immediately to: Law School Admission Council, Test Development, 662 Penn Street, Box 40, Newtown, PA 18940-0040. You may also contact us by e-mail at LSATTS@LSAC.org. The LSAC document, Policies and Procedures Governing Challenges to Law School Admission Test Questions PDF Icon can be found here.

(http://www.lsac.org/AboutLSAC/faqs-and- ... #questions)

Please write professionally and in a well thought-out manner if you decide to submit a challenge on this issue. LSAC can simply ignore flippant or poorly crafted emails and faxes. If the battle is to be won, it will be won using reason and convincing argumentation. Do your future profession proud, and fight this cause if you deem it justified. I know there are others out there who noticed this, and I hope you will be willing to speak up.

I thank you all for your time and I apologize for the length of this post. Feel free to contact me via PM.

OP, don't you think a petition would be more effective? I have a feeling that very few people will take the time to write LSAC a letter.

[Pahnda]

I'm not sure if a review would be very fruitful, unfortunately... I personally didn't find the term in this game (Intern/Assistant Q, if I'm not mistaken) very ambiguous. However, even if you took the definition to mean the 'location' rather than the 'type' of work, it would have essentially broken the game. Broken meaning that the game would boil down to two templates that are so specific that I know at least one of the questions would have no correct answers.

I agree that this was the most difficult game, but provided you understand the above and got a deduction about the two exclusion/inclusion rules combined with restrictiveness on one of the groups, it seems to follow quite logically.

[jdstl]

Come on man, the fourth game was tough, but not ambiguous. I mis-read the same wording as so many others, but when I went and looked at the actual wording it was quite clear.

[whuts4lunch]

Even if a whole game could be thrown out, the LSAC couldn't adjust for mistakes made on other parts of the test resulting from the flawed game. For instance, say a test taker left a few choices blank in an earlier game and planned to go back to them after finishing the 4th game. Say again that the test taker did not have time to go back after reworking the entire 4th game, but would have had enough time to go back had the rules in the 4th game been made clear. Or, what if the ambiguity of the game had a profound psychological impact on test takers that hurt performance on subsequent sections?

[citrustang]

"Ambiguity" in LG must necessarily refer to different potential interpretations, and those interpretations must be considered not according to their "legitimacy" but instead by "whether a reasonable level of uncertainty could lead to another, incorrect, yet supportable, interpretation." I can frame the issue in a different way: the issue is not whether the correct interpretation could result from the prompt but whether or not it must result from the prompt. No one is going to argue with you that there is only one correct interpretation and the other is incorrect. The problem is that a test-taker shouldn't need to rely upon unsuccessfully attempting questions in order to recognize his/her reasonable interpretation of a particular word was the wrong one of two possibilities.

LG is supposed to test many things, the above has not historically been one of them.

Taken from last night's thread. If you had no issue with the wording in the fourth LG, then congratulations. You are not being asked to send a letter of challenge. Those who are being asked know who they are.

[whatjusthappened]

Even if a whole game could be thrown out, the LSAC couldn't adjust for mistakes made on other parts of the test resulting from the flawed game. For instance, say a test taker left a few choices blank in an earlier game and planned to go back to them after finishing the 4th game. Say again that the test taker did not have time to go back after reworking the entire 4th game, but would have had enough time to go back had the rules in the 4th game been made clear. Or, what if the ambiguity of the game had a profound psychological impact on test takers that hurt performance on subsequent sections?

No, because they throw out LRs occasionally, which is the same thing but on a smaller level.